I love what you said about “learning to recognize a model from across the room”.
That’s crucial in identifying the weak spots in systems. [I generalize to “systems” as I’m not an AI focus in particular but systems in general] These don’t invalidate models per se, but they’re the point that deserves the most scrutiny as they have the greatest potential the smallest error to become magnified and become completely unnoticed either by the further parts of the systems or, more precisely, the people who are working with these systems.
I love finding those “spots”. So much to be learned there. I once did a study on paradoxes: what did they have in common? When I found the answer, it was a “duh of course!” moment – and I suspect you already know the answer. It doesn’t invalidate their importance _as_ paradoxes but it does put them in their proper place.
I start with a conception of things whereby: All is unique. But that’s just too much data isn’t it? Humans can’t process without patterns.
Once you start pattern seeking, you begin the reduction process. It’s necessary: without it, there’s just too much. So, where is the magic? It’s, to me in the category of “check your assumptions”. The one thing that gets “poo poo’d” as no big deal may turn out to be everything important about the thing.
It’s easy to get lost in a marvelous model that works perfectly “IF”.
What’s the “IF”?
===== [the following I removed] ====
My less tactful way of putting it is:
“Spot the Bullshit.”
[I’m not implying anybody’s bullshitting here; but in generally a good skill to acquire whenever someone is selling you something, whether it’s a used car or a concept. I try to do it carefully when someone is working on a lot of enthusiasm: I don’t like bursting bubbles… but if I think someone’s “going places” with an idea, I’d rather help them get through the rough patches of their model with me than find themselves slackjawed at a very important meeting and they’re stuck without an answer].
[but generally I just encourage people. It’s only when people are at that stage where they’ve got swords up and they’re ready for the challenge that I do it. Otherwise I give a lot of thumbs up, you can do it type stuff. [that’s usually all I want most of the time myself ]