Chomsky’s been involved in AI since the 1950s. His DNA is inside of all AI work at some level, particularly context-free grammar and how words are parsed on computers.
However, he was not a fan of neural networks as a model. He’s from the mainframe era where efficiency was important.
Neural networks are big and sloppy and imprecise and work on statistical inference.
He never liked neural networks, although he’s not as prone to extoll the praises of “hard-wired” as his student Pinker is. Chomsky’s always looking for the minimalist algorithm whereas Pinker was looking for an evopsych explanation for things.
I’m not a fan of either Pinker’s or Chomsky’s politics but being a linguist doesn’t make you good at political theory.
Same for George Lakoff. His embodied cognition is one I lean more strongly towards than either Pinker’s evopsych or Chomsky’s minimalist algorithm/language module thinking
(“George Lakoff, for example, holds that reasoning and language, arise from the nature of bodily experiences and, thus, even people’s own metaphors have bodily references”)
but Lakoff’s forays into political writing I try my best to ignore for while metaphors matter (and I am a fan of their power) and I’m aligned with a lot of that way of thinking, his analysis and suggestions for political speechwritings is cringy and feels a little too “17 words for snow” sloppy-thinking to me).
Each have their strengths and weaknesses.
I lean still towards connectionism which formed the basis of modern statistical inference engines like the various neural networks although it had a naivity to it.
I wasn’t crazy about Google’s groundbreaking 2014 “cheat” which inserted a “ground truth” notion to the neural network / statistical process , flipping the script into “Here’s what we already believe to be true and now let’s create some wiggle room around it and see how close we can get to it via prediction” rather than an open “what’s that?” which took up too many resources.
So yeah, in the gist of it, Chomsky’s not wrong here. the Chat engines work because they ‘liberated’ other people’s work and modified it ever so slightly so as to avoid copyright infringement. That’s what it’s very good at; But it does it so damn well that it becomes an slightly flawed Oracle of sorts and is really a blast to work with.
Microsoft invested $20 billion+ into a supposedly “non-profit” that wasn’t – all the sly games being played by various chat-engines honing in on the action… I’m fine with it because I’m enjoying it.
I’m not a purist like Chomsky. I know how it works and I don’t mind.