Geoffrey Hinton: The AI Godfather’s Final Warning | Digital vs Biological Intelligence

0:02 – 15:33

From backprop to meaning: the tiny model that foreshadowed modern language models

He opens with a threat disguised as a joke: if you sleep well tonight, you may not have understood the lecture. Then he goes back to an old split inside AI itself. One camp thought intelligence was mainly reasoning with symbols and rules. The other thought intelligence was mainly learning in networks of neuron-like units, with reasoning emerging later. Hinton places himself squarely in the second camp, and he starts from the simplest possible picture: artificial neurons take inputs, multiply them by weights, add them up, and learn by changing those weights. The key move is backpropagation. Instead of nudging one weight at a time and seeing whether performance gets better, you run the network forward, compare its answer with the target, then send error information backward so every connection can adjust in parallel. Hinton makes the contrast vivid with scale. In modern networks there can be about a trillion weights, so mutation-style trial and error would be hopelessly slow. Backprop was the practical breakthrough, and he points to the moment it became undeniable: in 2012, Alex Krizhevsky and Ilya Sutskever built AlexNet, which blew past older computer vision systems and opened the floodgates for neural nets to take over AI. Then he turns to language, where his real argument begins. Chomsky-style linguistics, he says, obsessed over syntax and missed the deeper function of language: words are building blocks for models of the world. To make that concrete, Hinton revisits a tiny network he built in 1985 to learn relations in two family trees, one English and one Italian. The network took a person and a relationship as input, turned each into a learned feature vector, then predicted the output person. What mattered was not memorizing sentences like "Colin has father James". What mattered was that the system learned features such as generation level and relation type, then combined them into rule-like behavior. If the input person is generation three and the relationship means one generation up, the output should be generation two. That is the unexpected angle of the whole section. Hinton says the old symbolic theory of meaning and the feature-based psychological theory are not enemies at all. They are two halves of the same picture. Relations between words can be captured through learned features and their interactions. He insists this is also how today’s chatbots work. They do not store strings of language and spit them back out. They store ways of turning words into features and using those features to predict what comes next. In his telling, the giant systems of today are descendants of that little 1985 model, just scaled up with more words, more layers, more data, and better architectures.