Andrej Karpathy: Software Is Changing (Again)

0:01 – 5:45

From code to weights to English: why software changed again

He opens with a jolt: software barely changed at a fundamental level for about 70 years, then changed "about twice quite rapidly in the last few years." That is the frame for everything that follows. Karpathy starts with a map of GitHub as a map of software itself, then introduces his older distinction between software 1.0 and 2.0. Software 1.0 is the familiar world of code, the instructions humans write directly for computers. Software 2.0 is different: the "code" is now the weights of a neural network, created not by typing logic line by line but by curating datasets and running optimization. Hugging Face, in this view, is basically GitHub for software 2.0. He makes it concrete with image models like Flux, where each fine-tune acts almost like a git commit in weight space. Then he turns the screw one more time. Neural nets used to be mostly fixed-function systems, image in, category out. The big shift is that large language models became programmable. That is why he calls them software 3.0. The program is now the prompt, and the striking part is that the programming language is English. His sentiment classification example shows the progression clearly: you can write Python, train a neural net, or prompt an LLM with a few short lines of natural language. What used to require specialized syntax can now be expressed in ordinary language. He ties this to his Tesla experience, and this is where the abstract model becomes memorable. In Autopilot, he watched the software stack change from the inside. There was a large amount of C++ code doing explicit logic, then neural nets grew in capability and size, and "all the C++ code was being deleted." Tasks like stitching information across cameras and across time moved into the neural net. His phrase is vivid because it is literal: software 2.0 "ate through the software stack of the autopilot." Now he sees the same thing happening again with LLMs. The lesson for students is practical, not philosophical. If software now exists in three forms, code, weights, and prompts, then fluency in all three becomes a career advantage. The future is not choosing one religion. It is knowing when to write explicit logic, when to train, and when to prompt.