Skip to content
Quotlr

Andrej Karpathy quote

To make the analogy explicit, in Software 1.0, human-engineered source code e.g. some .cpp files is compiled into a binary that does useful work. In Software 2.0 most often the source code comprises 1 the dataset that defines the desirable behavior and 2 the neural net architecture that gives the rough skeleton of the code, but with many details the weights to be filled in. The process of training the neural network compiles the dataset into the binary — the final neural network. In most practical applications today, the neural net architectures and the training systems are increasingly standardized into a commodity, so most of the active software development takes the form of curating, growing, massaging and cleaning labeled datasets.
Andrej Karpathy

Andrej Karpathy Slovak Computer scientist

53

Share: Facebook WhatsApp X Pinterest LinkedIn Email

Advertisement

More from Andrej Karpathy

  • Earth as a dynamical system is a really bad computer. A lot of information processing is concentrated in a few tiny compute nodes brains, chips with terrible interconnects, even as bad as use of physical translation and air pressure waves. And powered primitively by combustion.
    — Andrej Karpathy Open
  • A 4-year-old child actually has a few hundred million years of experience, not 4. Their rapid learning and generalization is much less shocking magical considering this fact.
    — Andrej Karpathy Open
  • It would be best if people made strong statements that are understood to be only 90% true, and ignore the counterexample police. This saves time and makes direction of statements clear.
    — Andrej Karpathy Open
  • For us it's much more becoming a story of How do humans and computers collaborate to actually create these vector space data sets?
    — Andrej Karpathy Open
  • If you vibrate the electromagnetic field just right, cars passively awash in the radiation for a while will suddenly drive better.
    — Andrej Karpathy Open
  • It's been a while since I last played Factorio, but I can't stop thinking about it, and of the entire economy as being an MMORPG version of it. Factory Town, RimWorld, Banished, Dawn of Man etc are all great too but somehow pack less long-term punch.
    — Andrej Karpathy Open

All Andrej Karpathy quotes