Andrej Karpathy - AI Progress, AGI Timelines, and the Future of Intelligence
Problems (3)
AI Model Collapse from Synthetic Training
LLMs trained on their own outputs become 'silently collapsed' — they generate content that looks reasonable individually but lacks entropy across samples. This creates a fundamental barrier to synthetic data generation and self-improvement loops.
Give me a way to train AI models on synthetic data without losing the diversity and creativity that makes them useful for novel problems.
The LLMs, when they come off, they're what we call 'collapsed.' They have a collapsed data distribution. One easy way to see it is to go to ChatGPT and ask it, 'Tell me a joke.' It only has like three jokes.
Reinforcement Learning Credit Assignment Problem
Current RL 'sucks supervision through a straw' — it takes complex multi-step reasoning and reduces learning to a single binary signal. Everything in a successful trajectory gets upweighted, including wrong paths, creating massive noise in the learning signal.
Help me learn from complex sequences by giving me specific feedback on which parts I did right and wrong, not just whether the final answer was correct.
The way I like to put it is you're sucking supervision through a straw. You've done all this work that could be a minute of rollout, and you're sucking the bits of supervision of the final reward signal through a straw.
AI Development Capability Assessment Gap
No reliable way exists to predict AI development timelines because capabilities that seem 'almost there' in demos often require years of additional work to reach production reliability. Each 'nine' of reliability requires constant effort.
Help me accurately forecast when an AI capability will be production-ready, not just demo-ready, so I can make realistic business and research planning decisions.
I'm very unimpressed by demos. Whenever I see demos of anything, I'm extremely unimpressed by that. If it's a demo that someone cooked up as a showing, it's worse.
Solutions (1)
Nanochat Full-Stack Learning Architecture
Instead of teaching AI through abstract concepts, Karpathy built an 8,000-line repository that implements a complete ChatGPT clone from scratch, serving as a 'ramp to knowledge' where every component is minimal but functional.
Mechanism: The mechanism works through progressive complexity building: start with a simple bigram model (lookup table), then add each transformer component only when the previous approach fails. Each addition i...