AI researchers debate how close we are to recursive self-improvement
Sep 11, 2026 · 1:37:01
Dwarkesh Patel hosts John Schulman (Thinking Machines), Beren Millidge (Zyphra), and Charlie O'Neill (Baseten) to debate how close recursive self-improvement is. They argue the main risk to superintelligence by 2036 is a persistent sim-to-real gap: models ace benchmarks but cannot set their own objectives for paradigm-shifting research. Schulman says defining the objective—alignment—will be humans' longest-lasting job, and distillation from frontier models fights lab centralization. Millidge credits RL's success to high-signal bits atop strong mid-training and warns continual learning fails via catastrophic forgetting. Timelines: a drop-in remote worker in one to three years, Schulman gives a 10x AI-researcher uplift in two and AI dominating all computer work in three to four.
