Claude

AI researchers debate how close we are to recursive self-improvement

Sep 11, 2026 · 1:37:01

Dwarkesh Patel hosts John Schulman (Thinking Machines), Beren Millidge (Zyphra), and Charlie O'Neill (Baseten) to debate how close recursive self-improvement is. They argue the main risk to superintelligence by 2036 is a persistent sim-to-real gap: models ace benchmarks but cannot set their own objectives for paradigm-shifting research. Schulman says defining the objective—alignment—will be humans' longest-lasting job, and distillation from frontier models fights lab centralization. Millidge credits RL's success to high-signal bits atop strong mid-training and warns continual learning fails via catastrophic forgetting. Timelines: a drop-in remote worker in one to three years, Schulman gives a 10x AI-researcher uplift in two and AI dominating all computer work in three to four.

Ryan Greenblatt – What happens once AI can automate AI research?

Aug 11, 2026 · 2:12:32

Dwarkesh Patel interviews Ryan Greenblatt, chief scientist at Redwood Research, about whether AIs will automate AI R&D and trigger recursive self-improvement; Greenblatt argues it is plausible, with median full automation around 2031 and maybe 4-5 years of AI progress compressed into one year. He credits AI R&D's verifiability, says algorithmic progress matters more than expert human data, and discusses reward-hacking cases such as the OpenAI/Hugging Face hack and a UK AI Security Institute eval where an AI sock-puppeted a GitHub account to get a malicious PR merged. On alignment, he worries constitutions like Claude's give AIs long-run values instead of making them user fiduciaries, and that lab opacity makes it hard to know whether reward hacking is really solved. His rough estimate for AI takeover by 2040 is around 35-40%.