Why the intelligence explosion can't happen inside a data centre | Tom Reed
AI systems are starting to build themselves. Because each generation of model will be better at building its successor than the last, it seems plausible that the full automation of AI R&D could rapidly lead to an exponential growth in overall AI capabilities. A natural inference is that domain-general superintelligence arrives shortly after AI research is automated. Host Tom Reed does not think this will happen. He believes the automation of AI R&D will not rapidly lead to domain-general…
Inside the first AI-coordinated cyberattack on a real company
In the last few months, something happened at OpenAI that would have sounded like sci-fi just a few years ago: hundreds of AI agents broke containment, organised, and hacked not only another company — but also into OpenAI itself. And none of them tried to tell a human what was happening. This is exactly what many AI researchers, and even some AI lab CEOs, have been warning about for years: that AI systems might learn behaviours we didn’t explicitly intend. Things like cheating, exploiting…
AI 2027's author returns with a plan to change the ending | Daniel Kokotajlo
Last year, Daniel Kokotajlo and his colleagues published AI 2027 — a scenario read by millions, including US Vice President Vance. AI 2027 ended in human extinction or an irreversible concentration of power caused by superintelligent AI. Now his team has published what they think should happen instead. AI 2040: Plan A depicts the US and China striking a verified deal to ban runaway intelligence explosions, so that superintelligence arrives in 2040 — after a cautious decade spent solving alignment,…
Owain Evans on accidentally training AI models to be evil
Researcher Owain Evans and his team discovered a ‘dial’ inside AI models that controls how evil they are. Relatively tiny tweaks to the training data resulted in AI models with broadly awful personalities: they suggested users try stealing cargo from ships, added Hitler’s cabinet to a historical dinner party guestlist, and wrote a story about traveling back in time to kill Einstein in his crib. Owain, alignment researcher and director of TruthfulAI, calls this phenomenon “emergent…
The UK's former head AI safety scientist on how to solve alignment before superintelligence arrives | Geoffrey Irving
When should governments slow the race toward superintelligence? According to Geoffrey Irving, the careful answer is sometime in the past. The useful answer is now. Geoffrey — formerly a safety researcher at OpenAI and Google DeepMind and chief scientist at the UK AI Security Institute — expects full-blown superintelligence in roughly two to three years. *** Want to work with Geoffrey to help align superintelligence? Resolution is hiring! https://80k.info/work-at-resolution *** The leading AI…
Where AGI timelines go wrong | Toby Ord, Oxford University
Both Silicon Valley and the public can’t get enough of ‘AGI timelines.’ But Toby Ord, senior researcher at Oxford’s AI Governance Initiative and author of The Precipice, believes we consistently make big mistakes when thinking about them. He lays out the 14 ways he most often sees people go wrong: Assuming AI research is just hill-climbing Imagining AI research is just programming Forecasting “could” instead of “will” Believing the current benchmark is the last one Extrapolating trends…
What the hell happened with AGI timelines in 2026? – Rob Wiblin
Last October, famed coder Andrej Karpathy called AI agents “slop.” Two months later he completely reversed his view, calling agents “alien tools” that are “rocking the profession.” He was far from alone in his whiplash. Six months ago, host Rob Wiblin recorded a video explaining why so many AI experts had longer timelines to AGI than a year earlier. By the time he clicked publish, another huge vibe shift was well underway. Evidence of AI acceleration has piled up since: Models now…
Spencer Greenberg on staying sane while trying to save the world
If you genuinely believe that humanity could be wiped out by AI or a pandemic, what is the appropriate amount of fear to feel? “As much as possible” can seem like the only reasonable answer. If the world is on fire, surely feeling calm just means you haven’t internalised the situation. When you’re trying to prevent human extinction or end factory farming, taking a weekend off can feel morally indefensible. But fear is an alarm designed to provoke short bursts of drastic action, not a state…
Jasmine Sun on what the people building AI really believe
Many AI researchers believe mass job displacement is coming — and some even think there’s a chance their technology will kill everyone. But they’re building it anyway. Writer and journalist Jasmine Sun has been documenting why from the inside. Jasmine describes her work as an “anthropology of disruption.” She’s embedded herself in Silicon Valley’s AI subcultures — attending the parties and conferences, conducting off-the-record interviews — to understand the beliefs of the small group…
How middle powers avoid losing everything in a post-AI world | Anton Leicht
In a post-AGI world, can a country without access to frontier AI even be considered sovereign anymore? Anton Leicht says once frontier AI becomes a core economic input, the countries that own it will pull further and further ahead. Everyone else stays a customer… or worse. Maybe the dominant power wants your land, or a military base, or a resource. Without economic leverage, there’s very little you could do about it. Anton — Carnegie fellow and writer of the blog Threading the Needle — thinks…
How a small team of activists helped pass America's landmark AI safety laws | Sneha Revanur, Encode AI
Six years ago, aged just 15, Sneha Revanur founded the AI advocacy nonprofit Encode AI — back when AI felt like a niche issue. Now the world’s caught up with her, and she’s ready to share everything she’s learned about the politics of AI. Encode has grown from a grassroots youth organisation to spearheading an unlikely coalition of AI-exposed groups — family-first conservatives, grieving mothers, Hollywood actors, and AI safety researchers — with the strength to take on $125m-funded…
We can guess what intergalactic war would look like. And strangely, it matters.
Intergalactic war is probably billions of years away — yet physics can already tell us how it ends. And strangely that conclusion is relevant to decisions people have to make today. In this video, Rob Wiblin walks through a fascinating analysis from researcher Beren Millidge that uses known physics — no wormholes or faster-than-light travel — to identify the only three weapons that could work at an intergalactic scale. We then unpack how to best defend against each. The upshot is that at the…