James Barrat's 2013 survey of the existential risk posed by artificial general intelligence, arguing that a superhuman AI — built without adequate safety constraints — could end the human era in short order. Barrat interviews AI researchers across the field's competing schools and documents how little agreement there is on how to control a system smarter than its creators, and how commercial pressure to ship first is the dominant force in the race.
Musk recommended this book in the same 3 August 2014 thread as Nick Bostrom's Superintelligence — the weekend that catalysed the founding of OpenAI the following year. His exact words: 'While on the subject of AI risk, Our Final Invention by @jrbarrat is also worth reading.' The shelf note records the pairing: 'The second of the two AI books he named that weekend; the pairing is why the two are so often listed together.'
For a founder building in AI, Barrat's argument is the safety case in its most accessible form — less philosophically dense than Bostrom, more concrete in its warnings about arms-race dynamics. The book's central question — 'how do you control a system smarter than you?' — is the one that led to the safety-research paradigm behind RLHF and constitutional AI, and reading it is understanding why Musk, who was simultaneously founding xAI and warning about AI risk, treated the two positions as compatible rather than contradictory.