Longreads + Open Thread

Pharmacometrics, Healthcare, Ahamed, GTA VI, DOGE, Fees, Efficiency, AGI

Longreads

You're on the free list for The Diff. This week, paying subscribers read about why low market concentration means one of several bad things ($), thoughts on espionage at AI labs ($), and the implications of better mechanistic interpretability ($). Upgrade today for full access.

Upgrade Today

Books

The AGI Chronicles: If you've been following a story closely enough that you could write a book on it, as Kevin Roose has with AI, one question you have is: when do you send your manuscript to the publisher? If the story has a definite start and end date—a book about a political campaign, say—then reality storyboards this out for you. You start your draft right away, keep adding to it and editing it, and as soon as it's done, you send. But what if the story isn't just changing all the time, but accelerating? The more events that can happen over a six-month period in some area, the more it's impossible for your book to be current events; by the time the reader has it, it's history.

There's a double-gonzo experience reading a book about events I was reasonably close to, in which many characters are friends-of-a-friend, and also reading a book by someone who was the star of one of the funnier anecdotes in the history of AI alignment and deployment. Kevin Roose was invited to try Bing's AI features, and, after a lengthy conversation, his chatbot started badgering him to leave his wife for her. So, he's close to the story.

Which is good! Because it's a story that's evolved a lot since it started with various plucky bands of researchers pursuing interesting ideas that were a long way from any commercial application. It's been striking to see how much of the research-lab culture has persisted as the labs have transformed into trillion-dollar companies: they still publish cool papers, and their employees and executives still make gnomic jokes on Twitter in a way that would seem wildly inappropriate if it came from the CEO of, say, Boeing or Exxon. The labs have certainly changed over time, just not as much as similar important organizations.

Some of the early history details are quite fun. GPT-1, for example, was trained in December, because enough people were on vacation that there were free GPUs. Thus, it joins a long list of inventions made under exactly those circumstances: the integrated circuit and Unix were both invented by people who kept working while people who might interrupt them were on vacation, and a lot of the early work on Microsoft, Facebook, and Stripe took place during Harvard's winter break and reading period. With institutions' calendars and models alike, it's been found that injecting a little randomness from time to time will shake things up and lead to a better outcome.

If you plot model capabilities and labs' valuations, you see a pretty smooth exponent up through the last few weeks, but if you zoom in, you'll see plenty of terrifying dips—disappointing training runs, models getting released with a lower version number than planned (GPT-4.5 was supposed to be GPT-5, but wasn't what people wanted from GPT-5). One thing I'd forgotten, because Anthropic hadn't publicly released a product at that point, was that the collapse of FTX and the release of ChatGPT happened in the same month, and that this would have been an incredibly grim one for Anthropic. They weren't sure if the investment would be clawed back, and OpenAI had shipped something that they could have, so it would be easy for them to envision a world in which AI capabilities raced ahead of alignment purely because of their bad luck. Even if shares of pre-IPO labs aren't regularly marked-to-market the way a public company's are, owning them means feeling the volatility even if there isn’t a daily mark.

Part of what this book does very well is that it puts all of these events together in a coherent timeline, whereas we've experienced most of them as one exciting release or headline at a time. An important feature of AI people, and something useful to keep in mind when talking about them, is that they live in a world where they used to be a tiny minority with views too weird to be worth addressing ("So you'll be able to tell your computer 'make me a picture of a dragon playing a guitar solo' and it'll just make one? And also this computer might decide to turn us into paperclips? C'mon..."), and those views have been vindicated. So, in their experience, people who call them crazy are just flat-out wrong and will be proven so in a few years. They have earned the right to be very, very weird, but the more legitimate that is—i.e. the more powerful models get, and the more profitable labs end up being—the more we'll have to ask the question of just how powerful odd people should be, and vice-versa. If caring about shrimp welfare is closely bundled with the ability to come up with creative ways to improve AI performance, and if AI keeps getting used more as a result, we approach an asymptote where the world is run by people who really care about shrimp. Which, in one sense, is fine—we ran a meritocratic process where there were many avenues for seeking power, and the best power-seekers happened to choose a character build more focused on linear algebra than on winning elections or whatever, so that's who we wound up with. But it will make politics very fraught, inside the labs and out: on the outside, voters are clearly spooked by AI in various ways. But inside the labs, there's reason to be equally concerned by the weird beliefs of the median voter, who cares a lot about AI, can influence the course of its growth through voting, and has basically no idea how it works. This book will give them some valuable context.

Open Thread

Diff Jobs

Companies in the Diff network are actively looking for talent. See a sampling of current open roles below:

Even if you don't see an exact match for your skills and interests right now, we're happy to talk early so we can let you know if a good opportunity comes up.

If you’re at a company that's looking for talent, we should talk! Diff Jobs works with companies across fintech, hard tech, consumer software, enterprise software, and other areas—any company where finding unusually effective people is a top priority.

And: we're now actively deploying capital into early-stage companies through Anomaly. Our focus is on defense, logistics, robotics, and energy. If you'd like to chat, please reach out.