Longreads + Open Thread

Longreads

• In the NYT, Elizabeth Dias has a good piece on Anthropic's efforts to, depending on your point of view, solicit the opinions of or morally compromise various religious figures. It includes the detail that Anthropic nearly pulled out of consulting with the Vatican on Magnifica Humanitas over concerns that the encyclical would declare AIs non-conscious. This has set off a very lively debate, over whether we should accept irresponsible claims from AI companies that AIs are conscious beings with moral worth, and the Church shutting that argument down. This controversy wildly exaggerates both sides' confidence. Here's Chris Olah of Anthropic: "To be clear... we don't know if A.I. models are conscious. I don't know. I'm genuinely uncertain. The thing that I care about is that we get to the right answer, whatever it is." And here's Cardinal Michael Czerny (via this thoughtful comment from a while back) "A related question much debated today is whether and in what sense we can speak of consciousness or conscience in relation to the most advanced artificial intelligence systems. It is a serious question, one that deserves attention and further study... The church welcomes these debates with respect and recognizes the value of the scientific and philosophical contributions." The contention that Silicon Valley weirdos are pushing this crazy AI-consciousness claim on us is somewhat complicated by the fact that the last survey I've seen on this showed that the belief that AI models are conscious was 5x more prevalent among the general population than among AI researchers, though the researchers are more open to the idea that computers could eventually be conscious. There are, of course, plenty of knock-down arguments that AI cannot be conscious. These fall into two categories: arguments you can use to demonstrate that everyone but you is not conscious, and arguments that use special cases to make everyone the speaker thinks is conscious count as conscious. It's a hard problem! Hard enough that there's a debate among philosophers about whether or not there is such a thing as the Hard Problem of Consciousness. They also have debates about what consciousness is, while neuroscientists and evolutionary biologists focus more on wondering how it works and when and why it arose. These problems are at some level unresolvable because the only consciousness you can observe is your own, and we aren't reliable narrators to ourselves, much less to everyone else. But! If sufficiently complex software systems can display traits that we associate with consciousness, it does present an opportunity to poke around in them and learn about some mechanisms by which this happens. This kind of question is one of the reasons that organized religion, as opposed to solitary meditation on religious texts, has stuck around so long: we're often faced with new situations whose specifics are not addressed in those texts, but whose general shape does match topics that religion speaks to. The question of whether something can be conscious but not ensouled, or seem smart but not be conscious, starts out as hypothetical, but it's a good idea to address it before people are worried that hitting control-C is a form of murder. (For what it's worth, my money is on: AI is not conscious, and we'll probably sharpen our definitions a lot as models exhibit more complex behaviors. A pretty likely outcome is that we gerrymander things in such a way that either we define them as not conscious or give them some special category with nothing else in it.) This is a controversial topic, and there's a lot to discuss, but if your view is that it's actually easy to determine what is and is not conscious, and exactly which substrates definitely can or can't support it, there's no need to let me know—you can feel free to make fun of me during the acceptance speech for your Nobel instead.
• Mark Koyama asks whether England in 1066 was unusually rich or unusually poor. It's a surprisingly tricky question, because we have narrow measures that make them seem rich—lots of Danegeld—and broader metrics by which they look quite poor, like the absence of large cities of note. This is a fun piece because it's trying to triangulate between very different ways of measuring wealth, some of which are broad-but-fuzzy, others of which are precise but narrow. The ultimate conclusion helps explain why England was a uniquely conquerable place around that time: a government that was disproportionately good at collecting taxes would have a high ROI, especially for someone who had relatively more military state capacity.
• Tom Dotan profiles Larry Ellison in Vanity Fair. Fun throughout. This piece is a very useful one from a media studies standpoint. Ellison is ridiculously, colossally, own-your-own-Hawaiian-island rich, to the point that basically any whim of his can be translated into action. There's a point in the story where the author is visiting Larry's island, approaches someone who works for one of his companies, and: "'They said there would be a journalist on the island today,' the employee told me before starting to walk away. 'They warned us about you in our morning meeting.' I hadn’t told anyone at Sensei Farms I was coming." When you put it like that, it sounds ominous. But put it like this: Larry Ellison was doing journalism, to a journalist: he was learning something interesting (i.e. that someone was visiting the island to do a profile of him) and then he published it to an interested audience (i.e. people who worked for him and thus wanted to stay on his good side). He was trying to shape their opinions or behavior in order to get them to do things he wanted, which is exactly what journalists are trying to do, too. It's fair play on either side, just fun to gawk at.
• In Asterisk, Jesse Smith writes a wonderful, much-needed article on the downsides of blue-collar jobs from the perspective of a blue-collar worker. It can simultaneously be true that: the marginal college matriculant would be better-off doing something with their hands, this won't always be true, and that blue-collar work has a physical toll that just isn't comparable to office jobs. One helpful feature of this piece is the blue-collar sociology: how they treat different newcomers, which jobs are uniquely desirable, etc. The piece also notes that you never know which specific task automation will hit next, so the experience of skilled tradespeople today is a kind of survivorship bias: every time another job gets automated, payroll as a percentage of total project cost goes down and so willingness to pay goes up. That can be a worthwhile gamble, but it's still a gamble.
• Katy Blumer, Kate Donahue, Katie Fritz, Kate Ivanovich, Katherine Lee, Katie Luo, Cathy Meng, and Katie Van Koevering present: An Abundance of Katherines: The Game Theory of Baby Naming. It's a wonderful send-up of a few different genres of academic papers. We've got models with bizarre assumptions, nonsensical citations, and an acknowledgements section that includes "We also want to thank the many dozens of people who have confused us for one another at conferences."
• In Capital Gains this week, we look at the phenomenon of abstract exports: when people say a country exports deflation, or risk, or whatever, what do we actually mean?
• A Read.Haus user asks if the big AI labs will ever be profitable, given that they keep spending so much and their products tend to get commoditized. The way to think about this is that there are ways to slice up the labs' business into parts that have positive contribution margins, and then some discretionary costs. These costs show up before the associated revenue. So, on average, the faster the labs think they should grow—the bigger the opportunity and the higher the odds that one of them ends up with a widening lead—the more it makes sense for them to lose now. They could, of course, get this wrong, but they have a decent track record so far. There are only so many order-of-magnitude increases in spending that they'll need to do, and if they haven't done most of them by now, the future is sufficiently strange that questions of GAAP profitability will be far from the most interesting ones you can ask.

Open Thread

• Drop in any links or comments of interest to Diff readers.
• White-collar workers often end up in a better position to see technological disruption coming, because processing information is a core part of their job. What does the world look like if they’re closer to blue-collar workers, where sometimes a technological change that they had no reason to see coming collapses their earning power? 

Diff Jobs

Companies in the Diff network are actively looking for talent. See a sampling of current open roles below:

• General Catalyst backed founder is looking for technical generalists who are excited about creating a fully autonomous firm. If you want to own the entire process of identifying, instrumenting, steering, and monitoring post-PMF businesses that run autonomously, and study how machine intelligence actually behaves in the real economy, this one is for you. (SF)
• a16z, Sequoia, Thrive-backed next generation pharma company that instruments the best drug hunters in the world with deterministic software, data and AI to accelerate the process of acquiring, developing, and monetizing promising therapeutics (three active immunology programs) is looking for senior software engineers. If you are excited about sublinear headcount scaling and building technology that drives therapeutic abundance, please reach out. (Boston, NYC, SF)
• A top prop trading firm is looking for people who combine exceptional quantitative ability with the judgment and relationship instincts to build critical financial partnerships and lead high-stakes negotiations. Candidates may come from markets, finance, or further afield; what matters is this combination of abilities, strengthened by the perspective that comes with experience. (NYC)
• A top prop trading firm is looking for people with exceptional strategic thinking and quantitative skills to help the firm model and manage its financing risk and strategy. Open to candidates from a range of analytically demanding fields. (NYC) 
• Ex-Bridgewater, Worldcoin founders using LLMs to generate investment signals, systematize fundamental analysis, and power the superintelligence for investing are looking for machine learning and full-stack software engineers (Typescript/React + Python) who want to build highly-scalable infrastructure that enables previously impossible machine learning results. Experience with large scale data pipelines, applied machine learning, etc. preferred. If you’re a sharp generalist with strong technical skills, please reach out. (Remote)

Even if you don't see an exact match for your skills and interests right now, we're happy to talk early so we can let you know if a good opportunity comes up.

If you’re at a company that's looking for talent, we should talk! Diff Jobs works with companies across fintech, hard tech, consumer software, enterprise software, and other areas—any company where finding unusually effective people is a top priority.

And: we're now actively deploying capital into early-stage companies through Anomaly. Our focus is on defense, logistics, robotics, and energy. If you'd like to chat, please reach out.