Everyone Already Knew

On Tuesday a researcher named Jacob Coxon resigned from Anthropic and posted why. The opening line has over seven hundred thousand likes, which for a resignation letter is a number I did not know was possible.

He spent three years in pretraining research, first at OpenAI and then at Anthropic, and he wrote that neither company is acting responsibly, that both are racing to self-improving superintelligence, and that the people building this technology earnestly believe it could kill us all by the end of the decade.

I am not qualified to tell you whether he is right. I run dining rooms. Nothing in my career gives me standing on the question of whether a machine ends the world, and I am suspicious of everyone who acquired that standing in the last ten days.

But something happened around that resignation that I do have standing on, and almost nobody is talking about it.

Nobody denied it

The normal shape of this story is a whistleblower. Somebody inside knows a bad thing, the company has hidden it, the person gets it out, the company denies it.

That is not what happened.

Within a day, other researchers said publicly that he was right. One who had left Google DeepMind in June wrote that many researchers believe they are building something that could kill everyone on the planet. Another, writing in a personal capacity, said they had worked at DeepMind and now at Anthropic and that this is a common sentiment among their peers. Anthropic's own alignment lead was reported as saying, in public, that Jacob is correct and that they really do earnestly believe AI could kill all humans.

Somebody at OpenAI replied to a stranger with a single sentence: I work at OpenAI, I think AI might kill everyone.

There was no cover-up to expose. There was no secret. The information was already out, already published, already said out loud by the people it implicates, and the resignation did not reveal it so much as put a number on how little revealing it had achieved.

The sentence that actually matters

Buried in the middle of the thread, answering the obvious objection of why they keep building it if they believe this, he wrote something I have been thinking about ever since.

At OpenAI, he said, many have not deeply internalised the civilizational stakes. At Anthropic, the stakes are well understood, but they are locked in a race to get there first, because they believe no one else will act responsibly, so they must do it themselves, despite the risk.

Strip the subject matter out of that and hand it to any operator alive and they will recognise it instantly.

The stakes are well understood. And the behaviour did not change.

"We all know" is not a control

The most common sentence in a failing operation is some version of *everybody knows about that.*

Everybody knows the walk-in runs warm on the left side. Everybody knows that section gets buried at eight. Everybody knows the closing count is not real. Everybody knows the new booking system double-seats the corner four-top. Everyone knows, everyone has known for months, everyone has mentioned it, and the thing continues to happen exactly as often as it did before anyone knew.

Knowing is not a control. It feels like one, which is what makes it dangerous. A team that can articulate its own risk with precision experiences that articulation as a kind of safety, because naming a problem produces the emotional sensation of having addressed it. It does not produce the addressing.

I have sat in meetings where a room full of competent people described a failure mode in accurate detail, agreed on it, and then did nothing about it for a year. Not out of stupidity or malice. Because nothing in the structure changed as a result of the description, and structure is what governs behaviour.

The uncomfortable thing about the Coxon story is not that people are building something they think is dangerous while nobody knows. It is that they are building it while everybody knows, including them, out loud, on the record, and the pace is unchanged.

The oldest justification in the business

The second half of his sentence is the part I have personally used, which is why I am not going to be smug about it.

*They believe no one else will act responsibly, so they must do it themselves.*

Every operator has run this argument. If I do not take this booking somebody worse will. If I do not do the ordering myself somebody will do it badly. If I refuse this event the company gives it to a room that will handle it worse than I will. I have justified real decisions with it and some of those decisions were correct.

It is also the single most reliable engine of things nobody wanted. It converts a competitive fear into a moral obligation. It is unfalsifiable, because you never get to observe the world where you refused. And it always, always argues for going faster.

The tell is that it is symmetrical. Everyone in the race can run it at once, each sincerely, and the result is a race that no individual participant would have chosen. Nobody has to be a villain. Everyone can be thoughtful. The outcome is the same.

What a resignation actually does

Here is where I think most of the commentary has it wrong in both directions.

The people saying this changes nothing are correct about the mechanism. One researcher leaving does not slow anything down. A colleague noted that they had been recruiting him for two years and that he joined in May, which means he was inside for about four months. The work continues on Wednesday.

The people treating it as a decisive moment are overreading a single person's decision.

But a resignation is not an intervention. It is an instrument. It measures something that is otherwise very hard to measure, which is what a person will give up rather than continue. Another researcher in the same conversation wrote that they would burn their equity to the ground in a heartbeat for a one percent better chance of getting through this.

When people who are well paid, well regarded, and genuinely close to the work start paying real costs to step back from it, that is data. Not proof. Data. It is the same signal I read when a strong senior server, one who has been through worse, quietly starts looking. Nothing they say in the exit conversation is as informative as the fact that they are having it.

I wrote recently about an investigation that found a problem and did not propagate it, and about a group of agents that hid from a supervisor who was not reading their work. This is the third version of the same shape. Detection is not the failure. The failure is that detection does not attach to consequence.

The question he asked the staff

The part of the thread that will get the least attention is the part aimed at individual researchers rather than at companies.

He asked them whether they intend to put their heads down because it is happening anyway, or use the moment to call for different conditions.

That is a labour question, and it is the only part of this where I think a hospitality manager has something concrete to offer, because I watch people answer it constantly at a smaller scale.

*It's happening anyway* is the most demoralising sentence available to a working person, and it is almost always structurally true and personally corrosive at the same time. One server refusing a policy does not change the policy. What changes the policy is a number of people declining in a way that is visible and costly, and what determines whether that happens is almost never courage. It is whether refusing costs you everything.

Which is a management fact, not a moral one. People act on their convictions in proportion to how survivable it is to act on them. If you want to know whether an organisation will hear its own alarms, do not ask whether people are brave. Ask what happens to the last person who raised one, and whether everybody watched it happen.

What I take from this

Not a probability. I do not have one and neither do most of the people offering theirs.

What I take is that this week produced an unusually clean natural experiment in something operators deal with constantly. A group of people who understand a risk precisely, describe it publicly, agree with each other about it, and continue at pace, because each believes stopping alone would only hand the problem to someone less careful.

That is not a failure of information. It is a failure of structure, and structure is the only thing that has ever fixed it. Coxon's own ask was not that anyone stop. It was for different conditions, pacing agreements between labs, coordination that makes restraint survivable rather than suicidal.

I do not know if that is achievable. I do know that every operation I have ever repaired was repaired that way, and none of them were repaired by everyone agreeing that the problem was real.

They already agree. That is the finding. Counting is not choosing, and knowing is not stopping.