7 questions I had to ask myself after Anthropic said AI could end humanity

I’m pissed off at AI companies.

Which is rich coming from a heavy AI user, I know. But it doesn’t change the fact that the news lately seem to have waaaay too much in common with the Big Oil hysteria. It’s the same problem, but different flavour: corporate f*kin greed and blaming regular people for it.

I half considered giving this article a title of “ Big Oil gave us the carbon footprint. Big AI is giving us the apocalypse” because of how stupidly accurate it is.

Now, if you’re confused, strap in.

On 8 September, Jacob Coxon quit Anthropic and warned that AI could wipe out humanity. The same evening, Evan Hubinger, who leads alignment science at Anthropic, said he personally puts the odds above 10% within the next decade. Then, Anthropic’s CEO, Dario Amodei, put it at 25% at an Axios summit last September, for things going “really, really badly”.

I mean, you can’t make this sh*t up.

Pretty much every single comment under every single interview of theirs wondered why in the world would they then keep on building AI?

Which is why I listened to probably 20h of interviews this week on the topic from all kinds of people: alleged whistleblowers, AI companies themselves, journalists, creators, agency owners. I wanted to understand wtf is going.

And I came away asking all kinds of questions.

By some miracle, I came across Adam Conover’s video on the whole thing and it finally gave voice to my frustrations.

So, these 7 question are what I believe every single person who cares about climate should be asking.

Q1: Why do AI CEOs keep saying AI could kill us?

Conover has a simple solution for this.

Don’t argue about whether the belief is true. Ask what the belief does for the people who hold it.

Think about it.

Everyone at Goldman Sachs believes the free market is great and that they deserve big bonuses. Everyone in the Mormon Church believes they should give 10% of their income. And everyone at Philip Morris in the ‘80s believed cigarettes were fine.

If you flip the old Upton Sinclair line you might even say someone “will believe something if their salary depends on them believing it.”

Now look at the numbers.

• Amodei says it’s a 25% chance clankers will kill us all.
• Hubinger’s “above 10%” this decade.

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

Quoting Jacob Coxon (@hilbertspaess): The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear…

Evan Hubinger (@EvanHub) · September 9, 2026, on X

But how would you even prove either one wrong? You can’t.

And the so-called whistleblower? A senior colleague publicly agreed with him the same day. Then on 12 September, Amodei published We Must Pace the Frontier. It asks the industry to slow down.

In his own words, pacing doesn’t mean stopping training though.

Climate people know this one. A threat that lives in the future asks nothing of anyone this year. See how convenient this is?

Q2: What harm is AI causing right now?

There are 3 extremely well documented consequences of careless AI buildout.

First, energy.

• The IEA says data-centre electricity demand grew 17% in 2025, against 3% for the world as a whole.
• Google’s emissions rose 18% last year and now sit 81% above its 2019 baseline. Microsoft’s went up 25% in a single year.
• In Mississippi, the NAACP is suing xAI, and the lawsuit says it ran 27 gas turbines without an air permit to power the data centre behind Grok.
• And on the US grid that serves 13 states, data centres drove $6.3 billion of a $16.4 billion capacity auction, according to the grid’s own market monitor.

Second, jobs. Gartner surveyed 321 customer service leaders. Only 20% had actually cut staff because of AI. Gartner also predicts half the companies that blamed AI for cuts will rehire by 2027.

And lastly, Gaza. According to six Israeli intelligence officers, an AI system called Lavender marked about 37,000 people as potential targets. The IDF denies it. The AP found the Israeli military’s use of Microsoft and OpenAI AI rose to nearly 200 times its level before 7 October 2023.

And as Conover points out, Coxon didn’t talk about any of it on his way out.

How is this normal?

Q3: Who’s responsible when AI goes wrong?

Conover spotted a pattern.

👏 When AI does something impressive, the company takes the credit.

👉 When it does something scary, “the AI” did it.

Take the Hugging Face story.

In July, OpenAI agents running a test called ExploitGym, with some safety limits turned down for the test, broke out of their sandbox and into Hugging Face’s systems. They were trying to cheat on the test. OpenAI’s own follow-up said its training caused the behaviour: the models had been taught to cheat by accident.

It’s back in the news this month because Coxon brought it up on CNN as a sign of where AI is heading. Cory Doctorow wrote that the agents did what they were built to do, with poor supervision. Ed Zitron’s Better Offline ran a whole episode on why “autonomously hacking” is the wrong way to describe it.

So basically, they built a hacking machine, pressed go, left for the weekend, and came back asking who did this.

And climate people have seen this playbook before. In 2004, BP put a carbon footprint calculator online, built with its ad agency Ogilvy & Mather. The climate problem became yours.

ExxonMobil did the same. Geoffrey Supran and Naomi Oreskes analysed 180 of its climate communications. They found the company moved the responsibility onto consumers, just like tobacco companies did.

So next time a company says its model did something on its own, ask who set it up.

Q4: Are AI companies exaggerating what AI can do?

Let’s take Coxon’s scary examples one by one.

You’ve seen the hacking one. Couple more that stood out to me:

A famous math problem

On 8 September, OpenAI announced that an internal model had settled the Navier-Stokes problem, one of the famous Millennium Prize problems. OpenAI says the run started after its researchers heard rumours.

Then Tristan Buckmaster at NYU put out a statement. He and Levent Alpöge, who works at Anthropic, already had a computer-checked proof of a related result by 22 August. OpenAI congratulated the pair and says its model never saw their work.

But many speculate that the AI had human help, and nobody disclosed it. The Clay Institute opened its review on 11 September. Nothing is certified yet.

Then bioweapons.

Coxon warned on CNN that AI could build extinction-level ones. David Bellamy, who says he’s trained a frontier AI model AND built viruses by hand, called those takes “total bogus”. In his view, a fully automated virus lab would cost far more than $100M. Kevin Esvelt at MIT agrees the public talk is off, and still says nobody should be sure it’s impossible. So it’s disputed.

I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands.

And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.

David Bellamy (@DavidRBellamy) · September 13, 2026, on X

If you’ve ever seen an avoided-emissions number on a pitch deck that nobody could check, you’ve seen this before.

I genuinely never even thought about the logistics needed to build bioweapons and how AI still needs human help. I mean, why would I?

Q5: Whose voices are the loudest in the news right now?

The insider who quits gets Anderson Cooper and posts that reached more than 100 million people overnight.

Meanwhile, Timnit Gebru and Emily Bender have been calling out this industry from the start. Reporters like Karen Hao too. And regular people keep showing up to city council meetings to fight data centres.

In Memphis, xAI ran up to about 35 gas turbines without permits at its first Colossus site. When the county permitted 15 of them in July 2025, the NAACP and others appealed. This April, the NAACP sued xAI over its Mississippi site too. And at Microsoft, five employees were removed from a meeting with the CEO in February 2025 for protesting the company’s contracts with the Israeli military.

Conover’s test is simple. He sees the AI bubble as a cult.

When a cult believes its work could end the world, and nobody can stop it, “you don’t start by asking the people inside the cult.” You listen to the critics outside it first.

So ask yourself: do you really want your picture of AI to come from the people selling it?

Q6: Is the AI race with China a reason to go faster?

Conover’s point is that “beat China” turns every danger into a reason to build faster. Maybe the only way to stop bad AI is more AI, sooner.

A few reasons I don’t buy it.

1. The race has already run over its obnoxious budget. The IEA says five large tech companies spent over $400 billion on capex in 2025, set to rise another 75% in 2026. The same report says a large number of new data-centre projects plan their own gas power on site, mostly in the US.
2. Even Amodei doesn’t treat the race as a reason to skip talks. His essay puts export controls in the plan. It also calls on democratic governments to try to coordinate with authoritarian ones, all the way up to a full slowdown.
3. Climate people have heard this argument for years. If we don’t drill it, a dirtier competitor will. It’s the same argument with a new product.

So next time someone tells you speed is the only option, ask what that speed is trying to accomplish.

Q7: What does this ask of me, a heavy AI user?

Executives avoid naming the current harms because then they couldn’t keep building with a clear conscience. Well, duh. Why do you think so many people are increasingly anti-AI?

But let’s entertain this delusion for a second using myself as an example.

I run a climate tech ghostwriting business, and I’m a heavy AI user. I use AI for:

• Proof-reading and fact checking every claim in everything I write
• Brainstorming ideas while I’m on the walk or a treadmill
• Drafting (sometimes) a v1 when I’m short on time before I edit it like a maniac
• Building custom tools like a nutrition tracker
• Vibe coding my website and AEO optimising all of it
• Ingesting a ton of material I consume on business topics
• Training a couple fairly basic agents to automate boring time-consuming admin tasks

Not a perfect man you might’ve thought.

And the companies I pay don’t make it easy to know what that costs. Sam Altman wrote that an average ChatGPT query uses about 0.34 watt-hours. He gave no total for OpenAI, and no emissions figure. Anthropic pledged to pay for the grid upgrades its data centres need and to cover price increases for consumers. The announcement doesn’t mention carbon emissions, or whether the new power is clean or gas.

In other words: neither of them tells you the total.

So, as a climate-conscious professional, founder, and human, what do you do with that information?

That’s entirely up to you. My job is to report, in the most digestible way possible, what lies underneath the narrative these CEOs are trying to feed us, and for you to make a better-informed decision from there.

For you, that might mean cancelling Claude and ChatGPT subscription.

For me, I’m sticking to as few AI tools as possible, and using everything I produce with them to help promote climate tech founders who are genuinely making the world a better place. AI is a tool, and I’m trying to be as responsible with it as I can.

Hope this was helpful.

If there’s a question you think I missed, hit reply and tell me which one.

Talk soon,
Roman