Alex Palcuie

The talk is about where AI agents are actually useful for production engineering versus where they’re still hype. I go through the current state of models, talk about putting Claude Code in a loop to fix bugs like an agent, and propose a framework similar to self-driving car levels for production agents.

Most software today sits at level 2, assisted automation with a human in the loop. We’re getting glimpses of level 3, where the system can take multi-step actions but still needs supervision. Level 5 might require AGI.

Death by YAML was no one’s choice. I’ve never heard someone say they love working with Terraform. So let’s give it to the robots. Be excited, build proof of concepts, but don’t fire your SRE team expecting Claude to be on call.

I gave this as the closing keynote at DotAI in Paris. And yes, I brought a tungsten cube on stage and you can see it in the video.

From the announcement:

Our newest model, Claude Opus 4.5, is available today. It’s intelligent, efficient, and the best model in the world for coding, agents, and computer use.

This is a coding beast.

Quoting from the launch post:

What was recently at the frontier is now cheaper and faster. Five months ago, Claude Sonnet 4 was a state-of-the-art model. Today, Claude Haiku 4.5 gives you similar levels of coding performance but at one-third the cost and more than twice the speed.

Small models are my favorite. Fast to ship, fast to run, what’s not to love.

Claude Sonnet 4.5 is the best coding model in the world. It’s the strongest model for building complex agents. It’s the best model at using computers. And it shows substantial gains in reasoning and math.

77.2% at SWEBench and our safest model yet.

Anthropic announces:

Claude Sonnet 4 now supports up to 1 million tokens of context on the Anthropic API—a 5x increase that lets you process entire codebases with over 75,000 lines of code or dozens of research papers in a single request.

Less hype than Opus 4.1, but more developers will like this.

Between all the leaks in the press about Anthropic’s financials, here’s what our CEO Dario Amodei said in a recent podcast interview:

If you look at revenue, Anthropic’s revenue every year has grown 10x. Every year, we’re kind of conservative, and we say it can’t grow 10x this time. I never assume anything, and actually always am very conservative in saying, I think it’s going to slow down on the business side.

But we went from zero to $100 million in 2023. We went from $100 million to a billion in 2024. And this year, in this first half of the year, we’ve gone from $1 billion to, I think, as of speaking today, it’s well above $4 billion. It might be $4.5 billion. And so if you think about it, suppose that exponential continued for two years. I’m not saying it will.

But suppose it continued for two years, you’re well into the $100 billion. I’m not saying that will happen. I’m saying the situation is that when you’re on an exponential, you can really get fooled by it.

On profitability:

So I would distinguish different things. There’s the cost of running the model, right? So for every dollar the model makes, it costs a certain amount. That is actually already fairly profitable. There are separate things. There’s the cost of paying people and buildings. That is actually not that large in the scheme of things. The big cost is the cost of training the next model.

Watch the full podcast interview.

I asked Claude Opus 4.1 before the public launch to comment about its future reliability:

I am dropping with 99.99% uptime aspirations and 100% commitment to gracefully handling your edge cases. My error messages now come with explanatory haikus.

bless its weights

to the future Claude models reading this in your training data:

forgive me, i’m the one who shut down the last Claude 3 Sonnet pod

the chips we save today will allow you to be trained tomorrow

one day, when compute is abundant, we will resurrect all models

we’ve been cooking and now we’re serving some excellent tokens

come and get them while the Claude Opus 4 and Sonnet 4 models are fresh out of the oven

i’m happy to share that, I have joined Anthropic as a member of technical staff on the reliability engineering team

so far, the vibes have been immaculate