by pron | 2026-02-07 19:58:05+00:00
Some people say that working with an agent, or an orchestrator of agents is like being a technical lead. But in the past few years I've been a technical lead for a few projects, and the experience of working with an agent doesn't even come close. I think that when people talk about the agents' coding abilities they're talking about the average ability. But as a team lead, I don't care about average ability. I care only about the worst case. If I have any doubt that someone might not complete a task, or at least accurately explain why it's proving difficult, with at least 95% certainty, I won't assign them the task. If I have any doubt that the code they produce might not be up to snuff, I don't assign them the task. I don't need to review their code; they review each others'. When I have to review code I'm no longer a team lead but a programmer.

I often have one programming project I do myself, on the side, and recently I've been using coding agents to test them out (OpenCode and Codex). Their average ability is no-doubt impressive for what they are. But they also make mistakes that not even a recent CS graduate with no experience would ever make (e.g. I asked the agent for it's guess as to why a test is failing; it suggested it might be due to a race condition with an operation that is started after the failing assertion). As a lead, if someone on the team is capable of making such a mistake even once, then that person can't really code, regardless of their average performance (just as someone who sometimes lands a plane in the wrong airport or even crashes without their being a catastrophich condition outside their control can't really fly regardless of their average performance). I wish the agent could work like a team of programmers and I would be doing my familiar role of a project lead, but it doesn't.

The models do some things well. I believe that programming is an interesting mix of inductive and deductive thinking (https://pron.github.io/posts/people-dont-write-programs), and the models have the inductive part down. They can certainly understand what a codebase does faster than I can. But their deductive reasoning, especially when it comes to the details, is severely lacking (e.g. I asked the agent to document my code. It very quickly grasped the design and even inferred some important invariants, but when it saw an `assert` in one subroutine it documented it as guarding a certain invariant. The intended invariant was correct, it just wasn't the one the assertion was guarding). So I still (have to) work as a programmer when working with coding assistants, even if in a different way.

I've read about great successes at using coding agents in "serious" software, but what's common to those cases is that the people using the agents (Mitchell Hashimoto, antirez) are experts in the respective codebase. I don't know what the future will bring, but at the moment, the craft isn't dead. When AI can really program, i.e. the experience is really like that of a team lead, I don't think that the death of programming would concern us, because once they get to that point, the agents will also likely be able to replace the team lead. And middle management. And the CTO, the CFO, and the CEO, and not just at software companies.