Quality, cost, time - pick all three

A lot of the hopes for agentic coding tools have been that they would take the engineers out of the loop: no more process, no more planning, just describe what you want and let the agents rip.

What's actually happened is the opposite. Point an agent at a codebase with no clear objectives, no definition of done, and no tests, and it will cheerfully spin you up a mess at a speed no human team could match. AI doesn't replace engineering discipline - it just amplifies whatever you've already got.

Which is unfortunate, if what you've got is none.

Over the course of the last year, we've been building a complex SaaS product with extensive use of AI tooling. We've evolved practices which seem to have got as close as I've ever experienced to being able to say: "Quality, cost, time - pick all three".

We had some advantages, true - it was a new greenfield project, so we had no legacy friction to overcome. But that hasn't turned out to be the biggest win - where we've benefitted is that all the humans involved in the project had already learned the hard way about what engineering discipline is, and why it's needed.

The tools we're using are this year's favourites and will probably change over time. The joints between them, though, are best practice from at least the last twenty years.

It starts before any agent runs

Requirements get worked up with the product team and captured in Notion (the choice of tool isn't the important factor here, but Notion works because it's lightweight and exposes a good surface to AI tools). Then before the agent writes a line of code, we make it interview us: a structured grilling session that works through every decision point in the requirement. While that's going on, the agent writes up its implementation plan, and the plan gets reviewed. It's exactly like the way that code is (or should be) reviewed in pull requests, but at the design stage.

Agents have turned out to be really good at this, because although they haven't got the world model that the humans have, they can skim through the current state of the code, the past designs and all the documentation in a fraction of the time it would take a human. We've found that pays off when the design session flags up a problem: "you want to do X, but that's going to clash with Y which we introduced three months ago".

None of this is new, of course - it's really just a definition of ready. Nobody starts building something that isn't understood yet. But what's changed is the ability to get that comprehensive overview, and the enforcement. The grilling and the agreed plan become the context the agent codes from, so skipping this step doesn't save time any more - it just orders up the confident mess from the first paragraph, at machine speed.

Agent-sized pieces

Once the plan's agreed, the agent creates a feature-level story in Linear and breaks it down into smaller units of work. Small iterations used to be about keeping humans focused and feedback loops short, and they still are. The difference is that today, the feedback loops are minutes long, and a badly-scoped piece of work doesn't get to drift for a fortnight before anyone notices. It drifts before lunch.

Done means tested, and documented

The agents work to a fixed set of rules: red/green/refactor, tests written first, every third-party dependency mocked, and test data in place for the happy path, the unhappy path and the edge cases before the feature code exists. As each story lands, it's picked up by a separate agent for an independent adversarial review. Meanwhile, other subagents write the documentation as the work progresses: architecture, data structures, test plans, runbooks for the devops work, straight into Notion.

That's a definition of done, enforced mechanically: tested, reviewed, documented, or it isn't finished. When everything's through, the main agent opens a pull request for a human, with a summary of the changes and a test plan. A person still signs off, but they're doing it with a much better overview of what's changed.

The exceptions get their own adversary

There are some key features which are critical and non-negotiable - one of ours is tenant isolation. Ensuring that can be surprisingly subtle, and it's easy for regressions to sneak in if the focus of the story doesn't seem to be specifically tenancy-related. To guard against that, we introduced a subagent that has exactly one job - try to BREAK tenancy isolation by any means possible. It doesn't build or fix anything - it attacks every time there's a commit and reports back what it finds.

That's not a new idea either - it's a red team. What's new is that it costs nothing to run repeatedly on every story, instead of being saved for the scary ones.

The bit that didn't work

The tempting next step is obvious: wire the whole thing together and let it run end to end. Stack up a backlog of stories, spin up an army of agents and burn tokens to parallelise the whole exercise. That didn't work all that well - without the human in the loop, the agents can and do run off on complete tangents that have to be spotted and brought back into line. They've got no sense of context beyond what you construct around them: the requirements, the plans, the rules and the documentation are their entire world.

The other problem with this approach is that you've reintroduced the big design upfront approach by stealth. For a service of anything more than trivial complexity, that just doesn't work. There are too many insights which crop up along the journey which need to get folded back into the design, and those need human judgement. We've yet to find any combination of skills or prompting that can replace the people entirely - what this process does is speed up the mechanical parts, but leaves the humans in charge of the decisions.

There's an irony around artificial intelligence that the first profession to be wiped out by AI could be software engineering, the one responsible for bringing it to life in the first place. Based on the last year's experience, I think that might be overblown.

How we build products HAS changed radically, and the tooling has improved at a dramatic pace, faster than anything I've experienced so far in my career. But so far that hasn't removed the need for humans in the design and oversight roles. If anything, it's made experience MORE important, and the faster the agents go, the more that seems to be the case.

People like me have been banging on about this for years - perhaps this time, you'll believe us?