If AI coding is reducing your code high quality, you’re not managing high quality proper

One frequent tackle the coding brokers that I see goes one thing like this: “Sure, AI helps you output extra code, however gained’t the standard endure?”

It actually will in the event you simply blindly merge the PRs and ship them off to prod. But in the event you take a considerate, layered method to managing high quality, I discover that it’s potential to not simply maintain the variety of bugs secure however truly cut back it—whereas nonetheless rising the output by 2-2x.

Many of those defensive layers are just about the identical as earlier than Claude/Copilot/Codex/and many others. (although they’re made simpler now by AI), whereas others are new. Here’s a defensive setup that I’ve seen efficiently utilized in apply, each on my crew and elsewhere.

One of the most important surprises after I started using spec-driven development was the drop in bugs within the freshly written code. Before spec-driven improvement, when constructing, e.g., a brand new characteristic, the groups I used to be on typically spent as much as a 3rd of the whole effort on the post-development “sharpening,” i.e., discovering and fixing numerous bugs. Many of those bugs occurred both as a result of we didn’t foresee sure interactions and edge instances, or as a result of the developer was drained that day and didn’t put in sufficient thought, or as a result of the designer or PM didn’t assume by means of sure situations. Some of those bugs had been missed and ended up in manufacturing.

After I began utilizing spec-driven improvement, the variety of these bugs in my code sharply dropped, and I’ve seen the identical drop for some (however not all) of my teammates. As far as I can inform, the primary explanation for this drop is one particular step within the course of: having the AI overview the necessities or the tech design and discover any gaps, edge instances, sudden interactions with the prevailing code, or different comparable issues.

The AI doesn’t get drained and, when prompted proper, is rather a lot much less doubtless to surrender looking for potential points. If something, it may possibly generally be overzealous, and I’ve to fastidiously overview its proposed edits to the necessities to guarantee that it doesn’t invent any points that aren’t there.

Coding brokers now make test-driven improvement (TDD) trivial to the purpose the place there’s no motive to not do it. However, it must be achieved proper: you don’t need the agent to blindly write passing assessments for any bugs it simply added to the code. So the very best planning and implementation expertise I’ve seen normally comply with this sample:

• Instruct the agent to assume by means of the take a look at situations and take a look at instances primarily based on the necessities,

• Write the take a look at instances,

• Write the implementation,

• Test the implementation in opposition to the take a look at instances and repair any points that come up,

• Maybe backfill any remaining protection gaps—however once more, preserving the necessities in thoughts.

Also, with the brokers writing the assessments, there’s no excuse to not shoot for near-universal protection or to attend on backfilling any lacking unit assessments.

There’s nonetheless no substitute for a human (you, QA, PM, or another person) truly making an attempt out the characteristic, going by means of all the sting instances, and seeing whether or not every thing works as anticipated or whether or not it’s worthwhile to make adjustments.

These guide assessments can take some time, particularly if the take a look at situations take some effort to arrange. This is among the steps that thus far has seen solely modest good points in productiveness, and it’s the primary motive that my output has elevated solely 2-3x as an alternative of one thing like 10x. Though now that I give it some thought, there could also be just a few alternatives for automation right here that I’ve missed.

End-to-end (E2E) assessments are arguably an important assessments within the codebase as a result of they confirm that new adjustments haven’t damaged any present performance as skilled by the top consumer. Ideally, they’d run on the PRs, within the take a look at/stage environments, and in manufacturing after each deployment. Ideally, they’d even be maintained by the identical builders who write common code, however I perceive that some organizations aren’t actually arrange for that.

AI does make it simpler to write down E2E assessments, however to do this successfully, it wants entry to the instruments or MCP servers that allow it debug take a look at failures—e.g., a browser software or MCP entry to the logs. However, it’s vital to remember the fact that E2E assessments aren’t an alternative choice to guide testing as a result of they’re only a tough, incomplete examine that nothing vital broke.

I discover that coding brokers aren’t nice at following advanced directions in AGENTS.md or CLAUDE.md. But they do fairly effectively in the event you add a separate move to seek out and repair particular points. These might be:

• Security points,

• Finding overcomplicated or duplicated code,

• Compliance with naming, file group, or formatting guidelines,

• A common code overview move to seek out any points with the logic,

• Overly lengthy feedback written in AI-ese as an alternative of standard English,

• Any different particular issues that you simply’d like to seek out and repair.

If added to the planning or implementation expertise, these might be just about “free” additions, including possibly 5-15 min to the implementation time with no further consideration required.

They might be additionally added to the PR opinions in the event you desire to check out the feedback earlier than making use of any fixes.

I feel I’m changing into satisfied that for minor tweaks and easy bug fixes, human opinions can change into optionally available. Provided that different defensive layers are nonetheless in place.

But for advanced adjustments, I discover that it’s nonetheless essential to overview AI-written code. I nonetheless commonly discover big-picture errors, missed opposed interactions with different options, overcomplicated or suboptimal implementations, and different issues. Not to say bizarre phrase selections like “mint” as an alternative of “generate” or “stamp” as an alternative of “set.”

AI code opinions have additionally been a very welcome addition. On my present crew, we run each Claude and Cursor opinions on the PRs, and surprisingly, every of them finds totally different issues. You may add different customized opinions from numerous angles, like safety, effectivity, interactions with different repos, and so forth, although remember that AI might be overly nitpicky in its opinions, so it’s vital to even have a move the place one other agent prunes the proposed AI-generated PR feedback that aren’t truly significant.

Once the code is in manufacturing, at a minimal, it’s good to have somebody periodically scroll by means of the logs or watch any consumer recordings in one thing like Fullstory, or overview numerous dashboards that observe error charges, latencies, and different points.

Even higher can be an error monitoring service like Sentry or GCP’s Error Reporting that detects and deduplicates errors.

The greatest method, nonetheless, can be to then have Claude/Cursor/no matter auto-diagnose these errors, determine the foundation trigger, and make PRs with the proposed repair.

I’m certain I’ve missed different vital parts of sustaining prime quality, however the primary thought is that with the fitting set of defensive layers, the elevated output doesn’t have to return at the price of reliability. If something, coding brokers now make it cheaper so as to add extra and deeper checks than earlier than: extra assessments, extra overview passes, quicker prognosis of manufacturing points.

So in the event you’re sufficiently targeted on high quality, I feel it’s fully potential to double the supply velocity whereas preserving the bugs underneath management. Or possibly even decreasing them.