After having managed business systems for a couple of years now, relying pretty heavily on coding agents to do so, I’ve lately been reflecting on the importance of the less “flashy” things that are becoming increasingly easy to forget in the age of AI. I mean the things like system resilience, testing. The “boring things.” It is so easy to stop holding code to a high standard and just ship on YOLO mode… and then blame the model.

Code is still code. The system built with AI, shipped without heavy testing, that then failed in production is the same system that would have been built by hand, shipped without testing, and then failed in production 20 years ago.

AI is a tool with limits. It’s both a product and a tool. There are legitimate reasons to not place blind trust in a product, and model regressions are real. But people may point the finger too much at the product, ignoring how it’s being used. They may not reason about the edge cases that bite in production weeks later. And when they do bite, the same person blindly tells Claude, “fix it no mistakes.” The model attempts to fix it inside a bloated, unorganized codebase. It of course breaks, making it feel like the model got worse. Then they wonder: “Is Opus 5.5 dumb now?”

So where do the habits actually go wrong? A few places I keep seeing.

Expecting the model to be the architect. These tools have limits, but I don’t think they’re generally bad. They are downright bad if you expect them to do the work of an engineer capable of reasoning about a well-architected system—but I don’t think that’s how they’re meant to be used. Architecture is not their forté. The human is supposed to be the architect who wields the tool.

Feeding it bloated context. A bunch of models really are getting a hell of a lot better, but people have been using a harness pulling bloated context for so long that irrelevant information is confusing the model when it’s supposed to focus on the present task. Imagine not being able to control your own working memory or attention. Imagine being forced to think about words irrelevant to the task you’re doing, losing focus while you’re doing it, and then still needing to pay attention to the objective. That’s what bloated context does to a model—irrelevant input literally stays in its “working memory” (i.e. its context window), when what it needed was very little of what it was fed. Context engineering is genuinely very important.

Using a nondeterministic system for a job that needs consistent output. LLMs are not deterministic. If a task is truly crucial and needs reliable output every time, I’d wonder if deterministic code is the better choice. If it’s a job that needs each run to produce identical output, an LLM may be the wrong tool for it.

Counting errors instead of tracking how often they happen. The chances of encountering an error increase as you continue to use a model. They’re nondeterministic, so odds are it’s eventually going to have a turn where the output is incorrect. It’s almost guaranteed to happen. So if your measurement is purely about whether errors happened, without tracking the frequency of those errors (the better indicator), you’ll just end up getting frustrated. And if you truly believe a new model stopped suiting a task, you can pin to one you prefer. Most frontier companies continue serving older models for quite some time. For example, the Claude Code harness still lets you go back as early as Opus 4.5.

All in all: AI or not, good engineering practices are still good engineering practices.