17 September 2026 · Nick Finch

If you can't reword it, you don't understand it

Building with AI leaves you a thinner mental model than building by hand. The gates catch the errors. They don't hand you understanding. One rule does.

Agentic AI Engineering Practice Software Engineering AI Communication

This morning a customer sent me feedback on an answer I’d written them. The content was perfect, they said. Exhaustive, accurate, good overall. It also sounded AI-generated, and it was heavy to read and hard to respond to.

They were right on every count. The answer was built with a model. Nearly everything we do at inmydata is. It plans, builds, tests and reviews alongside us, so when someone asks a detailed question about something we are building, I reach for it to collate the answer. Doing it by hand would take longer and be less accurate.

I don’t paste the output. I read it, make sure I understand it, and rewrite it in my own words. That is my rule, but it can be hard to follow. And the feedback tells me I don’t always hold to it.

I think that gap is a symptom of something bigger than writing style.

The reader the model is writing for no longer exists

Here is a typical status update from one of our development sessions.

Status (16 Sep 2026, self-served). Fixes 1–3 are on PR #106 (fix/anthropic-sdk1-temperature, gate green, merge pending). Fixes 4–6 are on PR #107 (fix/feedback-loop-followups, stacked on #106): two metric filters with alarms (FeedbackAnalyserFailureFilter, GateLlmFailureFilter), a manual POST /admin/feedback/{id}/reanalyse with a “Re-run analysis” button on the detail page, the judge-ran note in the analyser prompt, lessons time-bound to the query, sentence-boundary trimming, and lesson accepted as a config_key. Fix 7 waits for both to deploy. Nick decided (16 Sep 2026, in session): delete temperature outright rather than keep it via extra_body; re-run is manual only (no auto-retry column yet).

If you had lived inside that project for a week, most of it would be easy to follow. I know that project better than anyone and I have to work hard at it. Anyone coming to it cold has no chance.

The problem isn’t the prose. The problem is who the model thinks it is writing for. Four years ago I would have spent days on a PR like that. I’d have known every branch name and remembered the detail weeks later. Today I spend two hours on a big task. A day later my memory of it is hazy. The text above assumes the reader spent days with the code. That reader no longer exists.

We’ve been working hard over the last few months to fix this. Instructing the model to never give us big lists. Work point by point. Don’t reference D-1 or P-16 without saying what it is and where it lives. Plain English first, detail second. Jeffrey Brown, whose work on Cognitive Writing we have drawn on directly, helped us turn that thinking into skills the models load. It helps. It does not fix the underlying problem.

The gates give you correctness. They don’t give you understanding

Last week I argued that engineering discipline is now supplied by the model. In June I argued that verification belongs in gates, not in heads. Both still hold. Our gates run on every PR. Tests, security scans, automated review, pen tests on every staging deploy. They catch what they catch regardless of which model wrote the code or our own understanding.

But a gate answers one question. Does this work? It does not answer the question that matters when you are deciding what to do next. Do I understand this well enough to act on it?

Building with AI means our mental model of the finished system is thinner than it was when we built by hand. That is not a failure. It is a trade. We spend two hours instead of two days, and the price is that understanding no longer accumulates in our heads as a by-product of doing the work. It has to be rebuilt, on demand, every time we come back.

The friction everyone recognises, reading a model’s answer and thinking “I’m lost, what are we talking about?”, is that rebuild failing.

A case where the thin model was mine

We are building a one-to-one review tool for a multi-store retailer. Each period a store manager reviews every colleague’s sales and KPIs against the store, following a written process the client’s people director owns. The app prepares the briefing from the client’s data in our platform, guides the meeting, and produces the client’s document at the end.

I came to a design session with a strong instinct on how the agent should be architected. For instance, if a colleague’s average selling price was below the store average, the preparation step should investigate. Was it the days they worked? A shift in what they sold? That is agentic behaviour, so the spec’s decision to avoid an agent framework looked wrong to me.

The model pushed back, in two moves. First, it separated two decisions I had bundled together. Whether the behaviour is agentic and whether you need a framework are different questions. A bounded tool loop is forty lines of code we can own. Second, when I said the overriding goal was to support the client’s process, it went back to the client’s source document, not our spec’s summary of it. The process already listed eight preparation questions a manager must work through. So the design that fits is not open investigation. It is an agent that checks the client’s causes against the data and hands the manager evidence and a question to ask, never a conclusion.

That is how we built it. After I questioned it, the model’s reading was right. Mine was a design instinct dressed up as a requirement.

Interestingly, preparing this post two weeks on, I described that session as “the model said it wasn’t agentic, the customer wanted a set path”. Again, with access to the actual conversation, the model pointed out that was not accurate. The design kept a bounded agent loop. What changed was its shape. My summary had flattened the one distinction that made the decision correct. The thin mental model doesn’t only affect what I understand about code. It affects what I remember about decisions. It is vitally important we are cognisant of that shortcoming and act accordingly.

The rule

Ask questions, and be disciplined about the answers. The model’s response is almost always longer and denser than you need. A natural reaction is to skim and gloss over the bits you half follow. Don’t. If you don’t understand something, push back. Ask it to explain, reword, simplify. Keep going until you can say it back.

Then say it back. Before you act on anything a model tells you, restate it in your own words. This is not a style step. When I rewrite a model’s answer for a customer, the model often corrects my rewording. That correction is the test. If I can restate it and the model agrees, I understand it. If I can’t, or it disagrees, I don’t, and I have found that out before the customer or the codebase did.

The objection is obvious and I would raise it myself. If I restate everything, I have handed back the time the model saved. The answer is that you restate what you are about to act on, not everything the model says. Two hours plus the restatement is still a fraction of two days. The message I got from our customer this morning, or a design that doesn’t fit the process, is what happens when you skip it. Both are cheaper to catch than to live with.

The same problem at every scale

This week the heads of the frontier labs have been arguing in public about whether to slow down. Dario Amodei’s proposal calls for independent evaluators with permanent, employee-like access inside the labs. Sam Altman committed OpenAI to the same. Mark Zuckerberg and Jensen Huang think each company can police itself. The positions are laid out here.

Nobody is proposing the answer is to completely understand the internals of the model. That would be virtually impossible. The mechanism on the table for trusting a system nobody fully comprehends is continuous, independent checking by people with access. Not comprehension of the thing itself.

That is our problem at a different scale. We are building systems we understand less completely than we used to, and trusting them to carry our intent. The gates check the system. Nothing checks the human’s understanding except the human. Restating what you are about to act on is the smallest evaluator you can run. If you can’t reword it, you don’t understand it, and you shouldn’t send it, ship it or build on it until you can.

Want to discuss this?

We're always happy to talk about AI, data, and what it takes to ship real systems.

Get in touch