Decide if it's a toy before you build it

Two surveys landed this summer and they point in opposite directions.

McKinsey’s State of AI survey found that 32% of organisations have now declined to buy at least one software product because agentic coding tools let them build it themselves. In technology firms it is 41%. Among the companies McKinsey calls high performers, it is nearly half.

MIT NANDA’s State of AI in Business report found that internally built AI systems reach successful deployment about a third of the time. Purchased or partnered tools get there about two thirds of the time. Build it yourself and you fail twice as often.

One survey tells you to build. One tells you to buy. Both are right, because neither asked the question that decides the outcome.

Nobody asked who owns it

The McKinsey number measures a decision. Somebody chose not to sign a purchase order. The MIT number measures what is still standing a year later. Nothing in the first decision predicts the second, because the decision not to buy says nothing about who is going to keep the thing alive.

A few weeks ago we wrote about what the collapse of the build cost does to firms that sell software. This post is the other side of the counter. What it does to the firms that stop buying it.

When you decline a purchase and build instead, you do not remove the vendor from the arrangement. You become the vendor. The roadmap is yours. The patching is yours. The support queue, the model dependency that gets deprecated, the security posture, the decision about whether the tool should still exist next year. All of it moves onto your desk the moment the purchase order does not get signed.

Year one hides this. Year one is all honeymoon.

The arithmetic nobody re-ran

The maintenance literature has been consistent for decades. Depending on whose study you take, ownership accounts for somewhere between 60% and 90% of the lifetime cost of software. The Standish Group puts post-deployment enhancement at three to four times the original build.

Those are pre-agentic numbers, and that is the point. Agentic coding shrank the build term dramatically. It did not shrink the ownership term by anything like the same amount, because ownership is mostly judgement, not typing. Someone still decides what gets patched and when. Someone decides whether the tool has drifted from what the team needs. Someone decides when to swap the model underneath it and when to switch it off.

So the ratio has moved. If the build was ever a fifth of the lifetime cost, it is now a rounding error. Ownership is nearly the entire price of internal software, and the 32% have priced the build.

Klarna is the honest version of this at scale. They shut down around 1,200 SaaS applications and built their own stack. When the story became “Klarna replaced Salesforce with an LLM”, Sebastian Siemiatkowski corrected it publicly. They consolidated their knowledge into systems they now run themselves. They became the vendor and they staffed for it. That is what the decision costs when you make it consciously.

The boundary that matters

The failure MIT measured is not that building is hard. It is that nobody decided what the build was.

There are two kinds of internal tool. The first is a toy. You built it to save yourself an afternoon, it is allowed to break, and when it drifts you either fix it in an hour or throw it away. Nobody gets paged. The second is a product. Somebody owns it, somebody funds keeping it alive, and there is an agreed point at which you would kill it.

Both are legitimate. The disaster is the tool that was built as a toy and quietly became load-bearing. A team starts depending on it, nobody is accountable for noticing when it degrades, and it decays in production with real work running through it. That is the worst of both classifications, load-bearing and unfunded, and it is what a 33% success rate looks like from the inside.

The discipline is not paperwork. It is a conscious boundary. Before you build, know which of the two things you are building. If it is a product, build the structure around it that a product needs.

At inmydata the boundary is structural, which we did not set out to design but have come to rely on. Company products live in the company GitHub organisation, and anything there inherits the gating automatically. Security scanning and code review on every push. Module documentation. The test suite. Pen testing on every staging deploy. Our personal repositories are where the toys live. Moving something from one to the other is the classification decision, and everyone on the team knows it.

Three specimens

We have all three cases running, and they behave exactly as their classification predicts.

The pen test suite. In March we replaced a purchased service, commercial pen testing at £5,000 to £25,000 an engagement, with something built in a Friday afternoon. That is the McKinsey 32% behaviour, performed thirteen weeks before McKinsey started fielding the survey. It went into the company organisation, so it became a product the moment it was committed. Six months on it still runs after every deployment. It has needed maintenance, and it got it, because the structure it inherited made sure someone was looking.

Studio. We built an analytics platform that we expected to be the company’s flagship. It is not. The company is being built on something else. But Studio still serves paying customers, so it is a product whether we lead with it or not. It is maintained, gated, reviewed and pen tested like everything else in the organisation, because the alternative is being a vendor who does not show up. Deciding not to sell something does not release you from owning it.

This blog pipeline. The agent that proposes topics, researches them and briefs the drafts lives in my personal repository, outside the company organisation. It is a toy by design. Over the summer it ran thirteen cycles and I actioned two of them. Its proposals drifted into chasing the week’s news. One cycle was built on a peg that turned out to be false. Its evidence ledger fell eight posts behind while I published around it. I did not notice this as a failure, I just found myself using it less and writing more from scratch.

The execution machinery never missed a beat. Research ran every Monday all summer. What decayed was the product, because nobody owned the roadmap, and a toy does not get one. This week I noticed, spent a few hours re-specifying it, and it is useful again. That is the disposable classification working properly. Cheap to regenerate, allowed to drift, and the cost was slow erosion rather than a page at three in the morning.

Had a client team been depending on it, that same summer would have been the MIT failure.

The agents cannot make this decision

The strongest objection is that the same agents that built the tool can maintain it, so ownership is now cheap too.

Agents compress execution. They do not compress accountability. My pipeline is the proof. Its agents did their jobs flawlessly for two months while the product they were serving quietly stopped being worth using. Noticing that, and deciding it was worth a few hours to fix, was a roadmap decision. Nothing in the system was going to make it.

More to the point, an agent does not know which side of the boundary it is working on. It will build a toy with the same confidence it builds a product. You are the one who knows whether a team is going to depend on this next quarter, and the structure you put around it is how you tell the agents, and everyone else, which one it is.

Build the toy. Build it fast and throw it away without guilt. Just decide that is what it is before the first commit, because a toy that becomes load-bearing by accident costs you the same as a product, in time, money or performance, and you pay it without ever having chosen to.