terminal

TIM WARREN

chevron_leftBack to Insights
JUN 12, 2026

The AI Isn't Why Your Project Failed

Most AI projects die in the gap between a working demo and shipped value. That gap is project management, not model quality.

COVER_IMAGE
A developer's workspace with code on screen and a project board, illustrating AI work managed like a real project

Most AI projects don't fail because the model was bad. They fail because nobody managed the project. The demo worked, the room clapped, and then six months later there was nothing in production and no one could say why.

The numbers back this up. MIT's 2025 study of enterprise AI found that 95% of generative AI pilots delivered no measurable P&L impact. Gartner expects that by the end of 2025, at least 30% of generative AI projects will be abandoned after the proof of concept, and that over 40% of agentic AI projects will be canceled by the end of 2027. The reasons they give are not exotic. Unclear business value. Escalating costs. Bad data. Weak risk controls. None of that is a model problem. All of it is a management problem.

the gap between demo and delivery

Here is what the hype gets wrong. It treats the model as the hard part. The model is the easy part now. You can wire up a capable LLM in an afternoon and get something that looks like magic in a controlled demo.

The hard part starts after the demo. Where does the data come from, and who keeps it clean? What happens when the model is confidently wrong in front of a customer? How do you measure whether this thing is actually better than the spreadsheet it replaced? Who owns it when it breaks at 2am? A demo answers none of these. A shipped product answers all of them.

This is the gap where projects die. The team builds a thing that works once, on clean inputs, with someone watching. Then they try to make it work a thousand times a day on messy inputs with no one watching, and it falls apart. The failure looks like a technology failure. It's a planning failure wearing a technology costume.

why AI makes the management harder, not optional

People assume that smarter tools need less management. The opposite is true. AI projects are harder to run than normal software, for three reasons.

First, they're non-deterministic. Normal software does the same thing every time, so you can test it and trust it. An LLM gives you a slightly different answer each run. You can't just check that it works once. You need evals, which are repeatable tests that score the output across hundreds of cases, and you need to run them every time something changes. If your project plan has no line item for evals, your plan is fiction.

Second, the work is research, not construction. When you build a normal feature, you can estimate it because you've built things like it before. When you build with AI, you often don't know if the approach will work until you try it. That means short experiments with kill criteria decided in advance, not a six-month roadmap pretending to be certain. Managing research is a real skill, and most teams don't staff for it.

Third, the data is half the product. The MIT and Gartner findings both keep landing on the same root cause: data that isn't ready. A model is only as good as what you feed it, and getting clean, available, permissioned data into the thing is usually the longest pole in the tent. It's also the least glamorous work, so it gets skipped in the rush to a demo, and the project stalls there long before the model ever gets a fair test.

the take

I've watched this pattern up close as a fractional CTO, and the fix is boring on purpose. Treat the AI part as one risky component inside a normal, well-run project, not as a magic exception to the rules.

That means a few concrete things. Define what success looks like before you write a line of code, in a number you can measure. Not "use AI," but "cut ticket resolution time by 30% without raising the error rate." Build the eval harness early, because if you can't measure quality you're flying blind. Put the riskiest unknown first, prove or kill it in two weeks, and don't commit a roadmap until you've seen it work. Decide who owns the thing in production on day one, not day ninety. And spend the unglamorous time on data, because that's where the project actually lives or dies.

None of this is new. It's the same project discipline that made software projects succeed for the last thirty years. The AI doesn't excuse you from it. It raises the stakes, because the demo looks so good that it's easy to convince yourself the hard part is done when it has barely started.

The companies in MIT's 5% aren't smarter about models than everyone else. They're more serious about delivery. That's the whole difference, and it's available to anyone willing to do the dull work between the demo and the launch.

Common questions

Why do most AI projects fail?

Most fail in the gap between a working demo and a shipped product, not because the model is weak. MIT found 95% of generative AI pilots delivered no measurable P&L impact, and the usual causes are unclear value, bad data, and no clear owner.

Is managing an AI project different from a normal software project?

Yes, and it needs more management, not less. AI work is non-deterministic, closer to research than construction, and gated by data quality, so it needs evals, kill criteria, and clear ownership that a normal feature build can skip.

What is an eval in an AI project?

An eval is a repeatable test that scores a model's output across many cases, so you can tell whether a change made quality better or worse. If you can't measure quality, you can't manage the project.

Do better AI models remove the need for project management?

No. Better models make demos easier, which makes it more tempting to skip the planning. The hard parts (data, integration, ownership, measurement) are management problems that no model solves for you.

Sources

Need a CTO in your corner?

Book a CallView Services
chevron_leftAll Transmissions