Most companies we talk to have an AI pilot. Far fewer have AI doing real work every day. The gap between the two is now the main problem in enterprise AI, and the research from the past year says the same thing in different ways.
- MIT Project NANDA, in its 2025 report The GenAI Divide, found that about 95 percent of the organisations it studied saw no measurable return from their generative AI initiatives, despite an estimated $30 to 40 billion in enterprise spending.
- McKinsey's 2025 State of AI survey of nearly 2,000 respondents found 62 percent of organisations at least experimenting with AI agents, but only 23 percent scaling them anywhere, and no more than 10 percent scaling agents in any single business function. Just 39 percent reported any effect on enterprise EBIT.
- Gartner predicted in June 2025 that more than 40 percent of agentic AI projects will be cancelled by the end of 2027, citing rising costs, unclear business value and inadequate risk controls.
Each of these studies has its critics, and the MIT figure in particular uses a narrow definition of success. But they agree on the pattern: building a demo is easy, and getting it into production is where projects die.
Why pilots stall
When an AI project stalls, the cause is usually one of five things.
1. The goal was "use AI"
A pilot launched because the board asked for an AI strategy tends to look for a problem after the fact. Without a target like "cut the time to process a supplier invoice from 20 minutes to 2", nobody can say whether the pilot worked, so nobody approves the next stage.
2. The demo ran on clean data
Pilots often use a tidy export or a few sample documents. Production means scanned PDFs, half-filled CRM records, three versions of the same customer and an ERP with no proper API. The accuracy that looked great in the demo drops sharply when it meets real data.
3. Nobody could measure accuracy
"It looks good" is not a test. Without a set of real examples and a way to score the AI's answers, every model change, prompt edit or new edge case is a gamble. Teams lose confidence, and so do the people who would use it.
4. It sat outside the workflow
The MIT report describes this as brittle workflows and a lack of fit with day-to-day operations. A separate chat window that people have to remember to open will not change how work gets done. The AI has to live in the tools people already use: the ERP, the CRM, WhatsApp, email, the ticketing system.
5. Cost and risk were never planned
Token costs that look trivial in a pilot can grow quickly at volume, especially with agents that make many calls per task. And an agent that can write to your systems needs permissions, audit logs and approval steps. If those come up for the first time at the security review, the project stops there.
What the projects that make it do differently
McKinsey's high performers, the roughly 6 percent of respondents who attribute 5 percent or more of EBIT to AI, were about three times more likely than others to be scaling agents rather than piloting them. The projects that make it to production tend to share a few habits.
They start with one expensive workflow. Not "AI for finance", but "match incoming invoices to purchase orders and flag mismatches". Narrow scope makes it testable.
They build on real systems from day one. The pilot connects to the actual ERP or CRM, even if only in read-only mode at first. Integration problems surface in week two, not month six.
They build the test set first. We take 100 to 300 real past cases, record the correct outcome for each, and score every version of the system against them. That turns arguments about quality into numbers.
They keep humans in the loop where it matters. The agent handles the routine cases and sends exceptions and high-value actions to a person for approval. Automation coverage grows as trust is earned.
They watch unit cost. Cost per completed task is tracked from the first pilot run and compared with the cost of doing it manually. Routing simple steps to smaller models and caching repeated context keep it down.
They have a business owner. Someone in operations or finance owns the outcome and reviews the numbers monthly. IT and the AI team support them.
A production readiness checklist
Before you call an AI system production-ready, check that it has:
- A business metric with a baseline and a target
- Integration with the live systems it reads from and acts in
- An evaluation set of real cases, with an accuracy score that is tracked over time
- Guardrails: input validation, limits on actions, and a maximum number of steps per task
- Human approval for actions above a defined risk or value threshold
- Logging of every input, decision and action for audit
- Role-based access, so the agent can only see and change what it should
- Monitoring of cost per task, latency and error rates
- A fallback path when the AI is unsure or a system is down
- A named business owner and a monthly review
If more than two of these are missing, the system is still a pilot, whatever it is called.
Build, buy or partner?
According to coverage of the MIT research, projects that used purchased tools or external partners reached deployment about two-thirds of the time, against roughly one-third for purely internal builds. There is a caveat. Off-the-shelf products work well for generic tasks like meeting notes, writing help or standard customer support. When the work happens inside your own ERP, pricing rules or approval chains, you need something built for your process, whether your own team builds it or a partner does.
A practical split looks like this:
| Use case | Usual approach |
|---|---|
| Writing, summarising, meeting notes | Buy an existing product |
| Standard website or WhatsApp support | Platform, customised |
| Agents that act in your ERP, CRM or internal tools | Custom build |
| Processes that are a competitive advantage | Custom build |
| Regulated data or strict residency rules | Custom build, private hosting |
How we run AI pilots at Appmetry
Our AI solutions and AI agent development work follows the habits above. Every pilot has a fixed scope, one workflow, a success metric agreed up front, integration with your real systems, and an evaluation suite built from your own past cases. At the end you get a clear go or no-go decision backed by numbers, plus a running cost estimate for production.
For larger organisations rolling AI out across several functions, our enterprise transformation team helps sequence the work so that data and systems are ready before the agents arrive. If you are still choosing a first use case, our guide to AI agents for business covers where they work best, and the AI agent cost guide helps with budgeting.
If you have a pilot that is stuck, send us a note. We will look at what you have and tell you honestly whether it can get to production.
Sources
- MIT report: 95% of generative AI pilots at companies are failing, Fortune, August 2025: buy versus build deployment rates.
- MIT report: 95% of corporate generative AI pilots fail to deliver returns, Computing, August 2025: coverage of MIT Project NANDA, The GenAI Divide: State of AI in Business 2025.
- MIT report finds most AI business investments fail, reveals GenAI divide, Virtualization Review, August 2025: spending estimate, causes, and buy versus build findings.
- The state of AI in 2025: Agents, innovation, and transformation, McKinsey, November 2025: survey of 1,993 participants on agent adoption, EBIT impact and high performers.
- Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027, Gartner, June 2025.
- Gartner: more than 40% of agentic AI projects will fail by 2027, RCR Wireless, June 2025: independent coverage of the Gartner forecast.