From AI pilot to production in four steps

Why pilots stall

The reasons are rarely technical. Nobody wrote down what success meant. The demo ran on twenty clean sample invoices, not on the scanned, rotated, half-handwritten ones that arrive in a real accounts inbox. The security review was booked for the week before launch. And once the first excitement faded, nobody owned the thing.

Each of those has a simple fix, and each is built into one of four steps.

Step 1: Assess (1 to 2 weeks)

This step is about choosing one use case and sizing it realistically. It has a fixed scope and, like all our work, is paid after you accept it.

• Sit with the people who do the work and list the candidate tasks. Managers know where the pain is; clerks know where the time goes.
• For each task, estimate hours saved, errors avoided or revenue protected. Rough numbers are fine if the reasoning is written down.
• Look at the actual data: Tally, a shared drive, a WhatsApp group or someone's head. Confirm the system can get read access.
• Estimate running cost, including model usage and the time a person will spend reviewing output.
• Agree one success measure and the guardrails around it.

Gate: leadership signs off one use case, with its measure and a budget. Not three use cases. One.

Step 2: Pilot (2 to 3 weeks)

Build a working version on real data for a small group of real users, say four people in the accounts team.

• Connect to the real systems, even if read only at first.
• Build an evaluation set from real cases, and go looking for the awkward ones: the supplier who sends invoices as photos, the customer who writes in Gujarati.
• Put a human review step in front of any action that carries risk.
• Report against the success measure every week, including the weeks when it looks bad.

Gate: the pilot meets its measure and you accept it. Under our model, that is when the pilot is paid for.

Step 3: Production

From here the work is mostly ordinary engineering and change management, which is exactly why it gets underestimated.

• Harden the integrations, error handling and access controls.
• Add monitoring for quality, cost and failures, so problems show up on a dashboard before they show up in a complaint.
• Train users and write short runbooks in the language the team actually uses.
• Roll out in waves, one branch or one team at a time, and check the numbers after each.

Gate: the system stays stable at the agreed quality for an agreed period, usually a few weeks.

Step 4: Managed operations

An AI system does not stay finished. Providers retire model versions, your data shifts, a new product line brings questions the system has never seen. A monthly plan covers monitoring, quality checks, cost control, model upgrades and a small budget for improvements, under an SLA.

If nobody is named to do this work, the system will quietly get worse. Better to decide that before launch than after.

One test before you start

Ask the sponsor to state the success measure in a single sentence. "Post 70 percent of supplier invoices to Tally without manual typing" passes. "Use AI to improve finance" does not. If the sentence will not come, the pilot is not ready yet.