Skip to content
Cortaris

Insight

Why AI pilots don't reach production

Pilots rarely fail on the model. They fail on five things nobody scoped: write access, an owner, a control set, a baseline, and a rule for what happens when the agent is unsure.

6 min read

The pilot proved the wrong thing

Most pilots prove that a model can produce a plausible answer for a use case. That was never in doubt. What a pilot has to prove is that a workflow can run with the agent inside it: with the real input mix, real system access, real people releasing the output, and someone accountable when it is wrong. A pilot that reads from a data export and writes to a slide has proved nothing about production.

Five things that were never scoped

First, write access. The agent could read the CRM in the pilot; nobody asked whether it may write to it, and under which conditions. Second, an owner. The pilot belonged to the innovation team; the workflow belongs to the service desk, and nobody there signed up. Third, a control set. What does the agent do at low confidence, on a case out of scope, with an angry customer. Fourth, a baseline. Without handling times and error rates from before, nobody can say what improved. Fifth, an exit. Nobody defined the criteria for go-live, so nobody could say when the pilot was done.

The time goes into integration

In a build that reaches production, the model work is a small share of the effort. Most weeks go into integrating with the ticket system and the ERP, mapping permissions, building the test set from real cases, and agreeing with IT and compliance what gets logged. Pilots skip all of this. That is why they are fast, and why they are stuck.

The owner test

Before anything starts, one question: who in the business will look at the agent's output every day and be measured on it? If the answer is a role in IT or in an innovation function, stop. If it is the head of the service desk, the quote desk or the claims team, and they have said yes, you have a workflow. Everything else is a demo.

What to do with the pilot you have

Do not throw it away and do not scale it. Put it through the five questions. If write access, owner, control set, baseline and exit criteria can be defined in two weeks, scope a build. If not, the pilot has done its job: it told you which workflow not to start with. Pick the next one on evidence.

Next step

Start with one workflow.

Bring the process that costs you the most time. In 30 minutes we tell you whether it is worth automating, roughly what it would take, and which step comes first.