The model is the easy part

The demo went perfectly. You pasted a messy invoice into ChatGPT or Claude, it read the fields, answered the questions, and someone in the room said what everyone was thinking: this changes everything. Then the project moved toward production and quietly came apart. The same model that dazzled on ten clean examples started skipping fields at higher volume, choked on bad scans, and on Tuesday gave a different answer than on Monday to the same question.

A public model is built to sound plausible. Your operation has to be correct, secure and reliable every time. Those are different goals, and projects fall into the distance between them. The model was never the hard part. The hard part is everything around it.

Reading an invoice is not processing it

Take accounts payable at a company that receives thousands of invoices from hundreds of suppliers. In production, what the demo never showed comes out. Even when the electronic invoice has a standard format, the purchase order comes in the reference field, in the description, or nowhere at all. Credit notes look like invoices. Freight and additional taxes appear wherever each supplier decides to put them.

Reliable extraction requires validating it against many historical invoices, and even then it is only the beginning. Every invoice has to pass business validation, match against the purchase order and the goods receipt, follow the approval policy, be posted correctly in the ERP, recover if the transaction fails and leave a record. No finance department creates value by reading documents. The value is in closing the transaction, and if those steps are not built into the workflow, the model has only handed correct data to a person who still finishes the work by hand. The bottleneck just moved.

Why the demo works and the deployment collapses

The demo tests whether the model can do the task once, with clean inputs and a person watching. Production tests whether it can finish the whole job, with dirty data, at real volume, every day and without supervision.

A public model is extraordinary at producing a fluent, probably correct answer, and that is what a demo rewards. "Almost always right" is fine for a marketing draft. The operational backbone of a company does not have that margin. When invoices are posted or payments applied, almost always right means wrong often enough to hurt, and the errors do not announce themselves. They show up weeks later, at month-end close or in the audit.

And almost nobody has a way to catch them in time: according to McKinsey, in 2025 only about 1% of companies considered themselves mature in AI. The demo hides all of this because it is built to hide it.

A smarter model does not solve it

A frontier model can reason brilliantly through a hard case and still fail an operation's tests. It does not guarantee the same output twice, and a transaction requires it. On its own it does not stay within your policy, because it has no binding notion of what that policy is. And it cannot prove what it did or why, which is the first thing an auditor asks. Making it smarter improves the average answer, but it does nothing for consistency, security or accountability.

The difference, point by point:

• Accuracy: the model gives plausible answers; production needs transaction-grade accuracy, verified against your data.

• Consistency: the model varies from one run to the next; production needs the same correct result every time.

• Your data: with the public model, it leaves your company; production needs an isolated instance and data that trains no model.

• Systems: the model is a box you paste text into; production reads and writes live in your ERP, your CRM and your portals.

• Controls: the model comes with none; production needs approvals, segregation of duties, a record and rollback.

Control looks like red tape until the AI acts

An agent that only reads data seems low-risk, and controls seem optional. When that agent writes into your ERP, moves cash or files something in a portal, traceability, approvals and rollback become the difference between an asset and a liability.

When an AI agent fails inside a company, it is almost always for the same reasons: unchecked authority, flawed decision data, unpredictable outputs and no audit trail. All four are control failures. The team chose a capable model and skipped the layer that makes it safe to run.

An AI policy states an intention; control is applied at the moment of action. You do not put brakes on a car that is already rolling downhill.

What closes the distance

The infrastructure, the model and the frameworks for building agents can be rented by anyone with a credit card, and your competitors already have them. The operation lives in what sits on top. We organize it into the six decisions a company makes with any person who joins.

What it can see: the real process of the role, with its decision rules, its exceptions and its approval thresholds, written in a manual that belongs to the client.

What it can do: a closed list of named tools, with validated writes into your ERP and rollback when something goes wrong.

What it is forbidden to do: what it does not sign, what it does not promise and what has to pass through a person, also written in code so it does not depend on the model understanding it.

What it remembers: a folder per case that the client can open whenever they want, and not the model's memory, which is wiped when the conversation closes.

Who reviews: a manager with a name who approves what matters, after an exam with real cases whose outcome we already know.

What gets recorded: every action with its reason, its backing and its cost, and the business outcome as well as the task. The measure is that the invoice was paid and nobody complained.

On top of that comes the bill. In production, thousands of invoices a year with several calls each change the economics completely. That is why each step goes to the technology that fits it: a frontier model for judgment, a small one where the data cannot leave, vision for documents and fixed rules for whatever must never vary. What gets measured is the cost per transaction completed correctly.

None of this comes with the model, and all of it together is the operation: the intelligence of a frontier model with the reliability your transactions demand. That is what separates an agent that does tasks from a digital collaborator that takes charge of a role. And the model becomes a part that gets swapped when a better one appears.

Frontier models are remarkable and belong at the center, but they are not the system. The model is the engine. The operation is the engine plus the brakes, the steering, the dashboard and a driver who answers for where the car ends up.

The question that decides your project is not which model is best. That is the easy part, and it changes every few months. The question is what surrounds it. If you are weighing an AI project today, stop grading the model and start grading everything around it.

Sources

• McKinsey & Company, 2025: about 1% of companies consider themselves mature in AI.