Why pilots stall
A pilot is optimized to show that a capability is possible. It runs on chosen examples, with its authors present, and it is judged on impression. Those very conditions are what a production operation removes. In production the inputs are adversarial and unbounded, no author is watching, and the system is judged on its worst behavior, not its best.
So pilots stall not because the model got worse, but because the questions changed. "Can it do this?" becomes "Can we depend on it, prove it, and recover when it fails?" A pilot rarely has an answer to the second set, because it was never built to.
What a pilot proves, and what it does not
A good pilot proves that a capability exists and that a workflow is plausible. It does not prove reliability under distribution shift, behavior on adversarial input, cost and latency at volume, or safe failure. Treating the first as if it were the second is the single most common reason AI work does not reach production — or reaches it and should not have.
The six things production adds
Crossing the gap means engineering what the pilot skipped:
- Evaluation. Defined datasets, metrics, and thresholds, measured before release and monitored after — not a one-time impression.
- Controls. A boundary around the model: scoped permissions, allow-listed actions, policy checks, and a human gate on consequential actions.
- Observability. Privacy-safe traces, metrics, and logs so a regression is visible before it becomes an incident.
- Recovery. Retries, graceful degradation to a manual path, dead-lettering for poison cases, and a documented way back to a good state.
- Human authority. Clear ownership of consequential decisions, and the ability to override, recorded.
- Ownership. Someone accountable for the system in operation, with the ability to run, change, and retire it.
Evaluation is a loop, not a gate you pass once
The most important shift is treating evaluation as continuous. A build is measured against datasets and metrics, must clear an acceptance gate with thresholds set in advance, and only then operates — under monitoring that watches the same metrics. Operational outcomes feed back into the next improvement, which is evaluated again before it is accepted.
Evaluation loop
Text description of this diagram
Evaluation is a loop, not a one-time sign-off. A build is measured against defined datasets and metrics, then must clear an acceptance gate whose thresholds are set in advance. Only then does it operate — under monitoring that watches quality, cost, latency, and human overrides.
Operational outcomes feed back into the next improvement, and the changed system is evaluated again before it is accepted. The datasets, metrics, and thresholds are set out in the threat, evaluation, and acceptance specification.
A production-readiness checklist
Before a system earns operating authority, these should have real answers:
- What is measured, on what data, and what threshold must it clear?
- How does it behave on adversarial and out-of-distribution input?
- What is the worst action it can take, and what stops it?
- How is a failure detected, and what is the path back?
- Who is accountable for a consequential decision?
- Who owns it in operation, and can they change and retire it?
Where this is realized, and its limits
The Governed Case Operations System is built around this loop, and its threat, evaluation, and acceptance specification sets out the datasets, metrics, and criteria in full. This publication argues what production-readiness requires; it does not report production metrics for any deployment, and each operation must define its own thresholds with its owners.