Evidence — Engineering blueprint
Governed AI and agent architecture
How to place AI, and where justified an agent, inside a critical operation so that capability rises without authority outrunning consequence. The reusable form of the flagship's control boundary — with its tradeoffs, failure modes, and limits stated.
Engineering blueprint
A reference architecture, generalized from the flagship system. Not a product and not client work.
Purpose
What this architecture is for.
This architecture puts AI, and where justified an agent, inside a critical operation so that capability can rise without authority outrunning consequence. It is the reusable form of the control boundary realized in the Governed Case Operations System.
When it is appropriate
- High-volume work that involves reading unstructured documents, retrieving policy or context, and drafting a grounded assessment.
- Decisions that must be traceable to a source and defensible after the fact.
- Operations where some actions are routine and safe to automate, while others are consequential and must stop for a human.
Architecture
Components, relationships, and boundaries.
The model never runs alone. It is one component inside a deterministic system that decides what happens next and records every step.
Control boundary
Text description of this diagram
The model or agent sits inside a control boundary and is wrapped by the governance it needs to run safely: scoped permissions, the tools and retrieval it is allowed to use, evaluation, monitoring, and recovery.
Consequential actions do not leave the boundary on their own — they pass a human approval gate. Authority is proportional to consequence.
Components
- Workflow orchestrator
- The deterministic engine that advances work through named states and decides when the model is called. It, not the model, holds control.
- Retrieval
- Fetches the policy and prior context a task needs, with provenance, so the model works from grounded material rather than memory.
- Model gateway
- The single, observable path to the model. It enforces context minimization, limits, timeouts, and privacy-safe logging.
- Policy and rules engine
- Deterministic checks the model's output must pass — required citations, prohibited content, and business rules.
- Tool and action registry
- An allow-list of the actions the system may take, each scoped and invoked only by the orchestrator, never by model output directly.
- Approval gate
- A named state where a person with the right authority approves consequential actions before they leave the boundary.
- Evaluation harness and audit log
- Measures the system against datasets before release and records every actor, input, citation, and decision in operation.
Data and authority boundaries
The whole system runs inside a control boundary. Data and authority are scoped so that capability is contained.
- Least privilege: every service and role holds only the permissions it needs; the model gateway cannot write to a system of record.
- Scoped context: the gateway sends the model the minimum context required for the task, and records what was sent.
- Allow-listed actions: the system can only take actions in the registry, and only through integration adapters the boundary can see.
- Content is not a command: text inside a retrieved document can never cause a tool to run — instructions come only from the orchestrator.
- Secrets stay outside the model path and are provisioned as bindings, never placed in prompts or logs.
Human authority
Where a person stays in control.
Authority is granted in proportion to the consequence of the action. The human role is designed, not improvised.
- Low-consequence steps run automatically within explicit bounds.
- Medium-consequence work is decided by an operator with the model assisting.
- High-consequence actions require a senior approver at the gate.
- A person can always override; the override, its author, and its reason are recorded.
- Low model confidence, a failed policy check, or an unreadable input escalates to a human rather than proceeding.
Evaluation
How it is measured, and how uncertainty is handled.
The system is only trustworthy if it can be measured and held to a standard before and during operation.
What is measured
- Groundedness — is every model claim supported by a retrieved citation?
- Task success — does the output meet the acceptance criteria for the task?
- Policy-check precision and recall — are the right outputs flagged, without drowning operators in false flags?
- Human-override rate — how often do operators correct the draft, and why?
- Escalation correctness — do consequential or low-confidence cases reach a human?
- Latency and cost per unit of work — is it operable at volume?
Uncertainty
The model returns a calibrated confidence, not a bare answer. Below a set threshold it abstains and escalates rather than guessing. Calibration is itself measured, because an overconfident model is more dangerous than a cautious one.
Observability
Each step emits a privacy-safe trace: what state ran, which tools were called, how long it took, and the outcome — never the confidential field values. Metrics and traces make a regression visible before it becomes an incident.
Failure modes
How it breaks, and what holds.
| Failure mode | Mitigation |
|---|---|
| Ungrounded or fabricated output | Require citations, run deterministic policy checks, and gate consequential actions behind a human. An unsupported claim cannot pass. |
| Prompt injection through a retrieved document | Treat all content as data, never as instructions; tools run only from the orchestrator; the action registry is allow-listed. |
| Tool or action misuse | Scope every action's permissions, require confirmation for consequential ones, and log every invocation. |
| Model or vendor outage | Degrade gracefully to a manual queue; the workflow continues without the model rather than failing the operation. |
| Silent quality drift | Monitor the evaluation metrics in operation; drift past a threshold triggers review and, if needed, a rollback. |
| Sensitive data reaching the model | Minimize and, where possible, redact context; record exactly what was sent; keep the boundary auditable. |
Rejected alternatives
What was considered, and why it was not chosen.
-
A fully autonomous agent with no human gate
Considered It promises the highest throughput and the least human effort.
Rejected Authority would outrun consequence: a single wrong action could not be caught before it left the boundary. Autonomy is instead tied to consequence.
-
One large prompt that does everything
Considered It is fast to build and demonstrates well.
Rejected It cannot be inspected, tested, or governed step by step, and it fails opaquely. A deterministic workflow with discrete, testable states is used instead.
-
Fine-tuning as the primary control
Considered A tuned model can reduce obvious errors.
Rejected Tuning changes behavior on average but gives no runtime guarantee. Governance must be enforced by retrieval, policy checks, and the gate — not hoped for from weights.
-
Deterministic automation (RPA) only
Considered It is predictable and needs no model.
Rejected It is brittle on unstructured documents and ambiguous cases — exactly where this architecture earns its place. Where the input is fully structured, the simpler approach is correct (see below).
Deployment & cost
How it is deployed, and what drives its cost.
- Client cloud or VPC
- The system runs inside the client's environment with client-controlled keys; data does not leave their boundary.
- Hosted model API vs self-hosted model
- A hosted API is faster to operate and to update; a self-hosted model gives full data control and predictable cost at the price of operational burden. The gateway isolates this choice so it can change later.
- Isolated or on-premise
- For the most sensitive data, the whole path — including the model — runs in an isolated environment, accepting higher cost and slower model updates.
Operating cost
Operating cost is dominated by model inference, the retrieval and vector infrastructure, and human review time — not by the orchestration itself. Cost is described qualitatively here; a real figure depends on volume, model choice, and the review rate, and is estimated per mandate rather than asserted.
When not to use this architecture
The honest limits.
- When a deterministic rule, form, or report already solves the problem — do not add a model for its own sake.
- When the input is fully structured and unambiguous; conventional automation is cheaper and more predictable.
- When the consequence of any residual error is so high that a human must do the whole task, not merely approve it.
- When the data cannot be governed or the operation cannot be audited — the boundary is the point, and without it the architecture should not ship.
- When volume is too low to justify the evaluation and monitoring the architecture requires to stay safe.
What this does and does not prove
It shows
- That autonomy can be made proportional to consequence with explicit gates and boundaries.
- That governance components (retrieval, validation, policy, evaluation, audit) are architecture, not afterthoughts.
It does not show
- That a specific model or agent framework is best for a given problem.
- Quantitative reliability figures for any particular implementation.
How to inspect it
Read the architecture, authority boundaries, failure modes, and the explicit 'when not to use this' section. It is realized concretely in the flagship system.
Engagement
Begin with what must change.
If AI works in a prototype but cannot yet be trusted in your operation, a mandate begins with the control boundary it must run inside.
Or submit an RFP, or request an NDA first.