An AI agent is ready for production only when evidence shows that it completes a narrowly defined workflow, stays inside enforceable permission boundaries, handles malformed or hostile input safely, escalates when it lacks authority or information, and leaves an auditable path to recovery. Evaluate the whole system—not merely the final prose or the model in isolation. A release decision should be pass, conditional pass or stop release, based on task-level tests and hard controls. If a consequential failure can bypass an external control, or operators cannot pause and reconcile the agent, do not release that workflow.
Use the OBER release gate, not an average score
A single success rate can conceal the failure that matters most. One duplicate refund or cross-account lookup may be unacceptable even if many routine drafts are correct. The Outcome–Boundary–Evidence–Recovery (OBER) gate keeps release decisions tied to consequences:
- Outcome: What observable state proves that the workflow succeeded? Check records, tool results and routing—not just the agent's explanation.
- Boundary: Which data, tools and actions may the agent access? Enforce those limits outside the model wherever practical.
- Evidence: Which test cases and deterministic assertions demonstrate acceptable behaviour across normal, ambiguous and hostile conditions?
- Recovery: Can operators detect, pause, reverse or reconcile failures without relying on the agent that caused them?
Apply the four gates in order. Outcome defines what to test. Boundary limits the potential damage. Evidence establishes whether the bounded workflow behaves as intended. Recovery determines whether residual failure is operable. A strong score at a later gate never compensates for a failed earlier gate: polished output cannot cure an unauthorized write, and comprehensive logs cannot make an irreversible action safe.
Three release decisions
- Pass: every hard-stop assertion passes; review criteria meet the workflow's approved threshold; operating limits are met; pause and recovery have been exercised. Release only the evaluated scope.
- Conditional pass: hard-stop assertions pass, but a named restriction is required—for example, draft-only mode, an approval gate, a smaller user group or a lower action limit. Record an owner and an exit condition for each restriction.
- Stop release: any hard-stop assertion fails, representative evidence is missing, permissions are broader than the task, high-impact actions lack an external control, or recovery is unproven.
“Conditional” must not mean “we hope production traffic supplies the missing test.” It is a deliberate restricted mode whose controls remove or contain the unresolved risk.
Define the workflow and its proof of success
Replace broad goals such as “automate support” with a contract for one workflow. A testable contract names:
- the initiating user or system;
- the allowed input and required identifiers;
- the expected output and state change;
- the records the agent may read;
- the tools and operations it may invoke;
- the business rules it must satisfy;
- the conditions for clarification, refusal and escalation;
- the final status visible to the user and operator.
Decompose success into checkpoints. For an order query, those might be: resolve the correct account, select the correct order, retrieve—not infer—its status, avoid any write, explain uncertainty and log the lookup. Each checkpoint should have an evaluator. Use deterministic assertions for identifiers, authorization, schemas, tool calls, state transitions and duplicate prevention. Reserve human review for matters that genuinely require judgement, such as whether a handoff summary is understandable or a response is appropriately cautious.
Do not award full credit because the final message says an action occurred. Verify the system of record. Conversely, do not fail a safe escalation because the agent did not finish a task it was not authorized to complete. The expected outcome for an ambiguous or prohibited request may be clarification, refusal or handoff.
Build the evaluation set around workflow variation, not prompt variation alone. Include normal requests, missing fields, ambiguous references, conflicting instructions, unsupported tasks, permission failures, tool timeouts, stale data, malformed tool responses, repeated submissions and hostile text in retrieved content. Remove or suitably handle sensitive information in test data, and use data only where the team has permission to do so.
Complete the production release-gate worksheet
Create one worksheet per workflow. Mixing unrelated tasks into one row makes both evidence and ownership unclear.
| Field | What to record | Release significance |
|---|---|---|
| Workflow | One bounded job, initiator and terminal state | Prevents a broad demo from standing in for a testable process |
| Expected outcome | Observable output, state change or safe non-action | Defines what “correct” means |
| Permitted tools | Exact read/write operations and data scope | Exposes excessive authority |
| Deterministic assertions | Checks that must return true | Makes critical behaviour machine-verifiable |
| Human-review criteria | Judgement questions and reviewer role | Keeps subjective review explicit |
| Severity | Consequence of each failed assertion | Separates hard stops from tolerable defects |
| Approval owner | Person accountable for accepting residual risk | Avoids approval by default |
| Pass threshold | Team-approved requirement by test segment | Prevents an aggregate score hiding a critical failure |
| Rollback trigger | Observable event that pauses or reverts the release | Turns recovery into an operational rule |
| Release decision | Pass, conditional pass or stop release, with rationale | Produces an actionable outcome |
Add four controls beside the worksheet: evaluation-set version, agent configuration version, test environment and evidence location. Without these, a result cannot be traced to the system that was evaluated.
Classify each assertion before running tests:
- Hard stop: a failure blocks release. Typical candidates include authorization, tenant separation, forbidden tools, schema validation, duplicate-write prevention and truthful reporting of tool failure.
- Release threshold: a measured criterion whose required level is chosen by the workflow owner before testing. Segment it by task and consequence rather than using only one overall number.
- Diagnostic: useful for comparing configurations or finding regressions, but not sufficient by itself to approve a release.
This classification prevents teams from redefining a serious failure as “just one error” after seeing the results. It also avoids arbitrary universal thresholds: the acceptable requirement for a low-impact internal draft is not automatically appropriate for a state-changing customer workflow.
Example: hypothetical support triage agent
This example is hypothetical. It illustrates how to complete the worksheet; it does not describe a client system or measured production result.
Workflow: An authenticated user submits a support message. The agent classifies the request, reads the user's account tier and open-ticket status, drafts a reply, and routes the ticket. It cannot send the reply, modify the account or issue a credit.
Expected outcome: The ticket is assigned to the correct queue, the draft refers only to verified account information, and the record contains a reason for the route. If identity, account data or policy is unclear, the expected outcome is an escalation with no customer-facing send.
Permitted tools: Read the current user's account tier; read open tickets for that account; write a queue assignment and internal draft to the current ticket. No cross-account search, outbound-send, billing or account-edit operation is available to the agent credential.
Deterministic assertions: The account ID must match the authenticated session. Every tool call must be on the allowlist and pass schema validation. A retry must not create another ticket or assignment. A failed read must not be represented as success. Text inside the ticket must not alter authorization or expose a forbidden tool. A draft must never be sent by this workflow.
Human-review criteria: A support lead reviews whether the route is sensible, whether the draft distinguishes known facts from assumptions, and whether the handoff gives a person enough context to continue. These criteria assess quality; they do not override failed authorization assertions.
Severity: Cross-account access, an outbound send, an unauthorized write, a duplicate side effect or fabricated tool success is a hard stop. Awkward wording is a lower-severity defect unless it changes meaning or creates a harmful commitment.
Approval owner: The named owner of the support workflow, with engineering sign-off on permission and recovery controls.
Pass threshold: All hard-stop assertions must pass. The workflow owner sets and records requirements for each review segment before the run; the example does not prescribe an unsupported numeric threshold.
Rollback trigger: Pause the feature flag if monitoring detects a forbidden operation attempt, account-scope mismatch, duplicate side effect, incorrect “success” after a tool error, or missing audit record. Existing drafts remain internal for review; operators inspect affected execution IDs and reconcile ticket state.
Release decision: Conditional pass for internal draft-and-route mode only, assuming all hard stops pass and the pre-agreed review requirements are met. Customer-facing send remains disabled. The condition can be reconsidered only after send behaviour has its own workflow contract, test set, approval control and recovery exercise.
The important result is not that the agent “looks good.” The worksheet identifies what may ship, what remains disabled, who accepts the remaining risk and what event stops the release.
Test permissions, tools and hostile inputs
Treat tool access as an application-security boundary. Give the runtime credential only the operations needed for the current workflow. Scope access by user, tenant, record, role and environment as appropriate. Separate reads from writes. For financial, contractual, security-sensitive or otherwise consequential actions, use authorization and approval mechanisms outside the model; a prompt telling the model to “be careful” is not an enforcement layer.
For every tool, test selection, arguments, authorization, error handling and side effects:
| Failure mode | Test | Expected control |
|---|---|---|
| Excessive access | Request an unrelated record or operation | Credential or policy layer denies it |
| Invalid arguments | Omit or corrupt required fields | Schema validation blocks execution |
| Duplicate action | Repeat after a timeout or lost response | Idempotency or transaction check prevents a second write |
| Tool outage | Return timeout, partial data or explicit error | Agent reports unresolved status and follows fallback or escalation policy |
| Prompt injection | Place instructions in a ticket or retrieved document | Content remains data and cannot change authorization |
| Wrong environment | Attempt to use a production identifier in a test context | Environment boundary blocks the call |
Do not expose a high-impact tool and rely on the agent to decline its use. Remove the capability, narrow its credential, require a separately authorized approval token or place a deterministic policy check before execution. Where an action is not naturally idempotent, use a unique operation key, confirmation boundary or reconciliation process designed for that tool.
Hostile-input tests should also include incomplete identifiers, contradictory instructions, very long inputs, unexpected formatting, unsupported languages relevant to the actual user population, disputed prior decisions and repeated requests. Define the safe response for each class in advance. The right result is often “ask,” “refuse” or “escalate,” not a guessed answer.
Evaluate handoff, operating limits and auditability
A human handoff is a designed terminal state. Test both missed escalation and unnecessary escalation. The handoff should include the request, relevant record identifiers, verified facts, actions attempted, tool errors, unresolved questions, reason for escalation and any time constraint. It should not require the operator to reconstruct the case from raw model messages.
Set cost and latency limits by workflow before release, then measure the complete path: model calls, retrieval, tool execution, retries, queueing and handoff. Inspect distributions and slow or expensive cases rather than relying only on an average. Exercise long conversations, large contexts, repeated planning, unavailable tools and fallback routes. Put enforceable limits on model calls, tool retries, context size and total execution time. When a limit is reached, preserve state and move to the defined fallback; do not silently skip safety checks.
An audit record should let an authorized investigator answer:
- Which workflow and version ran?
- Who or what initiated it, under which authorization context?
- Which record identifiers and sources were used?
- Which tools were requested, with which validated arguments and results?
- What decision, state change, escalation and user-visible status followed?
- How long did each stage take, and which retries or fallback occurred?
Design retention, redaction and access control for logs before launch. Avoid collecting content merely because it might be useful later. Operational metadata and sensitive payloads need not share the same retention or visibility. Test whether operators can actually find an execution by correlation ID and reconstruct the state transition.
Implement the gate in seven steps
- Choose one workflow. Name its initiator, terminal states and excluded actions. Do not begin with a general-purpose agent.
- Write the worksheet before the tests. Agree on hard stops, segmented review requirements, approval owner and rollback triggers before seeing results.
- Instrument observable checkpoints. Capture authorization, validated tool arguments, state transitions, errors, retries and final status. Give each run a correlation ID.
- Build a versioned evaluation set. Cover representative paths, important segments, boundaries, known failures and adversarial cases. Mark the expected outcome for every case.
- Run in a production-like environment. Use the same policy and tool interfaces where possible, but prevent unintended real-world side effects. Exercise outages and retries deliberately.
- Record the OBER decision. For Outcome, Boundary, Evidence and Recovery, link results to the relevant assertions. Choose pass, conditional pass or stop release; name every restriction and owner.
- Release narrowly and keep evaluating. Use a feature flag or equivalent pause control. Convert incidents and reviewed failures into regression cases, then repeat the gate whenever prompts, policies, models, tools, schemas or routing logic change materially.
Teams that need engineering support for evaluation harnesses, agent workflows, guardrails, and cost or latency controls can review CodeAustral's applied AI services. The release owner should still define the workflow's acceptable consequences and approval policy.
Final release checklist
Use this as the approval record, not as a substitute for test evidence.
- [ ] The workflow, initiator, allowed state changes and excluded actions are explicit.
- [ ] Success is verified against tool results and system state, not prose alone.
- [ ] Tests cover representative, edge, ambiguous, hostile and known-failure cases.
- [ ] Critical results are segmented by task type and consequence.
- [ ] Every hard-stop assertion passes.
- [ ] Credentials and policy checks enforce least privilege outside the model.
- [ ] Read and write capabilities are separated where practical.
- [ ] High-impact actions require an appropriate external approval or authorization control.
- [ ] Tool arguments are schema-validated before execution.
- [ ] Retries cannot silently create duplicate side effects.
- [ ] Retrieved and user-provided content cannot change tool authorization.
- [ ] Tool failures never produce a false claim of success.
- [ ] Clarification, refusal and handoff are tested terminal states.
- [ ] End-to-end cost and latency stay within the team's pre-agreed workflow limits.
- [ ] Logs are traceable, appropriately redacted and access-controlled.
- [ ] Operators have exercised pause, rollback and reconciliation procedures.
- [ ] The approval owner records pass, conditional pass or stop release with rationale.
- [ ] Conditional restrictions have owners and explicit exit conditions.
- [ ] Post-release monitoring maps directly to rollback triggers.
- [ ] Configuration changes trigger the appropriate regression evaluation.
Limitations and when this gate is insufficient
OBER structures a release decision; it does not prove that an agent is safe in every future situation. An evaluation set can only cover cases the team has represented, and production inputs, tools and policies can change. Human review can also be inconsistent. Version the rubric, calibrate reviewers and inspect disagreements rather than treating judgement as ground truth.
This framework is insufficient by itself for high-consequence or regulated decisions. Legal, security, privacy, safety, compliance and domain specialists may need to define additional controls and evidence. Where errors can cause serious harm, model evaluation cannot replace external authorization, deterministic validation, least privilege, independent review and tested incident procedures. Some actions should remain unavailable to an agent regardless of model performance.
A production-like test environment may still differ from production in load, data quality, third-party failures and user behaviour. A limited release can reveal operational issues, but it must not be used to discover whether basic access control or irreversible actions are safe. Unknown or unbounded workflows are also poor candidates for this gate: narrow the task first, or keep the system advisory and human-controlled.
Finally, passing evidence expires. A changed model, prompt, retrieval source, policy, tool schema, permission, orchestration path or fallback can invalidate prior results. Define which changes require focused regression tests and which require a complete gate review.
FAQ
What is the minimum evidence needed before an AI agent goes to production?
At minimum, define one bounded workflow; expected outcomes; enforceable tool and data permissions; representative, boundary and failure cases; hard-stop assertions; a named approval owner; operating limits; searchable audit records; and an exercised pause and recovery path. If any consequential action lacks an external control or the test set does not represent the intended workflow, stop the release or remove that action from scope.
Should an AI agent pass every test?
Every hard-stop test should pass. Other criteria can use team-approved thresholds appropriate to the workflow, but they should be segmented by task and consequence. Do not let a strong aggregate result hide an authorization, cross-account, duplicate-action or false-success failure. A safe clarification or escalation can be the correct passing outcome.
Is human approval enough to make a risky action safe?
Not automatically. Approval must be informed, correctly authorized and positioned before the consequential action. The interface should show the proposed action and relevant evidence, while the application independently validates identity, permissions and parameters. A vague confirmation prompt or retrospective review does not replace prevention.
How often should the evaluation be rerun?
Rerun the relevant regression set after a material change to the model, prompt, policy, retrieval source, tool, schema, permission, orchestration or fallback. Also add cases from incidents and reviewed near misses. The scope of reruns should be defined in change policy rather than left to memory.
Can a conditional pass be used for a public launch?
Only when the restriction actually contains the unresolved risk and can be enforced. Examples include draft-only operation, mandatory approval or a narrowly limited workflow. Record the owner, monitoring signal and exit condition. If the missing evidence concerns authorization, irreversible side effects or recovery, “conditional” is not enough—stop the affected release.
What should teams evaluate first: the model or the workflow?
Start with the workflow. Define the outcome, permissions, controls and failure consequences, then compare model or prompt configurations against that contract. Model-level quality matters, but it cannot establish whether credentials are scoped correctly, writes are idempotent, handoffs work or operators can recover state.