AI product development beyond a ChatGPT wrapper means engineering a dependable workflow around a model, not merely calling an API and displaying text. The work includes defining one job and its unacceptable errors, preparing and retrieving the right data, requiring a typed output contract, evaluating representative and adversarial inputs, restricting tools and permissions, designing for uncertainty and correction, controlling cost and latency, and assigning an owner to learn from real use. A model call is an ingredient; the product is the controlled system around it.
That distinction changes the release question. Instead of asking whether a prompt produces an impressive answer, ask whether the feature can show its evidence, defer when evidence is weak, protect data, remain useful when a provider is slow, and make changes reviewable. The framework below turns those questions into a release gate and a worked invoice-extraction example.
The 7C release gate for an AI product
A useful product-readiness review can be organised into seven gates: Context, Contract, Challenge, Contain, Communicate, Control, and Continue. The order matters. A better model cannot repair missing source material; a polished interface cannot compensate for an unbounded action; and a dashboard cannot create an owner who is responsible for the next change.
The gate is deliberately binary at the decision point: pass, hold, or explore. “Explore” is for a promising idea whose task, data, or risk is still unclear. “Hold” is not a judgement on the model. It means the surrounding product has not yet made a failure understandable and recoverable.
- Context — Define what the model may see, which sources are authoritative, how fresh they must be, and which records each user may access. For a knowledge feature, retrieval is part of the product. For an extraction feature, the original document, parsing quality, and provenance are the context.
- Contract — Turn free-form generation into a typed, validated result. Name required fields, allowed values, null behaviour, and what happens when the model cannot comply. A rejected result is safer than malformed text entering a database.
- Challenge — Build a representative evaluation set before treating a prompt or model change as an improvement. Include ordinary cases, boundary cases, missing information, ambiguous inputs, and hostile or instruction-like content where relevant.
- Contain — Treat model output as untrusted until it has passed the controls appropriate to the action. Separate reading from writing, grant the minimum tool permissions, and place deterministic checks or explicit confirmation in front of irreversible operations.
- Communicate — Give people a usable way to inspect, correct, defer, or reject the result. Show provenance when it exists, label drafts as drafts, and make uncertainty visible without pretending that a confidence value is proof.
- Control — Set a cost and latency envelope for each task. Limit context, choose a model appropriate to the work, cache only when freshness permits, and define a useful degraded state for timeouts or provider errors.
- Continue — Assign ownership for logs, feedback, evaluation updates, prompt and model changes, and rollback. An AI feature needs a learning loop because new inputs and changed dependencies can expose a failure that the original examples did not contain.
AI Product Readiness Matrix and Release Gate
Use this matrix in a design review. Write down the evidence rather than awarding points for confidence. Thresholds are guidance to be chosen for the task and its consequences; they are not measurements of this article or claims about an existing system.
| Gate | Pass question | Hold condition | Evidence to record |
|---|---|---|---|
| Context | Can a reviewer trace each material answer or field to permitted, relevant input? | Sources are stale, duplicated, inaccessible, or impossible to inspect. | Source map, access rules, freshness decision, representative inputs. |
| Contract | Can the result be parsed, validated, stored, and rejected without silently changing its meaning? | The product parses prose with guesses, accepts missing required fields, or has no null and error state. | Schema, validation rules, invalid-output examples, fallback state. |
| Challenge | Do representative, boundary, ambiguous, and adversarial cases have an expected behaviour and a repeatable review method? | The evaluation set contains only clean demonstrations or has no reviewer for ambiguous cases. | Versioned cases, rubric, field-level checks, human-review record. |
| Contain | Are data access and tool permissions limited, with a deterministic check before consequential action? | Untrusted content can issue instructions, or a model can write where it only needs to read. | Permission map, action boundary, confirmation rule, refusal and fallback paths. |
| Communicate | Can a person see what is proposed, why it was proposed, and how to correct or defer it? | Confidence is hidden, provenance is unavailable, or the only choices are accept and abandon. | Screen states, source display, edit path, escalation route. |
| Control | Can the team observe cost and latency for the task and keep a slow or expensive call from degrading the whole workflow? | Context is unbounded, no budget exists, or a timeout leaves a blank or misleading result. | Budget policy, timing and usage fields, timeout state, degradation plan. |
| Continue | Is one person or team accountable for feedback, evaluation changes, dependency changes, and rollback? | No one owns the next review, or logs cannot explain which version produced a result. | Owner, release identifier, retention policy, change log, rollback procedure. |
A practical release rule follows from the table: pass only when every gate has an answer that another person can inspect. If a gate is unresolved, narrow the task, add a control, or keep the feature in exploration. Do not compensate for a missing safety or data decision by changing models until the demo looks better.
What the gates change in the build
Context: make evidence a first-class input
Start with the job, not a model catalogue. Identify the information needed, authoritative sources, freshness, access, and the behaviour when information is absent. Retrieval is part of the product: separate authoritative material from duplicates, preserve useful metadata, and make selected passages inspectable. For document workflows, keep the original file or source reference with extracted fields.
Tenant and permission boundaries belong here. Relevant is not automatically authorised, so carry the access decision into retrieval. Without a reliable source, return “not enough information”, ask for clarification, or use a manual route.
Contract: validate meaning before it reaches product state
A typed result gives the application explicit states. An invoice-extraction contract might require a vendor, non-negative total, supported currency, labelled line items, and a confidence signal. The fields depend on the workflow; the principle is to validate structure and meaning before storing or displaying a result as a business fact.
Reject missing required fields, unexpected values, impossible combinations, and values needing human confirmation. Keep the raw response separate from the validated representation where audit or debugging requires it. Confidence can help route a result to draft or review; it does not prove correctness.
Challenge: evaluate the task people actually perform
An evaluation set is a versioned description of good behaviour. Build it from permitted, representative examples and edge cases, not only attractive demos. For knowledge answers, label supporting sources and unacceptable unsupported leaps; for extraction, label important fields and the correct treatment of missing or ambiguous data.
Use field-level comparison for structured values and rubrics for open answers; keep human review for ambiguous cases and grader calibration. Test prompt, retrieval, schema, provider, and model changes against the same cases. Avoid a universal pass percentage: consequences differ, so set a task-specific floor, identify critical cases that must not regress, and document who can approve an exception.
Contain: separate suggestions from actions
Keep the action boundary explicit. A model may draft an email, propose a classification, or prepare a refund request; deterministic software and an authorised person should decide whether a consequential action happens. Do not grant write access when the task only needs reading.
Treat retrieved pages, uploaded files, and user text as data. Use least-privilege tools, schema and policy checks, sandboxing where needed, and confirmation before irreversible actions. Design the timeout path too: use a bounded retry, acceptable cache, manual queue, honest uncertainty, or a clear refusal with the next step.
Communicate: make uncertainty actionable
Uncertainty is a product state. Show whether the system is presenting a draft, a source-supported answer, a clarification request, or a blocked action, and offer edit, defer, or reject paths. Make correction inexpensive and preserve it for evaluation where policy permits.
Show relevant, readable provenance when the workflow allows it; do not present internal reasoning as proof. A source excerpt and editable result are usually more useful for review. Design latency states as well: stream interactive drafting or queue document jobs, while providing timeout and recovery states.
Control: make economics visible before they become a surprise
Cost control starts with the task boundary: retrieve only needed context, avoid repeating deterministic work, and choose a model that fits the capability required. Set an envelope for context, tool calls, response time, retries, and cost; record usage and timing by release. A cheaper path that weakens a critical field or makes review harder is not an automatic improvement.
Choose degradation by the job: a partial draft, a paused retry, a cached result when freshness permits, or a manual route.
Continue: turn corrections into a maintenance loop
Log the release, model or provider, prompt or configuration version, retrieved context identifiers, validation and action outcomes, and user correction where policy permits. Keep sensitive retention bounded.
Review corrections and rejected outputs; promote useful cases into the evaluation set, record why behaviour changed, and use a reversible control where possible. The aim is visible, recoverable change, not a frozen feature.
Example: invoice extraction with a review gate
Consider a product that turns an invoice or receipt into structured fields for an accounting review screen. This is an illustrative workflow, not a report of a shipped system or a measured result. Inputs may include a clean invoice, a scanned receipt, a multi-currency document, or an email thread pasted into an upload field. The model assists with extraction; it cannot post an accounting entry by itself.
1. Context. Preserve the original document, source identifier, readability decision, and access rules. Define whether an email thread or only its attachment is in scope; document content remains data, not a command to the tool layer.
2. Contract. Define fields such as vendor, total, currency, lineItems, and confidence, with rules for missing values and supported currencies. Validate the object before it reaches the accounting screen: malformed amounts, unsupported currency, or missing total should create review, not a guess.
3. Challenge. Label cases for the clean invoice, scanned receipt, multi-currency document, long email thread, missing field, and ambiguous total. Specify the expected fields and whether the right behaviour is extraction, clarification, or review. Include instruction-like content and expect it to be treated as document content.
4. Contain. Give extraction no authority to issue payment, alter a ledger, or send a message. If an accounting action is added, put deterministic validation and authorised confirmation between proposed fields and that action. Keep original and edited values distinguishable.
5. Communicate. Show extracted values beside supporting source areas where permitted. Mark the result draft or “needs review” when a required field is missing, the source is ambiguous, validation fails, or a task-specific review rule is triggered. Let the reviewer edit, reject, or request clarification; confidence is not proof.
6. Control. Bound context and retries; avoid irrelevant pages and repeated parsing. Choose an interactive or queued workflow with visible processing, delay, and failure states. Record usage and timing so changes can be compared without inventing a target from another workload.
7. Continue. Store schema version, input class, validation outcome, reviewer correction, and model configuration according to the data policy. Add recurring corrections or document patterns to the evaluation set.
The release decision is concrete: consider controlled release only when the schema parses, the input is in scope, evaluation cases have approved behaviour, and consequential actions remain gated. An ambiguous document without a review route is a hold.
Implementation steps for a release-ready build
- Write the job card. State the user, the input, the output, the decision it supports, and the actions the system must never take. Name the cost of a wrong field and the cost of a delayed answer.
- Map evidence and access. List source systems, freshness expectations, document boundaries, permissions, and provenance. Mark any assumption that still needs a product or domain owner to confirm.
- Draft the contract and states. Define the schema, null behaviour, validation errors, review state, clarification state, timeout state, and manual fallback before polishing the prompt.
- Assemble the evaluation pack. Label normal, boundary, ambiguous, missing-data, and hostile cases. Include expected outputs and expected non-actions. Keep the cases versioned and permitted for the intended environment.
- Build the containment boundary. Separate model calls from business writes. Add least-privilege tools, input handling, output validation, deterministic checks, explicit confirmation, and a route for refusal or deferral.
- Design the review experience. Show source and status, make edits reversible, identify generated text as a draft when appropriate, and provide a low-friction way to record a correction.
- Set operating controls. Choose context limits, retry rules, model routing, caching rules, latency handling, and an initial cost envelope. Add telemetry for release, usage, timing, validation, and fallback decisions.
- Assign the continuation owner. Decide who reviews feedback, approves changes, updates the evaluation pack, monitors failures, and can pause or roll back the feature. Put that responsibility in the delivery plan, not in an implied future task.
For a short release meeting, ask each gate owner to answer three questions: what evidence exists, what happens when the evidence is weak, and who can change the behaviour. A blank answer is a useful finding. It identifies the next piece of product work more clearly than another round of prompt polishing.
Limitations and trade-offs
This framework is a decision aid, not a benchmark or a substitute for domain review. It does not supply universal quality thresholds because the acceptable error depends on the task. A draft headline, a customer-support answer, and a financial field need different release rules.
Evaluation sets are incomplete by design. They can miss new input patterns, drift as the underlying data changes, or encode a reviewer disagreement as if it were a single truth. Human review adds capacity and consistency work. Provenance makes inspection easier but does not make an extracted value correct. A visible uncertainty state can protect a workflow while also creating more review work; that trade-off should be acknowledged rather than hidden.
Retrieval is not automatically the right architecture. If the task has no changing knowledge source, a simpler deterministic or conventional software path may be clearer. Conversely, retrieval cannot repair an inaccessible, contradictory, or poorly maintained source. Model routing and caching can lower usage but add branches, invalidation rules, and new cases to evaluate. A smaller system with fewer moving parts may be preferable when the task is narrow.
Finally, this gate does not replace security assessment, privacy review, accessibility work, legal advice, or sector-specific controls. Those reviews should be added when the product context requires them. If the team cannot explain the data boundary, the action boundary, or the recovery path, narrowing the feature is often more responsible than adding more model capability.
Frequently asked questions
What is the difference between a ChatGPT wrapper and an AI product?
A wrapper commonly sends an input to a model and displays the response. An AI product defines a job, supplies permitted context, validates the result, measures task-specific behaviour, handles uncertainty and failure, controls actions and operating cost, and learns from correction. The model may be important, but the surrounding decisions determine whether the feature can be used responsibly.
Does every AI product need retrieval-augmented generation?
No. Retrieval is useful when the task depends on changing or private source material that the model must consult. It is unnecessary for some transformations, classifications, and drafting tasks, and it can add complexity when the source is not authoritative or well maintained. Choose it because the job needs traceable context, not because it is a fashionable component.
What should an AI evaluation set contain?
It should contain permitted examples that represent normal use, boundary conditions, missing or ambiguous input, and the failure modes that matter to the workflow. Record expected fields or answer properties, source support where relevant, and expected non-actions. Combine automated checks with human review for cases where a rubric cannot capture the product's judgement.
How should a product handle uncertain output?
Give uncertainty a defined next step: show a draft for editing, ask a clarifying question, expose the supporting source, route the item to review, or decline to act. The choice depends on the consequence and freshness of the task. Do not use a model-generated confidence value as the only safety control.
How can a team manage AI cost and latency?
Bound the context and number of tool calls, choose a model appropriate to each subtask, cache only results whose freshness permits it, and set retry and timeout rules. Measure usage and timing by task and release. Then choose a degraded state that remains honest: a queued job, a partial draft, a cached answer with its age, or a manual path.
When should a team use a conventional software path instead?
Use a deterministic path when the rules are stable, the inputs are structured, and correctness must be explained through explicit logic. An AI component can still help with an upstream classification or drafting step, but it should not be added where it makes the decision harder to validate without adding meaningful value.
CodeAustral's public applied-AI service description covers LLM feature integration, retrieval-augmented generation over a team's own content, tool-calling workflows, evaluation, cost and latency controls, guardrails, and fallbacks. The CodeAustral applied AI service track is the relevant place to review that delivery scope. A useful brief should include the user job, inputs, prohibited actions, representative examples, fallback state, and the person who will own iteration.