Make AI API Responses Reliable with JSON Schema, Validation and Safe Retries

Why parseable JSON is not enough

An AI integration can return valid JSON and still break a CRM, catalog importer or invoice workflow. A field may have the wrong meaning, a number may use the wrong unit, or the application may treat a suggested action as an authorized command. OpenAI documents Structured Outputs for constraining model output to a supplied JSON schema, and function calling for connecting models to application tools. These solve different parts of the problem. A schema improves structural reliability; it does not establish factual correctness, permission to act, or immunity from malicious input.

Prerequisites and contract design

Choose a supported model and API path from the current documentation, then define the business contract before writing a prompt. For a support-ticket classifier, an illustrative response might contain category, priority, summary, requires_human_review and evidence. Specify allowed categories, required fields, string lengths and a schema version. Keep the original customer message separate from system instructions. Do not let a customer message alter the permitted schema or authorization rules. Prepare a representative evaluation dataset including empty requests, multilingual input, conflicting evidence and adversarial instructions.

Step 1: define a narrow schema

Prefer explicit object properties and enums over free-form prose that downstream code must guess how to parse. For example, priority can be low, normal or urgent; requires_human_review is a boolean rather than a loosely worded recommendation. Follow the API's supported JSON Schema subset and strict-mode requirements in the official documentation; not every JSON Schema feature is necessarily supported by every endpoint or model. Add an explicit unknown category when the evidence is insufficient. Keep internal IDs, prices and permissions under application control instead of asking the model to invent them.

Step 2: request structured output, not a text promise

Use the documented structured output configuration for the chosen endpoint and supply the schema as a machine-readable contract. A prompt such as 'please respond in JSON' is weaker because it does not enforce the schema. Parse the API response through the SDK or documented response shape and handle refusal, incomplete generation, request errors and output limits as distinct outcomes. Never assume that every response contains a successful completed object. If the SDK supports parsing into a typed structure, use it, but still validate business invariants after parsing.

Step 3: validate business meaning independently

After schema validation, apply deterministic checks: does the referenced order exist, does the caller own it, is the currency allowed, is a refund amount within the original payment, and is the requested transition legal? Reject or route to review when these checks fail. For extracted data, retain source references or snippets internally so a reviewer can inspect why the field was assigned. Avoid presenting model-generated citations as verified external evidence without checking the underlying sources. Use human approval for irreversible financial, legal or customer-facing actions.

Step 4: distinguish function calling from a final answer

Function calling is appropriate when the model needs to request a tool operation such as retrieving order status. Structured final output is appropriate when the application needs a predictable result object. The model proposing a function call does not mean it has authority to execute it. Implement allowlisted tools, per-tenant authorization, parameter validation, audit logs and bounded execution time. Return tool results to the model only after redacting unnecessary secrets. If a tool call changes state, use an idempotency key and a policy gate. Treat all retrieved documents and user messages as untrusted data.

Step 5: build a retry and evaluation policy

Retry transient transport errors with bounded exponential backoff and jitter, respecting documented rate limits. Do not blindly retry a completed side-effecting operation, and do not repeat a request indefinitely when the model refuses or the content violates a business rule. Track schema validation failures, business-rule rejections, human-review rate, latency, cost and downstream correction rate separately. Run regression tests on fixed examples before changing models, schemas or prompts. Compare performance by business outcome, not only by how often a JSON parser succeeds.

Troubleshooting and a safe rollout

If the API rejects the schema, reduce unsupported constructs and check required-field and strict-mode rules. If output is structurally correct but operationally wrong, tighten business validation and improve examples rather than expanding the prompt without measurement. If the response is truncated, adjust output budget or simplify the task; do not guess missing fields. Start with read-only or draft-producing workflows, then graduate to gated actions. For Brightery CRM, ecommerce and AI automation projects, the useful architecture is schema-constrained generation plus deterministic validation, observability and explicit human ownership of consequential decisions.

Primary documentation and further reading

OpenAI: Structured model outputs; OpenAI: Function calling. Consult the official documentation before deploying changes to production.