A destination-focused guide to model output validation. Official structured-output behavior, JSON Schema and OWASP guidance support separate checks for structure, business meaning, caller access and safe rendering or execution, with a consumer-boundary comparison and sink matrix.
At a glance
Key findings
- Valid JSON and schema adherence do not establish business truth, caller authority or safe interpretation at a downstream sink.
- Distinguish complete structured results from refusals, truncation and other non-success outcomes before consequential consumption.[3]
- Keep identity and permissions in trusted application context; model-supplied identifiers remain proposals to validate.
- Use destination-specific controls for text, rich HTML, databases, outbound requests and operating-system actions.
- Test schema-valid but unauthorized values as deliberately as malformed responses.
A valid object can still request the wrong thing
A model response can be valid JSON and still be unacceptable to the application that consumes it. A string can name another customer's record. A URL can point to a destination the service must not contact. A summary can contain markup that an unsafe renderer interprets as executable content. Parsing establishes that the response has a structure the parser recognizes. It does not establish business truth, caller authority, or safety at the destination where the values will be used.
The consequential boundary is the moment generated content becomes application action or interpreted output. OWASP's Improper Output Handling category identifies the risks of sending model results into downstream components without appropriate controls.[1] The application should decide what values it accepts and what operations those values may influence. A prompt asking the model to be careful can improve behavior, but it does not replace that decision. The consumer must remain safe when the model returns an unexpected or misleading result.
Consider a case-summary feature. The application already knows which case the employee may view. The model's role is to propose summary text and references to evidence supplied for that case. It does not need authority to select a different case, grant access, or choose an arbitrary rendering mode. A small consumer contract can preserve that distinction. If the model returns a plausible but unauthorized evidence identifier, the application should reject or remove the unsupported proposal according to an explicit policy rather than trust its apparent confidence.
This guide separates four checks: completion and structure, business meaning, caller authority, and destination-specific handling. They can be implemented in different components, but their responsibilities should remain visible. A schema validator cannot decide whether an identifier belongs to the current tenant. An authorization check cannot make an HTML string safe to render. An output encoder cannot establish that a summary accurately reflects the source. Treating each check as a separate boundary makes failures easier to test and harder to hide behind a single accepted flag.
Use the schema guarantee precisely
Use structured generation for the guarantee it provides. OpenAI's Structured Outputs documentation describes schema adherence for supported configurations, while also documenting refusals, incomplete responses, and the possibility of mistakes within structured content.[3] The application must distinguish a completed structured result from those other outcomes. A refusal is not a successful object with missing fields, and a truncated response should not be repaired into an apparently complete action without an explicit, bounded policy for doing so.
General JSON Schema can express required properties, object structure, and whether additional properties are allowed.[4] That helps prevent a consumer from accidentally accepting fields it was never designed to interpret. Define a small vocabulary and reject unexpected structure. However, supported schema features vary between model providers and configurations. An application validation schema is not automatically a valid provider request schema. Keep the two roles clear and verify any translation between them rather than assuming that a shared JSON document guarantees identical enforcement.
Schema-valid values can still be wrong. An evidence identifier may satisfy a string pattern while referring to a document outside the permitted set. A date may be syntactically valid but inconsistent with the source. A summary may meet the length limit while omitting a critical qualification. OWASP's input validation guidance distinguishes syntactic checks from semantic checks tied to the business context.[5] Apply that distinction after generation, where the application has the authoritative state needed to evaluate the proposed values.
Do not make completion detection depend on finding a closing brace in a stream. Use the provider's documented terminal state and the complete accepted result before allowing consequential consumption. Partial text may be displayed in a constrained preview if the product supports it, but it should not silently trigger the same operations as a finalized object. Record incomplete, refused, invalid, and accepted outcomes separately. That separation supports both safe behavior and useful investigation when a provider response does not match the normal path.
Put the boundary in the consuming application
The before-and-after comparison below shows where the application takes responsibility. In direct consumption, the response is treated as an instruction and passed toward an interpreter. In the proposed boundary, the response remains untrusted data until completion, structure, meaning, and authority checks succeed. Accepted values then enter a fixed interface appropriate to the destination. Failure leads to rejection or an explicitly supported fallback. This is an architectural comparison, not a measurement of attack reduction.
Place the boundary close enough to the consumer that later transformations cannot bypass it. Validating a response at the gateway is insufficient if another component later concatenates it into HTML, SQL, or a shell command. Document which component owns the final interpretation and which properties it requires. Where several consumers use the same model output, each may need a different destination control. A value safe for a plain-text preview may be unsafe when reused in a link, attribute, query, or executable expression.
Make accepted data a distinct application state. The internal object handed to a consumer should contain only the fields and normalized values that the application approved. Avoid carrying an unrestricted raw response alongside those values into a component that might accidentally use it. Retain the original under an appropriate evidence policy if needed, but separate that purpose from execution. This reduces the chance that a later feature bypasses validation by reading a convenient original field.
OWASP AISVS 1.0 separates schema rejection, output bounds, and outbound request controls in its model behavior chapter.[2] Those requirements support treating the boundary as several concrete properties rather than one generic sanitizer. Record which component enforces each property and how its failure is observable. If a system can reject malformed JSON but still lets a valid field select an arbitrary destination, the structural control is working while the destination boundary remains incomplete. The test report should describe both facts accurately.
The consumer boundary before and after validation
Conceptual comparison. Structured generation helps with format but does not replace application authorization or destination-specific controls.
Source. Cloud Security Desk conceptual synthesis, 2026-08-28, informed by OWASP and OpenAI. [1][3][5]
Method. Conceptual design, not measured data. Unit: process steps or control relationships; no numeric scale. Scope: The validation and authorization boundaries between model content and application actions. Limits: Conceptual comparison. Structured generation helps with format but does not replace application authorization or destination-specific controls.
Accessible table and figure data
| Stage | Direct consumption | Proposed consumer boundary |
|---|---|---|
| Model response | Response is treated as an instruction | Response is retained as untrusted data |
| Structure | Best-effort parsing | Complete result and schema checks |
| Meaning and authority | Model fields decide the operation | Application checks business rules and caller access |
| Destination | Generated content reaches an interpreter | Fixed safe interface receives accepted values |
| Failure | Retry or continue implicitly | Reject or use an explicit safe fallback |
| Stage | Direct consumption | Proposed consumer boundary |
|---|---|---|
| Model response | Response is treated as an instruction | Response is retained as untrusted data |
| Structure | Best-effort parsing | Complete result and schema checks |
| Meaning and authority | Model fields decide the operation | Application checks business rules and caller access |
| Destination | Generated content reaches an interpreter | Fixed safe interface receives accepted values |
| Failure | Retry or continue implicitly | Reject or use an explicit safe fallback |
Validate business meaning under the caller identity
Resolve caller identity and resource scope from trusted application context. The model may propose an identifier, but it should not determine which account owns the request or which permissions apply. In the case-summary example, build the authorized evidence set from the case the employee is allowed to view. Compare each proposed evidence identifier against that set. A well-formed identifier that is absent from the set remains unaccepted, even if the associated explanation sounds plausible.
Check business rules using authoritative state. A proposed status transition may be valid only from particular current states. A suggested value may need to agree with a source record or fall within a permitted domain. These checks are different from determining whether the output is fluent or helpful. OWASP's validation guidance supports separating syntax from semantics.[5] The application should own the rule and its data source, so an instruction embedded in the generated response cannot redefine what counts as acceptable.
Avoid accepting the model's explanation as proof of its own authorization. A field saying that approval was obtained is another generated claim unless it refers to an independently verified approval record. Similarly, a confidence value does not establish resource ownership. If a workflow requires human review, bind the approval to the exact proposed operation and current state. A generic approval of an earlier draft should not authorize a later changed payload that the reviewer never saw.
This output boundary complements tool authorization rather than replacing it. Tool permissions govern what a caller may invoke; consumer validation governs what returned or generated values may cause inside the application. Both are needed when a system has consequential capabilities. Keep the interfaces narrow enough that the model proposes data within a known operation instead of supplying a complete command language. The smaller vocabulary gives the application a tractable set of meanings to validate and makes unauthorized expansion easier to reject.
Choose the defense for the destination
The destination determines the primary injection defense. For a plain-text interface, use text rendering that does not interpret markup. If the product intentionally accepts rich HTML, use an appropriate maintained sanitizer and safe rendering design for the permitted tags, attributes, and links. OWASP's XSS guidance distinguishes output contexts and safe sinks.[6] Encoding for one context does not make content universally safe in JavaScript, CSS, HTML attributes, or URLs. Avoid moving an accepted value between those contexts without reconsidering the control.
For database operations, keep the statement structure under application control and bind accepted values as parameters. OWASP recommends prepared statements and appropriate allowlisting.[7] Parameters do not turn arbitrary generated SQL into a safe query, nor do they substitute for identifiers such as table names. A model can propose a limited query intent that the application maps to a fixed operation. The application must still check caller access and the meaning of the requested operation before using the accepted values.
For outbound requests, a syntactically valid URL is only an input to the destination policy. The service must consider allowed destinations, DNS resolution, redirects, and network boundaries according to its design. OWASP's SSRF guidance addresses these layers.[8] A model-generated link should not gain authority to reach internal services because it appeared inside valid JSON. Keep the controlled HTTP client and its policy at the final fetch boundary, where subsequent redirects or resolution changes cannot silently escape the intended restriction.
For operating-system work, prefer a dedicated library interface over direct shell execution. OWASP's command injection guidance also warns that argument handling and validation matter when an external command is unavoidable.[9] Escaping a string is not permission to execute an arbitrary generated command. Use a fixed operation, an allowlisted argument vocabulary, and appropriate process authority. The matrix below maps these destinations to distinct controls; it deliberately does not offer one sanitizer or wrapper that claims to make every downstream interpreter safe.
Different sinks require different controls
Conceptual control mapping informed by OWASP. The article does not provide a universal sanitizer, arbitrary SQL executor or shell wrapper.
Source. Cloud Security Desk conceptual synthesis, 2026-08-28, informed by OWASP. [6][7][8][9]
Method. Conceptual design, not measured data. Unit: process steps or control relationships; no numeric scale. Scope: Output destinations and the corresponding validation and authorization controls. Limits: Conceptual control mapping informed by OWASP. The article does not provide a universal sanitizer, arbitrary SQL executor or shell wrapper.
Accessible table and figure data
| Destination | Preferred interface | Additional boundary |
|---|---|---|
| Plain text UI | Text rendering or safe text node | Length and disclosure policy |
| Allowed rich HTML | Maintained sanitizer and context-aware rendering | Permitted tags, attributes and links |
| Database operation | Fixed parameterized statement | Allowlisted query intent and caller access |
| Outbound fetch | Controlled HTTP client and destination policy | DNS, redirect and network restrictions |
| Operating-system action | Dedicated library or fixed command interface | Allowlisted arguments and least privilege |
| Destination | Preferred interface | Additional boundary |
|---|---|---|
| Plain text UI | Text rendering or safe text node | Length and disclosure policy |
| Allowed rich HTML | Maintained sanitizer and context-aware rendering | Permitted tags, attributes and links |
| Database operation | Fixed parameterized statement | Allowlisted query intent and caller access |
| Outbound fetch | Controlled HTTP client and destination policy | DNS, redirect and network restrictions |
| Operating-system action | Dedicated library or fixed command interface | Allowlisted arguments and least privilege |
Make the consumer contract small and explicit
The example schema below defines a small application-side object for a case summary. It allows summary text and evidence identifiers, requires both fields, rejects additional properties, and bounds the strings and array. The limits of 2,000 summary characters and eight evidence identifiers are illustrative local design choices, not provider defaults or measured recommendations. The schema is general JSON Schema Draft 2020-12. It is not an OpenAI API request and should not be copied into a provider configuration without checking that provider's supported subset.[4][3]
The schema intentionally contains no tenant identifier, access level, destination URL, or executable operation. The application already knows the case and caller, so allowing the model to supply those values would add authority without helping the summarization task. Evidence identifiers remain proposals. After structural validation, the consumer must check them against the authorized evidence set and decide whether the summary's claims are adequately supported. The schema can reject an empty string; it cannot establish that a nonempty sentence is true.
Keep the contract versioned with the consumer. If a new field is added, decide what it means, which component validates it, and whether older consumers should reject or ignore it. Silent interpretation of an unknown field can create a new capability without a corresponding review. Test the actual serializer and validator behavior, including type coercion and duplicate-key handling where relevant. A schema file in the repository is not evidence that every request path uses it consistently.
The accompanying schema was checked locally with a Draft 2020-12 validator against valid and rejected specimens. Those checks establish only the example's structural behavior. A schema-valid specimen with an unauthorized evidence identifier was also checked to demonstrate that semantic rejection remains necessary. No model call or live application execution was performed. This distinction is useful in a release record: the schema test supports structure and bounds, while separate tests must support authority, source fidelity, and safe handling at the actual destination.
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"summary": {
"type": "string",
"minLength": 1,
"maxLength": 2000
},
"evidenceIds": {
"type": "array",
"maxItems": 8,
"uniqueItems": true,
"items": {
"type": "string",
"minLength": 1,
"maxLength": 80
}
}
},
"required": ["summary", "evidenceIds"],
"additionalProperties": false
}Test rejection and evidence as deliberately as success
Build rejection cases around each boundary. Supply a missing required field, an unknown property, an oversized string, a wrong type, and an incomplete response. Then supply values that satisfy the schema but violate the business contract: an evidence identifier outside the authorized set or a proposed transition that is not permitted from the current state. These cases should be tested independently of the model's likelihood of producing them. The consumer needs to behave safely whenever such data reaches it.
Test the destination after validation. For a text interface, a harmless markup specimen should remain inert text. For a database consumer, confirm that accepted values enter parameters in a fixed statement. For a fetch path, verify that destination policy is enforced at the actual network boundary. For an external process, confirm that the operation and arguments remain within the approved interface. The relevant OWASP guidance differs by sink, so the expected assertion should name the destination rather than merely say sanitized.[6][7][8][9]
Examine failure paths that attempt to improve availability. An automatic repair step can accidentally restore fields that the original validation rejected. A fallback model can return a different contract. A retry can repeat a consequential operation after the caller timed out. Decide which failures permit another generation attempt and keep that process bounded. Revalidation must occur on the final result, and no rejected or partial result should become actionable because an exception handler tried to keep the workflow moving.
Preserve concise evidence of the rejection and the relevant configuration. Record the contract version, failure category, request identifier, and whether any downstream action occurred. Retain sensitive original output only under the appropriate evidence policy. These records support both release testing and incident investigation without requiring unrestricted prompt logging. A useful test report distinguishes a model refusal, a schema rejection, a business-rule rejection, and a destination-control failure, because each describes a different property of the system.
Keep model suggestions inside an application contract
The consumer contract should describe what the application accepts, what authority remains outside the model, and which interface receives approved values. Start with the smallest useful output vocabulary. For the summary feature, text and references may be enough. Adding arbitrary URLs, SQL, or operation names changes the trust problem and should trigger a new review. The model's ability to produce a richer object is not itself a reason to grant that object more influence over the application.
Keep the layered checks visible in code and evidence. Completion handling determines whether there is a usable result. Structural validation checks the object contract. Business rules and caller context determine whether its proposed values are acceptable. Destination controls determine how those values can be rendered or used. A failure at any layer should lead to a defined state, with no hidden path that treats the rejected result as an instruction. This makes the boundary understandable to reviewers who did not write the original prompt.
Revisit the boundary when the destination changes, even if the model does not. A summary that previously appeared as plain text may later be exported to HTML, used in a notification, or inserted into an automated workflow. The earlier acceptance evidence may no longer cover the new interpretation. Likewise, a provider change can alter completion or refusal handling. Preserve the contract and source review dates so the team can identify which assumptions need to be checked again before expanding the capability.
Structured generation remains valuable because it can make the response easier to consume reliably. Its value increases when the application uses that structure as the beginning of a clear validation process. The final decision still belongs to the component that understands the caller, business state, and destination. Keeping that decision outside generated text allows the model to offer useful suggestions while preventing a syntactically correct response from silently acquiring authority the application never intended to grant.
Method and provenance
Reviewed OWASP output handling and AISVS 1.0 requirements, current official OpenAI Structured Outputs documentation, the JSON Schema object reference and sink-specific OWASP cheat sheets on August 28, 2026. The consumer comparison and control matrix are original conceptual designs. The example application schema was validated locally with Draft 2020-12 positive and negative specimens, including a schema-valid identifier that fails an independent authorized-set check.
No OpenAI API request, production application test or injection experiment was performed. The schema is a general application validation example, not a provider request format or complete authorization implementation. Illustrative bounds do not represent measured capacity or provider defaults. Schema validation does not establish factual accuracy, and every destination needs its own reviewed control.
AI assistance. AI assisted source research, drafting, schema-test preparation and original conceptual visual design. Product behavior and security guidance were checked against the cited official documentation and the local schema checks.
Published under the Cloud Security Desk organizational byline. Read the practitioner guide policy.
References
- LLM05 2025 Improper Output Handling OWASP. Accessed .
- Model Behavior, Output Control and Safety Assurance OWASP. Accessed .
- Structured model outputs OpenAI. Accessed .
- JSON Schema object reference JSON Schema project. Accessed .
- Input Validation Cheat Sheet OWASP. Accessed .
- Cross Site Scripting Prevention Cheat Sheet OWASP. Accessed .
- SQL Injection Prevention Cheat Sheet OWASP. Accessed .
- Server Side Request Forgery Prevention Cheat Sheet OWASP. Accessed .
- OS Command Injection Defense Cheat Sheet OWASP. Accessed .