Structured output
When code consumes the answer, prose is a liability. Ask for a structure, then check it.
Schemas
A JSON Schema states the fields, their types and constraints. Put it in the prompt or the API’s structured-output setting, then validate every response before using it: parse, check types, check ranges, and retry or fail cleanly when it doesn’t pass.
Constrained decoding
Many APIs can enforce a schema during generation by only allowing tokens that keep the output valid. That removes format errors entirely, but not wrong values: a perfectly formatted answer can still be false.
Pick a model output in the lab and see what the validator reports.
A worked example
The model returns {"name": "Lan", "age": "thirty", "email": "lan@x.vn"}. Name and email pass, but age must be an integer, so validation fails. The application can send the error back and ask for a corrected answer, or it can avoid the problem by constraining decoding so that only tokens allowed by the schema can be generated at each step.
{ "name": "Lan",
"age": "thirty",
"email": "lan@x.vn" }In practice
Most model APIs offer a JSON mode or schema-constrained output, and libraries can enforce a grammar during sampling for open models. Keep schemas small, mark required fields, and validate anyway: a valid structure can still hold wrong values.
Easy to confuse
- Valid JSON is not the same as valid against the schema; types, required fields and patterns are separate checks.
- Constrained decoding guarantees the format, not the content.
Why ask for JSON instead of prose when another program reads the answer?
Code can validate fields and types before using them; prose needs fragile parsing.
Constrained decoding guarantees…
It masks tokens that would break the format; it says nothing about whether the content is correct.
The model returns valid JSON with age = −4. What catches it?
Parsing only checks syntax; types and ranges need a schema validator, and business rules need code.
Read more in the books
- AI Engineering, Chip Huyen: Ch. 2, the section on structured outputs.
- AI Engineering, Chip Huyen: Ch. 5, “Prompt Engineering”: prompt engineering best practices.