LLM Lab

Structured output

When code consumes the answer, prose is a liability. Ask for a structure, then check it.

Where this sits in an LLM request
Instructions + context→Retrieved documents→Model generates→Output parsed and checked

Schemas

A JSON Schema states the fields, their types and constraints. Put it in the prompt or the API’s structured-output setting, then validate every response before using it: parse, check types, check ranges, and retry or fail cleanly when it doesn’t pass.

Constrained decoding

Many APIs can enforce a schema during generation by only allowing tokens that keep the output valid. That removes format errors entirely, but not wrong values: a perfectly formatted answer can still be false.

Pick a model output in the lab and see what the validator reports.

A worked example

The model returns {"name": "Lan", "age": "thirty", "email": "lan@x.vn"}. Name and email pass, but age must be an integer, so validation fails. The application can send the error back and ask for a corrected answer, or it can avoid the problem by constraining decoding so that only tokens allowed by the schema can be generated at each step.

{ "name": "Lan",
  "age": "thirty",
  "email": "lan@x.vn" }
name✓ string
age✗ must be an integer
email✓ matches pattern
Validating a model’s output against the schema. One field fails, so the app can retry with the error message, or use constrained decoding so that only valid tokens can be produced in the first place.

In practice

Most model APIs offer a JSON mode or schema-constrained output, and libraries can enforce a grammar during sampling for open models. Keep schemas small, mark required fields, and validate anyway: a valid structure can still hold wrong values.

Easy to confuse

Check yourself · 1 of 3

Why ask for JSON instead of prose when another program reads the answer?

Check yourself · 2 of 3

Constrained decoding guarantees…

Check yourself · 3 of 3

The model returns valid JSON with age = −4. What catches it?

Read more in the books

  • AI Engineering, Chip Huyen: Ch. 2, the section on structured outputs.
  • AI Engineering, Chip Huyen: Ch. 5, “Prompt Engineering”: prompt engineering best practices.
All books
Labschema validator
How to read this
  • The schema at the top says what a valid answer looks like. The tabs switch between example model outputs.
  • The output is checked against the schema; each error names the field and the rule it breaks.
{ "type": "object",
  "required": ["name", "age", "email"],
  "properties": {
    "name":  { "type": "string" },
    "age":   { "type": "integer", "minimum": 0 },
    "email": { "type": "string", "pattern": "^.+@.+$" } } }