Generative AI & LLM Apps
Topic 5 · Building LLM apps · ⏱ about 3 h

Structured Output & Tool Use

Get JSON your code can trust with schemas and validation, then let the model call your own functions: describe tools, run the calls it asks for, send back results, and keep the loop safe.

Show:
In this topic
Beginner

Why structured output?

Apps need data, not paragraphs. If you ask "What is the sentiment of this review?" you might get "Positive!", "The sentiment is positive." or a whole essay. Instead, ask for JSON that follows a schema:

{"sentiment": "positive", "score": 0.82, "topics": ["staff", "waiting time"]}

Three levels of reliability:

  1. Ask in the prompt and show an example. Works most of the time.
  2. Use the provider's structured-output or JSON mode with a JSON Schema. The model is constrained to produce valid JSON matching the schema.
  3. Always validate in your code anyway (for example with the pydantic library), and retry or fall back when validation fails. Valid JSON can still contain wrong values.
Beginner

Code: validate model output with pydantic

Pydantic turns a Python class into a schema and checks data against it. The same class can generate the JSON Schema you send to the model.

Python
import json
from typing import Literal
from pydantic import BaseModel, Field, ValidationError

class Review(BaseModel):
    sentiment: Literal["positive", "neutral", "negative"]
    score: float = Field(ge=0, le=1)       # must be between 0 and 1
    topics: list[str]

# the schema you could send to a structured-output API
print(json.dumps(Review.model_json_schema(), indent=1)[:300], "...")

# pretend these came back from a model
good = '{"sentiment": "positive", "score": 0.82, "topics": ["staff", "waiting time"]}'
bad = '{"sentiment": "very happy", "score": 1.7, "topics": "staff"}'

for raw in [good, bad]:
    try:
        r = Review.model_validate_json(raw)
        print("OK:", r.sentiment, r.score, r.topics)
    except ValidationError as e:
        print("Rejected:", len(e.errors()), "problems ->", [err["loc"][0] for err in e.errors()])
Open in playground ↗
Beginner

Tool use (function calling)

An LLM cannot check today's appointments or do exact maths by itself, but your code can. With tool use you describe functions to the model (name, what it does, and a JSON Schema for its arguments). The loop is:

  1. Your app sends the user's message plus the tool descriptions.
  2. The model either answers, or replies with a tool call: "call find_slots with {"date": "2026-11-20"}".
  3. Your code runs the function (the model never runs anything itself) and sends the result back as a new message.
  4. The model uses the result to answer, or asks for another tool. Repeat until it answers.

The model only ever asks; your code decides whether to run the call. That is your safety point.

Try it

Try it: one tool-use conversation, step by step

Step through the messages that pass between the app and the model when a patient asks to book an appointment.

Interactive: this part runs in your browser.

Beginner

Designing good tools

  • Clear names and descriptions: the description is a prompt. Say what the tool does, when to use it, and what each argument means, with units and formats ("date as YYYY-MM-DD").
  • Few, focused tools beat many overlapping ones.
  • Return useful errors ("no slots on that date; nearest free day is 2026-11-21") so the model can recover.
  • Validate arguments before running anything, exactly as you would for a web form.
  • Ask a human to confirm anything that changes the world: sending emails, payments, deleting data, booking. Reading is low risk; acting is not.
  • Limit the loop: a maximum number of tool calls per request, so a confused model cannot loop forever or spend without limit.

MCP (the Model Context Protocol) is an open standard for plugging tools into AI apps: a tool provider writes an MCP server once, exposing tools (actions), resources (data) and prompts (templates), and any MCP-aware app can use it. It is how assistants such as Claude connect to calendars, file systems and databases.

Check yourself

Quiz

1. The API guarantees valid JSON. Do you still need to validate in your code?

2. During tool use, who actually runs the function?

3. Which tool call should require a human to confirm first?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready