Generative AI & LLM Apps
Topic 9 · Making it reliable · ⏱ about 3 h

Evaluating LLM Apps

Know whether your app works and whether a change helped: build a test set, choose checks from exact match to rubrics and LLM judges (and their biases), measure retrieval and grounding separately, and monitor quality, cost and speed in production.

Show:
In this topic
Beginner

Why evaluation is the core skill

LLM apps fail quietly. A prompt tweak that fixes one example can break ten others, a provider can update a model, and users will find cases you never imagined. Without measurements you are guessing. The teams that build reliable LLM apps are the ones with a good evaluation set and the habit of running it on every change, just like unit tests in normal software.

Beginner

Build a test set

  • Collect realistic inputs: 30 to 100 to start, covering common cases, edge cases (very long, empty, wrong language, rude), and things the app must refuse.
  • For each, write the expected behaviour: an exact answer, the key facts the answer must contain, or which source it must cite.
  • Keep growing it: every bug report from users becomes a new test case.
  • Keep some cases held out that you never look at while tuning prompts, for the same reason you keep a test set in ML.
Beginner

Ways to score an answer

CheckGood forWatch out for
Exact match / code checksclassification, extraction, JSON validity, "does it cite a source?"only works when there is one right answer
Contains key factsRAG answers: are the 3 facts that matter present?wording varies; keep the list short
Human rating with a rubrictone, helpfulness, safetyslow and costly; write clear criteria so raters agree
LLM as a judgescaling rubric checks to thousands of answersjudges prefer longer answers, the first option shown, and their own style; check them against human ratings

For an LLM judge, give it a clear rubric with examples, ask for a reason before the score, use a simple scale (pass/fail or 1 to 3), and swap the order when comparing two answers.

Beginner

Evaluate retrieval and generation separately

For RAG, measure the two halves on their own so you know which to fix:

  • Retrieval: for each test question, mark which chunks are relevant. Recall@k is the fraction of questions where a relevant chunk appears in the top k; MRR (mean reciprocal rank) averages 1/rank of the first relevant chunk, rewarding finding it near the top.
  • Generation: faithfulness (is every claim supported by the retrieved sources?) and answer relevance (does it actually answer the question?). Tools such as Ragas automate these with LLM judges.
Try it

Try it: recall@k and MRR

Each row is a test question and the rank at which the first relevant chunk was retrieved (or "not found"). Edit the ranks and change k.

Interactive: this part runs in your browser.

Beginner

In production

  • Log inputs, retrieved chunks, outputs, latency, token counts and cost for every request (and remove or protect personal data in those logs).
  • Collect user feedback (thumbs up/down) and review a sample of conversations every week.
  • Track cost per request and the slowest 5% of response times, not just averages.
  • Re-run your evaluation set when you change prompts, models or retrieval, and before switching to a new model version.
Check yourself

Quiz

1. You switch to a cheaper model. How do you decide whether it is good enough?

2. A known bias of LLM judges is that they…

3. Recall@3 is 0.6. What does that mean?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready