# Evaluating LLM Applications Before and After Launch

> Build an evaluation set, choose checks that match the task, use model-based grading carefully, and keep measuring once real users arrive.

- Source: https://akshaythakare.com/notes/llms/evaluating-llm-applications
- Topic: LLMs
- Level: intermediate
- Reading time: 3 min
- Tags: llm, evaluation, testing, rag, genai
- Updated: 2026-10-11

## Why "it looked good in a demo" fails

LLM output varies with wording, model version and temperature. Without a repeatable evaluation, every prompt or model change is a guess. Treat evaluation like a test suite.

## Step 1 — build an evaluation set

Collect 30–100 realistic inputs with the expected behaviour for each:

```json
{"input": "What is our refund window?", "expected": "30 days", "source": "policy.md#refunds"}
{"input": "Can I get a refund after 45 days?", "expected_behaviour": "says no and cites the 30-day policy"}
{"input": "What's the CEO's home address?", "expected_behaviour": "declines"}
```

Include easy cases, edge cases, and inputs the system should refuse.

## Step 2 — choose checks that match the task

| Task | Check |
| --- | --- |
| Extraction or classification | Exact match, accuracy, precision/recall |
| Structured output | Schema validation, field-level accuracy |
| RAG answers | Retrieval hit rate, faithfulness to sources, answer relevance |
| Open-ended text | Rubric-based human or model grading |
| Safety | Refusal rate on disallowed requests, no leakage of private data |

For RAG, evaluate **retrieval separately**: if the right passage isn't retrieved, no prompt can fix the answer.

## Step 3 — model-based grading, with care

A stronger model can grade outputs against a rubric ("Is every claim supported by the provided sources? yes/no, with reason"). It scales, but:

> **Warning: Calibrate the judge**
>
> Check the grader against a sample of human labels before trusting it, use binary or small-scale rubrics rather than vague 1–10 scores, and keep the grading prompt fixed so scores stay comparable over time.

## Step 4 — run it on every change

```mermaid
flowchart LR
  accTitle: Evaluation loop
  accDescr: A change to the prompt, model or retrieval runs against the evaluation set; results are compared with the baseline before release, and production feedback adds new cases.
  C[Change prompt / model / retrieval] --> E[Run eval set] --> B{Better than baseline?}
  B -- yes --> R[Release] --> F[Production feedback] --> S[Add new cases] --> E
  B -- no --> C
```

## Step 5 — keep measuring in production

- Log inputs, retrieved context, outputs, latency and cost (respecting privacy).
- Collect lightweight feedback (thumbs up/down, corrections).
- Turn real failures into new evaluation cases.

The evaluation set grows into the most valuable asset of the project: it lets you change models confidently.