# Observability for LLM Applications

> What to log for every LLM and agent call — prompts, tokens, latency, tool calls, outcomes — and how to turn traces into fixes.

- Source: https://akshaythakare.com/notes/ai-tools/observability-for-llm-apps
- Topic: AI Tools
- Level: intermediate
- Reading time: 2 min
- Tags: ai-tools, observability, llm, production
- Updated: 2026-10-11

## Why normal logs are not enough

An LLM app can return HTTP 200 with a wrong, slow or expensive answer. You need to see what the model saw, what it did and what it cost.

## What to record per request

| Field | Why |
| --- | --- |
| Trace id, user / session, feature | Group steps and find a user's problem |
| Model and parameters | Reproduce behaviour |
| Prompt (or template + variables) and response | Debug wrong answers |
| Retrieved document ids | Tell retrieval failures from generation failures |
| Tool calls: name, arguments, result, duration | Debug agents |
| Input / output tokens, latency, cost | Budget and performance |
| Outcome: user feedback, eval score, error | Measure quality |

## Traces for agents

An agent run is a tree: plan → tool call → model call → tool call. A trace view shows where time and tokens went and which step went wrong.

## Turn traces into improvements

1. Sample failed or low-rated traces each week.
2. Label the failure type: retrieval miss, wrong tool, bad instruction, hallucination.
3. Add the case to the evaluation set.
4. Fix, re-run evals, and ship only if nothing else regresses.

> **Warning: Privacy**
>
> Prompts and responses can contain personal data. Redact or hash sensitive fields, control who can view traces, and set retention limits.