# RAG Pipeline Fundamentals

> Ingestion, chunking, embeddings, retrieval and generation — the parts of a retrieval-augmented generation system and where each one fails.

- Source: https://akshaythakare.com/notes/rag/rag-pipeline-fundamentals
- Topic: RAG
- Level: beginner
- Reading time: 3 min
- Tags: rag, llm, embeddings, vector-search
- Updated: 2026-10-11

## Why RAG

An LLM only knows what was in its training data. **Retrieval-augmented generation** fetches relevant passages from *your* data at question time and puts them in the prompt, so answers are grounded, current and citable — without retraining the model.

## The two pipelines

```mermaid
flowchart LR
  subgraph Indexing[Indexing — offline]
    D[Documents] --> P[Parse + clean] --> C[Chunk] --> E[Embed] --> V[(Vector index<br/>+ metadata)]
  end
  subgraph Query[Query — online]
    Q[User question] --> QE[Embed query] --> R[Retrieve top-k] --> RR[Re-rank] --> G[LLM generates<br/>with citations]
    V --> R
  end
```

Follow one question through the online path:

**RAG query path:** Question (user input) → Embed (query vector) → Retrieve (top-k chunks) → Re-rank (cross-encoder) → Generate (answer + citations)

## Each stage and its failure mode

| Stage | Key decision | Typical failure |
| --- | --- | --- |
| Parsing | Keep structure (headings, tables, code) | Tables flattened into nonsense |
| Chunking | Size, overlap, boundaries | Answer split across chunks |
| Embedding | Model choice, normalisation | Domain terms embedded poorly |
| Retrieval | Dense, keyword (BM25) or **hybrid** | Exact identifiers missed by dense-only search |
| Re-ranking | Cross-encoder on top candidates | Relevant chunk ranked 9th, cut at k=5 |
| Generation | Prompt with sources, instruction to cite and to say "I don't know" | Confident answer not supported by context |

## A minimal implementation

```python
chunks = split_markdown(doc, max_tokens=400, overlap=60)
vectors = embed([c.text for c in chunks])
index.upsert(ids=[c.id for c in chunks], vectors=vectors,
             metadata=[{"source": c.source, "heading": c.heading} for c in chunks])

hits = index.query(embed([question])[0], top_k=8)
context = "\n\n".join(f"[{i}] {h.metadata['heading']}\n{h.text}" for i, h in enumerate(hits))
answer = llm(f"Answer using only the sources below. Cite as [n]. "
             f"If the answer is not in the sources, say so.\n\n{context}\n\nQ: {question}")
```

## Measure it

Build a small evaluation set of real questions with known answers. Track **retrieval recall** (is the right chunk in the top k?) separately from **answer faithfulness** (is every claim supported by the retrieved context?). Most RAG quality problems are retrieval problems.