RAG Pipeline Fundamentals
Ingestion, chunking, embeddings, retrieval and generation — the parts of a retrieval-augmented generation system and where each one fails.
RAGBeginner3 min readUpdated 11 Oct 2026
Why RAG
An LLM only knows what was in its training data. Retrieval-augmented generation fetches relevant passages from your data at question time and puts them in the prompt, so answers are grounded, current and citable — without retraining the model.
The two pipelines
FlowchartSwipe to explore →Technical diagram
Rendering flowchart
Follow one question through the online path:
RAG query path
Questionuser input
Embedquery vector
Retrievetop-k chunks
Re-rankcross-encoder
Generateanswer + citations
Each stage and its failure mode
| Stage | Key decision | Typical failure |
|---|---|---|
| Parsing | Keep structure (headings, tables, code) | Tables flattened into nonsense |
| Chunking | Size, overlap, boundaries | Answer split across chunks |
| Embedding | Model choice, normalisation | Domain terms embedded poorly |
| Retrieval | Dense, keyword (BM25) or hybrid | Exact identifiers missed by dense-only search |
| Re-ranking | Cross-encoder on top candidates | Relevant chunk ranked 9th, cut at k=5 |
| Generation | Prompt with sources, instruction to cite and to say "I don't know" | Confident answer not supported by context |
A minimal implementation
chunks = split_markdown(doc, max_tokens=400, overlap=60)
vectors = embed([c.text for c in chunks])
index.upsert(ids=[c.id for c in chunks], vectors=vectors,
metadata=[{"source": c.source, "heading": c.heading} for c in chunks])
hits = index.query(embed([question])[0], top_k=8)
context = "\n\n".join(f"[{i}] {h.metadata['heading']}\n{h.text}" for i, h in enumerate(hits))
answer = llm(f"Answer using only the sources below. Cite as [n]. "
f"If the answer is not in the sources, say so.\n\n{context}\n\nQ: {question}")Measure it
Build a small evaluation set of real questions with known answers. Track retrieval recall (is the right chunk in the top k?) separately from answer faithfulness (is every claim supported by the retrieved context?). Most RAG quality problems are retrieval problems.
Finished this note?