# Embeddings and Vector Search, Explained for Engineers

> What an embedding is, how similarity is measured, why vector databases use approximate search, and when to add keyword search alongside it.

- Source: https://akshaythakare.com/notes/rag/embeddings-and-vector-search
- Topic: RAG
- Level: beginner
- Reading time: 3 min
- Tags: rag, embeddings, vector-search, llm
- Updated: 2026-10-11

## Embeddings: meaning as coordinates

An embedding model turns text into a fixed-length list of numbers — a vector — such that texts with similar meaning end up close together.

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")   # small open model, 384 dimensions
vectors = model.encode(["How do I deduplicate rows?", "Remove duplicate records", "Spark executor memory"])
```

The first two sentences share almost no words but land near each other; the third lands far away.

## Measuring similarity

| Metric | Idea | Note |
| --- | --- | --- |
| Cosine similarity | Angle between vectors | Ignores length; the common default |
| Dot product | Angle and length | Equals cosine when vectors are normalized |
| Euclidean distance | Straight-line distance | Smaller is closer |

Use the metric the embedding model was trained for — its documentation says which.

## Why "approximate" nearest neighbours

Comparing a query with every stored vector is exact but slow at millions of vectors. Vector databases (Qdrant, pgvector, Pinecone, Weaviate and others) build indexes such as **HNSW**, a layered graph that finds near neighbours quickly while occasionally missing the true closest one.

```mermaid
flowchart LR
  accTitle: Vector search flow
  accDescr: Documents are chunked and embedded into an index; a query is embedded with the same model and the index returns the nearest chunks.
  D[Documents] --> C[Chunks] --> E1[Embed] --> I[(Vector index)]
  Q[Question] --> E2[Embed with the same model] --> I
  I --> R[Top-k similar chunks]
```

You tune a recall-versus-speed trade-off with index parameters (for HNSW, values such as `ef` and `M`).

> **Warning: Same model on both sides**
>
> Query and documents must be embedded with the **same model and version**. Changing the model means re-embedding the whole corpus.

## Where vectors fall short

Vectors capture meaning but can miss exact tokens: error codes, product SKUs, rare names. **Hybrid search** combines vector similarity with keyword search (BM25) and merges the rankings, often with a reranking model on top.

## Practical defaults

1. Chunk documents by structure (headings, paragraphs).
2. Store metadata (source, date, permissions) next to each vector and filter on it.
3. Retrieve more than you need (say 20), rerank, then pass the best few to the model.
4. Measure retrieval quality on real questions before tuning anything else.