Embeddings and Vector Search, Explained for Engineers
What an embedding is, how similarity is measured, why vector databases use approximate search, and when to add keyword search alongside it.
Embeddings: meaning as coordinates
An embedding model turns text into a fixed-length list of numbers — a vector — such that texts with similar meaning end up close together.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2") # small open model, 384 dimensions
vectors = model.encode(["How do I deduplicate rows?", "Remove duplicate records", "Spark executor memory"])The first two sentences share almost no words but land near each other; the third lands far away.
Measuring similarity
| Metric | Idea | Note |
|---|---|---|
| Cosine similarity | Angle between vectors | Ignores length; the common default |
| Dot product | Angle and length | Equals cosine when vectors are normalized |
| Euclidean distance | Straight-line distance | Smaller is closer |
Use the metric the embedding model was trained for — its documentation says which.
Why "approximate" nearest neighbours
Comparing a query with every stored vector is exact but slow at millions of vectors. Vector databases (Qdrant, pgvector, Pinecone, Weaviate and others) build indexes such as HNSW, a layered graph that finds near neighbours quickly while occasionally missing the true closest one.
Rendering vector search flow
You tune a recall-versus-speed trade-off with index parameters (for HNSW, values such as ef and M).
Where vectors fall short
Vectors capture meaning but can miss exact tokens: error codes, product SKUs, rare names. Hybrid search combines vector similarity with keyword search (BM25) and merges the rankings, often with a reranking model on top.
Practical defaults
- Chunk documents by structure (headings, paragraphs).
- Store metadata (source, date, permissions) next to each vector and filter on it.
- Retrieve more than you need (say 20), rerank, then pass the best few to the model.
- Measure retrieval quality on real questions before tuning anything else.
Finished this note?