AI / ML · Guide

Vector Databases

Storing meaning as coordinates, finding neighbours fast, and the retrieval half of RAG.

— min read AI / ML

Meaning As Coordinates

An embedding model turns text, an image or audio into a vector of a few hundred to a few thousand numbers, positioned so that similar meanings land close together. A vector database stores those vectors and answers "what is nearest to this".

That is a genuinely different query from anything SQL offers. Keyword search finds documents containing your words; vector search finds documents that mean something similar, including ones sharing no vocabulary with the query at all.

MeasureNotes
Cosine similarityAngle only, ignores magnitude — the usual choice for text
Dot productEquivalent to cosine when vectors are normalised
Euclidean distanceStraight-line distance; sensitive to magnitude
Vectors are only comparable within the model that produced them. Change embedding model and every stored vector is meaningless — re-embedding the entire corpus is the migration, and it is not cheap.

Approximate Nearest Neighbour

Comparing a query against every vector is exact and linear — fine for ten thousand, hopeless for fifty million. ANN indexes trade a little recall for orders of magnitude of speed.

IndexIdeaTrade-off
FlatCompare against everythingExact, slow, no build time
IVFCluster, then search the nearest clustersFast; misses neighbours near cluster edges
HNSWNavigable small-world graphExcellent recall and speed; memory hungry
Product quantisationCompress vectors into codesBig memory saving, some accuracy lost

The knobs all trade the same three things: recall, latency and memory. Measure recall against an exact flat search on a sample before tuning for speed — a fast index that returns the wrong neighbours is worse than a slow correct one, and the difference is invisible without measurement.

Postgres with pgvector is often the right answer. If your corpus is in the millions rather than the billions, one fewer system to operate usually beats a specialised database, and you keep joins and transactions.

Chunking, Filtering & Hybrid Search

Retrieval quality is decided long before the index. Chunking is the biggest lever: too large and a chunk's embedding averages several topics into mush; too small and it loses the context that made it meaningful.

DecisionGuidance
Chunk sizeSplit on structure — headings, paragraphs — not a fixed character count
OverlapA little, so a sentence spanning a boundary survives
MetadataStore source, date, permissions — filtering needs them
Hybrid searchCombine vector and keyword: exact ids and names need lexical matching
RerankingRetrieve 50 cheaply, rerank to 5 with a stronger model
Filter by permission inside the query, not after it. Retrieving documents a user may not see and then dropping them still leaks through timing, counts and any summary built before the filter ran.

Retrieval In RAG

In retrieval-augmented generation the database supplies the context a language model answers from. When RAG answers badly, the fault is far more often retrieval than generation — the model cannot use a passage it was never given.

SymptomUsually
Confidently wrong answersNothing relevant retrieved; the model filled the gap
Right topic, wrong detailChunks too large, detail averaged away
Ignores recent informationIndex stale, or no recency weighting
Leaks other tenants' dataMissing metadata filter
Slow responsesRetrieving far more context than the answer needs
Evaluate retrieval separately from generation. Measure whether the correct passage appears in the top-k at all; if it does not, no prompt engineering downstream will recover it.

Interview Questions

What does a vector database do that SQL cannot?

Nearest-neighbour search over embeddings — finding items that mean something similar, including documents that share no words with the query.

Why is changing embedding model expensive?

Vectors are only comparable within the model that produced them, so switching requires re-embedding and re-indexing the entire corpus.

What do ANN indexes trade?

Recall for latency and memory. HNSW gives excellent recall and speed at a memory cost; IVF is cheaper but misses neighbours near cluster boundaries.

Why is chunking the biggest lever in RAG?

Chunks that are too large average several topics into one vector; too small and they lose the context that made them meaningful. Splitting on structure beats a fixed character count.

Why combine vector and keyword search?

Embeddings are poor at exact tokens — an order number, a product code, a person's name. Hybrid search covers both semantic similarity and literal matching.

A RAG system answers confidently but wrongly. Where do you look?

Retrieval first. Check whether the correct passage was in the top-k at all; if it was never retrieved, no amount of prompt work will fix the answer.

Quick Quiz

1. Cosine similarity measures…
2. Switching embedding models requires…
3. HNSW is characterised by…
4. Permission filtering should happen…
5. A RAG answer that invents details usually means…