Embeddings & Vector Databases

Embeddings turn text into numbers so that similar meanings land close together. A vector database stores millions of them and finds the nearest ones in milliseconds. In this session, let us learn how to choose both, and see a real speed test of three index types.

01Choosing an embedding model

An embedding model reads a chunk and outputs a vector, for example 1,024 numbers. Questions to ask:

• Dimension: more dimensions can hold more nuance but cost more storage and search time.
• Domain fit: is it good on banking, legal or medical text like yours?
• Languages: do you need Hindi, Marathi or mixed “Hinglish” queries?
• Max input length: must fit your chunk size.
• Benchmarks: check the MTEB leaderboard (Massive Text Embedding Benchmark), especially the retrieval scores, then test on your own data.

02What an embedding space looks like

03Cosine similarity

To compare two vectors, RAG systems usually use cosine similarity: 1 means same direction (same meaning), 0 means unrelated. The retriever returns the chunks with the highest similarity to the question.

04Vector databases compared

CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (id bigserial PRIMARY KEY, text text, department text, embedding vector(1024));
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

-- top 5 chunks for retail banking, nearest first (<=> is cosine distance)
SELECT text FROM chunks
WHERE department = 'retail_banking'
ORDER BY embedding <=> '[0.012, -0.031, ...]'::vector
LIMIT 5;

05Index types: Flat, IVF and HNSW

Comparing the query with every vector (Flat) is exact but slow at scale. Approximate indexes trade a tiny bit of accuracy for big speed gains.

We ran this benchmark on 100,000 clustered 384-dimensional vectors (CPU):

import time, numpy as np, faiss

d, n = 384, 100_000                                  # 384-d vectors, like all-MiniLM-L6-v2
rng = np.random.default_rng(0)
centers = rng.standard_normal((500, d))              # real embeddings form clusters of topics
xb = (centers[rng.integers(0, 500, n)] + 0.3 * rng.standard_normal((n, d))).astype("float32")
xq = (centers[rng.integers(0, 500, 200)] + 0.3 * rng.standard_normal((200, d))).astype("float32")
faiss.normalize_L2(xb); faiss.normalize_L2(xq)       # cosine similarity = inner product

flat = faiss.IndexFlatIP(d); flat.add(xb)

hnsw = faiss.IndexHNSWFlat(d, 32, faiss.METRIC_INNER_PRODUCT)
hnsw.hnsw.efSearch = 64; hnsw.add(xb)

quant = faiss.IndexFlatIP(d)
ivf = faiss.IndexIVFFlat(quant, d, 256, faiss.METRIC_INNER_PRODUCT)
ivf.train(xb); ivf.add(xb); ivf.nprobe = 8

_, truth = flat.search(xq, 10)
for name, idx in [("Flat (exact)", flat), ("IVF", ivf), ("HNSW", hnsw)]:
    t = time.perf_counter(); _, I = idx.search(xq, 10); ms = (time.perf_counter() - t) * 1000 / len(xq)
    recall = np.mean([len(set(a) & set(b)) / 10 for a, b in zip(I, truth)])
    print(f"{name:13s} {ms:5.2f} ms/query   recall@10 = {recall:.2f}")

HNSW was about 60× faster than exact search while still finding 97% of the true top-10. Timings depend on your machine; the pattern holds.

06Summary

• Choose embeddings by quality on your data, languages, dimension and cost; use MTEB as a starting point.
• Use the same model for documents and queries.
• Chroma for learning, Pinecone for managed production, FAISS for custom speed, pgvector if you live in Postgres.
• HNSW and IVF give huge speedups for a small recall cost.

07Quick quiz

08References

• MTEB leaderboard
• FAISS
• pgvector
• HNSW paper (Malkov & Yashunin)