Embeddings

Computers cannot compare meanings, but they are very good at comparing numbers. An embedding turns text into a list of numbers (a vector) so that similar meanings get similar numbers: “lost credit card” lands next to “stolen debit card”, far away from “biryani recipe”. That one idea powers semantic search, RAG and recommendations.

01The big idea: GPS coordinates for meaning

Cosine similarity measures how much two vectors point the same way: 1 = same meaning, 0 = unrelated. It is one line of NumPy:

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

02embed_query, embed_documents and providers

03Hands-on: run it here

The notebook’s similarity table and FAQ search, with a no-download stand-in embedding. With MiniLM in Colab the numbers are different (and unrelated pairs are not exactly 0), but the ranking idea is the same.

import numpy as np
from standins import KeywordEmbeddings
# In Colab: embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
embeddings = KeywordEmbeddings(dim=384)

vector = embeddings.embed_query("How do I block my lost card?")
print("dims:", len(vector), "| non-zero values:", {i: v for i, v in enumerate(vector) if v})

def cosine(a, b): return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
sentences = ["lost credit card", "stolen debit card", "best biryani in Hyderabad", "UPI payment failed"]
vectors = embeddings.embed_documents(sentences)
for i in range(len(sentences)):
    for j in range(i + 1, len(sentences)):
        print(f"  {sentences[i]:<26} vs {sentences[j]:<26} -> {cosine(vectors[i], vectors[j]):.3f}")

faq = ["How to reset my net banking password?", "What is the daily UPI limit?", "How to block a stolen card?",
       "How to update my mobile number?", "What are the home loan interest rates?"]
faq_vectors = embeddings.embed_documents(faq)
def search(question):
    q = embeddings.embed_query(question)
    scores = [cosine(q, v) for v in faq_vectors]
    best = int(np.argmax(scores))
    return faq[best], round(scores[best], 3)
for q in ["someone took my debit card", "forgot my login password", "loan rate for a house"]:
    print("search:", q, "->", search(q))

04Hands-on: Ajeetkumar’s Colab notebook

Outputs labelled stand-in embeddings came from re-running the notebook with the keyword stand-in (no 90 MB download): same cells and data, different numbers from MiniLM.

05Practice tasks

1. Embed 10 movie plots and find the closest one to “a robot falls in love”.
2. Compare the similarity of the Hindi and English versions of the same sentence.
3. Try BAAI/bge-small-en-v1.5 and compare scores with MiniLM.

Try these in the same Colab notebook after you finish the lab. Change one thing at a time and look at the output again.

06Summary

• An embedding turns text into a vector; close vectors = similar meaning.
• embed_documents for chunks (once), embed_query for questions (every time).
• Cosine similarity: 1 = same direction, 0 = unrelated.
• MiniLM (384 dims) is free on Colab; OpenAI, Gemini and Ollama have the same methods.
• Always use the same model for documents and queries; cache to avoid paying twice.

07Quick quiz

08References

• LangChain: Embedding models
• Sentence Transformers: all-MiniLM-L6-v2
• MTEB embedding leaderboard
• Ajeetkumar’s notebook: LC_08_Embeddings.ipynb (download)