Embeddings & Vector Search
Turn text into vectors that capture meaning, compare them with cosine similarity, search thousands of documents by meaning instead of keywords, and choose an embedding model and a vector database.
Meaning as a point in space
An embedding model turns a piece of text into a list of numbers, a vector, typically 384 to 3,072 numbers long. It is trained so that texts with similar meaning get nearby vectors: "How do I reschedule?" lands close to "Can I move my appointment?", even though they share no words. The NLP course explains how these models are trained.
Closeness is usually measured with cosine similarity, the cosine of the angle between two vectors (see vectors): 1 means the same direction (same meaning), 0 means unrelated. Many embedding models output vectors of length 1, so cosine similarity is simply the dot product.
Try it: search by meaning
A tiny 2-D picture of an embedding space (real ones have hundreds of dimensions, but the idea is the same). Pick a question and see which stored sentences are nearest.
Interactive: this part runs in your browser.
What embeddings are used for
- Semantic search: find the documents closest to a question. This is the "retrieval" in RAG.
- Clustering feedback or tickets into themes, and deduplication (near-identical vectors).
- Classification with very little data: embed the texts, then train a small model such as logistic regression on the vectors.
- Recommendations: "more like this".
Code: cosine similarity search with NumPy
Here the "embedding model" is a simple word-count vector so the code runs in the page. With a real embedding model only the embed function changes; the search code stays exactly the same.
import numpy as np, re
docs = [
"You can reschedule an appointment online or by phone.",
"Bulk-billing is available for children under 16.",
"Bring your Medicare card to your first visit.",
"The clinic is open 8am to 6pm on weekdays.",
"Cancel at least 24 hours ahead to avoid a fee.",
]
words = lambda t: re.findall(r"[a-z0-9]+", t.lower())
vocab = sorted({w for d in docs for w in words(d)})
def embed(text):
# stand-in for a real embedding model: word counts, scaled to length 1
v = np.array([words(text).count(w) for w in vocab], dtype=float)
n = np.linalg.norm(v)
return v / n if n else v
D = np.stack([embed(d) for d in docs]) # one row per document
def search(query, k=2):
sims = D @ embed(query) # cosine similarity (all vectors have length 1)
best = np.argsort(-sims)[:k]
return [(round(float(sims[i]), 3), docs[i]) for i in best]
for q in ["what are the opening hours", "is there a fee if I cancel"]:
print(q, "->", search(q))Choosing models and vector databases
- Embedding models: hosted ones from the major providers, or open models you can run yourself (for example from the sentence-transformers family). Compare them on the public MTEB leaderboard ↗, but test on your data, and check language support if your users write in Nepali, Hindi or other languages.
- Always embed queries and documents with the same model; vectors from different models are not comparable.
- Vector stores: for up to roughly a few hundred thousand chunks, a NumPy array or a library like FAISS is plenty. Beyond that, or when you need filtering and updates, use a vector database (pgvector inside PostgreSQL, Chroma, Qdrant, Weaviate, Milvus, Pinecone and others). They use approximate nearest-neighbour indexes such as HNSW to search millions of vectors in milliseconds.
Quiz
1. Two sentences share no words but mean the same thing. With a good embedding model their cosine similarity is…
2. You embedded your documents with model A and the questions with model B. What happens?
3. You have 5,000 short documents. What vector store do you need?
📚 Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- Sentence-BERT (Reimers & Gurevych, 2019) ↗ the classic paper on sentence embeddings for semantic search
- MTEB embedding leaderboard ↗ compare embedding models across many tasks and languages