Generative AI & LLM Apps
Topic 6 · Building LLM apps · ⏱ about 2.5 h

Embeddings & Vector Search

Turn text into vectors that capture meaning, compare them with cosine similarity, search thousands of documents by meaning instead of keywords, and choose an embedding model and a vector database.

Show:
In this topic
Beginner

Meaning as a point in space

An embedding model turns a piece of text into a list of numbers, a vector, typically 384 to 3,072 numbers long. It is trained so that texts with similar meaning get nearby vectors: "How do I reschedule?" lands close to "Can I move my appointment?", even though they share no words. The NLP course explains how these models are trained.

Closeness is usually measured with cosine similarity, the cosine of the angle between two vectors (see vectors): 1 means the same direction (same meaning), 0 means unrelated. Many embedding models output vectors of length 1, so cosine similarity is simply the dot product.

Try it

Try it: search by meaning

A tiny 2-D picture of an embedding space (real ones have hundreds of dimensions, but the idea is the same). Pick a question and see which stored sentences are nearest.

Interactive: this part runs in your browser.

Beginner

What embeddings are used for

  • Semantic search: find the documents closest to a question. This is the "retrieval" in RAG.
  • Clustering feedback or tickets into themes, and deduplication (near-identical vectors).
  • Classification with very little data: embed the texts, then train a small model such as logistic regression on the vectors.
  • Recommendations: "more like this".
Beginner

Code: cosine similarity search with NumPy

Here the "embedding model" is a simple word-count vector so the code runs in the page. With a real embedding model only the embed function changes; the search code stays exactly the same.

Python
import numpy as np, re

docs = [
    "You can reschedule an appointment online or by phone.",
    "Bulk-billing is available for children under 16.",
    "Bring your Medicare card to your first visit.",
    "The clinic is open 8am to 6pm on weekdays.",
    "Cancel at least 24 hours ahead to avoid a fee.",
]
words = lambda t: re.findall(r"[a-z0-9]+", t.lower())
vocab = sorted({w for d in docs for w in words(d)})

def embed(text):
    # stand-in for a real embedding model: word counts, scaled to length 1
    v = np.array([words(text).count(w) for w in vocab], dtype=float)
    n = np.linalg.norm(v)
    return v / n if n else v

D = np.stack([embed(d) for d in docs])          # one row per document

def search(query, k=2):
    sims = D @ embed(query)                     # cosine similarity (all vectors have length 1)
    best = np.argsort(-sims)[:k]
    return [(round(float(sims[i]), 3), docs[i]) for i in best]

for q in ["what are the opening hours", "is there a fee if I cancel"]:
    print(q, "->", search(q))
Open in playground ↗
Beginner

Choosing models and vector databases

  • Embedding models: hosted ones from the major providers, or open models you can run yourself (for example from the sentence-transformers family). Compare them on the public MTEB leaderboard ↗, but test on your data, and check language support if your users write in Nepali, Hindi or other languages.
  • Always embed queries and documents with the same model; vectors from different models are not comparable.
  • Vector stores: for up to roughly a few hundred thousand chunks, a NumPy array or a library like FAISS is plenty. Beyond that, or when you need filtering and updates, use a vector database (pgvector inside PostgreSQL, Chroma, Qdrant, Weaviate, Milvus, Pinecone and others). They use approximate nearest-neighbour indexes such as HNSW to search millions of vectors in milliseconds.
Check yourself

Quiz

1. Two sentences share no words but mean the same thing. With a good embedding model their cosine similarity is…

2. You embedded your documents with model A and the questions with model B. What happens?

3. You have 5,000 short documents. What vector store do you need?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready