Embeddings and Vector Similarity
How meaning gets turned into numbers you can compare mathematically.
What you'll learn
- Explain what an embedding vector represents
- Compute cosine similarity between two vectors by hand and in code
- Explain why similar meanings produce similar vectors
Prerequisites
Explanation
An embedding is a list of numbers (a vector) that represents the meaning of a piece of text, produced by a model specifically trained for this purpose. Instead of comparing text character-by-character, you compare these number lists mathematically — and texts with similar meaning end up with vectors that point in similar directions, even if they don't share any of the same words.
For example, "a happy dog" and "a joyful puppy" would likely produce embedding vectors that are close together, while "a happy dog" and "quarterly tax filing" would produce vectors far apart — the embedding model has learned, from massive amounts of text, which concepts tend to appear in similar contexts, and encodes that as geometric closeness.
Real embedding models (like those from OpenAI or open-source alternatives) produce vectors with hundreds or thousands of numbers. In this lesson's exercises, we use tiny hand-picked 2-4 number vectors purely to make the arithmetic traceable — a real system calls a hosted embedding API to get these vectors instead of hand-writing them.
The standard way to measure how similar two vectors are is cosine similarity: it measures the angle between two vectors, ignoring their length/magnitude, and returns a value from -1 (opposite) to 1 (identical direction). The formula is the dot product of the two vectors divided by the product of their magnitudes:
cosineSimilarity(a, b) = dot(a, b) / (magnitude(a) * magnitude(b))
Where dot(a, b) sums the products of matching positions (a[0]*b[0] + a[1]*b[1] + ...), and magnitude(v) is the square root of the sum of its squared values (its length). A cosine similarity near 1 means "very similar meaning," near 0 means "unrelated," and negative means "opposite" (rare in practice for text embeddings, which tend to cluster in a narrower positive range).
This single operation — comparing embedding vectors with cosine similarity — is the mathematical foundation underneath semantic search, recommendation systems, and retrieval-augmented generation, all covered later in this track.
Vectors as points, similarity as angle
Imagine each embedding as an arrow from the origin in space. Texts with similar meaning point in similar directions (small angle, cosine similarity near 1); unrelated texts point in very different directions (cosine similarity near 0).
Example
Computing cosine similarity between small hand-picked vectors standing in for real embeddings.
function dot(a, b) {
return a.reduce((sum, val, i) => sum + val * b[i], 0);
}
function magnitude(v) {
return Math.sqrt(v.reduce((sum, val) => sum + val * val, 0));
}
function cosineSimilarity(a, b) {
return dot(a, b) / (magnitude(a) * magnitude(b));
}
// Pretend these came from a real embedding API.
const happyDog = [0.9, 0.1];
const joyfulPuppy = [0.85, 0.15];
const taxFiling = [0.05, 0.95];
console.log(cosineSimilarity(happyDog, joyfulPuppy));
console.log(cosineSimilarity(happyDog, taxFiling));Try it yourself
Add a third vector and compare its similarity to happyDog.
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Guided exercise
Guided exercise
Complete `dotProduct(a, b)`, returning the sum of each pair of matching elements multiplied together.
Checks: Dot product of two 3-element vectors is correct · plus 1 hidden check
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Stuck? Get a hint.
Independent exercise
Independent exercise
Write `cosineSimilarity(a, b)` from scratch (dot product divided by the product of magnitudes), and use it to write `mostSimilar(query, candidates)` returning the index of the candidate vector most similar to query.
Checks: Identical vectors score ~1 · Perpendicular vectors score ~0 · plus 1 hidden check
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Stuck? Get a hint.
Common mistakes
- Comparing embeddings from two different embedding models — vectors from different models are not comparable to each other.
- Confusing cosine similarity (angle-based) with Euclidean distance (magnitude-based); they can rank differently.
- Assuming embeddings capture exact factual correctness rather than topical/semantic closeness.
Knowledge check
Takeaway
Embeddings turn meaning into numbers, and cosine similarity turns 'how similar are these meanings?' into simple arithmetic.
Summary
An embedding model converts text into a numeric vector positioned so that similar meanings land close together. Cosine similarity — dot product divided by the product of magnitudes — measures that closeness, forming the mathematical basis for semantic search and retrieval covered later in this track.
References
Your notes
Notes save automatically.
Finished this lesson?
Mark it complete to track your progress and schedule a future review.
AI tutor
The optional AI tutor isn't enabled in this deployment. All lessons, exercises, quizzes, and search work fully without it.