Embeddings Explained: Turning Meaning into Numbers

Introduction

Chat models generate text. Many AI features, though, need something different: a way for a program to tell that two pieces of text mean similar things. A help centre should match “I can’t sign in” with an article titled “Resetting your password”. A shop should show trainers when someone searches for “running shoes”. Embeddings are the tool that makes this possible, and they are the foundation of semantic search and RAG in the next two lessons.

Why Keywords Are Not Enough

Classic search matches words. Ask “How do I get my money back?” and a keyword search looks for pages containing “money” and “back”. The page that actually answers the question is titled “Refunds” and talks about “returning items” and “payment method” — it shares almost no words with the question. People describe the same idea in many different ways, and keyword search cannot see that “refund” and “get my money back” mean the same thing.

What we want is search by meaning. To do that with a computer, meaning has to become something a program can calculate with. That is exactly what an embedding is.

What an Embedding Is

An embedding is a list of numbers — a vector — that represents the meaning of a piece of text. An embedding model, trained on huge amounts of text, reads a sentence and outputs, say, 384 or 1,024 numbers. The individual numbers are not meant to be read by people, but the list as a whole has a powerful property: texts with similar meanings get similar lists of numbers.

To build intuition, imagine a tiny embedding with just three numbers, each scoring how much a text is about animals, food and technology. Real embeddings learn hundreds of directions on their own, but the idea is the same:

Picture each list as an arrow pointing into space. The cat and kitten arrows point in almost the same direction; the laptop arrow points somewhere else entirely. Notice that “cat” and “kitten” are different words — the embedding captures that they mean nearly the same thing.

Measuring Similarity

To compare two embeddings we measure the angle between their arrows using cosine similarity. It gives a score from -1 to 1: close to 1 means the texts mean nearly the same thing, around 0 means they are unrelated. The calculation is short — multiply matching numbers and add them up (the dot product), then divide by the lengths of both vectors:

# cosine.py
import math

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    length_a = math.sqrt(sum(x * x for x in a))
    length_b = math.sqrt(sum(y * y for y in b))
    return dot / (length_a * length_b)

# Our tiny hand-made embeddings: [animals, food, technology]
cat = [0.90, 0.10, 0.00]
kitten = [0.85, 0.15, 0.05]
laptop = [0.00, 0.10, 0.95]

print(round(cosine_similarity(cat, kitten), 3))  # 0.996: nearly the same meaning
print(round(cosine_similarity(cat, laptop), 3))  # 0.012: unrelated

A useful shortcut: if every vector is first scaled to a length of exactly 1 (called normalising), the two lengths are both 1 and cosine similarity is simply the dot product. Embedding libraries can normalise for you, which is why the real examples below only multiply and add.

Creating Real Embeddings

You do not need an API key to create embeddings. Small open models such as all-MiniLM-L6-v2 run on an ordinary laptop: the model downloads once (about 90 MB) and then works offline. The Python example uses the sentence-transformers library; the JavaScript one uses Transformers.js, which runs the same model in Node.js.

pip install sentence-transformers

# embed_demo.py
from sentence_transformers import SentenceTransformer

# A small, free model that runs on your own computer.
model = SentenceTransformer("all-MiniLM-L6-v2")

sentences = [
    "How do I reset my password?",
    "I forgot my login details",
    "What time does the shop open?",
]
# normalize_embeddings=True scales each vector to length 1,
# so cosine similarity becomes a simple dot product.
vectors = model.encode(sentences, normalize_embeddings=True)
print(vectors.shape)  # (3, 384): 3 sentences, 384 numbers each

query = model.encode("I've lost access to my account", normalize_embeddings=True)
scores = vectors @ query  # one similarity score per sentence
for sentence, score in zip(sentences, scores):
    print(f"{score:.2f}  {sentence}")

“I’ve lost access to my account” shares no words with “I forgot my login details”, yet that sentence scores highest, with the password question a close second, while the opening-hours question scores near zero. That is search by meaning in a dozen lines of code. For production apps you can also use hosted embedding APIs from providers such as Voyage AI, OpenAI, Cohere or Google; Anthropic does not offer its own embedding model, and its documentation points to Voyage AI. Whatever you choose, the concepts in this lesson stay the same.

Choosing What to Embed

A few practical rules make embeddings work well:

Key Takeaways

Summary

Embeddings turn meaning into numbers that a program can compare, and cosine similarity tells you how close two meanings are. You have created real embeddings on your own computer and watched a query match a sentence with completely different words. Next, you will use them to build a semantic search engine: splitting documents into chunks, storing their vectors and finding the best matches for a question.