Semantic Search: Find Documents by Meaning

Introduction

In the last lesson you compared a query with three sentences. A real app searches thousands of passages from help articles, documentation, policies or notes. In this lesson you will build a small but complete semantic search engine over a shoe shop’s help centre. It runs entirely on your computer, needs no API key, and is the exact retrieval step that RAG builds on in the next lesson.

How Semantic Search Works

Semantic search happens in two phases. Indexing runs once, ahead of time, whenever your documents change. Querying runs every time someone searches, and it is fast because the documents were already embedded.

Chunking Documents

A single embedding has to represent everything in its text, so long documents are split into chunks — passages small enough to be about one thing, but large enough to make sense on their own. Chunking well matters more than almost any other choice in a search system: bad chunks produce bad results no matter how good the model is.

Building the Search Engine

Here is the whole engine in one file. It holds four short help articles, splits them into chunks of whole sentences with one sentence of overlap, embeds the chunks in a batch, and stores everything in a simple in-memory index. The chunk size is deliberately tiny so you can see chunking happen on short articles; use a larger size for real documents.

# search.py — a tiny semantic search engine
import re
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

DOCS = {
    "shipping.md": (
        "We ship to the United States, Canada and the United Kingdom. "
        "Standard delivery takes 3 to 5 working days and is free on orders over $50. "
        "Express delivery arrives in 1 to 2 working days for $12.99. "
        "You get a tracking link by email as soon as your parcel leaves our warehouse."
    ),
    "returns.md": (
        "You can return unworn shoes within 30 days of delivery for a full refund. "
        "Shoes must be in their original box with all tags attached. "
        "Shoes that have been worn outdoors cannot be returned unless they are faulty. "
        "To start a return, open your order history and select Return item. "
        "Refunds reach your original payment method within 5 to 7 working days. "
        "Exchanges for a different size are free."
    ),
    "account.md": (
        "To change your password, sign in, open Account settings and choose Security. "
        "If you cannot sign in, use the Forgot password link on the sign-in page. "
        "We will email you a reset link that is valid for one hour. "
        "You can update your delivery address under Account settings and Addresses."
    ),
    "care.md": (
        "Clean leather shoes with a soft brush and apply leather conditioner once a month. "
        "Let wet shoes dry naturally at room temperature, away from radiators. "
        "Canvas trainers can be hand-washed in cold water with mild soap. "
        "Never put any of our shoes in a tumble dryer."
    ),
}


def chunk_text(text, max_words=40):
    """Group whole sentences into chunks of about max_words words."""
    sentences = re.split(r"(?<=[.!?])\s+", text.strip())
    chunks, current = [], []
    for sentence in sentences:
        if current and len(" ".join(current + [sentence]).split()) > max_words:
            chunks.append(" ".join(current))
            current = [current[-1]]  # overlap: repeat the last sentence
        current.append(sentence)
    if current:
        chunks.append(" ".join(current))
    return chunks


class VectorIndex:
    """Keeps chunks and their vectors in memory. Fine for a few thousand chunks."""

    def __init__(self):
        self.items = []  # each item: {"text": ..., "source": ..., "vector": ...}

    def add(self, text, source):
        chunks = chunk_text(text)
        vectors = model.encode(chunks, normalize_embeddings=True)  # one batch
        for chunk, vector in zip(chunks, vectors):
            self.items.append({"text": chunk, "source": source, "vector": vector})

    def search(self, query, top_k=3, source=None):
        q = model.encode(query, normalize_embeddings=True)
        results = []
        for item in self.items:
            if source and item["source"] != source:
                continue  # metadata filter
            score = float(item["vector"] @ q)  # cosine similarity
            results.append((score, item))
        results.sort(key=lambda r: r[0], reverse=True)
        return results[:top_k]


def build_index():
    index = VectorIndex()
    for name, text in DOCS.items():
        index.add(text, source=name)
    return index

Notice three design decisions. Chunks are embedded in one batch per document, which is much faster than one call per chunk. Each stored item keeps its source next to its vector, so results can say where they came from. And search accepts an optional source filter, applied before scoring — the same idea vector databases call metadata filtering, used to search only one product, language or customer’s documents.

Trying It Out

Save the engine as search.py or search.mjs, then create a second file next to it that builds the index and runs two searches — one across everything, and one filtered to the account help article:

# try_search.py
from search import build_index

index = build_index()
print(f"Indexed {len(index.items)} chunks")

for score, item in index.search("How do I get my money back?"):
    print(f"{score:.2f}  [{item['source']}]  {item['text'][:60]}...")

print("Only account help:")
for score, item in index.search("I can't log in", top_k=1, source="account.md"):
    print(f"{score:.2f}  [{item['source']}]  {item['text'][:60]}...")

This is the question from the start of the previous lesson. It never uses the words “refund” or “return”, yet all three results come from the returns article, led by the passage about refunds. Try your own questions — and try a few that the help centre cannot answer, such as “Do you sell handbags?”. You will still get results, but the best one scores only about 0.2. Search always returns its closest matches, even when nothing is truly relevant, which is why the next lesson adds a minimum score.

Vector Databases

Comparing a query with every stored vector is perfectly fine for a few thousand chunks. With millions of chunks it becomes too slow, and you also want your index saved to disk, updated without rebuilding everything, and shared between servers. That is the job of a vector database, such as Chroma, Qdrant, Pinecone, Weaviate, or PostgreSQL with the pgvector extension. They use clever indexes to find the nearest vectors without checking every one, and they support metadata filters like the source filter you just wrote.

The good news is that the concepts do not change: you still chunk, embed, store vectors with metadata, embed the query and ask for the top matches. Moving from the in-memory index to a vector database replaces the VectorIndex class, not the way you think about search.

Key Takeaways

Summary

You have built a working semantic search engine: it chunks documents, embeds them, ranks passages by meaning and filters by source. On its own it finds relevant passages, but users still have to read them. In the final lesson of this module you will hand those passages to a language model and have it write a direct, cited answer — the technique called retrieval-augmented generation, or RAG.