LearnAI ToolsCareerPractice BuildsPlayContact
Lesson 1323 min read

Vector Databases: Chroma & FAISS

Learn Chroma for local-first embedded vector storage and FAISS for pure in-memory similarity search, with a working example of each.

Introduction

Not every project needs a managed cloud vector database. Chroma and FAISS both run locally with no external service or account required — ideal for prototyping, small projects, or fully offline applications, at the cost of managing scaling yourself if the project grows.

What You Will Learn
  • How to install and use Chroma for local, persistent vector storage.
  • How to install and use FAISS for pure in-memory similarity search.
  • How to decide between a local library and a managed service like Pinecone or Weaviate.

A Real-Life Analogy First

Lesson 12's managed vector databases are like renting a professional, climate-controlled storage warehouse with staff who manage it for you. Chroma and FAISS, by comparison, are more like a filing cabinet you keep in your own home office — perfectly capable of organizing a reasonable amount of paperwork, free to set up, entirely under your own control, but you are the one responsible for it, and it won't scale to warehouse-sized volume on its own.

Chroma: Local-First Embedded Storage

Use case: Chroma is an open-source, embedded vector database — it runs in-process, persists to a local folder, and includes a built-in default embedding function so you can go from raw text to a searchable index without a separate embedding step.

pip install chromadb

Chroma in Action

import chromadb


client = chromadb.PersistentClient(path="./chroma-db")
collection = client.get_or_create_collection("product-docs")


collection.add(
    ids=["doc-1", "doc-2"],
    documents=[
        "Refunds are processed within 5-7 business days.",
        "Shipping takes 3-5 business days domestically.",
    ],
)


results = collection.query(query_texts=["how long do refunds take"], n_results=1)
print(results["documents"][0][0])
Terminal Output

Click Run to see what this code prints.

No Manual Embedding Step

Notice add() and query() take raw text, not vectors — Chroma's built-in default embedding function generates the vectors for you automatically. You can also plug in a specific embedding model (like OpenAI's) if you need consistency with another part of your stack.

FAISS: Pure In-Memory Similarity Search

Use case: FAISS (Facebook AI Similarity Search) is a lower-level library, not a full database — it does one thing extremely well: extremely fast similarity search over vectors you already have, entirely in memory, with no persistence, metadata storage, or server built in.

pip install faiss-cpu

FAISS in Action

import faiss
import numpy as np


# 4 example vectors, each of dimension 3, for simplicity
vectors = np.array([
    [0.02, 0.91, -0.13],
    [0.88, -0.12, 0.30],
    [0.10, 0.85, -0.09],
    [0.75, -0.20, 0.40],
], dtype="float32")


index = faiss.IndexFlatL2(3)  # L2 = Euclidean distance, dimension = 3
index.add(vectors)


query = np.array([[0.05, 0.88, -0.11]], dtype="float32")
distances, indices = index.search(query, k=1)


print("Closest vector index:", indices[0][0])
print("Distance:", distances[0][0])
Terminal Output

Click Run to see what this code prints.

Choosing a Vector Store

ToolPersistence & MetadataBest For
FAISSNone built in — pure similarity searchResearch, benchmarking, or as a component inside a larger system
ChromaBuilt in, local folderPrototyping and small-to-medium local/offline projects
Pinecone / Weaviate (lesson 12)Fully managed, built inProduction applications needing scale and reliability

Common Mistakes

Avoid These Mistakes
  • Using raw FAISS and expecting metadata storage or persistence — you must track document text and IDs yourself alongside it.
  • Forgetting to persist a Chroma PersistentClient's path, which by default falls back to an in-memory client that loses data on restart.
  • Scaling a local FAISS or Chroma setup far past what a single machine's memory can hold, instead of moving to a managed service.

Best Practices

  • Start with Chroma for any local prototype that needs persistence and basic metadata without standing up external infrastructure.
  • Reach for raw FAISS only when you need maximum control over the search algorithm itself, or are embedding it inside another system.
  • Move to a managed service (lesson 12) once a project needs to scale beyond a single machine or requires production reliability guarantees.

Frequently Asked Questions

Chroma is commonly used in production for small-to-medium workloads, and also offers a managed cloud option — evaluate it against your specific scale and reliability needs.

FAISS gives you lower-level control over the exact indexing algorithm (flat, IVF, HNSW, and others) for performance tuning — most projects do not need this level of control and are better served by Chroma.

Yes — the underlying concept (vectors plus metadata) is the same across tools, though you will need to re-generate or re-upload your data into the new system.

Chroma — it needs no account, no API key beyond your LLM provider, and no infrastructure decisions, so you can go from "nothing" to "a working search over my own documents" in a few minutes, exactly like the example above.

Key Takeaways

  • Chroma is an embedded, local-first vector database with built-in persistence and a default embedding function.
  • FAISS is a lower-level similarity search library with no built-in persistence or metadata storage.
  • Local tools are ideal for prototyping; managed services (lesson 12) fit production scale better.
  • The underlying "store vectors, search by similarity" concept is consistent across every vector store in this course.

Summary

Chroma and FAISS make it possible to prototype retrieval systems entirely locally, with no account or external service required, before ever touching a managed vector database.

Lesson 13 Completed
  • You can store and query documents locally with Chroma.
  • You can run a raw similarity search with FAISS.
  • You know how to choose between local and managed vector stores.
Next Lesson →

Embeddings & Semantic Search