Building a Chat App with Memory

Introduction

Chat is the most familiar way people use AI models, and it is the backbone of support bots, tutoring apps and writing assistants. In this lesson you will build a chat program that runs in your terminal. Along the way you will learn the one fact that surprises almost every beginner — the model does not remember anything between calls — and how real apps work around it.

The API Has No Memory

The API is stateless: every request is handled on its own, with no record of earlier requests. If you send “My name is Priya” in one call and “What is my name?” in the next, the second call contains no mention of Priya, so the model cannot know. Chat websites feel like they remember because the app quietly sends the earlier conversation again with every new message.

Call 1  →  "My name is Priya."
        ←  "Nice to meet you, Priya!"

Call 2  →  "What is my name?"
        ←  "I don't know your name — you haven't told me yet."

Memory Is a Messages List

The messages parameter is a list of turns, oldest first. Each turn has a role — user for the person, assistant for the model — and its content. To give a chat memory, keep that list in your program: append the user’s message, send the whole list, then append the model’s reply. Next turn, the list already contains everything said so far.

[
  { "role": "user", "content": "My name is Priya." },
  { "role": "assistant", "content": "Nice to meet you, Priya!" },
  { "role": "user", "content": "What is my name?" }
]

Here is the smallest chat with memory — a loop that reads a line, adds it to the list, sends the list and stores the answer. Press Ctrl+C to stop it:

# mini_chat.py
import anthropic

client = anthropic.Anthropic()
messages = []  # the whole conversation, oldest first

while True:
    user_text = input("You: ")
    messages.append({"role": "user", "content": user_text})

    response = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=16000,
        messages=messages,
    )
    reply = "".join(b.text for b in response.content if b.type == "text")
    print("AI:", reply)

    messages.append({"role": "assistant", "content": reply})  # remember it

Streaming Replies

A long answer can take several seconds to generate. With a normal call, the user stares at nothing until the whole reply is ready. Streaming sends the reply in small pieces as it is generated, so you can print words the moment they arrive — which is why chat apps appear to type. The total time is the same, but the app feels far faster, and streaming also avoids network timeouts on very long replies.

Both SDKs have a stream helper. You loop over the text pieces as they arrive, then ask the stream for the final, complete message, which you store in the history exactly as before. You will see it inside the complete program below.

Handling Errors

Calls over the internet sometimes fail. The SDKs already retry short-lived problems a couple of times, but your app still needs a plan for when a call fails for good. The common cases each have their own error type, so you can respond sensibly instead of crashing:

One detail matters for memory: if a call fails, remove the user message you just appended. Otherwise the history contains a question with no answer, and the user’s retry would be sent twice.

The Complete Chat App

Here is everything together: a system prompt, memory, streaming, error handling, a polite message when the model declines, and a limit on how much history is kept (explained in the next section). Type quit to stop.

# chat.py
import anthropic

client = anthropic.Anthropic()
MODEL = "claude-opus-5-5"
SYSTEM = "You are a friendly study buddy for beginner programmers. Keep answers short."
MAX_MESSAGES = 20  # remember the last 10 exchanges

messages = []  # the conversation so far, oldest first

print("Chat started. Type 'quit' to stop.")
while True:
    user_text = input("\nYou: ").strip()
    if user_text.lower() in {"quit", "exit"}:
        break
    if not user_text:
        continue

    messages.append({"role": "user", "content": user_text})
    print("AI: ", end="", flush=True)
    try:
        with client.messages.stream(
            model=MODEL,
            max_tokens=16000,
            system=SYSTEM,
            messages=messages,
        ) as stream:
            for text in stream.text_stream:
                print(text, end="", flush=True)  # show words as they arrive
            reply = stream.get_final_message()
    except anthropic.RateLimitError:
        messages.pop()  # forget the unanswered question
        print("[Too many requests. Wait a moment and try again.]")
        continue
    except anthropic.APIStatusError as e:
        messages.pop()
        print(f"[The API returned an error ({e.status_code}). Try again.]")
        continue
    except anthropic.APIConnectionError:
        messages.pop()
        print("[Could not reach the API. Check your internet connection.]")
        continue

    reply_text = "".join(b.text for b in reply.content if b.type == "text")
    if reply.stop_reason == "refusal" or not reply_text:
        messages.pop()
        print("[The model declined to answer that. Try asking differently.]")
        continue

    messages.append({"role": "assistant", "content": reply_text})
    del messages[:-MAX_MESSAGES]  # drop the oldest turns

Keeping Conversations Small

Because the whole history is sent on every turn, a long chat gets slower and more expensive with every message: turn 50 pays for the 49 turns before it. Eventually it could even exceed the model’s context window — the maximum amount of text it can read in one request. There are three common ways to keep conversations under control:

Key Takeaways

Summary

You now have a real chat application: it remembers the conversation, streams its answers, recovers from errors and keeps its history in check. The model can only talk about what it learned in training and what you put in the messages, though. To build apps that answer questions about your own documents, you first need a way for a program to understand what text means — which is what embeddings, the topic of the next lesson, provide.