AI in Real Apps
How AI models are integrated into real applications — the architecture they share, the six patterns behind almost every AI feature, and working code for each in Python and JavaScript.
The Architecture Every AI Feature Shares
Every AI feature in a real application — however different it looks to the user — follows the same basic shape. The user’s device never talks to the AI provider directly. It talks to your backend, and your backend, which holds the API key, builds the prompt, calls the model and decides what to send back.
Your backend is where the real work happens. It is the only place the API key can be kept safe, and it is where you add the system prompt, look up the user’s data, limit how often each user can call the model, and check what comes back. From the browser, calling an AI feature looks like calling any other endpoint in your app:
const res = await fetch("/api/chat", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ messages }), // the conversation so far});const { reply } = await res.json();Anything shipped to a browser or a mobile app can be extracted by its users, including keys hidden in bundled JavaScript or compiled apps. A leaked key lets anyone run up charges on your account. Keep keys in environment variables on your server.
Chatbots and Assistants
Conversations that remember context
- Customer support chat
- Tutoring and study helpers
- In-app help assistants
- Writing assistants
A chat assistant is a backend endpoint that receives the conversation, adds a system prompt that sets the assistant’s role and rules, and returns the model’s reply. Because the model API is stateless, the conversation is sent with every request. The endpoint below also validates what it receives, so a user cannot send unlimited history or fake roles, and turns provider errors into a friendly message.
In production you would usually stream the reply to the browser so words appear as they are generated, and store conversations on the server against the user’s account rather than trusting the history the browser sends.
Summarising and Extracting Data
Turn messy text into clean, structured data
- Summarising emails, calls and documents
- Pulling fields out of invoices and forms
- Turning reviews into structured feedback
- Meeting notes to action items
Much of the world’s useful information is trapped in free text: emails, support tickets, contracts, reviews. A model can read that text and return exactly the fields your program needs. Instead of hoping the model formats its answer correctly, you give the API a schema and use structured outputs: the reply is constrained to that shape and the SDK validates it, so your code works with a normal, typed object.
Click Run to see what this code prints.
The email is wrapped in tags and described as the thing to extract from, so instructions hidden inside it are treated as data. The same pattern summarises: add a summary field, or ask for “3 bullet points” in plain text when no program needs to read the result.
Smart Search and Recommendations
Find things by meaning, not keywords
- Product search that understands intent
- Help-centre and documentation search
- “Similar items” recommendations
- Finding duplicate tickets or questions
Semantic search uses embeddings — lists of numbers that capture meaning — to match what the user means rather than the exact words they type. You embed your catalogue or documents once, store the vectors, and at search time embed the query and rank items by similarity. No chat model is involved, so it is fast and cheap; the example below runs a free model locally.
Click Run to see what this code prints.
“Something comfy for a marathon” shares no words with “cushioned road running shoes for long distances”, yet it is the top match. With thousands of items you would store the vectors in a vector database such as pgvector, Chroma or Qdrant instead of a list.
Answering Questions from Your Documents (RAG)
Grounded answers with citations
- Support bots that know your policies
- Internal knowledge-base assistants
- “Chat with this PDF” features
- Answering questions about product docs
Retrieval-augmented generation (RAG) combines the previous pattern with a chat model. For each question, your backend searches your documents for the most relevant passages, puts them into the prompt as numbered sources, and instructs the model to answer only from them and cite them. The model supplies the language skills; your documents supply the facts, which keeps answers current and checkable.
Notice the two honesty checks: weak matches are dropped before the model is called, and the prompt tells the model to say it does not know rather than guess. Test both retrieval and answers with a fixed set of questions whenever you change the documents, chunking or prompt.
Classification and Routing
Sort, label and send things to the right place
- Routing support tickets to the right team
- Moderating user-generated content
- Tagging feedback by topic
- Detecting urgent or high-risk messages
Classification asks the model to choose from a fixed list of labels, and your code acts on the choice — sending a ticket to a team, flagging a post for review or tagging feedback. Defining the labels as an enum in the schema means the SDK validates the reply, so your routing code only ever receives one of your labels — a typo or an invented category can never slip through.
Click Run to see what this code prints.
Classification is often high-volume, so measure accuracy on a labelled sample of real messages, then try lower-cost settings or smaller models and keep the cheapest one that stays accurate enough. For decisions with real consequences, such as removing content, have a person review the cases the model marks as uncertain.
Agents That Use Tools
Let the model take actions through your code
- Assistants that check order status or book appointments
- Coding assistants that run tests
- Research assistants that search and read
- Automating multi-step back-office tasks
With tool use, you describe functions your code can run — look up an order, search a database, send an email — and the model decides when to call them. The model never runs anything itself: it replies with a request to call a tool, your code runs the function and sends the result back, and the loop continues until the model gives its final answer.
Agents are powerful but need the most care. Give each tool the smallest permissions that work, cap the number of steps, log every tool call, and require a person to confirm anything that spends money, sends messages or deletes data — a prompt injection hidden in a web page or email can try to trigger tools too.
- Module 5: Agents & Tools (coming soon)
- Lesson 9: AI Safety Basics →
Choosing the Right Pattern
| If your app needs to… | Use this pattern | Example |
|---|---|---|
| Hold a conversation | Chatbots and assistants | A support chat widget |
| Turn text into fields or a short summary | Summarising and extracting | Invoice email → order record |
| Find related items or passages | Smart search | “Shoes for a marathon” → running shoes |
| Answer questions from its own content | RAG | A bot that knows the returns policy |
| Put items into fixed categories | Classification and routing | Ticket → billing team |
| Take actions in other systems | Agents with tools | Look up and update an order |
Real products combine patterns. A support assistant might classify each message, answer common questions with RAG, extract order details into a ticket and use a tool to check delivery status — all behind one chat box. Start with the simplest pattern that solves the problem: a single well-prompted call is easier to test, cheaper to run and more predictable than an agent.
Before You Ship: Production Checklist
The patterns above are the core of each feature. Production apps also need these safeguards:
- Keep API keys on the server, in environment variables or a secrets manager, never in frontend code or Git.
- Validate input: limit message length and history size, and accept only the fields you expect.
- Rate-limit each user, and set a spending limit in your provider’s console.
- Handle failures: rate limits, timeouts, server errors and refusals should produce a friendly message, not a crash.
- Stream long replies so users see progress instead of a spinner.
- Control cost: log token usage per feature, cap max_tokens sensibly, cache repeated answers and reuse long system prompts with prompt caching.
- Protect privacy: send only the data the task needs, avoid secrets and unnecessary personal data, and check your provider’s data-retention terms.
- Defend against prompt injection: mark user content and retrieved documents as data, and limit what tools can do.
- Keep a person in the loop for high-stakes actions and decisions.
- Test with a fixed set of real examples before every prompt, model or data change, and keep monitoring after launch.
Frequently Asked Questions
Almost never. Most AI features call a hosted model through an API and shape its behaviour with prompts, examples and your own data (RAG). Training or fine-tuning is only worth considering once a well-prompted hosted model has clearly fallen short.
Any language that can make HTTPS requests works. Python and JavaScript (Node.js) have the most mature SDKs and examples, which is why this guide shows both. Use whatever your backend is already written in.
Hosted models charge per token, so cost depends on how long your prompts and replies are and how many requests you make. A single short request usually costs a fraction of a cent; log token usage from day one and estimate monthly cost from real traffic before launch.
The examples use Anthropic’s official SDKs, but every pattern here — chat, extraction, search, RAG, classification and tool use — works the same way with other providers. The request and response shapes differ slightly; the architecture and the ideas do not.
Build These Yourself
Module 4 of the free AI & ML track walks through each building block step by step — your first API call, a chat app with memory, embeddings, semantic search and RAG — with challenges in every lesson.