What Is an LLM? How AI Reads Your Prompt
Learn what a large language model actually does with your prompt: tokens, the context window, and why the exact wording you use changes the answer you get.
On this page 8 sections
Introduction
Chat assistants such as ChatGPT, Claude and Gemini feel like they understand you. Under the hood, each one is a large language model (LLM) doing something surprisingly simple: reading your text and predicting what text should come next, one small piece at a time. Once you understand that, “prompting” stops being a bag of magic phrases and becomes a skill you can reason about.
This first lesson of our free prompt engineering course explains what an LLM is, how it breaks your prompt into tokens, how much it can “see” at once, and why small changes in wording lead to very different answers. Every later lesson builds on these ideas.
- What a large language model is and how it was trained.
- What tokens are, and why they matter for cost and limits.
- What the context window is, and why the model “forgets”.
- Why precise wording produces better answers.
What Is a Large Language Model?
A large language model is a neural network trained on an enormous amount of text — books, websites, code and more. During training it sees billions of passages and repeatedly practises one task: given the text so far, predict the next piece. Getting good at that single task forces the model to pick up grammar, facts, writing styles, how code is structured and a great deal of general knowledge along the way.
After this first stage, assistant models are trained further on examples of helpful conversations, so that instead of just continuing any text, they respond to instructions the way a helpful assistant would. That is why you can write “Summarise this email in two sentences” and get a summary rather than a random continuation of your sentence.
The important point for prompting: the model does not look things up or run a hidden program when you ask a question. It generates the answer word by word, choosing what is most likely to fit everything it has been given. Your prompt is the main thing steering that choice.
Tokens: How the Model Sees Text
Models do not read letters or whole words. Text is first split into tokens — common words, parts of words, punctuation or spaces. A short common word such as “cat” is usually one token, while a long or rare word may be split into several pieces. Numbers, code and non-English text often use more tokens than you would expect.
A useful rule of thumb for English is that one token is roughly three-quarters of a word, so 100 tokens is about 75 words. Tokens matter in three practical ways:
- Cost: paid AI APIs charge per token, for both the text you send and the text you get back.
- Limits: every model can only handle a certain number of tokens at once (the context window, below).
- Odd mistakes: because the model sees chunks rather than letters, tasks like counting the letters in a word or reversing a string can trip it up.
Using the rule of thumb from this lesson, roughly how many tokens is a 750-word English article?
The Context Window
The context window is everything the model can see while writing its reply, measured in tokens. It includes any hidden instructions the app adds, the conversation so far, any documents you pasted, your latest message, and the reply being written. Modern models have large windows — often hundreds of thousands of tokens — but it is still a fixed limit.
Two consequences surprise beginners. First, the model has no memory outside the context window. In a chat app it seems to remember earlier messages only because the app sends the whole conversation again with every new message. Start a new chat and, unless the app has a separate memory feature, the model knows nothing about the previous one. Second, a very long context is not read equally well everywhere: important instructions buried in the middle of a huge paste can get less attention than those at the start or end. Keep prompts focused and put key instructions where they are easy to find.
You start a brand-new chat (with no memory feature turned on) and ask: “What name did I tell you yesterday?” What is the model most likely to do?
Why Wording Matters
Because the model predicts the most fitting continuation of everything in its context, the details you give shape the answer. A vague request leaves the model to guess your purpose, your audience and the length you want — so it falls back on a safe, generic answer. A specific request narrows down what “fitting” means.
Tell me about Python.I'm a beginner choosing my first programming language.In 5 bullet points, explain what Python is mainly used forand why beginners find it easy. Avoid jargon.The second prompt states who you are, what you want to know, the format and the reading level. None of these words are magic — they simply remove guesswork. Also remember that the output is sampled from likely options, so the same prompt can produce slightly different answers each time. Clear prompts make those answers consistently useful.
A student wants help preparing for a job interview. Which rewrite of their prompt is most likely to get a useful answer?
Give me interview questions.
What LLMs Cannot Do
- Know recent events: a model only knows what was in its training data, which stops at a cut-off date, unless the app gives it a search tool.
- Guarantee facts: it can produce fluent, confident text that is wrong — called a hallucination. Lesson 9 covers how to check.
- See your screen, files or previous chats unless that content is placed in its context.
- Reliably do precise counting or long arithmetic in its head; ask it to show steps or use a calculator tool.
Key Takeaways
- An LLM generates text by repeatedly predicting the next token from everything in its context.
- Tokens are word pieces; about 100 tokens ≈ 75 English words. They drive cost and limits.
- The context window is all the model can see — it has no memory beyond it.
- Specific wording removes guesswork and produces more useful, consistent answers.
Summary
You now know what an LLM is doing with your prompt: splitting it into tokens, reading everything in its context window, and predicting a fitting reply. In the next lesson you will turn that understanding into a practical structure for writing clear prompts: task, context, format and constraints.