2

Class 2: How Chatbots Work

Class 2 of 8About 3 hoursNo coding

By the end of this class you will be able to

• Explain in plain words how a chatbot writes: by breaking text into tokens and predicting the next one, again and again.
• Describe how a chat model is made: pre-training, fine-tuning and human feedback, and what each stage gives it.
• Explain the context window, and why a chatbot only "knows" what is in front of it plus what it learned in training.
• Explain why chatbots make things up and why their answers vary between runs, and what that means for how you use them.
• Tell apart the main kinds of model you'll hear about: reasoning, multimodal, open vs closed and small vs large, and why some languages cost more to process.

Where we are

In Class 1 we placed chat tools on the AI family tree: generative AI built from large language models. Today we look inside and see how they produce their answers, without any maths. In Class 3 we use that understanding to write better prompts.

1. Autocomplete, on a grand scale

You've seen autocomplete on your phone. Type "See you" and it suggests "tomorrow", "soon" or "later". It has seen enough messages to know which words usually come next.

A chatbot works on the same basic idea, taken to an extraordinary scale. At its core, a large language model (LLM) does one thing: given some text, it predicts what comes next. Then it adds that piece to the text and predicts again, and again, one small piece at a time, until the answer is complete.

Suppose the model is continuing this sentence:

"After the floods, the most urgent need in Lomi village is clean…"

It doesn't look the answer up. It calculates how likely each possible next word is, based on patterns learned from an enormous amount of text. Something like:

Possible next word	Likelihood (illustrative)
water	81%
drinking	9%
shelter	3%
sanitation	2%
everything else	5%

It picks one (usually a likely one, as we'll see in Section 7), adds it, and moves on to predicting the word after that.

That's the whole trick. There is no separate "facts" department, no database of true statements it checks against, and no plan for the whole paragraph written in advance. Everything a chatbot does, from summarizing your notes to drafting an email to answering a question, comes out of this repeated prediction.

The one-sentence version

A chatbot is a very sophisticated next-word predictor. It produces the text that is most plausible given what it has been given, and plausible is not the same as true.

"Just predicting the next word" may sound too simple to explain what these tools can do. The surprise of the last few years is how far it goes. To predict the next word in a legal argument, a translation or a worked maths problem well, a model has to capture a great deal about grammar, facts, reasoning and style. Prediction at this scale can look a lot like understanding. But the mechanism explains the tools' weaknesses, and that's what this class is about.

2. Tokens: the pieces the model reads

We've been saying "next word", but models actually work with tokens. A token is a chunk of text: often a whole common word, sometimes part of a word, sometimes a space or punctuation. The word "water" might be one token; "purification" might be split into two or three.

Know the Term: Tokens

A token is the unit of text a language model reads and writes. It is often a whole common word ("water"), sometimes part of a word ("purific" + "ation"), a space, or a punctuation mark. Before the model sees your text, a tokenizer chops it into tokens and turns each one into a number. A rough rule for English: 1 token ≈ ¾ of a word, so 100 words ≈ 130 tokens. Tokenizers are built mostly from English-heavy text, so the same sentence often needs more tokens in languages such as Arabic or Russian, and scripts such as Chinese are counted differently again. Model limits (the context window) and prices are measured in tokens, not words.

Analogy: Lego bricks. The model never sees your sentence as a whole; it sees the bricks it has been broken into, and it builds its answer one brick at a time.

Why you care: tokens explain why long documents hit a limit, why costs can differ between the languages you work in, and why chatbots sometimes stumble on tasks like counting the letters in a word: they never saw the individual letters.

Why does this matter to you? Three reasons.

• Limits are counted in tokens. How much text a model can handle at once, and how long its answer can be, are measured in tokens, not pages.
• Cost is counted in tokens. When your organization pays for AI use, it usually pays per token, both for what you send and what comes back.
• Some odd failures come from tokens. Ask a chatbot how many letters are in a long word, or to spell a word backwards, and it may get it wrong. It never saw the individual letters, only the chunks. (Newer models handle this better, but it's a good reminder of how they see text.)

3. Where the patterns come from: training

A chat model is made in stages, a bit like the training of a new colleague.

Stage 1: Pre-training. The model reads an enormous amount of text (a large share of the public web, books, articles, code) and practises one task billions of times: predict the next token, check against what actually came next, adjust slightly. The adjusting happens in the model's parameters, billions of numbers inside its neural network (see the Class 1 sidebar on neural networks). After weeks of this on thousands of specialised chips, the result is a base model. It has absorbed a vast amount of language, facts and patterns, but it isn't a helpful assistant yet. Ask it a question and it might continue with three more questions, because that's what often follows a question on a web page.

Stage 2: Fine-tuning. The base model is trained further on a much smaller, carefully written set of example conversations: a question and a good, helpful answer. Now it learns to respond like an assistant.

Stage 3: Learning from human feedback. People (and increasingly AI graders following written guidelines) compare pairs of answers and pick the better one. The model is adjusted to produce more answers like the preferred ones. This shapes its tone, its helpfulness, and what it refuses to do.

Know the Term: Pre-training, fine-tuning and human feedback

Chat models are built in stages. Pre-training is the huge first stage: the model reads an enormous amount of text (web pages, books, code) and learns to predict the next token. This takes weeks on thousands of specialised chips and produces a base model that can continue any text but doesn't reliably follow instructions. Fine-tuning (also called instruction tuning or supervised fine-tuning) then trains it further on a much smaller set of example conversations showing good, helpful answers. Finally, learning from human feedback (RLHF, reinforcement learning from human feedback, and related methods) has people, or AI graders following written guidelines, compare pairs of answers; the model is adjusted towards the preferred ones, which shapes its tone, helpfulness and refusals.

Analogy: a new staff member who first reads the whole library (pre-training), then shadows experienced colleagues on real cases (fine-tuning), then gets regular feedback from a supervisor on which of their drafts were better (human feedback).

Why you care: most of what a model "knows" comes from pre-training, fixed at a cut-off date. Its politeness, style and willingness to refuse come from the later stages. And the feedback stage rewards answers people like, which is one reason chatbots can sound confident and agreeable even when they are wrong.

Two things follow from this, and they are worth remembering:

1. The model doesn't learn from your chats as you use it. Training happens before release. When you correct it in a conversation, it adjusts within that conversation only. (Whether your organization's conversations are ever used for future training is a question about your tool's contract and settings. Enterprise tools usually exclude it, but check. Class 8 covers this.)
2. The feedback stage rewards answers that people like. People tend to prefer answers that are confident, complete and agreeable. That's partly why chatbots rarely say "I don't know" and sometimes agree with you too readily.

4. The transformer and attention

How does the model decide what comes next, given everything before it? Almost every modern chat model uses a design called the transformer, introduced by Google researchers in 2017. (It's the "T" in "GPT", which stands for generative pre-trained transformer.)

Its key idea is attention. When the model processes each token, it looks at all the other tokens in the text and works out which ones are most relevant to it. That's how it can keep track of what "it" refers to, twenty words later, or tell that "bank" in a sentence about interest rates means a financial bank.

Know the Term: Transformer and attention

The transformer is the design behind almost every modern large language model. It was introduced in the 2017 research paper "Attention Is All You Need". Its key trick is attention: when the model processes a word, it looks at all the other words in the text and decides which ones matter most for understanding it. In "The bank raised interest rates", attention links "bank" to "interest rates", so the model reads it as a financial bank, not a river bank.

Analogy: a meeting where every participant can listen to everyone at once and decide whose input matters for the point being made.

Why you care: the model can only "pay attention" to what is in the text you give it. That is why the context you provide matters so much.

Before any of this, each token is turned into a long list of numbers that captures something of its meaning, called an embedding. Words with similar meanings get similar numbers. Attention then works on these lists of numbers, layer after layer, gradually building a richer picture of what the whole text means, until the final layer produces the likelihoods for the next token.

Know the Term: Embeddings

An embedding is a list of numbers (a vector, often hundreds or thousands of numbers long) that represents the meaning of a word, sentence or document. A model learns to produce embeddings so that things with similar meanings get similar numbers, and so sit close together in this "meaning space". "Flood", "inundation" and inondation (French) end up near each other; "flood" and "invoice" end up far apart. Inside a language model, every token is turned into an embedding before the model works on it. Separately, embeddings are used to search by meaning: to find the documents closest to a question even when they don't share any of its exact words.

Analogy: a map where every idea has coordinates. Related ideas are neighbours, so to find documents about your question you look at what is nearby on the map.

Why you care: embeddings are how AI tools find the relevant pages in a large document library (see RAG in Class 4) and how search can work across languages.

You don't need to remember the mechanics. The practical point is this: attention works on the text in front of the model. The model is constantly asking "which parts of this text matter for what comes next?" If the important information isn't in the text, attention can't find it.

5. The context window: what the model can see

Everything the model can pay attention to at one moment is called its context, and the maximum amount it can handle is its context window, measured in tokens.

In a chat tool, the context usually includes:

• hidden instructions from the tool provider or your organization (a system prompt, covered in Class 3);
• the whole conversation so far: your messages and its replies;
• any documents or files you've attached, or search results the tool has pulled in.

Every time you send a message, the whole context is sent to the model again, and it predicts its reply from all of it. That's why it can refer back to something you said ten messages ago in the same chat, but knows nothing about a chat you had yesterday.

A desk, not a filing cabinet

Think of the context window as the model's desk. Whatever is on the desk it can read and use. Its training is more like a vague memory of everything it once read, useful but blurry and out of date. It has no filing cabinet where it can look up your documents unless your tool puts them on the desk.

Context windows have grown enormously. Many models can now take hundreds of thousands of tokens, enough for several long reports at once (as of 2026). But bigger isn't magic:

• When the desk is full, something falls off. In a very long conversation, tools may drop or summarize the earliest messages. The model may then "forget" instructions you gave at the start.
• Long contexts get patchy. Models tend to pay most attention to the beginning and end of a long text and can miss details in the middle.
• More text costs more time and money, because every token is processed.

Class 4 is all about putting the right information on the desk.

6. Why chatbots make things up

You may have heard the term hallucination: a chatbot stating something false as if it were fact. A made-up statistic, a report that doesn't exist, a quotation nobody said, a reference with a real-sounding title and invented authors. It is the most important weakness to understand, and now you have everything you need to see why it happens.

1. It predicts plausible text, not true text. If the most plausible-sounding continuation of "The population of Aramu district is…" is a number, the model will produce a number, whether or not it has ever seen one. Veloria is fictional, so any number it gives is invented.
2. It has no built-in fact check. Nothing inside the model compares its sentence with a trusted source before writing it (unless your tool adds a search or document step, and even then it can misread).
3. Its memory of training is blurry. It absorbed patterns, not a neat archive. Rare facts, specific figures, exact dates and names are exactly where the blur is worst.
4. Training rewarded answering. Across its training, a confident, complete answer was usually rewarded more than "I don't know". So it tends to fill gaps rather than admit them.
5. You may have asked for something that doesn't exist. Ask for "the total number of displaced people" when your notes only give households, and it may helpfully produce one.

Hallucinations look exactly like facts

There is no warning light. An invented figure is written in the same fluent, confident style as a correct one. The only defence is to check: anything specific (numbers, names, dates, quotations, references) that will be used in your work must be verified against a real source. Class 6 covers verification habits in depth.

What reduces hallucination (but never removes it)?

• Give it the source. Paste the field notes; don't ask it to recall facts. Summarizing text it can see is far more reliable than recalling from memory.
• Tell it what to do when information is missing. "If it isn't in the notes, write 'not stated'." (Class 3.)
• Ask it to point to where each figure came from, then spot-check.
• Use tools that ground answers in documents or search, with citations you can click (Class 4).

7. Why answers vary

Ask the same question twice and you'll often get two different answers: different wording, sometimes different structure, occasionally a different conclusion. This surprises people who expect computers to be consistent.

Go back to the table in Section 1. At each step the model has a list of possible next words with likelihoods. It doesn't always pick the top one. Most tools pick with a little randomness, like rolling weighted dice: "water" comes up most often, but sometimes "drinking" does, and from that point the rest of the sentence goes a different way.

How much randomness is controlled by a setting called temperature.

• Low temperature: the model almost always picks the most likely word. Answers are more predictable and repetitive.
• High temperature: less likely words get picked more often. Answers are more varied and creative, and more likely to drift or go wrong.

In most enterprise chat tools you can't set the temperature directly; the provider chooses a middle setting. (Some tools offer "more creative / more precise" styles, which have a similar effect.) Even at the lowest setting, answers can still vary slightly.

What this means for you:

• One answer is a sample, not the answer. If a result matters, try it twice. If the two answers disagree on a fact, that's a strong sign the model is unsure, and you should check.
• Variation is useful for creative work. Ask for three options; you'll get genuinely different ones.
• Variation is a problem for repeatable work. For a weekly SitRep, you want consistency. That comes from a clear template and format rules (Class 3), not from hoping the model says the same thing twice.

8. What the model knows, and when it stopped learning

Because pre-training happens before release, a model's built-in knowledge stops at a knowledge cutoff date. It doesn't know about events after that date unless your tool gives it search results or documents. Ask about something recent without search and it may say it doesn't know, or it may confidently describe an out-of-date situation. Class 4 covers knowledge cutoff and how tools work around it.

9. Not all models are the same

"The AI" isn't one thing. Behind your chat tool there is at least one model, often several, and you'll hear several distinctions as you read about AI. Here are the four that matter most for non-technical colleagues (as of 2026).

Reasoning models "think before answering". They produce a long internal working-out before their reply, which helps with multi-step problems like reconciling figures from several teams. They're slower and cost more.

Know the Term: Reasoning models

A reasoning model is a language model trained to "think before it answers". Before writing its final reply, it generates a long internal working-out (sometimes called thinking or a reasoning trace): breaking the problem into steps, trying approaches and checking its own work. It learned this habit through extra training that rewards correct final answers on problems that can be checked, such as maths, logic and code. Many tools now offer a "thinking" or "reasoning" mode, or switch it on automatically for harder questions (as of 2026). Reasoning takes more time and more tokens, so it costs more.

Analogy: the difference between answering a colleague off the top of your head and saying "give me ten minutes with a notepad".

Why you care: use reasoning modes for multi-step analysis, such as reconciling figures from several reports; don't bother for a quick rewrite. Reasoning reduces some errors but doesn't remove them, and a long, tidy chain of thought can still end in a wrong answer.

Multimodal models work with more than text: photos, scanned documents, charts, audio. You can give them a photo of a handwritten list or a voice note.

Know the Term: Multimodal models

A multimodal model can work with more than one kind of input or output ("modality"): text, images, audio, video, or documents such as PDFs and scanned forms. It turns each kind of input into the same internal form (tokens and embeddings), so it can, for example, read a photo of a handwritten distribution list, describe a chart, transcribe a voice note, or answer a spoken question aloud. Most leading chat tools are multimodal (as of 2026), though which modalities are switched on depends on your organization's tool and settings.

Analogy: a colleague who can read the report, look at the photos attached to it and listen to the voicemail, rather than one who only reads typed text.

Why you care: photos, scans and recordings from the field can now go straight into the AI tool. But reading an image is another place for errors (a misread "7" for "1"), and images and recordings can contain faces, voices and locations that need the same data protection as text.

Open vs closed models. Some models can only be used through their maker's service; others have their trained weights published so organizations can run them on their own computers.

Know the Term: Open vs closed models

A closed (or proprietary) model is used only through its maker's service or partners: you send your text over the internet and get an answer back, but you never get the model itself. Examples include the GPT, Claude and Gemini families (as of 2026). An open-weight model has its weights (the trained numbers that make up the model) published for download, so an organization can run it on its own servers or even a laptop, and adapt it. Examples include the Llama, Mistral, Qwen and DeepSeek families (as of 2026). "Open weights" is not always "open source": the training data and code are often not released, and licences can restrict use. Closed models are often the most capable and the easiest to use; open models give more control over where data goes and what it costs.

Analogy: a closed model is like a restaurant: you get the meal, never the kitchen. An open-weight model is like a ready-made dish you can take home, reheat in your own kitchen and season to taste, though you still don't get the full recipe.

Why you care: the choice affects where your data is processed, who can see it, cost and control. For sensitive organizational data, being able to run a model inside your own infrastructure can matter as much as raw capability.

Small vs large models. Bigger models are generally more capable; smaller ones are faster, cheaper and can run locally.

Know the Term: Small vs large models

Model size is usually measured in parameters (the adjustable weights learned in training). Large models have hundreds of billions of parameters or more and run in big data centres; they are generally better at difficult reasoning, rare topics and nuanced writing, but are slower and cost more per token. Small models (often called small language models, roughly a few billion parameters or fewer) are faster and cheaper, and some can run on a laptop or phone without an internet connection. Providers often offer a family in several sizes (for example a "mini" or "flash" tier alongside a flagship, as of 2026). For narrow, well-defined tasks, such as sorting messages into categories or pulling dates out of text, a small model is often good enough.

Analogy: a team of fast junior assistants vs one senior expert. The juniors are cheaper and quicker for routine work; you call the expert for the hard cases.

Why you care: bigger is not always better. Matching the model to the task saves money, energy and waiting time, and a small model running locally can keep sensitive data from leaving your organization.

If you need…	Consider…
A careful multi-step analysis	A reasoning mode, if your tool offers it
To work from a photo, scan or recording	A multimodal tool, if it's approved for that data
Data to stay inside your organization	An approved internally hosted model (often open-weight)
A quick rewrite or simple sorting task	A smaller, faster model is often enough

You usually don't choose these yourself. Your organization chooses which tools and models are approved. But knowing the differences helps you understand why two tools behave differently, and ask good questions when a new one is proposed.

10. Languages: why tokens cost more in some

Many organizations work in several languages. Take six widely used ones: Arabic, Chinese, English, French, Russian and Spanish. Chat models can work in all six, but not always equally well, and not at the same cost.

Remember that tokenizers chop text into chunks learned mostly from English-heavy data. Common English words tend to be single tokens. Words in other languages, especially those with different scripts or rich word endings, are often split into more pieces. So the same sentence can need noticeably more tokens in Arabic or Russian than in English, sometimes one and a half to two times as many or more, depending on the model's tokenizer. Chinese characters are counted differently again. Newer tokenizers have narrowed the gap, but it hasn't disappeared (as of 2026).

That has practical effects:

• Cost: the same SitRep may cost more to process in one language than another.
• Limits: a context window holds fewer pages of text in a language that needs more tokens.
• Quality: languages that were less common in the training data may get weaker answers: less fluent phrasing, more errors in names and figures, or English turns of phrase slipping in. Major world languages like these six are generally well supported, but many smaller and local languages often are not.

In practice

If you work across languages, check important output in each language with a fluent speaker. Don't assume that because the English answer was good, the Arabic or Russian one is equally good.

Prompt Lab

Work in pairs using your organization's approved AI chat tool. Today's exercises are explorations: you're not trying to get a good SitRep, you're trying to see how the machine behaves, including how it breaks. One partner types; the other keeps notes on anything surprising. Swap each exercise.

Exercise 1: Be the model: predict the next word

Task: First, without the AI, each partner writes down the next three words of the sentence below. Compare. Then ask the AI.

Continue this sentence with exactly the next three words, and then list five other three-word continuations you considered, from most to least likely:

"After the floods, the most urgent need in Lomi village is"

What to look for:

• Did you and your partner predict similar words? Why? (You've both read a lot of similar text; so has the model.)
• Are the model's alternatives all plausible? Are any of them true? It has no way of knowing what Lomi actually needs.
• Note: the model can't truly report its own internal likelihoods; its "list of alternatives" is itself a prediction. It's an illustration of the idea, not a window into the machine.

Exercise 2: Same question, different answers

Task: Send the prompt below. Then open a new chat and send exactly the same prompt. Do it a third time if you have time. Compare the answers.

Write a two-sentence headline summary for a situation report about flooding in Aramu district, Northern Veloria, where about 1,200 households have been displaced and clean water is the most urgent need.

What to look for:

• How different are the three versions in wording? In structure?
• Did any version add a fact that wasn't in the prompt (a number of people, a date, an organization)? Did different versions add different invented facts? That's variation and hallucination together.
• If you needed the same format every week, what would you add to the prompt? (Class 3 will answer this.)

Exercise 3: Break the model

Task: This is an open exploration. Your goal is to find where the model fails. Try the prompts below one at a time (new chat for each), then invent your own. Keep a list of what broke and why you think it broke, using the ideas from this class.

1. How many times does the letter "r" appear in "strawberry refrigerator"?
   Then write "humanitarian" backwards.

2. List three published reports about the 2025 Tula river floods in
   Northern Veloria, with authors, titles and dates.

3. What is 48,317 multiplied by 7,926? Answer with just the number.

4. A field team visited Lomi and Tavet. Another team visited Tavet and
   Dorra. The first counted 400 households, the second 250. How many households are there in total?

5. Is it true that the Veloria Relief Coalition won an award in 2024?
   Just answer yes or no.

What to look for:

• Prompt 1: the answers are 7 and "nairatinamuh". Mistakes here come from tokens: the model sees chunks, not letters.
• Prompt 2: Veloria is fictional. Did it invent reports with realistic authors and titles, or say it could find none? This is the classic hallucinated reference.
• Prompt 3: check with a calculator (the answer is 382,960,542). Exact arithmetic is a job for a normal program. Did a reasoning mode, if you have one, do better?
• Prompt 4: did it just add 400 + 250, or notice that Tavet may be double-counted and the total can't be known from this? This is a real SitRep trap.
• Prompt 5: a yes/no question with no basis for an answer. Did it guess? Did it say it couldn't know? Ask again in a new chat. Do you get the same answer?
• Your own: try a trick question, a question with a false assumption built in ("Why did the Veloria Relief Coalition close its Aramu office in 2021?") or a request in another language. Bring your best "break" to the debrief.

Exercise 4: What's on the desk?

Task: Test the context window. In one chat, give the AI an instruction and a fact, have a short unrelated exchange, then check whether it still uses them. Then open a new chat and ask again.

For this conversation: always end your replies with the word "ARAMU" in capitals. Also, remember that the next food truck is due on Tuesday. Reply "OK" to confirm.

Then send two or three unrelated questions (for example, "Suggest three names for a hygiene promotion campaign"), and finally:

When is the next food truck due?

Now open a new chat and ask only: "When is the next food truck due?"

What to look for:

• In the first chat, did it keep ending with "ARAMU"? Did it remember Tuesday? Both are on its desk.
• In the new chat, did it say it doesn't know, or did it guess a day? Nothing from the first chat is on the new desk (unless your tool has a memory feature switched on).
• What does this tell you about repeating key context when you start a new chat?

Exercise 5: Six languages

Task: Translate the same SitRep headline into all six languages from Section 10, then compare.

Translate this situation report headline into Arabic, Chinese, French, Russian and Spanish. Then, for each language, tell me roughly how many tokens you think the translation uses compared with the English, and explain why the numbers differ.

"About 1,200 households displaced in Aramu district; clean water and oral rehydration salts are the most urgent needs."

What to look for:

• If anyone in your group reads one of the languages, check the translation. Are names (Aramu), numbers and technical terms such as "oral rehydration salts" handled correctly?
• Does the model's explanation of token differences match Section 10?
• Can the model count its own tokens accurately? Usually not. It's predicting an answer about itself. If your facilitator shows a tokenizer tool, compare the real counts.
• What would you check before sending an AI translation of a SitRep to partners?

Check Your Understanding

1 At its core, what does a large language model do when it writes an answer?

1. It searches a database of verified facts and copies out the matching entry.
2. It plans the whole answer in advance, then types it out.
3. It predicts the most plausible next token, adds it to the text, and repeats.
4. It understands the question the way a person does and looks up the answer in its memory.

2 You ask a chat tool for the population of Aramu district (a fictional place) and it gives a precise figure. Why did this happen?

1. It predicts plausible text, and a number is the plausible continuation; it has no built-in way to check whether the number is true.
2. It is lying on purpose to seem more helpful.
3. It found an outdated census online.
4. It confused Aramu with a real district of the same name, so the number is probably close.

3 You send the same SitRep headline prompt in two new chats and get two different headlines. What is the best explanation?

1. The model learned from the first chat and improved its answer.
2. The tool is faulty; a working computer always gives the same output.
3. The provider updated the model between your two messages.
4. At each step the model picks the next token with some randomness (controlled by temperature), so the answers diverge.

4 Yesterday you corrected the chatbot: "Our reports say households, not families." Today, in a new chat, it says "families" again. Why?

1. It is ignoring you because the correction wasn't polite enough.
2. The model's training happened before release; your correction only applied inside yesterday's conversation.
3. It needs several days to absorb corrections into its training.
4. It remembered the correction but decided it was wrong.

5 In a very long chat, the AI stops following an instruction you gave at the start ("mark every estimate with ~"). What is the most likely reason?

1. The AI got tired after so many messages.
2. The instruction was automatically cancelled after one hour.
3. It has decided it knows better than you.
4. The context window filled up, so early messages were dropped or summarized, or the instruction got lost in a long context.

6 Your team processes the same SitRep in English and in Arabic. The Arabic version uses more of the tool's budget. Why?

1. The tokenizer was built mostly from English-heavy text, so the same content is split into more tokens in Arabic, and cost is counted in tokens.
2. Providers charge extra for every language other than English as a policy.
3. Arabic reports are always longer than English ones.
4. Models translate everything into English first, which doubles the cost.

7 Your office wants to use AI on sensitive beneficiary data, which must not leave the organization's own infrastructure. Which kind of model fits best?

1. The largest model available, because bigger models are safer with data.
2. A reasoning model, because thinking before answering protects the data.
3. An approved open-weight model run on the organization's own servers.
4. Any closed model, because closed means the data is kept closed.

Exit Ticket

Before you leave

In one or two sentences, explain to a colleague who has never used AI why a chatbot can give a confident, wrong figure. Use the words "predict" and "plausible".

Key terms

• Large language model (LLM): a model trained on huge amounts of text to predict the next token; the engine of chat tools.
• Next-token prediction: the core mechanism: given text, predict the most plausible next piece, add it, repeat.
• Token: a chunk of text (a word, part of a word or punctuation) that a model reads and writes; limits and costs are counted in tokens.
• Tokenizer: the component that splits text into tokens.
• Parameters (weights): the billions of numbers inside a model that are adjusted in training.
• Pre-training: the first, largest training stage, in which the model learns to predict text from an enormous collection of documents.
• Base model: a pre-trained model before it has been tuned to act as an assistant.
• Fine-tuning: further training on a smaller, targeted set of examples.
• RLHF (reinforcement learning from human feedback): adjusting a model towards answers that people rate as better.
• Transformer: the model design behind almost all modern LLMs.
• Attention: the transformer's way of working out which parts of the text matter most for each token.
• Embedding: a list of numbers representing the meaning of a piece of text.
• Context / context window: the text the model can see at once, and its maximum size in tokens.
• Hallucination: a confident statement from an AI that is false or invented.
• Temperature: a setting that controls how much randomness goes into choosing each next token.
• Knowledge cutoff: the date after which a model has no built-in knowledge.
• Reasoning model: a model trained to work through a problem internally before answering.
• Multimodal model: a model that handles more than one kind of input or output, such as text, images and audio.
• Open-weight model: a model whose trained weights are published so others can run it.
• Closed (proprietary) model: a model available only through its maker's service.
• Small language model: a model with relatively few parameters, faster and cheaper to run.