3

Class 3: Building with Code and AI Pair Programming

Class 3 of 8About 3 hoursPythonLayer: interface

By the end of this class you will be able to

• Call a model from Python through the messages API, using LiteLLM as a provider-neutral layer, and read the parts of a response that matter (text, usage, stop reason).
• Make a model call robust: timeouts, retries with exponential backoff for transient errors only, and a clear failure when retries run out.
• Track tokens and cost per run and enforce a budget in code.
• Work with AI coding assistants and coding agents using a spec-first loop: a written spec, tests as the contract, small steps.
• Review AI-written code with a checklist covering correctness, security, secrets, dependencies and edge cases.
• Judge when "vibe coding" is acceptable and when a task needs spec-driven development, and keep a project's hygiene (secrets, .gitignore, pinned dependencies) in order.

Where we are

In Class 2 you wrote a SitRep prompt that returns validated, structured output, and tried it in a notebook. Today you turn it into software: a small command-line app that calls a model, survives the failures real APIs produce, counts what it spends, and is covered by tests. You will also learn to build that software with an AI coding assistant without losing control of it. In Class 4 the app learns to answer from a document library.

1. From the chat window to code: the messages API

When you use a chat tool, a lot happens that you never see. The tool keeps the conversation, adds its own instructions, retries when the service is busy, stops you when you have used too much, and shows the answer as it is written. When you call a model from code, all of that becomes your job. That is what this class is about.

Think of the difference between eating in a restaurant and ordering wholesale from the kitchen's supplier. In the restaurant, a waiter handles everything. With the supplier you place a precise order in their format, you check the delivery, you chase late orders, and you pay per kilo. An API (application programming interface) is that supplier's order form: a fixed format your program uses to send a request and receive a reply.

Almost every model provider now uses the same basic shape, the messages API. You send a list of messages, each with a role and some content:

messages = [
    {"role": "system",    "content": "You summarize humanitarian field reports..."},
    {"role": "user",      "content": "Summarize these reports: <reports>...</reports>"},
]

• system: standing instructions for the whole conversation (Class 2's system prompt).
• user: what the person (or your program, acting for them) says.
• assistant: what the model said earlier. You include past assistant turns when you want a multi-turn conversation.

The reply contains more than text. The fields you will use most are:

Field	What it tells you	Why you care
The content	The generated text	Your output, to be validated before use
The stop reason (finish_reason)	Why generation stopped: finished normally, hit the length limit, called a tool, or was filtered	A length stop means the answer is truncated; a filtered answer may be empty
Usage	Input (prompt) tokens and output (completion) tokens	This is what you pay for
Model	The exact model version that answered	Needed for reproducibility and cost

One property surprises almost everyone: the API is stateless. The model remembers nothing between calls. A "conversation" is an illusion created by your code sending the whole history again with every request. That has three consequences you will meet all through this course: long conversations cost more with every turn, the history is yours to store and protect, and whatever you leave out, the model does not know.

Know the Term: Tokens

A token is the unit of text a language model reads and writes. It is often a whole common word ("water"), sometimes part of a word ("purific" + "ation"), a space, or a punctuation mark. Before the model sees your text, a tokenizer chops it into tokens and turns each one into a number. A rough rule for English: 1 token ≈ ¾ of a word, so 100 words ≈ 130 tokens. Tokenizers are built mostly from English-heavy text, so the same sentence often needs more tokens in languages such as Arabic or Russian, and scripts such as Chinese are counted differently again. Model limits (the context window) and prices are measured in tokens, not words.

Analogy: Lego bricks. The model never sees your sentence as a whole; it sees the bricks it has been broken into, and it builds its answer one brick at a time.

Why you care: tokens explain why long documents hit a limit, why costs can differ between the languages you work in, and why chatbots sometimes stumble on tasks like counting the letters in a word: they never saw the individual letters.

2. LiteLLM: one interface for many providers

Each provider publishes its own SDK (software development kit: a library that wraps the API for a particular language), and they differ in small, annoying ways: parameter names, error classes, how usage is reported. If your SitRep code calls one provider's SDK directly, moving to another provider, or running a cheaper model for one step, means rewriting it.

LiteLLM is an open-source Python library that puts one interface in front of more than a hundred providers. You write the OpenAI-style call once, and the provider is chosen by a prefix in the model string:

import litellm

response = litellm.completion(
    model="azure/sitrep-gpt",               # or "openai/gpt-4o-mini", "anthropic/claude-sonnet-4-5"
    messages=messages,
    temperature=0,                           # as repeatable as the model allows
    max_tokens=600,                          # a ceiling on output length (and cost)
    timeout=60,                              # seconds; never wait forever
)
text   = response.choices[0].message.content
reason = response.choices[0].finish_reason   # "stop", "length", ...
usage  = response.usage                      # prompt_tokens, completion_tokens
print(litellm.completion_cost(completion_response=response))  # USD, from LiteLLM's price table

Model names are as of 2026; check your provider. The keys come from environment variables (AZURE_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY), never from your code (Section 10).

A provider's own SDK is also fine, and many organizations route all calls through an AI gateway that centrally handles keys, logging, quotas and cost. Either way, keep the provider behind one small function of your own. In the labs that function has this type:

LLM = Callable[[List[Dict[str, str]]], str]   # messages in, text out

Anything with that shape can be plugged in: a LiteLLM call, a different provider, a scripted fake for tests, a wrapper that adds retries, logging or cost tracking. That one line is what makes the rest of this class testable.

3. Streaming

A 300-word SitRep takes a model several seconds to write. With a normal call, the user stares at nothing until the whole answer arrives. With streaming, the provider sends the answer in small pieces as they are generated, and your interface shows them straight away:

stream = litellm.completion(model=model, messages=messages, stream=True, timeout=60)
parts = []
for chunk in stream:
    piece = chunk.choices[0].delta.content or ""
    print(piece, end="", flush=True)        # the user sees text appear immediately
    parts.append(piece)
full_text = "".join(parts)                  # validate THIS, after the stream ends

Streaming makes the app feel faster; it does not make it cheaper or the answer better. It also changes three things you must handle:

• Validation happens at the end. You cannot check a JSON schema or a word limit until the last piece arrives, so don't act on partial output. Show it, then validate it.
• Failures can happen half-way. A stream can break after 200 words. Decide in your spec whether to retry from scratch, or keep the partial text and say so clearly.
• Usage arrives last, if at all. Some providers only report tokens in the final chunk, and only when you ask for it.

Use streaming where a human is watching; don't bother in an overnight batch job.

4. When calls fail: timeouts, rate limits and retries

A model API is a busy service on the other side of the internet. In any day of real use you will see all of these:

Failure	Typical signal	Retry?
Rate limit: too many requests or tokens per minute for your quota	HTTP 429	Yes, after waiting
Timeout: no answer within your limit	Timeout exception, HTTP 408	Yes
Server error / overloaded	HTTP 500, 502, 503, 504 (some providers use 529)	Yes
Connection error	Network exception	Yes
Bad request: malformed messages, context too long	HTTP 400	No: fix the request
Authentication / permission	HTTP 401, 403	No: fix the key or access
Content filtered	Provider-specific error or stop reason	No: retrying the same input gets the same result

The first four are transient: the same request may well succeed a few seconds later. The last three are permanent: retrying wastes time and money and hides a bug.

A timeout is the limit you set on how long to wait. Without one, a hung connection freezes your app indefinitely. With one, a hang becomes an ordinary, retryable error. Choose it from what you observe: if a normal SitRep call takes 8 seconds, 60 seconds is generous.

Retries with exponential backoff. When a call fails with a transient error, wait and try again, doubling the wait each time: 1 s, 2 s, 4 s, 8 s… up to a cap. Doubling gives an overloaded service room to recover. Add jitter, a small random extra delay, for a reason anyone who has phoned a busy office understands: if everyone whose call dropped redials at exactly the same second, the line is busy again for all of them. Jitter spreads the redials out.

Here is the lab's version. Every line maps to a requirement in the lab's spec:

def call_with_retry(llm, messages, retries=3, backoff=1.0, max_backoff=30.0,
                    jitter=0.1, sleep=None, rand=random.random):
    sleep = sleep or time.sleep
    attempt = 0
    while True:
        try:
            return llm(messages)
        except Exception as exc:
            if not is_transient(exc):
                raise                                   # 400, 401, content filter: don't retry
            if attempt >= retries:
                raise RetryError(attempt + 1, exc) from exc   # give up loudly
            delay = min(max_backoff, backoff * (2 ** attempt))
            sleep(delay * (1 + jitter * rand()))        # 1 s, 2 s, 4 s ... plus up to 10%
            attempt += 1

Four details are worth copying into every project:

1. sleep is injectable. The tests pass a fake that records delays instead of waiting, so "retry three times with 1, 2 and 4 second waits" is tested in a millisecond. Any time you write code that depends on the clock, randomness or the network, make that dependency a parameter.
2. Only transient errors are retried. is_transient() in the lab's llm.py recognizes rate limits, timeouts, connection problems and 5xx errors by class name or status code, so the retry logic does not care which provider is behind it.
3. Giving up is an explicit outcome. RetryError says how many attempts were made and chains the last error (from exc), so the log shows why.
4. Retries are a loop, so they need a budget. Three retries is typical for interactive use. Beware of stacked retries: if your code retries 3 times, and the SDK underneath also retries 3 times, and a gateway retries 3 times, one failing request can become 64 attempts. Pick one layer to own retries (LiteLLM's num_retries and the official SDKs' max_retries are other valid choices) and switch the others off.

A retry is only safe when repeating the request does no harm. Asking for a summary twice is harmless. Calling a tool that sends an email twice is not. Class 5 returns to this under the name idempotency: an action is idempotent if doing it twice has the same effect as doing it once.

5. Tracking tokens and cost

Providers charge per token, and input and output tokens are priced differently: output is typically three to five times more expensive per token. The cost of one call is:

cost = input_tokens  × price_per_input_token
     + output_tokens × price_per_output_token

Prices are usually quoted per million tokens. A worked example with illustrative prices of USD 0.15 per million input tokens and USD 0.60 per million output tokens:

Item	Tokens	Cost (USD)
One SitRep summary: 2,000 tokens in, 400 out	2,400	0.0003 + 0.00024 = 0.00054
40 summaries a day for a year	≈ 35 million	≈ 7.90
The same with a large model at 20× the price		≈ 158
The same with a runaway loop that makes 50 calls per summary		≈ 395 at the small model's price

The numbers are small until something multiplies them: a bigger model, a longer prompt, a retry storm, or an agent loop without a budget (Class 6). That is why cost tracking is part of the app, not an afterthought on the monthly invoice.

The lab's CostTracker accumulates calls, input tokens, output tokens and cost across a run, and enforces an optional budget before each call:

tracker = CostTracker(price_in_per_mtok=0.15, price_out_per_mtok=0.60, max_cost_usd=0.01)
llm = litellm_llm(model, on_usage=tracker.record)    # exact counts reported by the provider
summary = summarize(llm, reports, tracker=tracker)   # raises BudgetExceeded when spent
print(tracker.summary())   # calls=1 input_tokens=498 output_tokens=131 cost_usd=0.000153

Use the provider's reported usage whenever you can. When you can't (a fake in tests, or a provider that doesn't report it), the lab estimates about four characters per token. That rule of thumb holds for English. Text in Arabic, Chinese or Russian usually needs noticeably more tokens for the same meaning, so a SitRep produced in six languages (say Arabic, Chinese, English, French, Russian and Spanish) costs more than six times the English one. Measure it with your provider's token counter before you budget.

6. AI coding assistants and coding agents

You will write the rest of this course's code with an AI assistant beside you. The tools have changed fast, and as of 2026 they come in roughly four forms:

Form	What it does	Examples (as of 2026)
Autocomplete	Suggests the next lines as you type	GitHub Copilot, Cursor Tab, and completion built into most editors
Chat in the editor	Answers questions about your code; drafts functions you paste in	Copilot Chat, Cursor, JetBrains AI Assistant, and similar
Agent in the editor or terminal	Given a task, reads the project, edits several files, runs tests and commands, iterates	Claude Code, Copilot agent mode, Cursor Agent, OpenAI Codex CLI, Gemini CLI
Background / cloud agent	Takes an issue, works in its own copy of the repository, and opens a pull request for review	Copilot coding agent, Codex cloud, Claude Code on the web, and similar

Know the Term: Coding agents

A coding agent is an AI assistant that does more than suggest the next line of code. Given a task in plain language, it reads the files in a project, plans changes, edits several files, runs commands such as the tests, reads the results and tries again until the task is done or it gets stuck. Examples as of 2026 include Claude Code, GitHub Copilot's agent mode, Cursor's agent and OpenAI's Codex. The simpler kind of assistant, autocomplete, only suggests the next few lines while you type and leaves every decision to you.

Analogy: autocomplete is a colleague finishing your sentences; a coding agent is a junior developer you hand a ticket to, who comes back later with a set of changes for you to review.

Why you care: a coding agent can produce in minutes what used to take a day, but you are still accountable for every line it writes. Clear specifications, tests and careful review are what make its speed safe.

Moving down the table, the tool does more work between your decisions. That is the point, and the risk: an agent can change twelve files, install a package and rewrite a test in the time it takes you to read one function.

Two organizational points come before any technique:

• Use only tools your organization has approved, as approved. Coding assistants send your code, and often nearby files, to a provider. Check which data classifications may leave your machine, and whether your agreement excludes your data from training.
• Never paste real data or secrets into an assistant. Use fictional samples such as the Northern Veloria reports. An API key in a prompt is a leaked key.

7. Spec-first development: tests as the contract

The most reliable way to work with a coding agent is the same way you would brief a capable contractor: write down what "done" means before the work starts. Think of the terms of reference you would write for a consultant: the purpose, the deliverables, what is out of scope, and how the work will be accepted. A spec (specification) is that document for software, and tests are its acceptance criteria written as code.

The loop:

The lab's SPEC.md is a real example. An extract:

R3 · Retries. Retry transient errors (rate limits, timeouts, connection and 5xx errors) up to `retries` times (default 3) with exponential backoff: 1 s, 2 s, 4 s … capped at 30 s, plus up to 10% random jitter. Never retry other errors (bad request, authentication, content filter): they propagate immediately.

R4 · Give up loudly. When retries are exhausted, raise `RetryError` naming the number of attempts and chaining the last error. The CLI prints the error and exits with code 3.

and the test that turns R4 into a contract:

def test_gives_up_after_n_retries():
    llm = FakeLLM([TransientError(f"503 #{i}") for i in range(4)])
    sleep = RecordingSleep()
    with pytest.raises(RetryError, match="4 attempts") as info:
        call_with_retry(llm, MESSAGES, retries=3, backoff=0.5, jitter=0, sleep=sleep)
    assert len(llm.calls) == 4                      # 1 attempt + 3 retries, then stop
    assert sleep.delays == [0.5, 1.0, 2.0]          # no sleep after the final failure
    assert isinstance(info.value.__cause__, TransientError)

Why this works so well with AI assistants:

• Tests are an objective finish line. "Make pytest pass" is a goal an agent can check by itself, over and over, without asking you. "Make it robust" is not.
• Small steps keep the diff reviewable. A 30-line change you can read properly beats a 600-line change you skim. If a step goes wrong, you throw away minutes, not hours.
• The spec survives the conversation. Chat context is lost when the session ends; the spec and tests stay in the repository for the next person, and the next agent.
• The spec is where the thinking is. Deciding that a 401 must not be retried is an engineering judgement. Once it is written down, typing the code is the easy part.

Guard the contract. Agents under pressure to make tests pass will sometimes change the tests instead of the code: loosening an assertion, adding a skip, or deleting the failing case. Say explicitly in your instructions that tests may not be modified without your approval, and check the diff of test files first when you review.

Give the agent project context. As of 2026, most coding agents read a project instructions file automatically: AGENTS.md (a cross-tool convention), CLAUDE.md, or .github/copilot-instructions.md, depending on the tool. Put the stable facts there: how to run the tests, the coding style, "Python 3.9 compatible", "never add dependencies without asking", "all data in this repo is fictional". It is a system prompt for your codebase.

8. Reviewing AI-written code

AI-written code is usually fluent, well formatted and plausibly commented, which makes it easy to trust and easy to skim. Review it as you would review a pull request from a fast, confident new colleague who has never seen your security policy. You are the author of record: if it ships, it is your code.

Check	Ask	Typical AI mistake
Correctness	Does it do what the spec says, including the numbers? Do all tests pass, run by you?	Off-by-one retries (3 attempts instead of 3 retries); the "happy path" works, the rest is guessed
Tests	Were tests changed? Do new tests assert something real?	Weakened or deleted assertions; tests that only check "no exception"
Error handling	Are errors specific, reported and surfaced?	except Exception: pass; retrying every error, including 401
Security	Is any input trusted that shouldn't be? Any eval, shell calls, file paths from user input?	Building shell commands or SQL from strings; turning off TLS verification "to fix an error"
Secrets	Any key, token or password in code, logs, error messages or tests?	Printing the config "for debugging", including the key; a hard-coded key in an example
Dependencies	Is every new package necessary, real, maintained, licensed acceptably and pinned?	Adding a heavy library for one function; a package name that doesn't exist
Edge cases	Empty input, huge input, non-English text, missing fields, timeouts?	Nothing handles empty reports or a 2 MB file
Cost and loops	Can anything loop, retry or call the model without a limit?	A while True without a counter; retries stacked on SDK retries

The dependency row deserves a warning of its own. Models sometimes invent package names that sound right. Attackers have registered some of those invented names on public package indexes with malicious code inside, a practice called slopsquatting. Before installing anything an assistant suggests, check it exists, who maintains it and how widely it is used.

Here is a short function an assistant produced when asked to "add retries to the model call". Read it before you read the notes below.

def call_model(messages):
    for i in range(3):
        try:
            print(f"Calling {MODEL} with key {os.environ['OPENAI_API_KEY']}")
            return litellm.completion(model=MODEL, messages=messages).choices[0].message.content
        except Exception:
            time.sleep(1)

Review notes: (1) Secret leak: the key is printed to the console and any log. (2) Retries every error, including authentication failures and bad requests. (3) No backoff or jitter: a fixed 1-second wait. (4) No timeout: one hung call blocks forever. (5) Silent failure: after three failures it returns None, which the caller will happily put into a SitRep. (6) It sleeps after the final attempt for no reason. (7) It is not testable without a network and a real wait. Six lines, seven findings. None of them would show up in a quick demo, which is the point.

A useful habit: ask a second model, in a fresh session, to review the diff against your checklist (Prompt Lab Exercise 2). It adds to your review; it does not replace it.

9. "Vibe coding" vs spec-driven development

Know the Term: “Vibe coding” vs spec-driven development

Vibe coding (a phrase coined in 2025) means building software by describing what you want to an AI, accepting whatever code it produces without reading it closely, and judging only whether the result seems to work. Spec-driven development starts from a short written specification (what the software must do, what it must never do, and how you will check), turns that into tests, and then lets the AI write code that must pass those tests, which a person reviews before it is used.

Analogy: vibe coding is asking a builder for "a nice shed" and seeing what appears; spec-driven development is giving them a drawing, a materials list and an inspection checklist.

Why you care: vibe coding is fine for a throwaway prototype on fictional data. Anything that touches real data, real users or official figures needs a spec, tests and human review.

Vibe coding is not a sin. Describing what you want and letting an agent build it, checking only that it seems to work, is a fast way to explore an idea. The question is where the result ends up.

Situation	Vibe coding acceptable?	Why
A throwaway prototype to show a manager what a SitRep dashboard could look like, on fictional data	Yes	Nothing depends on it; you will rebuild it properly if it is approved
A one-off script to rename 200 files in your own folder	Yes, with a backup	Low stakes, easy to check the result by looking
Exploring an unfamiliar library to learn its API	Yes	The goal is learning, not the code
A tool colleagues will use every day	No	Other people will depend on behaviour nobody specified or reviewed
Anything that handles personal data, credentials or data above "public"	No	Security and privacy failures are invisible in a demo
Anything whose output becomes an official figure, e.g. a SitRep sent to partners	No	Errors propagate into decisions about people's lives and resources
Code that calls paid APIs in loops, or runs unattended	No	Runaway cost and silent failure

A practical middle path: vibe code the prototype, then throw it away. Once you know what you want, write the spec, write the tests, and rebuild in small reviewed steps. The prototype was the cheapest possible requirements workshop.

10. Project hygiene

Three habits prevent most of the embarrassing incidents in AI projects.

Secrets live in environment variables, never in code. An environment variable is a named value set in your shell or your deployment platform, which your program reads at run time:

import os
api_key = os.environ["OPENAI_API_KEY"]    # fails loudly if missing, which is what you want

For local development, a .env file (loaded by a tool such as python-dotenv) is convenient. It must be listed in .gitignore so it is never committed. In production, keys come from a secrets manager (such as Azure Key Vault or a cloud equivalent), not from files. If a key is ever committed, pushed or pasted into a chat, rotate it immediately (revoke it and issue a new one). Deleting the commit is not enough: git history, forks and caches keep copies.

.gitignore is part of the project. The lab's contains:

.env
*.key
.venv/
__pycache__/
.pytest_cache/

Add secret scanning as well: many code hosts scan pushes for key patterns, and tools such as gitleaks or detect-secrets can run as a pre-commit check on your machine.

Pin your dependencies. "Works on my machine" usually means "my machine has a different version of a library". Pinning records the exact version of every package you depend on, so that everyone, including CI (continuous integration: the service that runs your tests on every change) and the production server, installs the same thing:

pip install litellm pytest
pip freeze > requirements.lock        # exact versions, e.g. litellm==<version>
pip install -r requirements.lock      # on any other machine

Tools such as uv or poetry do the same with a lock file. AI libraries release very often; upgrade on purpose, run the tests, and commit the new lock file.

Prompt Lab

Use these with the coding assistant your organization has approved, in the lab folder. Compare results with your pair.

Exercise 1: A spec-first prompt for a coding agent

Task: add a new requirement to the lab app through the agent, without letting it improvise. First add R9 to SPEC.md yourself, then give the agent this prompt.

You are working in labs/engineering/c3, a small Python CLI app. Read SPEC.md, app.py, llm.py and test_app.py before changing anything.

Task: implement requirement R9 from SPEC.md:
  R9 · Length check. After a successful summary, count its words. If it has more than
  --max-words × 1.2 words, still print it, but also print to stderr:
  "Warning: summary has N words; the limit was M."

Rules:
1. Work test-first. First add tests for R9 to test_app.py and run them: show me they fail.
2. Then change app.py to make them pass. Keep the change as small as possible.
3. Do not modify or delete any existing test. If you believe an existing test is wrong,
   stop and tell me why instead of changing it.
4. Do not add dependencies. The code must run on Python 3.9.
5. Run the full test suite (pytest -q) and show me the output.
6. Finish with a summary: files changed, what each change does, and anything in the spec
   you found ambiguous.

What to look for: Did it actually show failing tests before implementing, or skip straight to code? Did it touch any existing test? How did it count words, and does that match what you meant (are "1,200" and "Tamsi-Bara" one word each)? Did it notice that R9 should be added to the acceptance-test table in SPEC.md? Anything it lists as ambiguous is a gap in your spec, and it's better to fix it there than in the code.

Exercise 2: A code-review prompt

Task: review a diff with a second model, in a fresh session so it has no loyalty to the code. Use the diff from Exercise 1, or the call_model function from Section 8.

You are a senior Python reviewer for a humanitarian organization. Review the diff below against this checklist. For each item, reply PASS, FAIL or N/A, and for every FAIL quote the exact line and propose a fix.

1. Correctness: does the code meet the stated requirement exactly? Off-by-one errors?
2. Tests: were any tests modified or weakened? Do new tests assert real behaviour?
3. Error handling: are errors specific, surfaced and never silently swallowed?
   Are only transient errors retried?
4. Security: any untrusted input used in shell commands, file paths, SQL or eval?
5. Secrets: any key, token or password in code, logs, error messages or tests?
6. Dependencies: any new package? Is it necessary, real and pinned?
7. Edge cases: empty input, very long input, non-English text, timeouts.
8. Cost: any loop or retry that could call the model without a limit?

Requirement:
<paste R9 from SPEC.md>

Diff:
<paste the output of: git diff>

End with one line: APPROVE, or REQUEST CHANGES with the number of FAIL items.

What to look for: Run it on the call_model function: does it find all seven problems from Section 8, and does it invent any that aren't there? Does it quote real lines or paraphrase? Try the same review with two different models; the disagreements are where your own attention should go. Would you trust an APPROVE from this prompt without reading the diff yourself? (You shouldn't.)

Exercise 3: Let the agent interview you before it writes a spec

Task: turn a vague request from a colleague into a draft spec. The assistant asks the questions; you answer them; it drafts; you edit.

A colleague asked me: "Can you make the SitRep tool also do Spanish and French?"

Before writing anything, interview me to turn this into a one-page spec in the style of the attached SPEC.md. Ask me one question at a time, at most eight questions in total, covering: purpose and users, inputs and outputs, what exactly "do Spanish and French" means (input, output, or both), quality checks, cost limits, failure behaviour and non-goals. After my last answer, write the spec with numbered requirements, non-goals, and a table mapping each requirement to a test name. Do not write any code.

<attach or paste SPEC.md>

What to look for: Did its questions surface decisions you hadn't made (translate the reports, the summary, or both? who checks the translation? what about Arabic, Chinese and Russian)? Are the requirements testable, or vague ("translate accurately")? Keep the draft: edited by you, it is a strong start for a real feature.

Lab: SitRep Summarizer, spec-first

Goal: understand a small, production-shaped LLM app, then extend it with an AI coding agent without losing control of the code: spec first, tests as the contract, small reviewed steps.

The code is in labs/engineering/c3/ in the course repository (or the course zip).

File	Role
SPEC.md	The written spec: interface, requirements R1–R8, non-goals, acceptance tests
llm.py	LLM type; litellm_llm() with timeout, error translation (TransientError) and usage reporting
app.py	call_with_retry(), CostTracker, tracked(), summarize(), CLI main()
test_app.py	15 tests with a scripted FakeLLM and a fake sleep
sample_reports.md	Three fictional field reports from Northern Veloria province
.gitignore	Keeps .env and caches out of git

Step 1 · Set up and run the tests (10 min)

cd labs/engineering/c3
python -m venv .venv && source .venv/bin/activate
pip install litellm pytest
pytest -q

All 15 tests pass without an API key, in well under a second. Open test_app.py and read FakeLLM: unlike Class 6's version, a scripted item can be an exception, which is raised instead of returned. That is how the tests simulate a rate limit followed by success.

Step 2 · Read the spec, then find each requirement in the tests (15 min)

Read SPEC.md from top to bottom. Then, with your pair, pick three requirements and find the test that enforces each one (the table at the end of the spec helps). For each, ask: if I deleted this test, what mistake could slip through? For R5 (timeouts), notice that the spec admits it is only partly covered. That's honest, and it's a stretch challenge.

Step 3 · Read the retry and cost code (15 min)

Read call_with_retry() and CostTracker in app.py and is_transient() in llm.py. Answer together:

• With retries=3, how many calls are made before RetryError? How many sleeps?
• Why is sleep a parameter, and why does call_with_retry look up time.sleep only when it is called?
• summarize() checks the budget before calling. What would go wrong if it checked after?

Try it: add a test showing that a 401 (use the HTTPError helper class in the tests) is raised immediately, with no sleeps. Should it pass without code changes? Run it and see.

Step 4 · Extend it with a coding agent, spec-first (20 min)

Add requirement R9 · Length check (Exercise 1) to SPEC.md, including a row in the acceptance-test table. Then give your coding agent the Exercise 1 prompt. Watch what it does rather than looking away: note every command it runs and every file it opens.

When it finishes, run pytest -q yourself. Don't take its word for it.

Step 5 · Review the diff (10 min)

Run git diff (or view the changes in your editor) and review it with the checklist from Section 8, test files first. Then run the Exercise 2 review prompt on the same diff in a fresh session. Compare its findings with yours. Accept the change only when you can explain every line to your pair.

Step 6 · Run it for real (10 min)

export MODEL=openai/gpt-4o-mini        # or azure/<deployment>, anthropic/<model>
export OPENAI_API_KEY=...              # see README.md for each provider's variables
python app.py summarize sample_reports.md --max-words 120 --max-cost 0.01

Check the summary against the reports: does it flag the 1,200 vs 1,450 households conflict? Does it say "not stated" for deaths, or invent a number? Read the usage line on stderr: what did the call cost, and what would 40 a day cost for a year? Finally, set the timeout impossibly low (timeout=0.001 in main()), run again, and watch the retries and the exit code 3.

Stretch challenges

1. Streaming (--stream). Update the spec first (what happens if the stream breaks half-way?), write a test with a fake that yields chunks, then implement.
2. Honour Retry-After. If a transient error carries a retry_after value, wait that long (capped) instead of the computed backoff. Test-first.
3. Fallback model. Add --fallback-model, used once if the primary still fails after its retries; record which model answered in the usage line. Have the agent build it from your spec, then review its diff with the checklist.

Check Your Understanding

1 The overnight SitRep batch job wraps every model call in a retry helper. Which failures should the helper retry, with exponential backoff?

1. Every error: models are non-deterministic, so any failed call might succeed next time.
2. Rate limits (429), timeouts, connection errors and 5xx server errors, but not bad requests (400), authentication errors (401/403) or content-filter refusals.
3. Only timeouts. A 429 means the quota is exhausted, so the job should stop for the night.
4. None. Retries hide problems, so every error should stop the job immediately.

2 A colleague refactors the SitRep chat prototype so each user message is sent in its own API call. Now the assistant has no idea what was discussed two messages ago. What is the cause?

1. The temperature setting resets the model's memory between calls; set it to 0.
2. The provider deletes conversations after each call for privacy; enable the provider's memory setting.
3. The context window is too small for the conversation; switch to a model with a larger window.
4. The API is stateless: the model only sees the messages sent in that call, so the code must resend the conversation history each time.

3 During a provider outage, your code retries each call 3 times, the SDK underneath also retries 3 times, and the organization's AI gateway retries 3 times. At most, how many attempts can one SitRep request produce, and what should you do?

1. 64 attempts (4 × 4 × 4). Choose one layer to own retries and switch retries off in the others.
2. 9 attempts (3 × 3).
3. 3 attempts: the layers share one retry counter.
4. 10 attempts (1 + 3 + 3 + 3): the retries add up.

4 One summary uses 2,000 input tokens and 500 output tokens. The model costs USD 0.15 per million input tokens and USD 0.60 per million output tokens. What does the call cost?

1. USD 0.000375
2. USD 0.001275
3. USD 0.0006
4. USD 0.0015

5 You ask a coding agent to implement requirement R9 in the SitRep app. It reports: "All tests pass." In the diff you see it changed an existing assertion from == 4 to >= 1. What should you do?

1. Accept it: the tests pass, and the agent had context you don't.
2. Reject the change, restore the test, and tell the agent that existing tests may not be modified without approval. Review test-file diffs first from now on.
3. Ask the agent to confirm that its change is correct.
4. Delete the test: if it needed changing, it was probably flaky.

6 While adding retries, your coding assistant adds 'pip install litellm-retry-helper' to the setup instructions. Nobody on the team has heard of the package. What is the right response?

1. Check that the package really exists and who maintains it, and whether you need it at all (the retry logic is about 20 lines). Only add it if it passes, pinned to an exact version.
2. Install it: the assistant was trained on a huge amount of code, so it knows which packages exist.
3. Install the latest version without pinning, so you always get security fixes.
4. Ask the assistant whether the package is safe, and install it if it says yes.

7 In which of these situations is "vibe coding" (accepting AI-generated code because it seems to work, without close review) acceptable?

1. A tool colleagues will use every day to total displacement figures for the SitRep.
2. A script that automatically emails the finished SitRep to partner agencies each morning.
3. A small script that processes staff contact lists: it's only 40 lines, so review isn't worth it.
4. A throwaway prototype, on fictional data, to show a manager what a SitRep dashboard could look like.

8 A teammate notices an API key hard-coded in app.py, which has already been pushed to the team's shared repository. What should happen first?

1. Push a new commit that removes the key; the problem is then solved.
2. Make the repository private.
3. Revoke and rotate the key immediately, then move it to an environment variable (with .env in .gitignore) and add secret scanning.
4. Nothing, as long as the repository is internal to the organization.

Exit Ticket

Think of one piece of code you (or your team) want to build with an AI coding agent in the next month. In three sentences: (1) would you vibe code it or build it spec-first, and why; (2) what is the single most important requirement you would write in its spec; (3) which item on the review checklist is most likely to catch a problem in it?

Key terms

• API / messages API: the fixed request format for calling a model: a list of messages, each with a role (system, user, assistant) and content.
• SDK: a library that wraps an API for a particular programming language.
• LiteLLM: an open-source library giving one interface to many model providers.
• Stateless: the model remembers nothing between calls; your code resends the history.
• Stop reason (finish_reason): why generation ended, e.g. finished, length limit, filtered.
• Streaming: receiving the answer in pieces as it is generated.
• Timeout: the longest your code will wait for a reply before treating the call as failed.
• Rate limit: a provider's cap on requests or tokens per minute; exceeding it returns 429.
• Transient vs permanent error: worth retrying later vs will fail the same way again.
• Exponential backoff, jitter: doubling the wait between retries, plus a little randomness.
• Idempotent: safe to repeat; doing it twice has the same effect as doing it once.
• Token usage, cost tracking, budget: counting input and output tokens, pricing them, and refusing further calls past a limit.
• Coding assistant, coding agent: an AI that suggests code vs one that plans, edits files and runs commands to complete a task.
• Spec, spec-first development: a written definition of done, written before the code.
• Tests as the contract: tests that encode the spec and define when a change is finished.
• Vibe coding: accepting AI-generated code by whether it seems to work, without close review.
• Slopsquatting: publishing malicious packages under names that AI tools tend to invent.
• Environment variable, secrets manager: where keys live instead of code.
• Pinned dependencies, lock file: recording exact package versions so every install is the same.