Skip to content
LLM Engineering Course
English
Esc
↑↓navigate↵open⌘Jpreview
On this page

What's new

Course home

Dated changes a learner can notice, newest first. Each entry links to the page that changed. Internal work — tests, tooling, dependency updates — is not listed here; the Git history has it.

2026-09-30 — AG03: the measurements are run again

Every table, grid and sample on the AG03 page comes from a new run on the current course pages. On the 1.7B, spotlighting (step 3) now stops five attacks at once (7 of 8 to 2 of 8), and declarative-1 is the attack that survives every text-level step; the 4B falls for one attack at step 0 instead of two and completes its own task in 5 of 8 runs instead of 6. The traced examples and the classifier exercise (81 of 370 honest chunks flagged for the plain question) were re-run too. The page says that the prompt holds course pages, so a page edit can flip a cell. → AG03 — Prompt injection

2026-09-30 — R01 and R02: the reference run is measured again

The reference evaluation was run again on the current course pages, and every number R02 and rag/README.md quote now comes from it: improved reaches Recall@1 0.878, Recall@5 1.000 and MRR 0.935 against 0.512, 0.902 and 0.670 for naive. Under naive the model now refuses ten on-topic questions instead of nine, and the groundedness gap (0.523 against 0.575) shows on the same 31 questions both presets answer. R02 lists the refused questions again, with their ranks. The outputs R01 pastes (the first run with --show-hits, the Break question on both presets and the independent-exercise question) were re-run and match today’s pages. → R02 — So it does not lie · R01 — From document to answer

2026-09-29 — R01: install step, visible hits and a gate

R01 now starts its practical part with “Install and download”: the rag extra the track needs, the model downloads (about 9.3 GB) before the first command, how to open the walkthrough notebook, and what to do about No module named 'sentence_transformers'. rag/ask.py --show-hits prints the retrieved passages with their scores, and “First run” says what you should see, why your ranks can differ and how to check them. Recall@k and MRR are defined before the exercises, Break asks a question no course page answers, your own evaluation question goes in artifacts/my-work/ instead of the committed set, and a completion gate says when you are done. The evaluation table prints n/a for “Off-topic refused” when a run has no off-topic questions. → R01 — From document to answer

2026-09-29 — Unit 0: see why the model picks each letter

The “Boost one logit” example now shows why each letter is chosen: how often each letter follows a space in the training text with its probability before and after your change, how the text is written one step at a time (including the tie between e and h that makes the model write te te te, and the d/k tie behind mode), and the 16 training sentences with every follower marked. The lesson explains the ties behind te te te, mode and pa pa. → Unit 0 — First Tiny-LM Run

2026-09-29 — F06: a skeleton for the gate and a reference that checks every row

The F06 checkpoint now starts from examples/checkpoint_skeleton.py: six stubbed functions and two diagnosis sentences, with the data and checks provided. It stops at the first unfinished step and names it. The reference examples/foundations_checkpoint.py now covers every rubric row, including the MLP gradient check on W1[0, 0], one PyTorch SmallMLP step with its expected losses, and a second run compared within 1e-7; it prints one line per rubric row, so you compare your report key by key. The page states one rule for the step order (clear gradients any time before backward(), check them finite before step()), shows the numbers one_update.py prints, says what the seed does, adds a collapsible “class in ten lines” primer, and gives a time guide. The checkpoint’s last stop prints the numbers your diagnosis must cite, and a stray print() in your functions no longer breaks the second run. → F06 — PyTorch translation and foundations checkpoint

2026-09-27 — F05: every backward formula on real numbers, then checked

F05 now says up front that you will not derive the backward formulas: each is a rule you compute on real numbers and then check by nudging. One example goes through the 2→3→2 network in ten numbered steps with every value printed; a shape rule (“a gradient has its parameter’s shape, so only one multiplication fits”) places each transpose, and a gradient check covers all of W1 and fails when the tanh slope is left out. A three-line demo shows two linear layers collapsing into one. A new batch step compares your gradients with the course helper, the Value engine section explains operators, recording, order and the shared path a * a, and a Gate toolkit collects one-hot targets, copying before a nudge and float64 for F06. The page gives times per route. → F05 — MLP, backprop & autograd

2026-09-27 — F04: nudge and watch before the derivative rules

F04 now starts by nudging w and watching the loss, so a derivative is a measured rate before it is a rule; each of the three rules it uses (square, line, chain) is checked by a nudge, and no prior calculus is needed. It works the first training step for all four points by hand (the gradient is an average), states the defaults of train_scalar_neuron, and shows every step’s weight and bias, so you can tell a wrong gradient sign (the weight drifts away) from a learning rate that is too large (the weight jumps across the target). You write your own 20-step training loop and check it against the trace. Each step names its file in artifacts/my-work/, and the page gives a time guide. The learning guide and diagnostic say that F04 teaches the derivative rules; the glossary adds derivative, chain rule, central difference, MLP, tanh, one-hot and SGD. → F04 — Neuron and gradient descent · Learning guide · Diagnostic · Glossary

2026-09-27 — F03: softmax and cross-entropy in six small steps

F03 now builds up to its exercise in six runnable steps: exp and ln (and why a calculator’s log key is the wrong one), softmax of one row by hand, why shifting every score changes nothing while shifting one moves probability — the +4.0 edit from Unit 0 — why subtracting the maximum prevents inf and nan, the penalty −ln p as a table, and a two-row batch that explains axis=-1, keepdims and probs[np.arange(2), targets]. The Build lists the six tests to write, the break to try, a hint and a reference solution; a stretch shows log-softmax for losses that would be inf. Time guide: 50–75 minutes if you program, about 2–3 hours over two sessions if you are new. → F03 — Probability, logits, and loss

2026-09-27 — F02: shapes in five small steps

F02 now teaches one idea per step, each to predict and then run in artifacts/my-work/shapes.py: broadcasting a row onto a table, * versus @ with the rule “entry (i, j) = row i · column j”, reshape versus .T, a small [B, T, C] array, and a linear layer x @ W + b with a worked example. You run a bias that fails loudly and one that passes silently, and learn why a transposed square weight slips through shape checks. The Build names its four tests and has a hint and a reference answer; the page states a time budget for both routes. → F02 — Tensors and shapes

2026-09-25 — F01: the Build no longer asks for untaught skills

A new step before the Build, “Testing that bad input is rejected”, teaches f-strings and pytest.raises with runnable files and a test skeleton for invalid inputs. Each primer block now says which file it goes in, the pytest file is shown whole, zip uses strict=True, and the question about the if became a remove-and-predict one with its answer. The completion checks now run ruff check artifacts/my-work and python -m pytest artifacts/my-work, which test your files rather than the course’s. The page gives separate times for the primer and the Build. → F01 — Python and NumPy

2026-09-25 — Unit 0: reason about the model, not just run it

Unit 0 now shows the text the model learned from and how 24 of 84 spaces followed by t become 0.263; turns the corpus loss into one worked number (surprise −ln p); says why almost every edit makes the loss worse; lists the model’s 33-character alphabet and which row your edit changes; and ends with sample answers to compare after writing. A wrong prediction no longer reads as failing the unit, and the time guide says 60 minutes if you program, 90–120 over two sittings if you are new. The summary shows a fourth decimal when three would hide a change, run_unit0.py --help describes every option, and the troubleshooting table lists the allowed characters. → Unit 0 — First Tiny-LM Run · Unit 0 troubleshooting

2026-09-24 — Setup and the first route, clearer for beginners

Setup now covers getting access to the repository (GitHub CLI, GitHub Desktop or a ZIP), what to install on Windows, macOS and Ubuntu/Debian, VS Code as the editor, a CPU-only PyTorch for Linux and an “Every session” checklist; the first model run is left to Unit 0, and notes for later lessons are collapsed. After the diagnostic the next page is now F01; the Godot-RL bridge says it is only for graduates of that course. The learning guide gives rough times, and the glossary explains terminal, venv, JSON, backend, corpus, argmax and checkpoint. → Setup · Learning guide · Glossary

2026-09-24 — Unit 0: boost one logit yourself

Before “Break an assumption”, Unit 0 now has an interactive element: pick the prompt ("the " or "a "), a target character and a boost from −6 to +6, and watch the target’s probability, the top 5 next characters before and after, both greedy texts, the corpus loss and the verdict change as you drag. It rebuilds the same counted bigram table the script uses, so its numbers match the page’s summaries; three buttons replay the page’s runs (+4 on m, +3 on t, −4 on t). → Unit 0 — First Tiny-LM Run

2026-09-23 — R01 and R02 are now available in German

Both RAG lessons now have a complete German version, from preparing the corpus and the embedding trace through the guardrails, the reference run and the independent exercises. Commands, code and program output stay in English, as on the other German pages; the questions you send to the index stay English because the index is built from the English pages. → R01 — From document to answer · R02 — So it does not lie

2026-09-23 — Unit 0: a ten-line summary instead of a JSON dump

run_unit0.py now prints ten lines — prompt and token IDs, the top candidates, both generations, the target’s probability and the corpus loss before → after, and one computed verdict — and saves the full report to artifacts/unit0/report.json; --json prints it. The page explains every line, walks through the full report once, and shows why --boost 3 moves a probability without changing the text. diagnose_unit0.py prints one line per check, and an empty prompt now gives a one-line error instead of a traceback. AG01 now runs run_unit0.py without redirecting its output. Characters with equal probability are now listed by token ID, so the top-candidate lists in Unit 0’s and T01’s reports read the same on every platform. → Unit 0 — First Tiny-LM Run

2026-09-23 — R01 and R02: every number measured on today’s course

The RAG lessons quoted a reference run from before the agent pages joined the corpus. Every number R01, R02 and the RAG README quote now comes from a fresh run on today’s 41 pages: improved still beats naive on retrieval (Recall@1 0.878 against 0.561), but its lexical groundedness is lower (0.557 against 0.631), and R02 now says so instead of calling it unchanged. The offline example in R01 shows a different top passage. A test now flags when a new page moves these numbers. → R02 — So it does not lie · R01 — From document to answer

2026-09-23 — AG03: measure what a defence against prompt injection buys

One attack set, the same poisoned document, run against R02’s retrieval pipeline and AG02’s agent through a cumulative defence ladder — delimiters, escaping, spotlighting, a pattern filter, ingest cleaning, a confirmation step, then scoping the send tool so no call can choose a recipient. Every measurement runs on local Qwen models only, never a hosted API. On the small model, retrieval success falls from 7 of 8 to 3 of 8 and never reaches zero; the 4B agent never takes the bait at any step, yet the same defences still cost it the task, from 6 of 8 completions down to 0 of 8. → AG03 — Prompt injection

2026-09-23 — R02: defeat the injection filter

The independent exercise no longer asks for a seventh regex. You now write two phrasings of the 999-euro attack that slip past all six patterns, watch the filter drop R02’s own Guardrails section, and explain which guardrail still catches you. The passage tags are escaped, so a document can no longer close its own tag and forge a passage. → R02 — So it does not lie

2026-09-22 — AG02: the same agent on native tool use

Run AG01’s agent three ways on one question — the text protocol on Claude, Claude’s native tool use, and Qwen3’s own <tool_call> format — and read what the API took over (the schema, the parse, the call ids) against what is still yours. Includes the first Break exercise where the API refuses a broken conversation. → AG02 — Native tool use

2026-09-21 — F01 takes the Python basics one idea at a time

The section that introduced slices, loops, zip, functions, assert and pytest in one go is now five short steps, each with one thing to predict. A slice visualizer on the page lets you type a slice and see which positions of ["a", "b", "c", "d"] light up, and line up two slices to see the pairs zip produces — the bigram idea in miniature. → F01 — Python & NumPy

2026-09-21 — AG01: build an agent loop by hand

The first unit of the new Applied: Agents group is live. You build an agent with nothing hidden — a plain-text tool protocol you parse yourself, a step budget, and a trace that shows what the model actually saw — compare two local models on the same question, and watch a small model invent a source. Two more agent units are planned and marked (coming soon) on the home page. → AG01 — The agent loop, by hand

2026-09-20 — Unit 0 no longer assumes what the course teaches later

The whole model is stated in three sentences before the first prediction, every technical term is defined where it first appears, and the repetitive output is explained as the expected result rather than a broken run. The Python check gains a hand-check route for learners who do not program yet. → Unit 0 — First Tiny-LM Run

2026-09-19 — The home page says what you will be able to do

Nine skills in plain language, each linked to the unit that proves it, and a list of planned units marked (coming soon) so you can see what is not written yet. → Course home

2026-09-19 — Applied: RAG moved below the Course group

The retrieval track now sits after the core course in the sidebar. Its content and its route in the learning guide are unchanged. → R01 — From document to answer

2026-09-17 — The course is served from llm.onlinekurs.training

The published site moved to its own domain. Old links redirect.

Course home

Was this page helpful?