LLM Engineering Course
Learn how language models work by building the important pieces yourself, then use those pieces to develop a model for a realistic company use case.
Start with setup and the learning guide. The guide names prerequisites and checkpoints; the glossary explains unfamiliar terms. No prior machine learning is required.
Last updated 2026-09-30 — see What’s new.
What you will be able to do
Every skill below is proved by a unit you can run on a CPU, not by a reading list.
- Explain how text becomes a prediction — byte-level tokenization, embeddings, self-attention and a decoder, built by hand in T02, T03, T04 and T05.
- Train a small GPT and resume it — a training loop you can interrupt and reproduce, in T05, T06 and E01.
- Control how a model generates text — temperature, top-k/top-p, stop rules and the KV cache, in I01.
- Tell a real improvement from noise — held-out evaluation across seeds, with uncertainty and contamination checks, in E02 and E03.
- Fit the work to a budget — parameter counts, memory and time estimated before you spend them, in E04 and the hardware preflight.
- Adapt a model instead of retraining it — SFT, LoRA, DPO and continued pretraining, compared on evidence, in A01–A04 and C04.
- Build a retrieval system over your own documents — chunking, sentence embeddings, a vector store and answers that cite their sources, in R01.
- Keep a system honest — refusing when the answer is not in the sources, enforcing citations, detecting prompt injection and measuring the refusal rate, in R02.
- Build an agent loop with nothing hidden — a plain-text tool protocol you parse yourself, a step budget, and a trace that shows what the model actually saw, in AG01.
- Put the same agent on a native tool-use API and say what the API took over — schemas, structured calls, call ids — and what it did not, measured on the same question three ways, in AG02.
- Measure what a defence against prompt injection actually buys, one step at a time — a cumulative ladder run against R02’s retrieval pipeline and AG02’s agent, on local models only, in AG03.
- Decide whether to train at all — turn a requirement into a measured baseline gap and a reversible decision record, in C01 and the capstone.
The Applied track continues beyond retrieval. These units are planned and not written yet:
- Your own MCP server (coming soon) — connect the agent to the course’s MCP server, then build and serve one yourself.
- Prompt engineering (coming soon) — score prompt variants against an evaluation set instead of trusting patterns.
- Guardrails as a unit of their own (coming soon) — input and output checks for any LLM system, not only for retrieval.
- Conversational AI (coming soon) — multi-turn answers, memory and the context budget.
- Vector databases (coming soon) — exact search against approximate, and the point at which you switch.
Two entry paths
- Standalone path: includes Python, mathematics, neural-network, and PyTorch foundations.
- Godot RL bridge: uses a diagnostic to skip concepts already demonstrated in the Godot RL course (a separate course; if you have not taken it, use the standalone path).
Both paths start with Unit 0, continue through the diagnostic and learning-path check, and converge before text tokenization. Every skipped foundation keeps a re-entry link.
The standalone foundations track F01–F06 covers Python/NumPy, tensor shapes, probability and cross-entropy, a trainable neuron, MLP/backpropagation, a tiny autograd engine and a PyTorch checkpoint. Godot-RL graduates take the B01 bridge and prove only the missing LLM-specific concepts. Both paths continue through the core: the T01 bigram baseline, T02 byte-level BPE, the T03 data pipeline and embeddings, T04 self-attention, the T05 Mini-GPT decoder and its T06 core gate, I01 decoding, sampling and the KV cache, then E01 reproducible training, E02 data quality, E03 evaluation and E04 profiling. The company track C01–C04 turns a requirement into a model decision, auditable data, protected evaluation and continued pretraining; the adaptation track A01–A04 covers SFT/LoRA, the adaptation comparison, DPO and the RLHF bridge; the capstone ties them into a reversible company-model decision. The Applied: RAG track (R01–R02) is a standalone entry that needs only Python and one LLM call.
Every required outcome has a CPU route. Run the hardware and cost preflight before choosing a larger profile.
What you will build
- a bigram language-model baseline;
- a BPE tokenizer;
- self-attention and a decoder-only transformer;
- a small GPT-style model with training and evaluation;
- instruction and preference fine-tuning;
- a synthetic-data pipeline with quality gates;
- a retrieval system over the course material, with its own evaluation set, guardrails and a served API;
- an agent loop with two tools over your own reports and the course text;
- a reproducible company-model capstone.