---
title: "LLM Engineering Course"
sidebar:
  label: "Home"
---

<span id="llm-engineering-course" />


Learn how language models work by building the important pieces yourself, then
use those pieces to develop a model for a realistic company use case.

Start with [setup](/setup) and the [learning guide](/learning-guide).
The guide names prerequisites and checkpoints; the [glossary](/glossary)
explains unfamiliar terms. No prior machine learning is required.

Last updated 2026-09-30 — see [What's new](/changelog).

## What you will be able to do [#what-you-will-be-able-to-do]

Every skill below is proved by a unit you can run on a CPU, not by a reading list.

- **Explain how text becomes a prediction** — byte-level tokenization, embeddings,
  self-attention and a decoder, built by hand in [T02](/bpe-tokenizer),
  [T03](/data-pipeline), [T04](/attention) and [T05](/mini-gpt).
- **Train a small GPT and resume it** — a training loop you can interrupt and
  reproduce, in [T05](/mini-gpt), [T06](/mini-gpt-gate) and [E01](/pretraining).
- **Control how a model generates text** — temperature, top-k/top-p, stop rules and
  the KV cache, in [I01](/inference-decoding).
- **Tell a real improvement from noise** — held-out evaluation across seeds, with
  uncertainty and contamination checks, in [E02](/data-quality) and
  [E03](/evaluation).
- **Fit the work to a budget** — parameter counts, memory and time estimated before
  you spend them, in [E04](/profiling) and the [hardware preflight](/hardware).
- **Adapt a model instead of retraining it** — SFT, LoRA, DPO and continued
  pretraining, compared on evidence, in [A01–A04](/sft-lora) and
  [C04](/continued-pretraining).
- **Build a retrieval system over your own documents** — chunking, sentence
  embeddings, a vector store and answers that cite their sources, in
  [R01](/rag-build).
- **Keep a system honest** — refusing when the answer is not in the sources,
  enforcing citations, detecting prompt injection and measuring the refusal rate,
  in [R02](/rag-quality).
- **Build an agent loop with nothing hidden** — a plain-text tool protocol you
  parse yourself, a step budget, and a trace that shows what the model actually
  saw, in [AG01](/agent-loop).
- **Put the same agent on a native tool-use API and say what the API took over** — schemas, structured calls, call ids — and what it did not, measured on the same question three ways, in [AG02](/agent-native).
- **Measure what a defence against prompt injection actually buys, one step at a
  time** — a cumulative ladder run against R02's retrieval pipeline and AG02's
  agent, on local models only, in [AG03](/agent-injection).
- **Decide whether to train at all** — turn a requirement into a measured baseline
  gap and a reversible decision record, in [C01](/company-strategy) and the
  [capstone](/company-capstone).

The Applied track continues beyond retrieval. These units are planned and not
written yet:

- **Your own MCP server** *(coming soon)* — connect the agent to the course's MCP server, then build and serve one yourself.
- **Prompt engineering** *(coming soon)* — score prompt variants against an
  evaluation set instead of trusting patterns.
- **Guardrails as a unit of their own** *(coming soon)* — input and output checks
  for any LLM system, not only for retrieval.
- **Conversational AI** *(coming soon)* — multi-turn answers, memory and the
  context budget.
- **Vector databases** *(coming soon)* — exact search against approximate, and
  the point at which you switch.

## Two entry paths [#two-entry-paths]

- **Standalone path:** includes Python, mathematics, neural-network, and
  PyTorch foundations.
- **Godot RL bridge:** uses a diagnostic to skip concepts already demonstrated
  in the Godot RL course (a separate course; if you have not taken it, use the
  standalone path).

Both paths start with [Unit 0](/unit-00), continue through the
[diagnostic and learning-path check](/diagnostic), and converge before text
tokenization. Every skipped foundation keeps a re-entry link.

The standalone foundations track [F01–F06](/foundations-01-python-numpy)
covers Python/NumPy, tensor shapes, probability and cross-entropy, a trainable
neuron, MLP/backpropagation, a tiny autograd engine and a PyTorch checkpoint.
Godot-RL graduates take the [B01 bridge](/foundations-godot-rl-bridge) and
prove only the missing LLM-specific concepts. Both paths continue through the
core: the [T01 bigram baseline](/bigram-baseline), [T02 byte-level BPE](/bpe-tokenizer),
the [T03 data pipeline and embeddings](/data-pipeline), [T04 self-attention](/attention),
the [T05 Mini-GPT decoder](/mini-gpt) and its [T06 core gate](/mini-gpt-gate),
[I01 decoding, sampling and the KV cache](/inference-decoding), then
[E01 reproducible training](/pretraining), [E02 data quality](/data-quality),
[E03 evaluation](/evaluation) and [E04 profiling](/profiling). The company
track [C01–C04](/company-strategy) turns a requirement into a model decision,
auditable data, protected evaluation and continued pretraining; the adaptation
track [A01–A04](/sft-lora) covers SFT/LoRA, the adaptation comparison, DPO and
the RLHF bridge; the [capstone](/company-capstone) ties them into a reversible
company-model decision. The [Applied: RAG track](/rag-build) (R01–R02) is a
standalone entry that needs only Python and one LLM call.

Every required outcome has a CPU route. Run the
[hardware and cost preflight](/hardware) before choosing a larger profile.

## What you will build [#what-you-will-build]

- a bigram language-model baseline;
- a BPE tokenizer;
- self-attention and a decoder-only transformer;
- a small GPT-style model with training and evaluation;
- instruction and preference fine-tuning;
- a synthetic-data pipeline with quality gates;
- a retrieval system over the course material, with its own evaluation set,
  guardrails and a served API;
- an agent loop with two tools over your own reports and the course text;
- a reproducible company-model capstone.

<aside className="course-note" aria-label="Development status">
<strong>Development status</strong>

Every lesson linked on this page is written and runs on a CPU, in English
and German; the units marked "(coming soon)" are not written yet.

</aside>
