Diagnostic and learning path
← Unit 0 troubleshooting · F01 — Python & NumPy → · Glossary
Learning outcome
In 45–60 minutes, you will collect evidence about the foundations you already control and receive a conservative route through the course. This is routing, not an exam: choosing more foundation work is always allowed and every shortcut is reversible.
Complete Unit 0 first. Every path joins again before text and tokenization.
Two entry paths
| Path | Start here when | What the diagnostic changes |
|---|---|---|
| Standalone | Little or no Python, mathematics, ML, or PyTorch experience | Marks foundation blocks Required and still recognizes isolated strengths |
| Godot-RL bridge | You completed all or part of the Godot RL course | Verifies transferable skills and removes repetition only where evidence is strong |
Course completion and self-report never unlock a skip on their own.
1 · Experience inventory
Mark each item new, practised, or can explain and implement:
- Python functions and tests;
- tensor shapes;
- logits, softmax, logarithms, and cross-entropy;
- derivatives and the chain rule;
- parameters, activations, loss, and backpropagation;
- a PyTorch forward/backward/update cycle;
- seeds, splits, and baselines;
- observation, action, policy, reward, and trajectory.
Do not score this inventory. It helps distinguish environment trouble from a missing concept.
2 · Short theory check
Answer without searching first, then verify the reasoning.
- What does a list comprehension produce, and when is a generator safer?
- Why should a test check behaviour rather than a private implementation detail?
- X has shape [4, 8, 32] and W has shape [32, 64]. What is the shape of XW?
- In [B, T, C], which axis normally indexes token position?
- What probabilities does softmax assign to [0, 0], and what happens after adding the same constant to both logits?
- Why does a confident wrong prediction have a large cross-entropy loss?
- For L = (wx - y)², what is the derivative of L with respect to w? F04 teaches this without prior calculus; if derivatives are new, say so and move on.
- Why does an early layer’s gradient depend on later operations?
- What is the difference between a parameter and an activation?
- What information from the forward pass is needed for backpropagation?
- Why are gradients normally cleared before the next optimizer step?
- Name an operation that can differ between training and evaluation.
Optional bridge prompt: map observation, action, policy, trajectory, and reward to LLM concepts, then name one place where the analogy fails.
3 · Coding check
Documentation is allowed for a Review result; record when you used it.
Task A — Python sequences
Write a function that turns the following sequence into adjacent input/target pairs:
["a", "b", "c", "d"]
[("a", "b"), ("b", "c"), ("c", "d")]Add tests for an empty sequence, one item, and the normal case. Explain the time complexity.
Task B — stable probabilities and shapes
With NumPy or PyTorch:
- demonstrate a numerically stable softmax;
- show that adding one constant to all logits leaves probabilities unchanged;
- calculate cross-entropy for one correct target;
- add shape and finite-value assertions.
Task C — one PyTorch update
Create a small Linear(3, 2) classifier and a batch shaped [4, 3]. Run a full training step: forward pass, cross-entropy against four class indices, gradient clearing, backward pass, finite-gradient check, and optimizer step. State the logit shape and explain what happens if gradients are never cleared.
4 · Score the evidence
| Score | Meaning |
|---|---|
| 0 · Not yet | No correct runnable result, or a critical shape/loss/gradient error remains |
| 1 · With support | Correct after documentation, a hint, or one material correction |
| 2 · Independent | Correct without help, with assertions and a clear explanation |
Use the lower of theory and coding evidence for a domain.
5 · Build your route
| Result | Rule | Action |
|---|---|---|
| Required | A controlling skill is 0, evidence is missing, or a critical error occurred | Complete the full block and checkpoint |
| Review | No controlling skill is 0, but at least one is 1 | Complete the compact review and checkpoint |
| Skip + checkpoint | Every controlling skill is 2 | Skip lessons, retain the re-entry link, and take the common gate |
| Stretch | Offered only after Skip | Try a derivation, gradient check, or comparison |
When uncertain, choose Review.
| Block | Evidence | Re-entry link |
|---|---|---|
| F01 Python and development | Python theory + Task A | Re-enter F01 |
| F02 Tensors and shapes | shape questions + Task B assertions | Re-enter F02 |
| F03 Probability and loss | probability/loss questions + Task B values | Re-enter F03 |
| F04 Neuron and gradient descent | derivative question + traced parameter update | Re-enter F04 |
| F05 MLP and autograd | backprop questions + gradient check | Re-enter F05 |
| F06 PyTorch and reproducibility | Task C + repeatable checkpoint | Re-enter F06 |
| B01 Godot-RL bridge | optional bridge prompt + verified experience | Re-enter B01 |
All routes complete tokenization, causal modelling, attention, and evaluation.
6 · Example routes
| Learner | Recommended route |
|---|---|
| Complete beginner | Unit 0 → F01–F06 Required → common gate → LLM core |
| Godot-RL graduate with rusty probability/PyTorch | Unit 0 → F02–F05 Review → short B01 bridge → gate |
| Python developer new to ML | Unit 0 → F02 Review → F03–F05 Required → F06 checkpoint → gate |
| ML engineer new to LLMs | Unit 0 → foundation checkpoints → optional stretch → gate |
7 · Common gate
Before text and tokenization, everyone must infer one batched linear-operation shape, verify one stable cross-entropy value, run one backward/update cycle, explain gradient clearing, and reproduce the result with the supplied seed.
If one item fails, follow its re-entry link, complete only that focused block, and retry the item. You never restart the whole course.
Re-entry catalog
Python and development
Return here when functions, iteration, tests, exceptions, or environment commands block progress. Take F01 and rerun Task A.
Tensors and shapes
Return here when batch/time/channel axes, broadcasting, matrix multiplication, or shape assertions become unclear. Take F02 and retry the shape item.
Probability and loss
Return here when logits, stable softmax, log probabilities, or cross-entropy are unclear or non-finite. Take F03 and rerun Task B.
Gradients and neural networks
Return here when chain-rule paths, parameter updates, or loss movement are unclear. Take F04 and trace one update by hand and in code.
MLP and autograd
Return here when MLP shapes, graph paths, local derivatives, or gradient checks are unclear. Take F05 and verify one parameter.
Reproducible experiments
Return here when PyTorch update order, seeds, baselines, or comparisons are unreliable. Take F06 and reproduce the checkpoint twice.
Godot-RL bridge
| Godot RL | LLM | Do not assume |
|---|---|---|
| observation | token context | context is a sequential token window |
| action | next token | the action becomes future context |
| policy | vocabulary distribution | pretraining primarily learns likelihood |
| trajectory | generated sequence | boundaries differ from environment termination |
| reward | preference/verifier signal | not every LM stage uses reward |
| behaviour cloning | supervised fine-tuning | masking and chat formatting matter |
| PPO | RLHF update | reference-policy and KL controls are added |
| ONNX inference | optimized inference | decoding benefits from a KV cache (I01) |
What’s next
Record your Required, Review, Skip, and Stretch blocks, keep every re-entry link, and start F01. Only if you completed the Godot RL course, take the compact Godot-RL bridge instead. All paths meet at F06.
← Unit 0 · → F01 · Godot-RL bridge
Reference answers after your first attempt
Theory key and where to practise
- A list comprehension materializes a list; a generator yields items on demand and can avoid storing every item. See F01.
- A behavior test verifies a promised result; private names may change without changing the result. See F01.
[4,8,64]: the shared axis 32 is summed away. See F02.T, the middle axis, indexes token position. See F02.[0.5,0.5]; a shared shift cancels during normalization. See F03.- The true class has small probability, so its negative logarithm is large.
2(wx-y)x: the square rule gives2(wx-y),wxchanges at ratex, and the chain rule multiplies the two. F04 teaches these rules and checks them by nudgingw.- Changing an early value changes all downstream calculations; their local derivatives multiply and branch contributions add. See F05.
- A parameter is learned and stored; an activation depends on this input.
- Graph connections and the local values needed by each derivative.
- Backward adds gradients to stored values; without clearing, previous batches alter this update unless accumulation was intentional. See F06.
- Dropout drops values only during training; evaluation disables it.
Coding references: F01 shows adjacent pairs and tests; F03 and
src/llm_course/foundations.py show stable probabilities and cross-entropy;
F06 contains a complete runnable PyTorch update. Compare after attempting
your own answer and mark any help honestly in your routing evidence.
← Unit 0 troubleshooting · F01 — Python & NumPy → · Glossary