Skip to content
LLM Engineering Course
English
Esc
↑↓navigate↵open⌘Jpreview
On this page

Unit 0 five-learner pilot protocol

Course home · English source reference

Status: ready to run; human pilot evidence is still required for MIN-76

Acceptance rule

Five fresh learners must reach their first generation within 60 minutes without an instructor editing code or issuing learner-specific commands. Every blocker, elapsed time, standardized hint, and recovery is recorded in unit-00-results.md. MIN-76 must remain open until all five rows pass or a failed row is repaired and repeated with a fresh learner.

Learner coverage

Recruit five people across these profiles where possible:

ID Useful profile coverage
P01 no Python or ML experience
P02 Python developer new to ML
P03 Godot-RL learner
P04 Windows/PowerShell user
P05 macOS MPS or Linux/CUDA user, while retaining CPU fallback

Do not collect names, employer data, prompts containing private information, or generated checkpoint files. The anonymous IDs above are sufficient.

Moderator script: 45–60 minutes

Minute Learner action Moderator action
0–5 read outcome; predict continuation and intervention start timer; do not explain model
5–15 complete Setup; run diagnose_unit0.py record exact blocker and generic hint ID only
15–25 run run_unit0.py; locate first generation record time-to-first-generation
25–35 identify prompt IDs, top probabilities, matrix shape, and loss ask the six standard explanation questions
35–45 run own prompt and positive boost record prediction and measured delta
45–50 run negative boost; interpret failure do not edit command or code
50–55 repeat seeded sample and compare record reproducibility result
55–60 give six-sentence explanation score completion gate

Standardized hints

Only these reusable hints are allowed; record their IDs:

  • H1: activate the environment shown in Setup;
  • H2: copy the diagnostic command exactly from Unit 0;
  • H3: use --device cpu after an accelerator failure;
  • H4: follow the first matching troubleshooting row;
  • H5: use only supported lowercase characters in the first prompt.

Any code edit, custom shell command, or direct answer from the moderator counts as individual correction and fails that run.

Pre-pilot engineering observations

Observation Risk Implemented mitigation
PyTorch installation can be much longer than the model run learner interrupts setup troubleshooting says to wait for final line; timer separates setup blocker
accelerator discovery can fail independently of model code learner cannot start explicit CPU diagnostic and full CPU fallback
stale generated assets can fail validation confusing checkpoint error fresh --artifact-dir recovery without committing artifacts
unsupported prompt characters collapse to ? surprising trace tokenizer contract and supported-character hint

These are engineering observations, not substitutes for five learner sessions.

Was this page helpful?