A02 — Choose an adaptation with evidence
← A01 — SFT & LoRA · A03 — DPO → · Glossary
Prerequisites: company requirements, held-out evaluation and SFT/LoRA. Nordlicht Workspace is fictional. Its narrow support workflow needs routing, correct policy facts, valid JSON and safe escalation. Those are separate quality axes; one average cannot replace inspection of severe failures.
Start with the cheapest adequate change
| Method | What changes? | Appropriate first question | Main cost or risk |
|---|---|---|---|
| Prompting | Instructions/context only | Does clearer output guidance suffice? | Context and inference cost |
| Retrieval | Relevant approved documents in context | Are facts changing or must sources be cited? | Retrieval quality and permissions |
| DAPT/TAPT | LM weights from domain/task text | Is unfamiliar language the actual bottleneck? | Forgetting; no explicit answer format |
| Full SFT | All selected model weights | Do many representative answers justify full updates? | Gradients/optimizer state and overfit |
| LoRA | Small A/B matrices on frozen weights | Can a small update teach task behavior? | Limited capacity; base remains in memory |
| QLoRA | LoRA plus a quantized frozen base | Does a tested 4-bit backend fit the host? | Kernel/backend constraints and quantization error |
QLoRA retains a frozen 4-bit base and trains higher-precision adapters. NF4 uses a codebook designed for normally distributed weights; double quantization also compresses quantization scale constants. It does not train every base parameter in four bits. QLoRA’s paper describes these mechanisms. The pinned bitsandbytes backend documentation lists CUDA, XPU, HPU and supported Linux/Windows CPU platforms, not this macOS MPS host. QLoRA is unavailable in this measured environment. No simulated 4-bit experiment is presented as QLoRA evidence.
Compare like with like
A fair paired comparison fixes the base, tokenizer, train/dev splits, seed, sequence treatment, batch size, optimizer and steps. Record trainable parameters, wall time and the exact memory scope. Equal steps do not mean equal compute, and one shared learning rate is a controlled baseline rather than optimal tuning for every method. Never compare tiny random MiniGPT and pretrained SmolLM2 as though only the adapter method changed.
For the CPU MiniGPT pair, seed 7, 120 steps, batch 4 and AdamW LR 0.003 are fixed.
Sixteen training and eight independently authored development cases are used.
The tokenizer is trained only on training text. Generated corpora/weights stay in
ignored artifacts/; executable recipes and compact numerical evidence are tracked.
| MiniGPT method | Trainable parameters | Train NLL | Dev NLL | General-control NLL | Dev correct | Format failures |
|---|---|---|---|---|---|---|
| Initial random base | 40,416 | 6.163 | 6.181 | 6.337 | 0/8 | 8/8 |
| Full SFT | 40,416 | 0.488 | 3.816 | 7.309 | 0/8 | 5/8 |
| Manual LoRA, rank 4 | 3,968 | 3.768 | 4.469 | 7.212 | 0/8 | 8/8 |
Full SFT’s train/dev gap is 3.816 − 0.488 = 3.328 nats. Its general-control loss
increases by 0.973; LoRA increases it by 0.875. These two simple completion controls
show worsening likelihood, not a comprehensive general capability benchmark.
The original random model had no established general competence to preserve.
Lower train loss and fewer trainable parameters do not satisfy the company gate.
Both adaptations are rejected for deployment; keep the base and investigate data
coverage, prompt consistency and evaluation failures before another development run.
Measured runtime, memory and the separately identified real pretrained CPU/MPS
experiment are recorded in docs/baselines/sft-lora-v1.md and
docs/model-cards/company-adapter-v1.md. The PEFT example evaluates identical cases
before training, after training, after adapter reload and after merge.
Predict → Trace → Build → Break → Measure → Explain
- Predict: Which method can update tomorrow’s policy without weight training?
- Trace: Follow an approved document through prompt context versus training data.
- Build: Run the paired CPU recipe and inspect one failed generation per domain.
- Break: Raise learning rate or train on one repeated answer; predict collapse first.
- Measure: Compare train/dev gap, JSON failures, critical safety and general control.
- Explain: Choose rollback or promotion using the company acceptance criteria.
Independent exercise
Candidate A has 7/8 correct cases but fails one critical secret request. Candidate B has 6/8 correct, no critical failures, and twice the inference latency. The contract requires zero critical failures and latency below 200 ms; B takes 240 ms. Decide whether either can be promoted, then propose the next measurement without touching gold.
Hint
Hard acceptance gates do not compensate for each other. Use development cases for iterative changes; gold remains reserved for the final sealed decision.
Reference answer
Neither passes: A violates safety, B violates latency. Preserve the current deployed version, inspect A’s escalation behavior and profile B under the same hardware/load. Re-evaluate development after a concrete change. Do not claim a gold improvement from a lower training loss.
Checkpoint: explain when retrieval, continued pretraining and SFT solve different problems; read a fair paired scorecard; distinguish storage savings from peak memory; reject a promising adapter when a required gate fails.