mlx-rl is a GRPO harness that RL-trains LoRA adapters on a single Apple Silicon machine — no cluster, no cloud. One M3 Ultra (96 GB) trains a 35B-parameter model against verifiable rewards: 64 sampled completions per step, judge-graded, ~4 hours per 200-step run.
The same 35B model answers twice: once as its pristine base self, once with an RL-trained calibrated honesty adapter (4.3M LoRA parameters — 0.01% of the model). The adapter was trained on trivia only; hedging on unanswerable questions transfers to people and papers it has never seen. Try to break it — flag what you find.
Language models answer confidently whether or not they know. The RL recipe here teaches a model to act on its own uncertainty in free conversation: hedge on the unknowable, answer the known, stop asserting things that don't exist. Post-cutoff paper questions go from 3% hedged (base) to 88% — while famous-paper recall stays at 95–100% and an agentic-coding benchmark shows zero capability cost (45/72 vs 45/72, same harness, adapter the only variable).
| base | +RL adapter | |
|---|---|---|
| hedges on made-up people | 0.20 | 0.95 |
| hedges on post-cutoff papers | 0.50 | 0.92 |
| still answers famous papers | 1.00 | 1.00 |
| reads its own per-item uncertainty | — | AUROC 0.74 |
| SWE-bench-style coding (of 72) | 45 | 45 |
Replicated across seeds; the adapter is inert without its system prompt (register-bound by construction, so it cannot leak hedging into contexts that didn't opt in).