The model copies the number correctly and then concludes wrong: move the reasoning into code
A 2B model given a raw record copies the numbers faithfully and then draws the wrong conclusion from them: at the 4th percentile it still says intermediate. So the pipeline moves the reasoning into deterministic code and leaves the model only the prose, with numbers as symbols the harness substitutes. The paired ablation puts a number on it: remove the mediation and 30% of readings carry a factual defect, against 0% with it, McNemar p = 2.9e-11. What breaks in a small model is not transcription. It is ranking.
Read the post →