Self-improving AI models have long promised to learn from their own mistakes, but a new technique reveals why that process often fails silently. Researchers have discovered that standard self-distillation, a method where a model learns from its own verified solutions, actually suppresses the very thinking process it needs to succeed. By flipping the script, they have created a system that accelerates mathematical reasoning dramatically.

The problem lies in how models process information during training. When a student model tries to mimic a teacher model that has already solved a problem, the teacher becomes overconfident in obvious structural words like "therefore" or "let." Meanwhile, it ignores the crucial deliberation tokens—the "wait," "maybe," and "let's see" moments that drive complex, multi-step search. This imbalance causes the student to skip the hard work of reasoning, leading to inconsistent results in math tasks even when the approach works elsewhere.

The solution is Anti-Self-Distillation, which does the opposite of traditional methods. Instead of pulling the student toward the teacher, it pushes them apart, creating a divergence that highlights the missing reasoning steps. This approach naturally bounds the learning signal and includes a smart gate that stops the process once the teacher becomes too confident. The result is a drop-in replacement for existing systems that forces the model to engage with the actual logic of the problem rather than just copying the final answer.

The impact is significant. Across five different models ranging from 4 billion to 30 billion parameters, Anti-Self-Distillation achieved baseline accuracy in just 2 to 10 times fewer training steps. It also boosted final accuracy by up to 11.5 points. This breakthrough offers a scalable path for language models to bootstrap their own reasoning capabilities, turning self-correction into a powerful engine for intelligence rather than a bottleneck.