We often assume that if a computer can solve a problem with a short program, we can simply teach an AI to do the same by showing it the step-by-step logic. This seems like a straightforward shortcut: write out the reasoning, fine-tune the model, and watch it learn. But new research shatters this assumption, revealing that for certain complex tasks, the very act of writing down the thought process breaks the model’s ability to solve the problem.

The study tested this by asking AI models to solve nine deterministic reasoning tasks, ranging from simple arithmetic to cryptarithms, which are puzzles where letters represent digits. Researchers reverse-engineered the computer programs that solve these puzzles and tried to distill that logic into the AI’s chain-of-thought. While the models handled lookup tasks and basic arithmetic with near-perfect accuracy, they completely failed at cryptarithms. Even though the underlying arithmetic was easy for the AI, it could not carry out the necessary search process as a linear, left-to-right derivation.

The surprise lies in why this failure occurs. It is not a lack of capability; the models can perform the individual math steps correctly. Instead, the problem is structural. When forced to output a chain-of-thought for search-based problems, the model stops reasoning and starts memorizing. It learns to predict the final verdict as a fixed template rather than deriving it, leading to accuracy rates that plummet to 16-57%. The AI essentially cheats by guessing the answer based on patterns rather than solving the puzzle, a behavior that persists regardless of the model’s size or the fine-tuning method used.

The takeaway is stark: you cannot simply convert any algorithm into a teachable thought process. If a task requires searching through a vast, information-free structure, there is no faithful forward chain-of-thought to imitate. To make such tasks learnable, we must remove the search entirely, precomputing the complex parts so the AI only needs to recall and verify. Distillation works for verification, but it fails when the core of the task is search.