Back to blog

AI Fails at Simple Shapes

Based on research by Amirreza Mohseni, Mona Mohammadi, Morteza Saghafian, Naser Talebizadeh Saradari

Look at a simple drawing of nested circles. To you, it is obvious which shape contains which. To the most advanced AI models, it is a nightmare. Researchers have created CurveBench, a benchmark designed to test if artificial intelligence can truly understand spatial relationships or if it is just guessing based on patterns.

The benchmark presents models with 756 images of non-intersecting curves, ranging from simple polygons to dense, maze-like configurations. The goal is deceptively straightforward: the AI must analyze the image and reconstruct the exact tree of containment relations between the shapes. It is a test of hierarchical topological reasoning, requiring the model to map out exactly how regions are nested within one another without any lines crossing.

The results are startling. Even the most powerful proprietary model, Gemini 3.1 Pro, achieved only 71.1% accuracy on the easy set and a dismal 19.1% on the hard set. This reveals a critical blind spot in current vision-language models. They can recognize objects, but they struggle to reason about the precise geometric logic of how those objects relate in space. The gap between human intuition and machine performance is vast, especially as complexity increases.

However, there is a path forward. By using reinforcement learning with verifiable rewards, researchers fine-tuned the open-source Qwen3-VL-8B model. This specific training method boosted its accuracy from a mere 2.8% to 33.3% on the easy tasks, outperforming giants like GPT-5.4 and Claude Opus 4.5 under their evaluation protocol. While this is a significant leap, the remaining difficulty on hard tasks proves that exact, topology-aware visual reasoning is far from solved. AI still has a long way to go before it truly understands the geometry of the world it sees.

Source: arXiv:2605.14068

This post was generated by staik AI based on the academic publication above.