We trust AI to remember everything, but what happens when it cannot see what it needs to act? As multimodal models move from passive chatbots to active agents in complex environments, a critical flaw emerges: they struggle to recall information that is no longer visible. This isn't just a minor glitch; it is a fundamental barrier to deploying these systems in real-world scenarios where context is fleeting and hidden.

Researchers have introduced RNG-Bench, a new benchmark suite designed to test this specific capability. Unlike existing tests that either show the full state of a game or only check memory at the very end, RNG-Bench forces models to reconstruct past observations during multi-step interactions. The suite features two challenging games: Matching Pairs, where card identities are briefly revealed and must be recalled later, and a 3D Maze, where the model must integrate egocentric views into a spatial map. These tests are rigorous, with the hardest configurations requiring the model to process contexts of roughly 128,000 tokens and 350 images per episode.

The results reveal a surprising bottleneck. While models can make good decisions when they have the right information, most residual errors stem from forgetting earlier observations rather than poor decision-making. In other words, the issue is not that the AI is stupid, but that it is forgetful. This distinction is crucial because it isolates memory reconstruction as a distinct skill that current frontier models have yet to master, leaving them far from saturated even in these controlled environments.

However, there is a clear path forward. Fine-tuning models like Qwen3.5-9B on optimal-policy rollouts and filtered model demonstrations significantly improves performance on RNG-Bench. Remarkably, this targeted training boosts memory capabilities without degrading the model's general multimodal abilities. This suggests that while current AI struggles with non-Markovian tasks, targeted reinforcement learning can bridge the gap, paving the way for more reliable, memory-aware agents.