Imagine speeding up your AI’s responses by five times without sacrificing accuracy. That is the promise of a new framework called Domino, which tackles one of the biggest bottlenecks in modern large language models: the slow process of generating text token by token. By rethinking how these models draft and verify text, researchers have found a way to make AI feel instantly responsive, even for complex tasks.

Large language models typically generate text autoregressively, meaning they predict the next word based on the previous ones. This sequential nature is inherently slow. To speed things up, a technique called speculative decoding allows the model to guess multiple tokens at once and then verify them in parallel. However, this creates a tricky trade-off. If the model guesses sequentially, it is accurate but slow. If it guesses in parallel, it is fast but often makes mistakes because it ignores the causal link between words. This tension has long limited how much faster these models can actually run in practice.

Domino solves this by decoupling the two processes. It first uses a parallel backbone to generate a rough draft of the entire text block quickly. Then, a lightweight component refines this draft by adding the necessary causal context that parallel methods usually miss. To ensure this hybrid approach remains stable and accurate, the researchers introduced a specialized training method that gradually shifts focus from the fast parallel draft to the precise causal correction. This clever separation allows the system to enjoy the speed of parallel processing without the penalty of poor accuracy.

The results are significant. Tests on Qwen3 models show that Domino achieves up to a 5.49 times speedup in end-to-end inference under the Transformers backend and up to 5.8 times throughput improvement under SGLang serving. This breakthrough suggests that we can finally have our cake and eat it too: the rapid generation of speculative decoding combined with the precision of autoregressive modeling, making AI interactions noticeably faster and more efficient for everyday users.