Diffusion models

Plain English. Models that generate by starting with noise and refining the whole output in parallel over several passes — rather than producing it one token at a time. The technique behind image and video generation, now being applied to text.

Why it moves money. Parallel generation is a throughput play. An autoregressive model's speed is capped by producing tokens sequentially; a diffusion language model refines everything at once, and the claimed speeds — over a thousand tokens a second on a single GPU — would rewrite inference economics if quality holds. For agents that consume millions of tokens per task, generation speed is a cost and latency line, so an architecture that trades a little quality for a large multiple of throughput has a real market even if it never leads benchmarks.

What to watch. Independent quality evaluations at the claimed speeds. Vendor demos measure the easy half; the question is whether diffusion text holds up on long, structured outputs where autoregressive models are strongest.

From the signals. DiffusionGemma claimed ~1,500 output tokens a second on a single H100. Mercury 2.5 claimed frontier-lite quality at 1,107 tokens a second. O'Reilly read Google's reshuffle as a bet on diffusion over the frontier.

Further reading. Ho et al., Denoising Diffusion Probabilistic Models (2020)