Why Diffusion Language Models Are Suddenly Everywhere

Why it’s suddenly everywhere

1. 01

   On October 19, Inception launched Mercury 3, reporting 1,068 tokens per second and a one-million-token context window. It was a commercial claim that diffusion could scale without losing its speed advantage.

2. 02

   DiffuSpace followed with Clover-35B-A3B-Preview on October 28. Its 35 billion total parameters include about 3 billion active per token; the company reports 680 tokens/s on one GB300 GPU at batch size one.

3. 03

   Clover’s open model card makes the experiment unusually inspectable: it is derived from Qwen3.6-35B-A3B, uses 32-token blocks by default, and fills confident masked positions in parallel.

4. 04

   A viral Xiaohongshu post compressed those releases into “diffusion will replace large language models within three years.” The claim then became a Zhihu hot question, turning an inference-architecture debate into a mainstream one.

The timeline

1. 2021-07Diffusion learns discrete states

   D3PM extends denoising diffusion beyond continuous pixels. Its absorbing-state variant connects diffusion with masked and autoregressive generation, and the paper demonstrates character-level text generation.

2. 2022Diffusion-LM targets controlled writing

   Xiang Li and colleagues iteratively denoise Gaussian vectors into word vectors. The work stands out on six fine-grained control tasks, but it is not yet a general chatbot at modern LLM scale.

3. 2025-02LLaDA scales masked diffusion to 8B

   An 8-billion-parameter model trained from scratch masks tokens during its forward process and predicts them during reverse generation. It proves competitive with LLaMA3 8B on several in-context-learning evaluations.

4. 2025-05The serving bottleneck gets a name

   Fast-dLLM identifies two practical obstacles: bidirectional denoising cannot simply reuse an autoregressive KV cache, and naïve parallel decoding damages token dependencies. Block-wise caching and confidence-aware decoding address both.

5. 2026-10-19Mercury 3 makes the commercial case

   Inception says its new model is more than 40 times larger than Mercury 1, yet still exceeds 1,000 tokens/s in its own tests. Browser agents, tutoring and search become the showcase workloads.

6. 2026-10-28Clover brings open weights to the argument

   DiffuSpace releases a block-diffusion MoE model with code and weights. Official results put it ahead of similarly sized Qwen models on selected agent and general benchmarks; independent reproduction is the next test.

7. 2026-11-01The nuance becomes a hot take

   Discussion jumps from model forums to a Chinese hot list. “Will diffusion replace LLMs in three years?” overtakes the more useful question: which parts of generation benefit from parallel refinement?

The backstory

The image trick, rewritten as a fill-in-the-blanks game

The popular mental picture of diffusion comes from images: begin with visual noise, then remove some of that noise again and again until a picture appears. Language needs a translation of that idea. Tokens are discrete symbols, not pixel values with a convenient continuum between them.

Researchers pursued two main routes. Diffusion-LM mapped words into continuous vectors and denoised those vectors. D3PM instead corrupted discrete states directly. Its absorbing state offered an especially durable analogy: replace a token with a special masked state, then learn what belongs in the blank. Modern systems such as LLaDA use this masking family.

Generation therefore resembles a repeated fill-in-the-blanks exercise. The model starts with a canvas containing many unknown positions, predicts several of them, keeps some answers and revisits the uncertain remainder. It can look to both the left and right of a blank. An autoregressive model has a stricter routine: read the prefix, emit the next token, append it, and repeat. That routine is easy to understand and extraordinarily successful, but the answer’s length sets a floor on sequential decoding steps.

Parallel does not mean instantaneous

The phrase “generate all the tokens at once” hides the hard part. A diffusion model still performs multiple denoising passes. Update too few positions per pass and the speed advantage evaporates; update too many and separately predicted words may not fit one another. A sentence is not a bag of independent blanks.

Fast-dLLM’s confidence-aware method makes that trade-off explicit. Tokens above a confidence threshold can be revealed together, while uncertain ones wait for another pass. Clover uses the same broad idea with a default threshold of 0.95. Its blocks are also ordered: once a 32-token block is resolved, it joins the prefix for the next block. This is parallel work inside a sequence, not the abolition of sequence.

Caching needs new plumbing too. In an autoregressive transformer, the prefix is fixed, so its key-value activations can be cached while the next token is computed. Bidirectional denoising changes states within the active block across passes, preventing a direct copy of that trick. Fast-dLLM uses approximate block-wise caching; Clover adapts a hybrid Gated DeltaNet backbone and recommends its own SGLang path for serving. These are signs of a maturing stack, but also evidence that established inference infrastructure cannot simply be swapped out untouched.

The comparison with DeepInfer’s serving stack is instructive. DeepInfer squeezes the conventional autoregressive route with cache reuse, scheduling and separated prefill and decode machines. Diffusion attacks the same latency problem by changing the decoding algorithm. A headline tokens-per-second figure says little until both approaches are measured on the same hardware, workload, batch size and quality setting.

What Clover changed—and what it did not

Earlier diffusion language models often carried an asterisk: promising control or parallelism, but weaker general capability than strong autoregressive models. Clover’s significance is that its developers put competitive agent scores and open weights into the same release. The reported 73.6 on SWE-bench Verified and six leading agent evaluations make it more than a decoding curiosity.

The asterisks have changed rather than disappeared. The published results are company-reported. Clover begins with Qwen3.6 weights, so it inherits abilities built by an autoregressive model instead of demonstrating that a diffusion model must be trained from zero. Its model card also acknowledges repetition in very long generations and includes controls to suppress it. Those details do not negate the release; they tell us what is actually being tested.

Nor does “diffusion will replace LLMs” make sense as a literal category statement. A diffusion language model is still a large language model. The proposed replacement is autoregressive generation, one particular way of producing text with an LLM. Even that contest may end in hybrids: block diffusion already retains ordered blocks, while other systems combine autoregressive planning with faster parallel execution.

The likely near-term division of labor is visible in the vendors’ own examples. Agents repeatedly wait for a model before they can open a page, call a tool or inspect a result, so every decoding delay compounds. A fast decision layer such as Jev handles bounded judgments; diffusion models now promise faster general generation within related loops. The useful question is not which architecture wins everything. It is where waiting time can be removed without moving errors somewhere less visible.

Key facts

Conventional decoding

Left to right

Masked diffusion

Iterative refinement

Clover parameters

35B total / 3B active

Clover block default

32 tokens

Mercury 3 claim

1,068 tokens/s

Open research milestone

LLaDA 8B, 2025

What people are saying

Common questions

Is a diffusion language model still an LLM?

Yes. It is large, models language and can perform familiar chat, coding or reasoning tasks. “Diffusion replacing LLMs” is shorthand for diffusion replacing autoregressive LLMs. The more precise distinction is between decoding and training paradigms, not between language models and some unrelated technology.

Does parallel decoding make a whole answer appear in one step?

No. The model refines a masked canvas over several passes, and block-diffusion systems process blocks in order. Parallelism means several positions can be resolved in one pass. The number of passes depends on the decoding policy and the model’s confidence.

What evidence would settle the replacement debate?

Independent comparisons on identical hardware and workloads, with quality held constant, would be a start. They should include long outputs, high concurrency, total compute and end-to-end agent completion rather than a peak tokens-per-second number. Reliable results across those conditions would matter more than a three-year prediction.