Chapter 2 breaks down the mechanics; chapters 3 and 4 answer “what it changed about the output” and “what it changed for you.”

2. Inside the Model: What Text-to-Video Actually Does

2.1 Two roads to creating something from nothing: adversarial vs denoising

Before 2020, the mainstream answer to “make something from nothing” was the GAN (Generative Adversarial Network): a generator fakes, a discriminator calls out fakes, and both improve through opposition. Elegant, but with three hard flaws — unstable training (any imbalance collapses it), mode collapse (the generator learns one trick and emits the same few faces forever), and poor scalability (bigger models were harder to train, not better).

DDPM (Denoising Diffusion Probabilistic Models, see the table above) took a completely different view: don’t learn how to generate — learn how to denoise. During training, a real image is gradually corrupted with Gaussian noise until it becomes pure static (the forward process), and a network learns at each step to predict the noise that was mixed in. Generation starts from pure noise and subtracts predicted noise step by step (reverse sampling) until an image emerges. The cost is many sampling steps (original DDPM: 1,000), but the payoff is two things GANs never had: extremely stable training and quality that scales smoothly with model size — the precondition for everything scaling-law related that followed.

2.2 The two costs of diffusion: steps and compute

The denoising paradigm works, but it’s expensive in two places:

2.3 Latent diffusion: compress 8× first, generate second

In 2022, Latent Diffusion Models (LDM — the base of Stable Diffusion) made the most “engineering” and most consequential decision in the field: don’t diffuse in pixel space — compress first. A VAE (variational autoencoder) compresses the image 8× per side into a latent space (64× fewer pixels); all diffusion happens in that compact space; the VAE decodes back to pixels at the end.

The significance for users is hard to overstate: an order-of-magnitude drop in compute meant consumer GPUs could run generative models for the first time — and API marginal costs could finally come down. The 2026 sub-$0.05/sec price war, and the entire local-generation ecosystem (ComfyUI and friends), rest on this compression.

2.4 U-Net → DiT: making “add compute” predictable

Early diffusion models used U-Net (a convolutional network) as the denoising backbone. In 2023, DiT (Diffusion Transformer, see the table above) showed that swapping the backbone for a Transformer makes quality rise smoothly and predictably with compute (FID falls monotonically as GFLOPs grow). In 2024, OpenAI’s Sora technical report brought this scaling law into video: chop video into spacetime patches, feed them to a Transformer as tokens, scale parameters, data, and compute together — and capabilities climb in steps.

2.5 From image to video: the time axis and 3D VAE

An image model produces one still frame; video needs a stack of frames that stay coherent. Three technical pieces:

• 3D VAE: compression extends from “one image” to “a block of spacetime” (time × height × width), so an entire clip is compressed at once and frame information naturally shares the latent space.
• Spatiotemporal attention: every DiT token sees both spatial and temporal position, so the model learns “what this person looks like next frame” and “how the background moves as the camera pans.”
• Progressive resolution & frame rate: generate content at low resolution first, then upscale to 1080p and high frame rate — the design that keeps inference costs manageable.

Video Diffusion Models (VDM, Google) and Make-A-Video (Meta) established this route in 2022–23: lean on massive unlabeled video plus a pretrained image model, then learn the time dimension. First-generation output looked like “two seconds, flickering, six-fingered hands” — but the direction was right.

2.6 How text steers the picture

Your prompt isn’t fed directly to the denoising network. A large language model (T5-class) or CLIP encodes the text into vectors; at every denoising step the network reads them via cross-attention, deciding which way to push the current step; and at inference, classifier-free guidance (CFG) computes both the “with prompt” and “without prompt” directions and extrapolates between them, so the result hugs your prompt. Every tuning trick you know — CFG strength, prompt phrasing — is really adjusting the signal-to-noise ratio of this control channel.

3. What Iteration Changed About the Output: Six Dimensions

Twelve years of iteration on one table — typical output level per dimension in three eras (qualitative stars, based on blind-arena reputation and our own tests; not official benchmarks):

4. What Iteration Changed for the User: Wait, Price, Barrier, Failure Modes

4.1 Wait time: from “make coffee” to “just click”

Sampling steps are the direct source of waiting. Three stacked generations changed the wait for a 5-second clip:

The wait didn’t just shrink the annoyance — it changed how you create: with minute-long waits you generate once and curate carefully (slot-machine mode); with second-level waits you iterate live (conversation mode). That shift is the direct technical cause of the interactive creation tools that appeared in 2026.

4.2 Price: the compound interest of latent space and distillation

The price curve is the product of three stacked layers: latent compression (2022, order-of-magnitude compute cut) → architectural efficiency (DiT: more quality per FLOP) → distillation & acceleration (same quality, fewer steps). Add Chinese platforms’ open-source push, and the 2026 landscape looks like this:

• Mainstream API prices span $0.02–0.75/sec, a spread of over 30× (full data in our pricing report).
• The price floor is on Chinese platforms: Jimeng 3.0’s 720P tier costs ¥0.28/sec (~$0.039); blind-test favorite Seedance 2.0 runs about $0.15/sec — 3–5× cheaper than the Western flagship Veo 3.1 Standard ($0.40/sec). Quality and price have decoupled.
• The same model can cost up to 31× more on a different shelf (Seedance 2.0/2.5 across channels: $0.022–0.682); Veo 3.1 spans roughly 13×. Iteration lowers production cost, but shelf markups don’t hand the savings to you — you have to comparison-shop.
• Cautionary tale: Sora 2’s API shut down on 2026-09-24 with no official replacement — however leading the technology, vendor continuity risk belongs to the user.

4.3 Barrier & workflow: from “spells” to storyboards

Note the other side of the falling barrier: the competition barrier fell too. In 2022, knowing how to use the tools was a moat; in 2026 everyone can produce clips — scarcity has moved from “can operate the tool” to “can write the script, compose the shot, and pick the topic.”

4.4 How failure modes evolved: what got fixed, and what didn’t

5. Acceleration: Same Model, Different Experience

Acceleration techniques deliver 2–10× inference speedups on the same base model — the most active layer of the 2026 value-for-money battlefield:

• Consistency distillation: a student model jumps straight to the teacher’s “1,000-step endpoint” in one or a few steps. The landmark work, LCM (Latent Consistency Models), compressed Stable Diffusion sampling to 4–8 steps; DMD (Distribution Matching Distillation) showed one-step sampling can approach teacher quality. The cost is a small loss of diversity.
• Quantization: weights compressed from FP16 to INT8/INT4 cut memory and bandwidth proportionally, with quality loss usually imperceptible. Both cloud and local inference rely on it heavily (see the ONNX Runtime quantization guide).
• Accelerated LoRAs: low-rank adapters tuned for speed on a frozen base model. We tested this route in our MiniMax H3 accelerated LoRA report: in specific scenarios it buys double-digit percentage speedups at very low deployment cost.
• Engineering layer: KV-cache reuse, progressive generation (content at low resolution → upscale), cross-frame KV inheritance — pure efficiency work that changes no principles.

6. What This Means When You Choose a Model

Principles, translated into selection rules — four questions that actually matter:

7. Key Takeaways

• The through-line of the whole evolution: make denoising cheaper. Latent compression (order-of-magnitude compute cut) → DiT (more quality per FLOP) → distillation (order-of-magnitude step cut) — three layers together lowered the cost of a clip by two orders of magnitude.
• The single most output-changing technology was spatiotemporal modeling. Resolution and length improved gradually; frame-to-frame coherence appeared “from zero” — and it’s what let editing software accept AI video at all.
• For users, the biggest change wasn’t picture quality — it was wait time. Second-level generation turned creating from “slot-machine” into “conversation,” a shift that matters as much as any model upgrade.
• Quality and price have decoupled. The blind-test favorite (Seedance line) and the price floor (Jimeng, Volcano) both sit on Chinese platforms; the Western flagship no longer automatically means “best.”
• Native audio starts the next reshuffle. With dialogue and SFX inside the model, dubbing and sound workflows are partially moving upstream into generation — structural, not incremental.
• Technical dividends don’t reach users automatically. A 31× cross-channel spread for the same model, and Sora 2’s unannounced shutdown, prove that “where you buy and whether it survives” matters more than “which is strongest.”