The Value of Covariance Matching in Gaussian DDPMs and the Lanczos Sampler
Researchers have identified a fundamental limitation in standard diffusion model reverse processes and proposed a solution that improves convergence rates. Isotropic covariance assumptions in Gaussian DDPMs incur path-space KL divergence errors that scale as Omega(1/T), but matching the full posterior covariance reduces this to O(1/T^2), a quadratic improvement. The Lanczos Gaussian Sampler makes this theoretically superior approach computationally tractable without requiring matrix inversion or additional training. This advance matters for classifier-guided generation and any application where trajectory fidelity across all denoising steps, not just final output, affects downstream performance. The result tightens the theoretical foundations of diffusion models and opens practical paths to higher-quality conditional generation.62
