AdaSplash-2: Faster Differentiable Sparse Attention

The real contribution here is narrower than 'faster sparse attention' suggests: AdaSplash-2 specifically targets the iterative solver used to compute the normalization constant in α-entmax attention, replacing expensive convergence loops with a histogram-based warm start that lands close enough to the answer that only one or two iterations remain. The sparsity mechanism itself is unchanged.

This fits into a cluster of work on making transformers cheaper to run without discarding their expressive properties. The 'Stability and Generalization in Looped Transformers' paper covered here the same day is a useful counterpoint: that work shows architecturally that recall plus normalization is necessary for stable fixed points, while AdaSplash-2 is attacking the computational cost of normalization from the opposite direction, treating it as an engineering problem rather than a theoretical one. Neither paper cites the other's framing, but together they highlight that normalization in attention is both a correctness requirement and a performance bottleneck worth solving independently.

The credibility test is whether AdaSplash-2's iteration reduction holds at sequence lengths above 32k tokens in a publicly reproducible benchmark. If an independent group reproduces the 1-2 iteration claim on a standard long-context suite within the next two quarters, the histogram initialization approach will likely be adopted more broadly.