NeurIPS 2025

Fourier Token Merging: Understanding and
Capitalizing Frequency Domain for Efficient Image
Generation
Jiesong Liu
Department of Computer Science
North Carolina State University
Raleigh, NC 27606
jliu93@ncsu.edu
Xipeng (Gracen) Shen
Department of Computer Science
North Carolina State University
Raleigh, NC 27606
xshen5@ncsu.edu
Abstract
Image generation requires intensive computations and faces challenges due to long
latency. Exploiting redundancy in the input images and intermediate representa-
tions throughout the neural network pipeline is an effective way to accelerate image
generation. Token merging (ToMe) exploits similarities among input tokens by
clustering them and merges similar tokens into one, thus significantly reducing the
number of tokens that are fed into the transformer block. This work introduces
Fourier Token Merging, a new method for understanding and capitalizing frequency
domain for efficient image generation. By introducing frequency token merging, we
find that transforming the token into the frequency domain representation for clus-
tering can better exert the ability of clustering based on the underlying redundancy
after de-correlation. Through analytical and empirical studies, we demonstrate the
benefits of using Fourier clustering over the original time domain clustering. We
experimented Fourier Token Merging on the stable diffusion model, and the results
show up to 25% reduction in latency without impairing image quality. The code is
available at https://github.com/Fred1031/Fourier-Token-Merging.
1
Introduction
Image generation has achieved impressive results through advanced generative models such as stable
diffusion [22], but these models face significant computational challenges and long latency, especially
when deployed on resource-constrained devices [5, 12, 27]. A primary avenue for addressing
these issues involves exploiting redundancies within input images and intermediate neural network
representations to streamline computation. T...