ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image
  Restoration
Authors: name: Yongsheng Yu, name: Haitian Zheng, name: Zhifei Zhang, name: Jianming Zhang, name: Yuqian Zhou, name: Connelly Barnes, name: Yuchen Liu, name: Wei Xiong, name: Zhe Lin, name: Jiebo Luo

Recent progress in generative models has significantly improved image
restoration capabilities, particularly through powerful diffusion models that
offer remarkable recovery of semantic details and local fidelity. However,
deploying these models at ultra-high resolutions faces a critical trade-off
between quality and efficiency due to the computational demands of long-range
attention mechanisms. To address this, we introduce ZipIR, a novel framework
that enhances efficiency, scalability, and long-range modeling for high-res
image restoration. ZipIR employs a highly compressed latent representation that
compresses image 32x, effectively reducing the number of spatial tokens, and
enabling the use of high-capacity models like the Diffusion Transformer (DiT).
Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that
structures the latent space into sub-bands to ease diffusion training. Trained
on full images up to 2K resolution, ZipIR surpasses existing diffusion-based
methods, offering unmatched speed and quality in restoring high-resolution
images from severely degraded inputs.