TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models
Authors: name: Gilad Deutch; name: Rinon Gal; name: Daniel Garibi; name: Or Patashnik; name: Daniel Cohen-Or
Published: 2024-08-01 17:27:28

Abstract: Diffusion models have opened the path to a wide range of text-based image
editing frameworks. However, these typically build on the multi-step nature of
the diffusion backwards process, and adapting them to distilled, fast-sampling
methods has proven surprisingly challenging. Here, we focus on a popular line
of text-based editing frameworks - the ``edit-friendly'' DDPM-noise inversion
approach. We analyze its application to fast sampling methods and categorize
its failures into two classes: the appearance of visual artifacts, and
insufficient editing strength. We trace the artifacts to mismatched noise
statistics between inverted noises and the expected noise schedule, and suggest
a shifted noise schedule which corrects for this offset. To increase editing
strength, we propose a pseudo-guidance approach that efficiently increases the
magnitude of edits without introducing new artifacts. All in all, our method
enables text-based image editing with as few as three diffusion steps, while
providing novel insights into the mechanisms behind popular text-based editing
approaches.