FlowOpt: Fast Optimization Through Whole Flow Processes for
  Training-Free Editing
Authors: name: Or Ronai, name: Vladimir Kulikov, name: Tomer Michaeli

The remarkable success of diffusion and flow-matching models has ignited a
surge of works on adapting them at test time for controlled generation tasks.
Examples range from image editing to restoration, compression and
personalization. However, due to the iterative nature of the sampling process
in those models, it is computationally impractical to use gradient-based
optimization to directly control the image generated at the end of the process.
As a result, existing methods typically resort to manipulating each timestep
separately. Here we introduce FlowOpt - a zero-order (gradient-free)
optimization framework that treats the entire flow process as a black box,
enabling optimization through the whole sampling path without backpropagation
through the model. Our method is both highly efficient and allows users to
monitor the intermediate optimization results and perform early stopping if
desired. We prove a sufficient condition on FlowOpt's step-size, under which
convergence to the global optimum is guaranteed. We further show how to
empirically estimate this upper bound so as to choose an appropriate step-size.
We demonstrate how FlowOpt can be used for image editing, showcasing two
options: (i) inversion (determining the initial noise that generates a given
image), and (ii) directly steering the edited image to be similar to the source
image while conforming to a target text prompt. In both cases, FlowOpt achieves
state-of-the-art results while using roughly the same number of neural function
evaluations (NFEs) as existing methods. Code and examples are available on the
project's webpage.