Title: CL-Gen: An Inference-Time Iterative Optimization Framework for Reference-Consistent Image Generation Based on Closed-Loop Control
Primary area: generative models
Keywords: Image generation, Closed loop control, Iterative optimization, Reference consistency

Abstract: Controllable image generation technology enables precise content synthesis based on user-provided reference conditions, garnering significant research attention. However, existing methods still face significant challenges in maintaining reference consistency, as they lack the observation and error correction for the reference consistency of generated images. Inspired by the concept of closed-loop systems in control theory, we propose a framework that enhances reference consistency through an iterative optimization scheme during inference time. It takes the image generation model as the control plant, observes and feeds back the actual generation state, and then adjusts the input of the generation model through a modified PID controller to enhance reference consistency. This framework supports a variety of controllable generation methods and different types of control conditions. Moreover, it is easy to implement, requiring only the addition of a few lines of code without the need for extra training. We validate the application of this framework in three key tasks: identity-preserving portrait generation, pose-controlled generation, and depth-controlled generation. For identity-preserving portrait generation, our method improves facial similarity by 12.07\%. For pose-controlled and depth-controlled generation, errors are reduced by 32.64\% and 33.49\%, respectively. This work not only provides a solution for reference-consistent image generation but also offers a new perspective for broader applications. Our code will be released upon publication.

TL;DR: This paper proposes a closed-loop framework to iteratively optimize reference consistency in generated images at inference time.