EchoDistill: Bidirectional Concept Distillation for One-Step Diffusion
  Personalization
Authors: name: Yixiong Yang, name: Tao Wu, name: Senmao Li, name: Shiqi Yang, name: Yaxing Wang, name: Joost van de Weijer, name: Kai Wang

Recent advances in accelerating text-to-image (T2I) diffusion models have
enabled the synthesis of high-fidelity images even in a single step. However,
personalizing these models to incorporate novel concepts remains a challenge
due to the limited capacity of one-step models to capture new concept
distributions effectively. We propose a bidirectional concept distillation
framework, EchoDistill, to enable one-step diffusion personalization (1-SDP).
Our approach involves an end-to-end training process where a multi-step
diffusion model (teacher) and a one-step diffusion model (student) are trained
simultaneously. The concept is first distilled from the teacher model to the
student, and then echoed back from the student to the teacher. During the
EchoDistill, we share the text encoder between the two models to ensure
consistent semantic understanding. Following this, the student model is
optimized with adversarial losses to align with the real image distribution and
with alignment losses to maintain consistency with the teacher's output.
Furthermore, we introduce the bidirectional echoing refinement strategy,
wherein the student model leverages its faster generation capability to
feedback to the teacher model. This bidirectional concept distillation
mechanism not only enhances the student ability to personalize novel concepts
but also improves the generative quality of the teacher model. Our experiments
demonstrate that this collaborative framework significantly outperforms
existing personalization methods over the 1-SDP setup, establishing a novel
paradigm for rapid and effective personalization in T2I diffusion models.