DICE: Discrete Inversion Enabling Controllable Editing for Multinomial
  Diffusion and Masked Generative Models
Authors: name: Xiaoxiao He; name: Ligong Han; name: Quan Dao; name: Song Wen; name: Minhao Bai; name: Di Liu; name: Han Zhang; name: Martin Renqiang Min; name: Felix Juefei-Xu; name: Chaowei Tan; name: Bo Liu; name: Kang Li; name: Hongdong Li; name: Junzhou Huang; name: Faez Ahmed; name: Akash Srivastava; name: Dimitris Metaxas
Published: 2024-10-10 17:59:48

Abstract: Discrete diffusion models have achieved success in tasks like image
generation and masked language modeling but face limitations in controlled
content editing. We introduce DICE (Discrete Inversion for Controllable
Editing), the first approach to enable precise inversion for discrete diffusion
models, including multinomial diffusion and masked generative models. By
recording noise sequences and masking patterns during the reverse diffusion
process, DICE enables accurate reconstruction and flexible editing of discrete
data without the need for predefined masks or attention manipulation. We
demonstrate the effectiveness of DICE across both image and text domains,
evaluating it on models such as VQ-Diffusion, Paella, and RoBERTa. Our results
show that DICE preserves high data fidelity while enhancing editing
capabilities, offering new opportunities for fine-grained content manipulation
in discrete spaces. For project webpage, see
https://hexiaoxiao-cs.github.io/DICE/.