The Promise of RL for Autoregressive Image Editing
Authors: name: Saba Ahmadi, name: Rabiul Awal, name: Ankur Sikarwar, name: Amirhossein Kazemnejad, name: Ge Ya Luo, name: Juan A. Rodriguez, name: Sai Rajeswar, name: Siva Reddy, name: Christopher Pal, name: Benno Krojer, name: Aishwarya Agrawal

We explore three strategies to enhance performance on a wide range of image
editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and
Chain-of-Thought (CoT) reasoning. In order to study all these components in one
consistent framework, we adopt an autoregressive multimodal model that
processes textual and visual tokens in a unified manner. We find RL combined
with a large multi-modal LLM verifier to be the most effective of these
strategies. As a result, we release EARL: Editing with Autoregression and RL, a
strong RL-based image editing model that performs competitively on a diverse
range of edits compared to strong baselines, despite using much less training
data. Thus, EARL pushes the frontier of autoregressive multimodal models on
image editing. We release our code, training data, and trained models at
https://github.com/mair-lab/EARL.