RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language
  Models
Authors: name: Yeongtak Oh, name: Jisoo Mok, name: Dohyun Chung, name: Juhyeon Shin, name: Sangha Park, name: Johan Barthelemy, name: Sungroh Yoon

Recent multi-modal large language models (MLLMs) often struggle to generate
personalized image captions, even when trained on high-quality captions. In
this work, we observe that such limitations persist in existing
post-training-based MLLM personalization methods. Specifically, despite being
post-tuned with large-scale caption data through supervised fine-tuning (SFT),
these models frequently fail to produce faithful descriptions in real-world
scenarios, such as multi-concept image captioning. However, acquiring
large-scale, high-quality captions for such complex settings is both costly and
difficult. To address the data-centric nature of SFT, we propose a
reinforcement learning (RL)-based post-training framework. To the best of our
knowledge, this is the first RL-based approach to post-train MLLMs for
personalized image captioning. Our method significantly enhances both visual
recognition and personalized generation capabilities of MLLMs, and consistently
outperforms existing SFT-based baselines, especially in the challenging
multi-concept image captioning task.