SymDPO: Boosting In-Context Learning of Large Multimodal Models with
  Symbol Demonstration Direct Preference Optimization
Authors: name: Hongrui Jia; name: Chaoya Jiang; name: Haiyang Xu; name: Wei Ye; name: Mengfan Dong; name: Ming Yan; name: Ji Zhang; name: Fei Huang; name: Shikun Zhang
Published: 2024-11-17 08:29:14

Abstract: As language models continue to scale, Large Language Models (LLMs) have
exhibited emerging capabilities in In-Context Learning (ICL), enabling them to
solve language tasks by prefixing a few in-context demonstrations (ICDs) as
context. Inspired by these advancements, researchers have extended these
techniques to develop Large Multimodal Models (LMMs) with ICL capabilities.
However, existing LMMs face a critical issue: they often fail to effectively
leverage the visual context in multimodal demonstrations and instead simply
follow textual patterns. This indicates that LMMs do not achieve effective
alignment between multimodal demonstrations and model outputs. To address this
problem, we propose Symbol Demonstration Direct Preference Optimization
(SymDPO). Specifically, SymDPO aims to break the traditional paradigm of
constructing multimodal demonstrations by using random symbols to replace text
answers within instances. This forces the model to carefully understand the
demonstration images and establish a relationship between the images and the
symbols to answer questions correctly. We validate the effectiveness of this
method on multiple benchmarks, demonstrating that with SymDPO, LMMs can more
effectively understand the multimodal context within examples and utilize this
knowledge to answer questions better.