InternLM-XComposer2: Mastering Free-form Text-Image Composition and
  Comprehension in Vision-Language Large Model
Authors: name: Xiaoyi Dong, name: Pan Zhang, name: Yuhang Zang, name: Yuhang Cao, name: Bin Wang, name: Linke Ouyang, name: Xilin Wei, name: Songyang Zhang, name: Haodong Duan, name: Maosong Cao, name: Wenwei Zhang, name: Yining Li, name: Hang Yan, name: Yang Gao, name: Xinyue Zhang, name: Wei Li, name: Jingwen Li, name: Kai Chen, name: Conghui He, name: Xingcheng Zhang, name: Yu Qiao, name: Dahua Lin, name: Jiaqi Wang

We introduce InternLM-XComposer2, a cutting-edge vision-language model
excelling in free-form text-image composition and comprehension. This model
goes beyond conventional vision-language understanding, adeptly crafting
interleaved text-image content from diverse inputs like outlines, detailed
textual specifications, and reference images, enabling highly customizable
content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach
that applies additional LoRA parameters exclusively to image tokens to preserve
the integrity of pre-trained language knowledge, striking a balance between
precise vision understanding and text composition with literary talent.
Experimental results demonstrate the superiority of InternLM-XComposer2 based
on InternLM2-7B in producing high-quality long-text multi-modal content and its
exceptional vision-language understanding performance across various
benchmarks, where it not only significantly outperforms existing multimodal
models but also matches or even surpasses GPT-4V and Gemini Pro in certain
assessments. This highlights its remarkable proficiency in the realm of
multimodal understanding. The InternLM-XComposer2 model series with 7B
parameters are publicly available at
https://github.com/InternLM/InternLM-XComposer.