Expanding Performance Boundaries of Open-Source Multimodal Models with
  Model, Data, and Test-Time Scaling
Authors: name: Zhe Chen; name: Weiyun Wang; name: Yue Cao; name: Yangzhou Liu; name: Zhangwei Gao; name: Erfei Cui; name: Jinguo Zhu; name: Shenglong Ye; name: Hao Tian; name: Zhaoyang Liu; name: Lixin Gu; name: Xuehui Wang; name: Qingyun Li; name: Yimin Ren; name: Zixuan Chen; name: Jiapeng Luo; name: Jiahao Wang; name: Tan Jiang; name: Bo Wang; name: Conghui He; name: Botian Shi; name: Xingcheng Zhang; name: Han Lv; name: Yi Wang; name: Wenqi Shao; name: Pei Chu; name: Zhongying Tu; name: Tong He; name: Zhiyong Wu; name: Huipeng Deng; name: Jiaye Ge; name: Kai Chen; name: Min Dou; name: Lewei Lu; name: Xizhou Zhu; name: Tong Lu; name: Dahua Lin; name: Yu Qiao; name: Jifeng Dai; name: Wenhai Wang
Published: 2024-12-06 18:57:08

Abstract: We introduce InternVL 2.5, an advanced multimodal large language model (MLLM)
series that builds upon InternVL 2.0, maintaining its core model architecture
while introducing significant enhancements in training and testing strategies
as well as data quality. In this work, we delve into the relationship between
model scaling and performance, systematically exploring the performance trends
in vision encoders, language models, dataset sizes, and test-time
configurations. Through extensive evaluations on a wide range of benchmarks,
including multi-discipline reasoning, document understanding, multi-image /
video understanding, real-world comprehension, multimodal hallucination
detection, visual grounding, multilingual capabilities, and pure language
processing, InternVL 2.5 exhibits competitive performance, rivaling leading
commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is
the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a
3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing
strong potential for test-time scaling. We hope this model contributes to the
open-source community by setting new standards for developing and applying
multimodal AI systems. HuggingFace demo see
https://huggingface.co/spaces/OpenGVLab/InternVL