How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal
  Models with Open-Source Suites
Authors: name: Zhe Chen, name: Weiyun Wang, name: Hao Tian, name: Shenglong Ye, name: Zhangwei Gao, name: Erfei Cui, name: Wenwen Tong, name: Kongzhi Hu, name: Jiapeng Luo, name: Zheng Ma, name: Ji Ma, name: Jiaqi Wang, name: Xiaoyi Dong, name: Hang Yan, name: Hewei Guo, name: Conghui He, name: Zhenjiang Jin, name: Chao Xu, name: Bin Wang, name: Xingjian Wei, name: Wei Li, name: Wenjian Zhang, name: Lewei Lu, name: Xizhou Zhu, name: Tong Lu, name: Dahua Lin, name: Yu Qiao

In this report, we introduce InternVL 1.5, an open-source multimodal large
language model (MLLM) to bridge the capability gap between open-source and
proprietary commercial models in multimodal understanding. We introduce three
simple improvements: (1) Strong Vision Encoder: we explored a continuous
learning strategy for the large-scale vision foundation model -- InternViT-6B,
boosting its visual understanding capabilities, and making it can be
transferred and reused in different LLMs. (2) Dynamic High-Resolution: we
divide images into tiles ranging from 1 to 40 of 448times448 pixels
according to the aspect ratio and resolution of the input images, which
supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we
carefully collected a high-quality bilingual dataset that covers common scenes,
document images, and annotated them with English and Chinese question-answer
pairs, significantly enhancing performance in OCR- and Chinese-related tasks.
We evaluate InternVL 1.5 through a series of benchmarks and comparative
studies. Compared to both open-source and proprietary models, InternVL 1.5
shows competitive performance, achieving state-of-the-art results in 8 of 18
benchmarks. Code has been released at https://github.com/OpenGVLab/InternVL.