LLaVA-OneVision: Easy Visual Task Transfer
Authors: name: Bo Li; name: Yuanhan Zhang; name: Dong Guo; name: Renrui Zhang; name: Feng Li; name: Hao Zhang; name: Kaichen Zhang; name: Yanwei Li; name: Ziwei Liu; name: Chunyuan Li
Published: 2024-08-06 17:59:44

Abstract: We present LLaVA-OneVision, a family of open large multimodal models (LMMs)
developed by consolidating our insights into data, models, and visual
representations in the LLaVA-NeXT blog series. Our experimental results
demonstrate that LLaVA-OneVision is the first single model that can
simultaneously push the performance boundaries of open LMMs in three important
computer vision scenarios: single-image, multi-image, and video scenarios.
Importantly, the design of LLaVA-OneVision allows strong transfer learning
across different modalities/scenarios, yielding new emerging capabilities. In
particular, strong video understanding and cross-scenario capabilities are
demonstrated through task transfer from images to videos.