Gemini vs GPT-4V: A Preliminary Comparison and Combination of
  Vision-Language Models Through Qualitative Cases
Authors: name: Zhangyang Qi; name: Ye Fang; name: Mengchen Zhang; name: Zeyi Sun; name: Tong Wu; name: Ziwei Liu; name: Dahua Lin; name: Jiaqi Wang; name: Hengshuang Zhao
Published: 2023-12-22 18:59:58

Abstract: The rapidly evolving sector of Multi-modal Large Language Models (MLLMs) is
at the forefront of integrating linguistic and visual processing in artificial
intelligence. This paper presents an in-depth comparative study of two
pioneering models: Google's Gemini and OpenAI's GPT-4V(ision). Our study
involves a multi-faceted evaluation of both models across key dimensions such
as Vision-Language Capability, Interaction with Humans, Temporal Understanding,
and assessments in both Intelligence and Emotional Quotients. The core of our
analysis delves into the distinct visual comprehension abilities of each model.
We conducted a series of structured experiments to evaluate their performance
in various industrial application scenarios, offering a comprehensive
perspective on their practical utility. We not only involve direct performance
comparisons but also include adjustments in prompts and scenarios to ensure a
balanced and fair analysis. Our findings illuminate the unique strengths and
niches of both models. GPT-4V distinguishes itself with its precision and
succinctness in responses, while Gemini excels in providing detailed, expansive
answers accompanied by relevant imagery and links. These understandings not
only shed light on the comparative merits of Gemini and GPT-4V but also
underscore the evolving landscape of multimodal foundation models, paving the
way for future advancements in this area. After the comparison, we attempted to
achieve better results by combining the two models. Finally, We would like to
express our profound gratitude to the teams behind GPT-4V and Gemini for their
pioneering contributions to the field. Our acknowledgments are also extended to
the comprehensive qualitative analysis presented in 'Dawn' by Yang et al. This
work, with its extensive collection of image samples, prompts, and
GPT-4V-related results, provided a foundational basis for our analysis.