IQBench: How "Smart'' Are Vision-Language Models? A Study with Human IQ
  Tests
Authors: name: Tan-Hanh Pham; name: Phu-Vinh Nguyen; name: Dang The Hung; name: Bui Trong Duong; name: Vu Nguyen Thanh; name: Chris Ngo; name: Tri Quang Truong; name: Truong-Son Hy
Published: 2025-05-17 13:24:08

Abstract: Although large Vision-Language Models (VLMs) have demonstrated remarkable
performance in a wide range of multimodal tasks, their true reasoning
capabilities on human IQ tests remain underexplored. To advance research on the
fluid intelligence of VLMs, we introduce **IQBench**, a new benchmark designed
to evaluate VLMs on standardized visual IQ tests. We focus on evaluating the
reasoning capabilities of VLMs, which we argue are more important than the
accuracy of the final prediction. **Our benchmark is visually centric,
minimizing the dependence on unnecessary textual content**, thus encouraging
models to derive answers primarily from image-based information rather than
learned textual knowledge. To this end, we manually collected and annotated 500
visual IQ questions to **prevent unintentional data leakage during training**.
Unlike prior work that focuses primarily on the accuracy of the final answer,
we evaluate the reasoning ability of the models by assessing their explanations
and the patterns used to solve each problem, along with the accuracy of the
final prediction and human evaluation. Our experiments show that there are
substantial performance disparities between tasks, with models such as
`o4-mini`, `gemini-2.5-flash`, and `claude-3.7-sonnet` achieving the highest
average accuracies of 0.615, 0.578, and 0.548, respectively. However, all
models struggle with 3D spatial and anagram reasoning tasks, highlighting
significant limitations in current VLMs' general reasoning abilities. In terms
of reasoning scores, `o4-mini`, `gemini-2.5-flash`, and `claude-3.7-sonnet`
achieved top averages of 0.696, 0.586, and 0.516, respectively. These results
highlight inconsistencies between the reasoning processes of the models and
their final answers, emphasizing the importance of evaluating the accuracy of
the reasoning in addition to the final predictions.