PhyX: Does Your Model Have the "Wits" for Physical Reasoning?
Authors: name: Hui Shen, name: Taiqiang Wu, name: Qi Han, name: Yunta Hsieh, name: Jizhou Wang, name: Yuyue Zhang, name: Yuxin Cheng, name: Zijian Hao, name: Yuansheng Ni, name: Xin Wang, name: Zhongwei Wan, name: Kai Zhang, name: Wendong Xu, name: Jing Xiong, name: Ping Luo, name: Wenhu Chen, name: Chaofan Tao, name: Zhuoqing Mao, name: Ngai Wong

Existing benchmarks fail to capture a crucial aspect of intelligence:
physical reasoning, the integrated ability to combine domain knowledge,
symbolic reasoning, and understanding of real-world constraints. To address
this gap, we introduce PhyX: the first large-scale benchmark designed to assess
models capacity for physics-grounded reasoning in visual scenarios. PhyX
includes 3K meticulously curated multimodal questions spanning 6 reasoning
types across 25 sub-domains and 6 core physics domains: thermodynamics,
electromagnetism, mechanics, modern physics, optics, and wave\&acoustics. In
our comprehensive evaluation, even state-of-the-art models struggle
significantly with physical reasoning. GPT-4o, Claude3.7-Sonnet, and
GPT-o4-mini achieve only 32.5\%, 42.2\%, and 45.8\% accuracy
respectively-performance gaps exceeding 29\% compared to human experts. Our
analysis exposes critical limitations in current models: over-reliance on
memorized disciplinary knowledge, excessive dependence on mathematical
formulations, and surface-level visual pattern matching rather than genuine
physical understanding. We provide in-depth analysis through fine-grained
statistics, detailed case studies, and multiple evaluation paradigms to
thoroughly examine physical reasoning capabilities. To ensure reproducibility,
we implement a compatible evaluation protocol based on widely-used toolkits
such as VLMEvalKit, enabling one-click evaluation.