Interpretable Physics Reasoning and Performance Taxonomy in
  Vision-Language Models
Authors: name: Pranav Pawar, name: Kavish Shah, name: Akshat Bhalani, name: Komal Kasat, name: Dev Mittal, name: Hadi Gala, name: Deepali Patil, name: Nikita Raichada, name: Monali Deshmukh

As Vision-Language Models (VLMs) grow in sophistication, their ability to
perform reasoning is coming under increasing supervision. While they excel at
many tasks, their grasp of fundamental scientific principles, such as physics,
remains an underexplored frontier. To reflect the advancements in these
capabilities, we introduce a novel and accessible framework designed to
rigorously evaluate VLMs on their understanding of 2D physics. Our framework
features a pragmatic scenario generator that creates a diverse testbed of over
400 problems across four core domains: Projectile Motion, Collision Dynamics,
Mechanics, and Fluid Dynamics. Through comprehensive evaluation of four
state-of-the-art VLMs, we demonstrate a strong correlation between model scale
and reasoning ability, with our top-performing model, Qwen2.5-VL-7B, achieving
an overall score of 0.815. We find that while models excel at formulaic
problems, they struggle significantly with domains requiring abstract spatial
reasoning. By designing this framework, we aim to democratize the study of
scientific reasoning in VLMs and foster deeper insights into their capabilities
and limitations.