Title: ICPRL: Acquiring Physical Intuition from Interactive Control
Primary area: foundation or frontier models, including LLMs
Keywords: VLM Agents, Physical Reasoning, Dynamic Environments

Abstract: VLMs excel at static perception but falter in interactive reasoning in dynamic physical environments, which demands planning and adaptation to dynamic outcomes. Existing physical reasoning methods often depend on abstract symbolic inputs or lack the ability to learn and adapt from direct, pixel-based visual interaction in novel scenarios. We introduce **ICPRL** (In-Context Physical Reinforcement Learning), a framework inspired by In-Context Reinforcement Learning (ICRL) that empowers VLMs to acquire physical intuition and adapt their policies in-context. Our approach trains a vision-grounded policy model to internalize a policy-improvement process by learning across diverse multi-episode interaction histories. 
This enables the agent to adapt its strategy at test time to unseen environments by conditioning on recent interactions, without requiring any weight updates. This adaptive policy works in concert with a separately trained world model that provides explicit physical reasoning by predicting the results of potential actions. 
At inference, the policy proposes candidate actions, while the world model functions as an in-context planner, guiding a search procedure to select the most promising action. Evaluated on the diverse physics-based puzzle-solving tasks in the DeepPHY benchmark, **ICPRL** demonstrates significant improvements across both its policy-only and world-model-augmented stages. Notably, these gains are retained in unseen physical environments, demonstrating that our framework facilitates genuine in-context acquisition of the environment's physical dynamics from interactive experience.