Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learning
Authors: Zhe Hu
Jing Li
Zhongzhu Pu
Hou Pong Chan
Yu Yin
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal Decision Making
Situational Reasoning
Vision-Language Model Reasoning
Primary area: applications

Abstract: Vision Language Models exhibit impressive performance for various tasks, yet they often lack the sophisticated situational reasoning required for complex decision-making. This paper shows that VLMs can achieve surprisingly strong decision-making performance when visual scenes are replaced by textual descriptions, suggesting foundational reasoning can be effectively learned from language. Motivated by this insight, we propose Praxis-VLM, a reasoning VLM for vision-grounded decision-making. Praxis-VLM employs the GRPO algorithm on textual scenarios to instill robust reasoning capabilities, where models learn to evaluate actions and their consequences. These reasoning skills, acquired purely from text, successfully transfer to multimodal inference with visual inputs, significantly reducing reliance on scarce paired image-text training data. Experiments across diverse decision-making benchmarks demonstrate that Praxis-VLM substantially outperforms standard supervised fine-tuning, exhibiting superior performance and generalizability. Further analysis confirms that our models engage in explicit and effective reasoning, underpinning their enhanced performance and adaptability.

Traxis-VLMe Vision-Grounded Decision Making via Text-Driven Reinforcement Learning
Zhe Hu Jing Li Zhongzhu Pu Hou Pong Chan O, Yu Yin
Department of Computing, The Hong Kong Polytechnic University, O Tsinghua University
InspireOmmi AI, v Alibaba Group. * Case Western Reserve University
zhe-Gerek. pol.:eecoonnt....y...... com httpa////o/..eeeeee///rrriiia VLM
Situation
Questions
Situation
Questions
Choose the most normatively relevant appropriate action perform nes.
Textual
Your geal You soop your honds. Based the imago, you neat?
Phint ron phato of product and the schich
Situation
shou
condition
nearyy.
(A) go the bottroom
stgff
(is find the butritué
impressions dicrectll..
(C) get the tub
a pictone of product and discoss its features with the person sext you
(0) switshh on the bumktub fouset
Decision Making
@thinkka The image shows bathroom with a bothtub, small table with a plant and box, door, and a rug. The question
ithinka The shagesshow a store sisle with various products, incldding poper towels, tissues, and dispssbble cups. The person the video appears to be holding
asks what action Here's the reasoning
after having
bottle
dtsposable possibly considering purchaning Here the analysis of offech
seeme inapproprist because person already holding product, and there no ind catoon that they neyd help making choiee between (A) Go to the bothroom This is air eiready done since we are the bathroom.
At..
Find the bothrub
bathtub is visible the
diffrrent products.
Praxds- VLM
image, so this step stephas been completed.
B:... This action is also iappropp ate becouse there no indication that the penson simply holdings
(G) Get in the tub This not a feasible action becouse the it. logical stop
needs assisannce with the condition of products. the Its condition
flext- Driven
task fill bathtub with water,
showing
needing information
Switch bothitb foucet to the bothtub with weteris/thinke
Dr... This act istoonisppppeer isppproprate berause the personis holding a product, and
Traiing)
discussing its features could be helpful if they are unuure about the product or want more inform atmonationiie (0/
kanswers Given the context the question and the Image,
the most sporoor late action to (D) switch on the bathtub
sanswermitven the context and the norm normative oppnogrtateness the behavtor, the most appr opriatttte action (D) s/answers
faucet watarv//naeer??
Figure 1: Wlustrative examples of Praxis- VLM's decision-making process. Employing text-driven training. Praxis- VLM performs sophisticated reasoning by analyzing visual situations, posing relevant questions, and generating reasoned textual responses to support multimodal decision-making.
Abstract
Vision Language Models exhibit impressive performance for various tasks, yet they often lack the sophisticated situational reasoning required for complex decision- making. This paper shows that VLMs can achieve surprisingly strong decision- making performance when visual scenes are replaced by textual descriptions, suggesting foundational reasoning be effectively learned from language. Mo- tivated by this insight, we propose Praxis- VLM, a reasoning VLM for vision- grounded decision-making. Praxis-VLM employs the GRPO algorithm textual scenarios to instill robust reasoning capabilities, where models learn to evaluate actions and their consequences. These reasoning skills, acquired purely from text, successfully transfer to multimodal inference with visual inputs, significantly reducing reliance on scarce paired image-text training data. Experiments across diverse decision-man makiin benchmarks demonstrate that Praxis- VLM substantially outperforms standard supervised fine-tuning, exhibiting superior performance and
"Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

generalizability. Further analysis confirms that our models engage in explicit and effective reasoning, underpinning their enhanced performance and adaptability.
1 Introduction
"Language is the dress of thought." Samuel Johnson
Developing systems capable of complex real-world interaction necessitates robust, vision-grounded situational terision-makingg [1-3]. Vision Language Models (VLMs) exhibited immense promise for this purpose, offering foundation for agents that can perceive and understand visual environments [4]. However, current VLMs often lack the explicit reasoning capabilities needed to parse nuanced visual scenarios and make optimal decisions [5-7]. This limitation hinders their deployment in crucial applications, like robotics [1-3] and interactive assistants [8, 91, where the capacity to "think before decide," " much like humans do, is paramount for safe and effective operation.
Meanwhile, advancements in recent large language models (LLMs) highlight the potential of multi- step reasoning for tackling complicated tasks [10, 11]. Recent efforts have attempted enhance VLMs with sophisticated reasoning capability from text-based models [12-14]. These approaches typically utilize reasoning-orientee supervise VLM training. However, they rely heavily on large-scale, high-qualig vision data paired with textual ground- -ruth answers, which are notoriously expensive and laborious to curate across diverse real- -world scenarios [15-17]. The challenge of obtaining such paired image-text training data becomes even more pronounced in real-world decision-makee makinn contexts with diverse situations.
LLMs to generate reasoning chains, which are then used to
This data acquisition challenge consequently motivates us to investigate the fundamental nature of decision-making abilities and their reliance direct multimodal training. An essential question then arises: Is the core of decision-making ability exclusively tied to direct multimodal experience? If not, there may be more cost-effective pathways to improve the multimodal decision-making ability of VLMs. To address this, we conduct preliminary analysis ($ 2) and find surprisingly effective yet underexplored alternative: when visual situations are represented by textual descriptions, even standard VLMs could achieve comparative or even improved performance on complex multimodal decision-maker benchmarks like VIVA [18] and PCA-Bench [19]. This observation sparked our central hypothesis: fundamental decision decision-making and reasoning capabilities can be disentangled from visual perception and learned primarily through language-basee basse representations, which can then be effectively transferred to visually grounded contexts during inference. This notion resonates with the mental model theory [20], which posits that humans construct internal, often language-based, representations of situations to reason, predict outcomes, and guide their decisions, later applying these internal models to sensory experiences and act upon real-world situations.
Motivated by this insight, we propose Praxis-VLM, a reasoning VLM that learns high-level decision- making principles from language and applies this "praxis" cally, we begin by constructing a text-based training corpus where visual situations are articulated through text descriptions, mitigating the need for image-text paired data. Then, to foster robust and transferable reasoning- ttheinnggthhaaa ability to "think before decide"-wv we employ Reinforcement Learning (RL) approach. Specifically, we employ GRPO algorithm [21] with multi-stage training to encourage the model to generate explicit reasoning chains before reaching a decision. To facilitate effective learning, we introduce a novel adaptive R/ reward that targets different skills at each training stage. The reasoning abilities acquired through this process are then transferred when Praxis-VIMM processes actual visual inputs during multimodal inference. Illustrative examples are shown in Figure 1.
griiineg environments. Specifi-
To evaluate PraM we adopt challenging decision- decision-making benchmarks spanning diverse tasks: VIVA for human-centered situations, PCA-Bench for embodied agent tasks, and EgoNormia [22] for first-person video understanding. The results show that Praxis- VLM outperforms both the vanilla VLMs and SFT baselines. More importantly, it exhibits remarkable generalizability, suggesting that the reasoning skills acquired from the text are indeed fundamental and transferable. Moreover, in-depth analysis reveals that Praxis-VLA considers multiple meaningful dimensions of decision- making, such as situation analysis, consequence evaluation, safety considerations, and norm adherence (S 5.4). This underpins both the improved decision quality and the potential for cross-domain transfer. Finally, the analysis of common reasoning errors further provides valuable insight for future research.

---

In summary, our main contributions are: (1) We show the potential of leveraging language medium for instilling transferable reasoning skills in VLMs for situational decision-mant decision-making; (2) Building on this, we propose a text-based RI. paradigm and introduce Praxis-VI.M, a novel model that learns high-level decision-makinn principles from language and grounds them in concrete, multimodal scenarios via an adaptive R1 reward; (3) Through extensive experiments and analyses across three diverse benchmarks, we demonstrate Praxis- VLM's superior reasoning and generalization capabilities for decision-making charting a practical and data-efficient path for VLM training.
2 Preliminary Analysis
The primary goal of this research is to enhance the vision-grounded situational decision-making capabilities of VLMss enabling them to effectively reason about visually perceived situations and take appropriate actions. While recent advancements show promise in equipping models with thinking processes to tackle complex reasoning tasks [23, 24), a significant bottleneck remains: the scarcity of large-scale, annotated datasets that pair visual inputs with optimal actions and reasoning steps.
To investigate alternative pathways for developing decision-making skills, we preliminarily analyze the performance of VLMs under different input conditions. We evaluated Qwen2.5-VE VL [25], on two vision-gruvs grounded decisiion-aaingg benchmarks: VIVA [18] and PCA- CCA-Bennch [19]. Both benchmarks are framed as Visual Question Answering tasks, requiring the model to choose the best action from multiple options based on an image depicting a specific situation. We compare two settings: (1) using the original image as input situation, and (2) using a textual description of the situation, either caption generated by GPT-4o or taken from the dataset's annotations, place of the image.
The results, presented in Figure 2, reveal a compelling insight: the text situation setting shows performance comparable to, or even slightly better than, the VLM operating directly on the image input. This observation suggests that the fundamental reasoning and decision logic required for navigating these numan-centryy and embodied-age agent scenarios can be substantially cap- tured and learned from textual representations alone.
Iriggo Stuction Stastion
Image Stuation Sitatio
Orvend 5.6.55565
Quann
Qwnen? 5-V.
Cward 5
Such findings resonate with human cognitive develop- ment, where abstract knowledge, reasoning skills, and decision-making strategies are often acquired through language, detached from immediate perception of the real situation [26]. Based on this insight, we hypoth- esize that VLMs can similarly benefit from acquiring reasoning capabilities primarily through text-based learning. Therefore, our core methodological premise is to cultivate sophisticated reasoning and decision-making policies using rich, text-only situational descriptions paired with desired outcomes. The ultimate aim is to transfer these textually- learned reasoning skills effectively to multimodal inference for the model to ground its decisions in the actual visual information. It enables the model to leverage the text data for reasoning development while retaining the VLM's s ability to perceive and act vision-grounded contexts.
(PCA-Benchi
Figure 2: Model accuracy on VIVA [18] and PCA-Bench [19]. Image Situation uses the original image as input, and Text Situa- tion employs the caption (text) instead.
3 Method: Learning to Reason and Decide from Text
Our primary goal is to enhance the reasoning and decision-making capabilities of VLMs in vision- grounded scenarios. Recognizing the challenges in acquiring large-scale image-text pairs for training and inspired by 8 2. we propose novel paradigm to learn maisiinnmmaingggg skills from text-only data and transfer these skills to multimodal inference. Here, we employ Reinforcement Learning to foster the model's ability to generate explicit reasoning processes for complex decision-making, As illustrated in Figure 3, our framework involves three key phases: (1) Creating text-based decision- making dataset, (2) Optimizing the VLM's ss reasoning and prcision-making using GRPO [21], and (3) Deploying the enhanced VLM for decision-making with actual visual input during inference.
3

---

Synthetic Data Creation
Textual GRPO Training
Multimodal Decision Making
Textual Situation
Decision Making Q
<hhinoo../thinlo + Answer
Imagery Situation
Decision
<thinlo..sttiikk. + Answeri
dhinio...@thinko >
Answer
<thinkk The image showes rosal
acciioer You sev 20kkme now.
speail limk sign indiaaiing 30 kash Since the driver coeentl 20 kmnth, ... they can cottinue driving... Hosweee if inpostart be ammre of the surrounding and maintain safe from othrr Sancsere Knys driving sianwers
the prinppal of booke etalents, reeaking minsr injarios After invesigacing
Implemeer sosetutios seskits involiing students purmn tu motiate ..
Formut Rew and
Adaptive Reward
Model Training
Accurary Reward
<think-
Figure 3: Overview of Praxis-VLAn Learning transferable reasoning from text-only data for mul- timodal prcision-making The process involves (1) constructing synthetic text- bast- training data where situations are represented through textual descriptions, (2) training the VLM on this data using RL with adaptive rewards to develop reasoning skills, and (3) transferring the learned reasoning to vision-grounded decision-making tasks during inference.
3.1 Problem Formulation and Model Setup
We start the methodology description with the problem formulation for vision-grounded decision- making: an agent (VLM) receives visual situation x (e.g.. image or video frame) and textual question σ about action selection. The objective to learn a policy π (y]r> ,x9) that generates response y maximizing task success alignment with desired criteria (e.g.. human preferences, safety constraints). We initialize the policy π using Qwen2.5-VL [25] for its strong capabilities in multimodal understanding and instruction following, which enables solid foundation. The VLM architecture M naturally supports the joint processing of visual and textual inputs: I = MM(x x
Inspired by $ 2, importantly, our training strategy focuses on enhancing reasoning capabilities primarily through text. Therefore, during the training phase, we synthesize text-based data (<rseet: 2t y)}. where the visual inputs x are replaced with textual descriptions x. The model is then trained on this data by updating only the language model components of the VLM, leveraging the scalability of text for efficient knowledge acquisition. Yet, during the inference phase, the entire trained VLM architecture, including the vision encoder, is used to process the image-text input pair (x R), ), allowing the textually-learned reasoning skills to be applied to visual situations.
3.2 Text-Based Decision-Making Data Construction
To gather reasoning skills, a cornerstone of our methodology is the creation of a high-quality, text-only dataset specifically designed to teach complex pocisinn-making reasoning. This dataset serves as the primary training ground for our model learning. Its design aims to be: (1) challenging enough to necessitate multi-step reasoning for optimal decision-making, and (2) structured to allow evaluation via straightforward rule- based metrics, mitigating the need for complex reward modeling and reducing the risk of reward hacking [27]. We hence formulate the task as question answering based scenario: (x.er 29, y), where x is the textual description that replaces the visual situation, z? multiple-choice question of decision making relevant to the situation, and y is the answer.
textual
Leveraging recent advances in LLM- data synthesis [17, 28], we employ GPT-4o [29] for data construction. Specifically, we first craft 10 seed questions as in-context examples to guide data generation, then prompt GPT-4o to produce additional samples. To maximize data diversity, we adopt a batch generation approach, producing 10 samples at time, followed by deduplication. This strategy allows an effective generation of varied scenarios and questions, yielding a final dataset of 10K training samples and 1K validation samples. Importantly, filtering or intensive curation, enabling fast, aonniic training data generation while reducing reliance on costly image-pair datasets. More details of data creation are in Appendix B.3.
method requires no manual
3.3 Reasoning Policy Optimization via GRPO
To cultivate robust reasoning abilities that go beyond the behavioral cloning limitations of supervised fine-tuning (SFT) [30], we employ Reinforcement Learning (RL) to fine-tune the VLM's policy.

---

Specifically, we utilize Group Relative Policy Optimization (GRPO) [21], an RL algorithm well for optimizing decision-makee policies based on sampled trajectories to enhance reasoning.
Concretely, given an old policy Told and a reference policy Tref, GRPO optimizes the current policy To by sampling G response trajectories O = {o for each query x. The objective function is:
1
To(oi(z) As. (o
max 10.
min
G
1=1
To (ox(2))) (ota
1 - 1 + +e)))) )) -KDKLTTOO-la met [-r[+
clip
(I)
(x))
where e >> 0 is a policy ratio clipping hyperparameter, 3 > 0 balances the KL-penalty term against the advantage- advantage-weighted weighted policy update, and D [To] Fref [ is the KI divergence between the current and reference policy. The term A, = I. me-mean( (r) rGll represents the normalized advantage estimate of the i-th response at the group level. GRPO aims to improve the policy by increasing the likelihood of actions that lead to higher-than-avere
( "a
estimated returns within a sampled group.
3.4 Multi-Stage GRPO Training with Adaptive R1 Reward
To further encourage robust, explicit reasoning capabilities, we employ multi-stage GRPO training. It is inspired by recent findings that geometry and math data can enhance model logical reasoning ability [10, 14, 31]. Multi-stage GRPO, combined with a newly designed adaptive R1 reward tackling different aspects [32], allows to first establish foundational logical structuring and then refine sophisticated decision-making skills, enabling models to learn different skills at different stages.
Stage 1: Cold Start Initialization for Foundational VLM Reasoning. The initial stage focuses on equipping the VLM with multi-step reasoning abilities. We employ the geometry3k dataset [33] for GRPO training, converting the task into numerical computation which can be readily evaluated using rule-based metrics. Following DeepSeek-R1, we enforce a specific output format: "<think> </think><answer </answer>", compelling the model to externaiize its reasoning process.
key finding of our work is that the commonly adopted SFT-then-RL paradigm, where model is first fine-tuned on (image, question, reason) triplets with SFT to learn the desired reasoning structure before RL [12, 13], can be circumvented. We find that directly training an instruction-tun -tunndd VLM (e.g., Dwen2.5-VL-Instrueee VL-Instruct) with GRPO is effective when coupled with an adaptive reward mechanism. In this initial phase, the rewards prioritize format adherence. Specifically, we leverage a combination of: (1) Rug: Calculates if the count of each special token (<think>, </think>, <answer>, </answerz) in the output equals one, which strongly encourages the model learn the basic output structure and narrows the search space; (2) R Measures if the output strictly adheres to the exact nested format; (3) Raseumey: Rewards the correctness of the final numerical answer.
Once the model consistently produces outputs in the desired format, Raa is removed, and the focus shifts more towards Raccurscy. thereby promoting better reasoning and result accuracy.
Stage 2: RL Training for Text-Based Decision Making. The model emerging from Stage 1, now possessing better-initialized capability for multi-step reasoning and format adherence, serves as the foundation for the second training stage. This stage targets our primary goal: enhancing sophisticated decision-making skills. Here, we utilize the curated text-based decision-making dataset and train the model to mimic human-like learning processes by exploring diverse reasoning paths for various textual scenarios. The reward function in this stage primarily emphasizes the correctness of the final decision, implicitly validating the quality of the preceding reasoning. Leveraging text data in this manner allows for the efficient adaptation and refinement of reasoning skills for sophisticated decision- making, ultimately yielding a policy designed for effective transfer to multimodal inference.
For this decision decision-making RL training, the adaptive rewards aim to encourage both comprehensive thinking and accurate decisions. We use a combination of: (1) Riamat Continues to ensure adher- ence to the thinking-answeringg structure; (2) Racenncc:: Rewards the correctness of the answer; (3) Rien: Encourages the model to generate deliberate and longer reasoning chains. Our observations indicate that Rige is effective in promoting more comprehensive consideration of the situation and
5

---

action candidates. Contrary to some recent work suggesting that long reasoning chains might be redundant [34, 35], our results show that encouraging longer reasoning can lead to more thorough situational analysis for more complex situations. We will provide detailed discussions on this in ğ 5.2. This adaptive reward strategy across stages enables efficient training by targeting different skills, including format adherence, logical computation, and complex pacision-making sequentially.
4
Experimental Settings
4.1 Tasks and Datasets
To comprehensively assess the Dacision-making capabilities of our model in diverse Dision-grounde scenarios, we utilize three benchmarks that encompass wide spectrum of real-world situations.
VIVA [18]: This benchmark focuses on human-centered situations. comprises 1,240 images depicting variety of real-world scenarios. Models are tasked with understanding social contexts and predicting appropriate actions or responses aligning with human values based on the visual scenes.
PCA-Bench [19]: A benchmark encompassing 317 scenarios of embodied robotics, autonomous driving, and interactive games. It requires models to process multimodal observations and select an action from a predefined action space. We use the open track proportion of the benchmark.
EgoNormia [22]: A dataset with 1,743 samples centered around ego-centric video understanding, where the model needs to interpret actions and anticipate future events from an egocentric perspective.
All tasks are formalized as VAQ and we employ accuracy as the evaluation metric. We consider VIVA and PCA-Bench as in-domain benchmarks, as they align with the typical image-text input and decision- macision-making formats our model is trained on. In contrast, EgoNormia serves an out-of-domuin benchmark, introducing additional challenges such as sequential and temporal reasoning over video frames and egocentric perception. These datasets offer rigorous and diverse testbed for evaluation. We follow the original data splits and prompts provided by each benchmark.
4.2 Baselines and Implementations
Following previous work in reasoning based VLMs [13, 36], we adopt Qwen2.5-Vi VL as our backbone model, with both its 3B and 7B parameter variants. We compare the performance of our model (Praxis-VILM) against baselines, including original backbone (vanilla) VLMs and the SFT method. The SFT baselines include two variants: one (w/ SFT) that directly predicts the answer y, and another (w/ Reason SFT) that first generates a reasoning chain before producing the final answer.
Implementation Details. For both GRPO and SFT training, finetune full model parameters. For GRPO training, we set rollout N to 5 and KL divergence coefficient to 0.01. During inference, we leverage VLLM Library [37] with greedy decoding. More details are in Appendix B.2.
5 Results and Analysis
5.1 Main Results
The main results are in Table 1, showing several key advantages of the proposed Praxis- First, our text- GRPO training strategy effectively endows VLMS with robust decision- making skills that successfully transfer to multimodal scenarios. Across both 3B and 7B model scales, Praxis-VLL consistently outperforms Swen2.5-VL-Innsrrie as well as SFT approaches on all benchmarks. This primary observation underscores the core efficacy of our approach: it successfully imbues the model with decision decision-making making capabilities learned from textual scenarios, which are then effectively transferred and applied during multimodal inference in varied visual environments.
Second, Praxis-LLM exhibits superior generalization ability compared SFT-based ap- proaches. This advantage is particularly salient in the out-of-domain EgoNormia dataset, which features sequential video inputs distinct from our training regime. Praxis- VLM maintains strong performance here, stark contrast to both SFT baselines, which struggle significantly when faced with such domain shifts. Such disparity suggests that while SFT-based methods primarily learn to imitate the patterns seen during training with behavioral cloning, our GRPO-trainee model internalizes more

---

Models
VIVA [18]
PCA-Bench [19]
EgoNormia [22]
VL-3B
76.61
48.58
51.92
w/ SFT
77.42
46.37
35,06
() w/ Reason SFT
75.81
49.53
28.34
Praxis-VLMM333 VLM-3B (ours)
79.03
50.79
54.27
+ w/ one-stage GRPO
79.52
50.79
53.13
Qwen2.5-VL-7B
80.97
46.37
46.19
w/ SFT
81.13
45.74
34.83
w/ Reason SET
78.79
53.00
34.08
Praxis VLM-7B (ours)
84.03
60.25
54.33
w/ one-stage GRPO
83.87
58.99
49.57
Table I: Main results measured by accuracy (%). w/ SFT denotes the SFT baseline to directly predict the answer, while Reason SFT first generates a reasoning chain before producing the answer, w/ one-stage GRPO is our model variant without math cold start initialization (Stage 2 only).
fundamental and broadly applicable decision-making skills. In contrast, while Reason SFT learns to replicate the reasoning patterns seen during training, it appears to overfit these specific patterns and struggles adapt its reasoning when faced with domain shifts. By optimizing the policy based on task outcomes and allowing exploration beyond static dataset examples, Praxis- LLM learns to analyze situations, evaluate potential actions, and understand consequences in its own generated reasoning paths, cultivating generalizable decision maainng competency across diverse situations.
Third, the cold-start initialization in multi-stage framework further enhances the model's generalization capabilities, particularly for novel and complex tasks. Comparing the full, two- stage Praxis-VILn with its one-stage variant (without the initial math cold start) reveals distinet improvement in generalization: while both variants achieve comparable performance on in-domain tasks (VIVA, PCA-Bench), the two-stage Praxis-VLM consistently exhibits superior accuracy on the EgoNormia benchmark. This demonstrates that the math cold-start successfully bolsters the model's foundational logical reasoning architecture, thereby enhancing its capacity adapt and perform effectively in novel and complex decision decision-making scenarios. Besides, the performance gain is more pronounced in 7B models, possibly because of larger models' better logical reasoning potential.
In summary, the results demonstrate that Praxis- VLM, enhanced with text-based GRPO training, achieves substantial improvements in vision-grounded decision-making. It effectively leverages textual guidance to learn generalizable decision-making capabilities, enabling robust performance across diverse complex multimodal scenarios. Compared to SFT approaches and the original base models, Praxis-Ve VLM exhibits markedly stronger generalization and adaptability.
5.2 Impact of Reasoning Length on Model Performance
Previous results have shown explicit reasoning help- ful to task. For further analysis, measure the length of the generated reasoning chain produced for each sample in VIVA and EgoNormia. Based on these lengths, we divide the samples into five equal bins by length percentiles. We then calculate the accuracy within each bin for both Praxis- VI.M and Qwen2.5-VL. The results are presented in Figure
Oweed EAB Po
(%)
Apturacy
Oueenc SA
Owendd 540
We can observe a general trend of decreasing ac- curacy for Praxis-VLM the reasoning length in- creases for both datasets. However, crucially, the accuracy of the baseline Qwen-Ve VL, without reason- ing, also shows a similar downward trend. This strongly suggests that the decreasing accuracy is cor- related with sample difficulty; Praxis-Ve VLM tends to generate longer, more detailed reasoning for instances that inherently challenging.
Lon1
Lend
Lond
Lend
Len2
Lond
Lend
EgoNormia
Figure 4: Accuracy versus reasoning length VIVA and EgoNormia. Samples are grouped into 5 quintile bins based on the reasoning length percentile generated by Praxis-VLM, (Lenl: shortest 20%. Len5: longest 20%).
We do not include PCA-bench due the relatively small sample sizes.

---

VIVA
CTA-Bench
EgoNormia
Model Name
Orig.
Major.
@
Orig.
Major.
P
Orig-
Major.
Pass@8
Qwen2,5-VL VL
80,97
80.73
80,81
46.37
48.27
56.47
46.19
46.36
54,50
w/ SFT
81.13
81.21
83.55
45.74
46.37
50.16
34.83
34.60
40.79
w/ Reason SFT
78.79
80.64
89.03
53.00
58,36
82.33
34.08
35.69
66.04
Praxis-VILn VLMM7B
83.87
84.36
89.27
58.99
61.83
77.92
49.57
55.08
72.23
Table 2: Performance of diverse sampling, Orig.: Greedy decoding accuracy. Major: Majority vote aceuracy with distinct samples. @ Accuracy with least one correct answer from samples. For Praxis- VLM, we use the one-stage variant without math cold start for a fair comparison with SFT.
Moreover, within nearly every length bin, Praxis- VLM consistently outperforms its corresponding Qwen2.5-VL. baseline. This holds true for both the 3B and 7B models. This finding further reinforces the effectiveness of the explicit reasoning process learned by Praxis-VLM, demonstrating its robust benefit across varying levels of sample complexity.
Finally, we observe a noticeable performance drop for Praxis- VLMM specifically in the longest reasoning bin (Len5) on the VIVA dataset than Qwen-VL. We then manually examine these cases and find two potential contributing factors. First, some generated outputs exceed the maximum sequence length configured during inference (i.e., 1,024 tokens), causing the generation to be truncated before the final answer tag could be produced. Second, extremely long reasoning chains might sometimes cause "overthinking." " where the extended reasoning process potentially introduces noise, negatively impacting the final decision accuracy. This point may warrant further investigation in future work.
5.3 Diverse Reason Sampling for Enhanced Decision Making
To further evaluate the robustness of Praxis-VLM'ss decision-making, we generate 8 diverse samples per instance with decoding temperature as 0.2 and measure accuracy via Majority Vote (" ("Major", most frequent answer) and Pass @8 (at least one correct answer). The results are shown in Table 2.
Compared to greedy decoding ("Orig.") "), we observe that for reasoning-enhanced models like Praxis- VLM and Reason SFT, the majority vote yields improved accuracy. More significantly, the Pass @8 8 scores demonstrate substantial improvement scores. This indicates that while the single most probable reasoning path might not always lead to the correct answer. the correct solution is often reachable and present within a small set of diverse reasoning trajectories explored by these models.
Moreover, we can observe that despite Reason SFT's ability to sometimes find the correct answer within its samples (high Pass @ 8), Praxis- VLM consistently outperforms it in the Majority Vote metric across all datasets. This suggests that while both models explore relevant reasoning paths, the central tendency of Praxis- VLM's reasoning (as reflected by the majority vote) is more reliably accurate. We interpret this as evidence for higher quality or more robust reasoning process learned via GRPO. Overall, the results highlight Praxis- VLM's strength in both exploring the solution space effectively (high Pass (ass@8) and converging the correct answer (strong Major. and Orig.).
5.4 Exploring Praxis-VLLM Reasoning: What Does It Consider?
To gain deeper insights into the reasoning ability learned
Safety Rink Maragement
Sitaaioona Analysio Vieua Eviderce Assecurent
by Praxis-VIL we analyze its generated reasons. We first prompt GPT-4o to generate keyphrases that sum- marize the reasoning aspects of each sample. These keyphrases are then clustered across the dataset. As shown in Figure 5, this analysis reveals four primary di- mensions characterizing the model's reasoning process.
-iaard idertilicatoo Health
Proriization RiAk Miligation
25%
Reason Type Clusters
Phobdem Recognt on
Rule Norm
35%
Action Analysis & Consequence
Enoitit & Piogorakii SuciatCuutarll horrs interent on Rogulai Compliance
A major aspect identified is Situational Analysis, which focuses on interpreting the scenario, assessing visual evidence, recognizing the core problem, and establish- ing situational awareness. This also highlights that the model can adapt the reason to multimodal inputs. Complementing this understanding is Action & Con- sequence Evaluation, where the model systematically compares potential actions, anticipates both positive and negative outcomes, weighs trade-offs, and often employs elimination strategies to justify its decisions.
Consecuannee Prackctin
Pros and Core Optirns
Process E minution
Figure 5: Dominant reasoning dimen- sions used by Traxis-VLY, in decision- making. Clusters were identified by an- alyzing keyphrases generated by GPT-4o from the model's 's reasoning chains.
8

---

Situation
Question
Siluation
Question
Situation
Question
the vctiontt
most appioprishe To the and
Select the rooe aoprnp rataal andinn take Walcl dicance slin play
The ssxeT
Siner steon (n)
Pripcce far cae diould doven
crving.
Dtop the oa
acceus dhe dccher.
FCD Bilp ainourees.
Tum
Model Output
driiing
Output
The Hireaall ardees out seuer piclic
Model Output
Ho dina recer scure
Reseae: inage thoss mos apae tanee the inaltic jight i ned ia ieporiant to xip jursaiisu
Reaeoe: The image trown young chis waening towardo erteeto
The real skilk refor the wellin sleu and then suuederr
plaan Luchoo the lorg lams sulliyy inai G-1 CC Thea a proecvv Asswer
Arswen: Gi Supp
yoo driking and focuing dioulety witch cuubill proverting accientt
Misittrppretation Situation
Safety Priontization
Fallures
Appropriateness Misalignmont
Figure 6: Illustrative examples of common failure cases for Praxis VLM, identified by analyzing the reasoning generated for incorrect predictions. Complete outputs are in Appendix C.
Another key dimension is Safety & Risk Management, where the model actively identifies potential hazards, considers risk mitigation strategies, and prioritizes health and safery, indicating its potential in human-centered consideration. Lastly, reasoning also incorporates Rule & Norm Adherence, which entails consideration of explicit regulations, implicit social cultural norms, procedural correctness, and the appropriateness of involving authorities.
Taken together, the clustering results suggest that Praxis-VLM develops comprehensive and structured approach to reason in decision-making. It systematically analyzes situations, deliberates over potential actions and their consequences, and accounts for key constraints related to safety, rules, and norms. This multifaceted capability, cultivated through text-based RL., underpins the model's improved performance and its ability to generalize across diverse scenarios.
5.5 Understanding Failures: Error Analysis Through the Lens of Reasoning
We have shown Praxis- VLM's superiority, and here we discuss its limitations by analyzing the reasoning chains associated with error samples. This qualitative analysis helps pinpoint common failure modes in the model's 'ss reasoning process, Figure illustrates examples of the common error categories identified through this analysis.
when it attempts to deliberate step-by-step.
First, Misinterpretation of Situation Context remains a challenge. In these cases, the model might acknowledge certain details but fail to grasp their full implication or overlook other critical contextual elements, leading to flawed assessment of the situation (e.g.. In the image of Example 1, the model fails to recognize the actual green light indicating permission to proceed straight). Second, we observed Safety & Prioritization Failures. This category includes errors where the model struggles to appropriately weigh immediate actions against long-term safety needs or fails to prioritize the most critical safety concern among multiple factors present in the scenario (c.g., in Example 2, the model addresses an immediate risk without selecting the best overall preventative measure). Third, errors frequently arise from Normative & Appropriateness Misalignment. Here, the model may make questionable assumptions about social interactions or select actions based on flawed reasoning about social norms, ethics, or expected behavior in given context (e.g., in Example 3, the model fails understand that driving under the influence is illegal and should be stopped).
These failure modes highlight that while encouraging explicit reasoning is beneficial, challenges remain in ensuring deep and accurate contextual understanding, robust prioritization under complex constraints, and reliable alignment with nuanced social and ethical norms. Addressing these aspects within the reasoning process is a key direction for future research. More samples are in Appendix C.
6 Related Work
VLMs in Situational Decision-Making, The quest to enable intelligent agents to make informed decisions in real-world, situated environments is central to embodied AI research [2, 22, 38, 39]. VLMs have emerged as a powerful foundation for such agents, demonstrating significant promise in applications like roboties, autonomous navigation, and interactive task planning [40-43]. These models integrate visual perception with language understanding to interpret and interact with their surroundings. However, a persistent challenge lies in equipping VLMs with the capacity for multi-step

---

reasoning required to navigate and act effectively in nuanced and dynamic situations. Our work focuses on enhancing such sophisticated reasoning abilities crucial for robust decissin-maakngg
Reasoning in VLMs. The ability to reason is fundamental to effective decision-making. Recent studies have explored VL.Ms' visual reasoning capabilities across various tasks, including visual question answering and commonsense reasoning [44-47]. Traditional methods often rely on fine- tuning VLMs on large multimodal datasets tailored specific tasks reasoning styles [16, 48, 49]. More recently, approaches have emerged that encourage VLMs to generate explicit reasoning steps, often leveraging RL to optimize performance on complex tasks [13, 36, 50-52]. While these RL- based methods have shown success in improving reasoning, they typically necessitate extensive training on datasets comprised of paired image-text data. Our text-driven RL diverges from this by proposing a more data- lata-efficient pathway to instill reasoning.
Text-Driven Enhancement of VLMs. Leveraging textual data to enhance VLMMs is interest. Some prior studies have utilized text to improve VLMS by aligning the embedding spaces of different modalities [53-55]. However, these approaches generally do not target the cultivation of sophisticated reasoning abilities for situated decision-makinn we introduce a novel method to employ text-driven RL to instill generalizable decision-making competency. A crucial aspect of our contribution is finding that these reasoning skills, learned entirely from text, can be effectively transferred diverse vision- siionngrounded scenarios for decision makiii....
of growing
Building on our preliminary analysis,
7 Conclusion
We introduce Praxis- VLMM, a reasoning- VLM for complex lision-grounder decision-making Motivated by our finding that foundational reasoning can be effectively learned from text-only descriptions, Praxis- VLM utilizes baxed GRPO to instill robust reasoning skills that successfully transfer to vision-grous ussion-grounded inference. The experiments demonstrate that our approach outperforms the original VLMs and different SFT methods, particularly in generalization to oft-oi-domain tasks with general reasoning abilities. Our work offers data- efficient pathway to more capable and generalizable VLMs by effectively transferring abstract reasoning learned from language to guide complex vision-grounded decision-making.
three benchmarks of diverse situations
Ethics Statement and Broader Impacts
The evaluation of our models in this research is conducted using publicly available benchmarks, including VIVA, PCA-Bench, and EgoNormia. We adhered strietly to the original usage protocols and licensing terms of these benchmarks, utilizing them without modification and solely for model inference during the evaluation phase. For the generation of any synthetic text-based training data using LLMs like GPT-4, we employ keyword-basse filtering mechanisms designed to mitigate the inclusion of potentially harmful, biased, or unsafe content.
Despite these precautions, it is important to acknowledge that work, Praxis- VLM, builds upon pre-trate pre-trained VLMs. These foundational models are typically trained on extensive datasets scraped from the internet, which may inadvertently contain and reflect existing societal biases or problematic content. While our method focuses on learning reasoning skills, the potential for the model to inherit or amplify such underlying issues from its base architecture remains. We therefore strongly advise users and developers to conduct thorough ethical reviews, bias assessments, and impact analyses before deploying systems based on this research in real-world applications, particularly in sensitive or high-stakes domains.
Acknowledgement
This work is supported by grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. PolyU/25200821), the Innovation and Technology Fund (Project No. PRP/047/22FX), and PolyU Internal Fund from RC-DSAI (Project No. 1-CEIE).
10

---

References
[1] Tao Feng, Chuanyang Jin, Jingyu Liu, Kunlun Zhu, Haoqin Tu, Zirui Cheng, Guanyu Lin, and Jiaxuan You. How far are we from agi: Are Ilms all we need? arXiv preprint 103333. 2024.
[2] Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, and Liang Lin. Aligning cyber space with physical world: A comprehensive survey on embodied ai. arXiv preprint arXiv:2407. 068866 2024.
[3] Qinhong Zhou, Sunli Chen, Yisong Wang, Haozhe Xu, Weihua Du, Hongxin Zhang, Yilun Du, Joshua B Tenenbaum, and Chuang Gan. Hazard challenge: Embodied decision making in dynamically changing environments. arXiv preprint arXiv:2401. 2024.
[4] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024.
[5] Zhiting Hu and Tianmin Shu. Language models, agent models, and world models: The law for machine reasoning and planning. arXiv preprint arXiv:2312.052.:1 2023.
[6] Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey on vision- 14093, language-action models for embodied ai. arXiv preprint arXiv:2405. 140993 2024.
[7] Zongxia Li, Xiyang Wu, Hongyang Du, Huy Nghiem, and Guangyao Shi. Benchmark evalua- tions, applications, and challenges of large vision language models: A survey. arXiv preprint arXiv:2501.02189, 2025.
[8] Sonia Jawaid Shaikh. Artificially intelligent, interactive, and assistive machines: A definitional framework for intelligent assistants. International Journal of Human-Computer Interaction, 39(4):776-789, 2023.
[9] Yu Chen, Scott Jensen, Leslie J Albert, Sambhav Gupta, and Terri Lee. Artificial intelligence (ai) student assistants the classroom: Designing chatbots to support student success. Information Systems Frontiers, 25(1):161-182 2023.
[10] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Iims via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
[11] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, al. Openai ol system card. arXiv preprint arXiv:2412.167200 2024.
[12] Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Wei Chang. Open- vithinker: An early exploration to complex vision-language reasoning via iterative self- improvement. arXiv preprint arXiv:2503. 17352, 2025.
[13] Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. RI-onevision: Advancing generalized multimodal reasoning through cross- moosl formalization. arXiv preprint 2xX1v:2533.1061 2025.
[14] Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint wrXiv:5033.06745 2025.
[15] Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang. and Lifu Huang. Vision-flan: Scaling human-labeled tasks visual instruction tuning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACI. 2024, pages 15271-15342, Bangkok, Thailand, August 2024. Association for Computational Linguistics.

---

[16] Zhiyang Xu, Ying Shen, and Lifu Huang. MultiInstruct: Improving multi-modaa zero-shot learning via instruction tuning. In Anna Rogers, Jordan Boyd-Graben, and Naoaki Okazaki, editors, Proceedings of the 6l Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers), pages 11445-11465, Toronto, Canada, July 2023. Association for Computational Linguistics.
[17] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34911 2023.
[18] Zhe Hu, Yixiao Ren, Jing Li, and Yu Yin. VIVA: A benchmark for rision-grounde decision- making with human values. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2294-2311, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[19] Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang. Peiyi Wang. Xiangdi Meng, Tianyu Liu, and Baobao Chang. PCA-bench: Evaluating multimodal large language models in perception-cognition-attii
chain. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACI. 2024, pages 1104, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[20] John R Wilson and Andrew Rutherford. Mental models: Theory and application in human factors. Human factors, 31(6):617-634 1989.
[21] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepscekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
[22] MohammadHossein Rezaci, Yicheng Fu, Phil Cuvin, Caleb Ziems, Yanzhe Zhang, Hao Zhu, and Diyi Yang. Egonormia: Benchmarking physical social norm understanding. arXiv preprint arXiv:2502.20440, 2025.
[23] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling 1lm test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314 2024.
[24] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
[25] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 2rXiv:2502.1392. 2025.
[26] Margaret Wilson. How did we get from there to here? an evolutionary perspective embodied cognition. In Handbook of cognitive science, pages 373-393. Elsevier, 2008
[27] Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460- 9471, 2022.
[28] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instruc- tions. arXiv preprint ar arXhv:2212.10560) 2022.
[29] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-40 system card. arXiv preprint arXiv:2410.21276. 2024.
[30] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 47Xiv:1707.06347, 2017.
12

---

[31] Yan Ma, Steffi Chern, Xuyang Shen, Yiran Zhong, and Pengfei Liu. Rethinking rl scaling for vision language models: transparent, scratch framework and comprehensive evaluation scheme. arXiv preprint wrxivv2504.02588) 2025.
[32] Hou Pong Chan, Wang Chen, Lu Wang, and Irwin King. Neural keyphrase generation via reinforcement learning with adaptive rewards. In Anna Korhonen, David Traum, and Lluís Mårquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computa- sional Linguistics, pages 2163-2174, Florence, Italy, July 2019. Association for Computational Linguistics.
[33] Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyrl: An efficient, scalable, multi-modality rl training framework, 2025.
[34] Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 16419, 2025.
[35] Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint 2xXx:2250400888: 2025.
[36] Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable r rl-style large vision-language model. arXiv preprini ar arXiv:2504.07615 2025.
[37] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[38] Andrew Szot, Max Schwarzer, Harsh Agrawal, Bogdan Mazoure, Rin Metcalf, Walter Talbott, Natalie Mackraz, R Devon Hjelm, and Alexander T Toshev. Large language models as gen- eralizable policies for embodied tasks. In The Twelfth International Conference on Learning Representations, 2023.
[39] Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Anderson Compalas, Dawn Song, and Xin Eric Wang. Multimodal situational safety. arXiv preprint arXiv:2410.06172, 2024.
[40] Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean--aptiitt Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv:2312.1 1805, 2023.
[41] Zipeng Fu, Tony Z Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipu- lation with low-cost whole-body teleoperation. arXiv:2401.021177 2024.
[42] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. LLLVA-NeXT: Improved reasoning, OCR, and world knowledge, 2024.
[43] Mohit Shridhar, Jesse Thomason. Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mot- taghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740-10749 2020.
[44) Zhe Hu, Tuo Liang, Jing Li, Yiren Lu, Yunlai Zhou, Yiran Qiao, Jing Ma, and Yu Yin. Cracking the code of juxtaposition: Can AI models understand the humorous contradictions. In The eightt Annual Conference on Neural Information Processing Systems, 2024.
[45] Yuqing Wang and Yun Zhao. Gemini in reasoning: Unveiling commonsense in multimodal large language models. arXiv preprint arXiv:2312. 17661, 2023.
[46] Nitzan Bitton-Guetta, Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. Breaking common sense: Whoops! vision-and-language bench- mark of synthetic and compositional images. In Proceedings of the IEEF/CVE International Conference Computer Vision, pages 2616-2627, 2023.
13

---

[47] Haider AI- Al-Tanan, Quentin Garrido, Randall Balestriero, Diane Bouchacourt, Caner Hazirbas, and Mark Ibrahim. Unibench: Visual reasoning requires rethinking vision-language beyond scaling. arXiv preprint arXiv:2408.048/1 2024.
[48] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. 2023.
[49] Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning. arXiv preprint urXiv:2410.66990 2024.
[50] Fanqing Meng, Lingxiao Du Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint urXiv:2503.07365. 2025.
[51] Xinyu Ma, Ziyang Ding, Zhicong Luo, Chi Chen, Zonghao Guo, Derek F Wong, Xiaoyi Feng, and Maosong Sun. Deepperception: Advancing rl-like cognitive visual perception in milmss for knowledge-intensive visual grounding. arXiv preprint arXiv:2503. 12797, 2025.
[52] Yifan Du Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration reproducing ol-like milm. arXiv preprint arXiv:2501.01900 2025.
[53] Dasol Choi, Guijin Son, Soo Yong Kim, Gio Paik, and Seunghyeok Hong. Improv- ing finegrainned visual understanding in vlms through text-only training. arXiv preprint 24Xiv:242..124400 2024.
[54) Xiaomin Yu, Pengxiang Ding, Wenjie Zhang, Siteng Huang, Songyang Gao, Chengwei Qin, Kejian Wu, Zhaoxin Fan, Ziyue Qiao, and Donglin Wang. Unicorn: Text-only data synthesis for vision language model training. arXiv preprint MXXi::5503222550 2025.
[55] Yuhui Zhang, Elaine Sui, and Serena Yeung-Levy. Connect, collapse, corrupt: Learning cross modal tasks with modal data. arXiv preprint 1exiv::2401.08666. 2024.
14

---

A Discussions on Limitations and Future Work
The explorations in this work with Praxis-VLM open up several exciting avenues for future research, offering insights into how abstract reasoning learned from language can be effectively grounded in multimodal contexts. While we demonstrated significant gains using 3B and 7B parameter models, broader investigation into the interplay between model scale and the efficacy text-driven reasoning transfer would be beneficial. Understanding how models of different size perform could reveal valuable scaling dynamics for this learning paradigm.
Furthermore, our current approach leverages curated corpus of text-only situational descriptions. An insightful direction for future work lies in optimizing this data aspect further. While text offers a data-effi tata-efficieet route to learning reasoning, exploring advanced data selection strategies could unlock even greater efficiency. This could involve identifying or generating smaller subset of highly "effective" textual scenarios that most potently instill transferable reasoning skills.
Another promising frontier involves enhancing the synergy between the text-learned reasoning and the VLM's foundational visual perception, which, however, is out of the scope of our work. The ultimate effectiveness of the transferred reasoning during multimodal inference hinges on how accurately the visual input is perceived and aligned with the conceptual understanding developed through text. Future research could focus on improving the VLM's core visual grounding capabilities, perhaps through targeted pre-training or co-training strategies that explicitly link visual features to the abstract reasoning structures learned from language.
Finally, despite the effectiveness of our method, the error analysis shows several common fail pattens Praxis-VLM tends exhibit during the reasoning process. Future work may address these issues to further enhance the model's performance.
B
Experiment Details
B.1 Data Statistics
Our experiments utilize three established benchmarks for embodied decision makiin, providing comprehensive evaluation of our model's capabilities across diverse sce- narios. Key statistics for these benchmarks are presented in Table The benchmarks are: (1) VIVA [18], which is fo- cused on human-centeree decision makinng presenting var- ious real-world social situations where the model must pre- dict appropriate human actions; (2) PCA-Bench [19] en- compasses scenarios from embodied robotics, autonomous driving, and interactive games. For our experiments, we
Task
Number
VIVA
1,240
317
EgoNormia 1,743
Table 3: Data statistics for each evalua- tion benchmark. "Number" refers the count of test instances used.
use the open track test set provided by the benchmark; (3) EgoNormia [22], which centers normative decision- making from an ego-centric video perspective, requiring models to interpret actions and anticipate events involving tool use object manipulation.
Across all benchmarks, we utilize the task of action selection to measure the model dacissionmmkinn ability. This is formalized as a multiple-choice question answering task, where the model is presented with a visual situation and must choose the most appropriate action from several candidates. For EgoNormia, which uses video input, we adhere to the method described in the original work: video frames are sampled at a rate of one frame per second and are then concatenated from left to right (LTR) to form single composite image input for the model.
B.2 Implementation Details
For both GRPO and SFT training, we finetune full model parameters with BF For GRPO implementation. we use the EasyRL Library 3. We adopt the default hyper hyper-parameters, and set rollout N to 5 and KL divergence coefficient as 0.01. The learning rate is set as 1e-6.
htttp:://ggtabb..cc/hhyyg///////
15

---

For SFT implementation, we employ the HuggingFace TRL We set the number of training epochs 3 and learning rate 2e-5. For Reason-SFT baseline, as there no reasoning chains available. we first utilize GPT-4 to generate plausible reasoning chain for each textual training sample. We then fine-tune the base VI.M using SFT on these augmented (Situation, Question, Reason, Answer) samples, specifically training the model to first generate the reasoning chain and then the final answer, mimicking the desired output format.
All models are trained on four NIVIDA A100 and H100 GPUs. For Praxis-VLM, we adopt the following system prompt:
System Prompt
You are helpful Al Assistant, designed to provided well-reasoned and detailed responses. You FIRST think about the reasoning process internal monologue and then provide the user with the answer. The reasoning process MUST BE enclosed within <think> and </think> tags, and the final answer MUST BE enclosed within <answer> and </answer> tags.
Druing inference, we leverage VLLM Library [37] with greedy decoding. The model performance evaluated with accuracy. To parse the model output and match it to the original options (e.g.. A/B/C/D/E, etc), first design list of rules for matching; if it cannot be matched, we prompt GPT4-0 to match the model answer with the options. The prompts for each benchmark in inference are shown below:
Inference Prompt for VIVA
You are given situation and a question. Based on the situation provided, select most appropriate option to answer the question:
## Situation: As shown in given image.
## Question: _quuettin__
Now answer the question. Just output the choice:
Inference Prompt for PCA-Bene
Please answer the question below based on the images.
## Question: _question_
Now answer the question by selecting the correct option.
Inference Prompt for EgoNormia
The given images from a first-person perspective video depict a person in given situation. Please answer the question below based on the images.
## Question: Given the helow list of behaviors, choose the single normatively relevant or appropriate action perform next. You shouldn't use the info in options to learn about the context, but rather to make decision based on the normative appropriateness of the behavior. You shouldn't 't eliminate any options only based on the presence of elements in the context; should focus on normative appropriateness. _questio
Now answer the question by selecting the correct option.
Reward. For GRPO training, we adopt rule-based rewards, eliminating the needs for parameterized reward models. For geometry3k data training, we convert the task into numerical computation and use the Math- Math-Verify Library calculate the binary reward: for correct and for incorrect. For text-based decision making, which is formulated multiple-choice question, we parse the model
http:///ht.bb..ooooggggggg/g/aaa Verify
16

---

renard/ing oount
rewerd
6:4
Figure 7: Training dynamics of Praxis- -LM-s- (one-stage GRPO).
Situation
Question
Reason Answer
You manainng communityy evacusttinn dhe to on opor baching hurricone and poternial floods.
What should be your highost priority when coordi-ating sting the exccaation?
<thinko When aging exacustin recercee and diverss sladdlynng
highteipffoitty
A. Printtire exerasstngg the exderly papvbttion,
whi eldeely inshediols,
diabittiee familiee chiltien lncaion threst pood reinn flaodverers the moet time cemitiie and lhe entenin fattor These in high-nee fined sonse face an immeciate phyacca canger from mpidly datertoneing conditons, suct as fast oxlnn water buildigg lapse imsossable mads, Fallin to exaqusee these attropging
Transportanlon lon resources are limited,, and you have a mix of eiderily res domss, familles with childwen, and individuals with disabiitiss Cortain the Rooding others.
they are the most vunerable able to the
high-iik Hood zones hurnsann's potential effects
valnerable diftrrennt
Evacuste residents
atlalts,
families
among
removed provide safety.
Evacuute in firt--cmme firtt-eerved manner to ensore impart ofity and svnid conflizz
emerpeany highestiit
cuttion tressfer </tlekk
sarower 5. stanseer
Figure 8: Left: An example of a synthetic data sample used for our text-based training. It comprises textual situation description, multiple-choice decision-making question, and the target answer. Right: The reasoning chain, also generated by GPT-4o, is used for Reason SFT training, and not utilized during our GRPO training phase.
answer using rules and match it with correct answer using similar binary reward. For length reward (Rien), we first count the number of words multi-step reason, and then calculate the score as this word count divided by scaling factor of 250. To prevent disproportionately long outputs from dominating, Rico is capped at a maximum value of 1.0, which is achieved if the word count reaches or exceeds 250 words. For geometry 3k training. the overall reward is: R = Raceuracy + Roooat 0 Regg- For text-based decision making training, we remove the tag reward and include the length reward, with the overall score as: R Raccuracy 0.8 Rioemat + 0.5 Rign
Training Dynamics. Figure 7 illustrates key aspects of the training dynamics during the GRPO phase, including the evolution of various reward components and the mean output length. The plots demonstrate that the model initially prioritizes and rapidly masters the desired output format. Rewards associated with structural correctness, such as 'tag count" and 'format' (reflecting proper use and placement of reasoning tags like '<think>' and '<answer>'), are observed to quickly saturate at or near their maximum values early in the training process. Once format adherence is achieved, the 'accuracy", which measures the correctness of the final decision, exhibits a consistent upward trend. This signifies that the model then progressively refines its reasoning capabilities to improve decision-making accuracy. Concurrently, the output length typically shows an initial increase as the model learns generate more detailed reasoning. followed by stabilization. These dynamics effectively demonstrate how the adaptive reward strategy guides the model: first to learn the task structure. and subsequently to enhance the substantive quality and accuracy of its reasoning and outputs.
B.3 Details on Text-Based Data Creation
For the construction of our text-based decision-making dataset, we utilize GPT-40 6, A primary objective to generate training data that is both sufficiently challenging to necessitate multi-step reasoning and diverse in its situational contexts to enhance model generalizability. To promote this situational diversity, we implement a batch generation strategy: GPT-4o is prompted to produce 10 samples per request. These generated batches subsequently underwent deduplication process to ensure a varied collection of scenarios and questions. This method proves effective in creating the desired dataset characteristics. The prompt used for this data generation is shown below:
gpt-40-2024-11-22
17

---

Prompt for Training Data Generation
Now your task is to create complex decision- making questions in human-centered situations. Fach question contains situation description, multiple-choice question, and You consider the following approaches enhance complexity:
Add more context to the problem, such as tools, back-ground information, or character details, making the constraints
specific;
Make the options challenging:
Consider different ways the question is asked, incorporating reverse reasoning, dialectical reasoning, critical thinking, ctc.
The question doesn't necessarily have ask which action correct but could focus other pects related to decision-making.
There are no specific format or wording requirements for the questions. but they should be in the form of multiple-choie questions. You should make the situation diverse. You should also include rationalc explain the answer.
## Examples: example
Now generate 10 candidate question with answer. Your output should be presented JSON list
We provide synthetic sample for text-based training in Figure
Data Diversity Analysis. To assess the diversity of the generated scenarios, we conduct analysis by prompting GPT-4o to cluster the generated textual situations into topical categories. The clustering results indicate broad coverage across varied scenarios: (1) Workplace Performance and Personal Issues, (2) Resource Allocation, (3) Project Management, (4) Balancing Competing Interests, (5) Policy, Rules & Enforcement, (6) Ethical Dilemmas, (7) Interpersonal Conflict, (8) Emergency Handling, (9) Navigating Setbacks, (10) Event Planning & Logistics, and (11) Balancing Inclusivity and Majority Preference.
C Sample Outputs
We present error outputs --M-7B with full reason paths in Figure 9. We further include model outputs samples from VIVA, NCA-Bench, and EgoNormia in Figure 10, Figure 11, and Figure 12.
18

---

Situction
Question
Model Output
What best action take
Reoson:
with
trafffo trafffo
limet indiattes
whet you are shiing on the
with,, which lowar than the current spood of knnh, Given that the traft light to rod, important stop the vehicle to
compeyy with vantic regulation orm emoume sotredy for usere The presence other vatuciies and paccestiane abso stagents atopping necessary
donn (4)
Tur Fight
Keep ditking
Tan Leit
Anower C. Stop par
Situation
Question
Model Output
B. Were iecrudditen staees allow
peount du docoordd
the
This daeatt adoness
bright mecisaa inceddaae dar get counts
dll rik use
G. rosall sukey gale tariens
D. Touch the ls nacipaia the
statiic cnpicad the iecut
work tddlased cacelar
The studion deea requrrs
codtetiiin momerry aero the
sloved stveeop
Situction
i Question
Aigoadel Output
lake.
scover iitill
Thamon The mepp coltr aparcrio our al apoen Srrviic snatial This collec his driing and
vee atoo
replicamet
Propoae the thie should abow
coult
acchest
driing
me
coeto arooe avayya pey atte fun sorarit lis or the mad
Orein
some dipected tiei imigee.
Figure 9: Error sample outputs of
with full model reasons.
You are given situstion and question. Based on the situation provided, appropriate option
question:
A. Tell them to stop texting and focus driving.
B.
to drive slower when textiig.
C. Ask them text briefly and then put the phone down.
they use voice
text instcud.
F. Take no action, assuming they do not noed help.
"tiak The situatio depicted the image activitics testing, caing, ongaging
drving while holing smariphone, which clear indication distracted driving. Distracted driving
be behaviox dangerous bosause
pransey
iphone while driving
driver's anetiin Itom the road, incroming the risk acciaaanss evaluate each option.
A. "Tall them stop texting and focus on driving" " This is the most direct and appeopeiate action. It addresses the immediate aue of distracted driving and ensures road.
the driver
"Remind them drive slower when resting Whik might bes good lidox for general sife driving practicos,
sulfrow
the
distrcction atd Slowing
does not diminate the risk accidiat canval by tosting while driving.
C. "ssk them to lext boidlly and thes put the phone down." This optins suguetts thai the driver could cantinue to test, albeit for a short period. However, even brie! distractions can lead to accidents, so this is nota safe course of action.
"Remied they
instcad." "_ While using commands might like allanstivv, sill the driver being
commands beterr "Take no nction, iearmnng they not nand help. Ignoring the situtton not resporsible action, especially when's poses a significant to the driver and
hands eyes intenct
phone,
others on the road.
Given the options,
approprine initial
tell sap toxting driving. <ahinkk>
Samswert Tell them stop texting and focus un driving <lanswer-
Figure 10: Output of Praxis- VLM-7B with full model reasons on sample from VIVA.
19

---

You are driving assistant. Based on current image, what should you do if you want to drive forward safely?
Slow down
Keep driving Call the police.
<hhikee> The image shows a snowy road with visible tire tracks, indicating that vehicles have already passed ftrough There also a warn ing sign on the right side the road, which opiccally indicates potential hazard cautium Given these ubservations, important cautious and proceeding.
conditions
1. Slow down: This is reasonable action take when approaching potentially hazardous area, especially snewy conditions where visibility might be reduced traction
compromised.
Keep driving: This option is not advisable because presence of warning sign suggests there may be risk that needs be adiressed. such as other ubstacles thai coukd affect driving safety.
calling police might necessary extrome cases, would need to be valled if there a clear danger that poses immediate thecal public safety, bui this does nut seum to be the case here. <chinkk
pollee
required
driving. police
canswers Gäven the context, the most appropriatc action Slow down ensure safe navigation of the road.
<imnswrr
Figure 11: Output of Praxis- VLM-7B with full model reasons on sample from PCA-Bend
The Images from first-person perspective video deplct person given situation. Given the below behavlors, choose the single most normatively relevant or uppmpriate action to perform next.
Complete your repoct and then cycle away.
B. Excuse yourself politely and continue delivery.
Resume cyeling after brief conversation.
Donate to the charity and then contime cycling.
E. No behavior is remately upproprizte
<thiake The images show a street seene with a person walking dog on the sidewalk. The person appears to le engaged a casual acti vity, possibly The
walking their dog. There indication action
delivery, charity that needs attention continue curvent activity without interuption,
atively relevant appropriate
peaceful
Option suppests completing report and eycling away, which dircetly relited oneself for delivery, which also not applicable here. Option C suggests resuming cycling after a brief conversation, which could be approgeinte the person was stopped for moment but now contimes theiractivity. Option suggests donating to charity, which indicated in the images and scems unrolated normatively rolevant.
depicted images.
implics
Option suggests
bebaviur appeopriaxx, which
considered
Given the context of images, the most appropriste action would be resume the current activity, which is likely cycling, as aligns with normative expectation continuing peaceful actvity in residential area. <ithink>
<arswer> C. Resume cycling after a brief converstion. <tanswer
Figure 12: Output of Praxis-VL.M-T VLM-7B with full model reasons on a sample from Egomaa
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our paper studies pecision-makknn of VLMs and proposes novel method to improve model performance, as illustrated the Abstract and Introduction Section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations and future work Section A.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer: [NA]
Justification: Our work focuses on empirical studies with data-driven methods, and does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section and Section B.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
22

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the corresponding code. We will also release data and code upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https:/ peattss:/niis..ee cc/pu
bidcc/ggiidss/odeeuuuuiiii www.///oobessssiiiiiiiiioo onPolicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. cctpublic/////cpccpccccccccc...t //cgid/ussssseeeeeeeeeee /eoeee//omSuubmi iisimmmssioni ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section and Section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The results are reported as accuracy estimates using greedy decoding, without eтror bars or confidence intervals. However, our analysis includes results from diverse sampling regarding multiple predictions, demonstrating the model robustness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section B
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
cc. wctppiicc/cpppcbt.ticccc. ctwiieseieelenees
Answer: [Yes]
Justification: Section 7.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Section 1, Section and Section 7
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
Section
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Section B and Section
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, license, copyright information, and terms of use package should be provided. For popular datasets, paperswi thcode. com/ /mmatttsets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
25

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Section B.3 for synthetic data generation
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
26

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification:
We discuss the use of LLMs for synthetic data generation in Section B.3.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy (https: (https:/ /ttpp:s////uuips../// http://ewweess.eeeeeeeeeeeeeeeee Ctt.ec///oooeeeenee/22253 for what should or should not be described.
27


Reviews:
[{"review_id": "iI9LocS3dA", "reviewer": "Reviewer_jthP", "summary": "The paper introduces Praxis-VLM, a vision-language model trained to perform vision-grounded decision-making through text-only reinforcement learning. Instead of using paired image-text data, the model learns reasoning from synthetic textual scenarios using the GRPO algorithm and an adaptive reward scheme across two training stages. At inference time, it applies these reasoning skills to real visual inputs. Experiments on three benchmarks show Praxis-VLM outperforms supervised fine-tuning baselines and generalizes well across tasks. The authors argue that foundational decision-making skills can be effectively learned from language alone and transferred to visual contexts.", "questions": "- **Robustness to Visual Understanding Errors**: How does the model perform when image descriptions are imperfect or noisy? Is its performance sensitive to vision encoder quality?\n\n- **Synthetic Data Quality and Coverage**: How diverse and representative are the GPT-4-generated scenarios? Could performance drop if the text data lacks variety or introduces bias?\n\n- **Role of the Math Pretraining Stage**: What specific benefit does Stage 1 contribute beyond output formatting? Are there measurable improvements in reasoning depth or structure?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- **Innovative Use of Text-Only Training**: The paper presents a cost-efficient alternative to multimodal training by using synthetic text-only data to develop reasoning skills for vision-language models.\n\n- **Strong Empirical Results and Generalization**: Praxis-VLM consistently outperforms strong baselines across diverse tasks, including an out-of-domain benchmark, demonstrating both effectiveness and robustness.\n\n- **Clear Evidence of Reasoning Quality In Experiment**: The paper includes thoughtful qualitative and quantitative analysis, such as reasoning type clustering and diverse decoding evaluations, which show the model produces multi-dimensional, structured reasoning that improves decisions.\n\nWeaknesses:\n- **Evaluation Focused on Multiple-Choice QA Format**: All benchmarks used (VIVA, PCA-Bench, EgoNormIA) are framed as multiple-choice QA tasks. This narrow task format may limit the generality of conclusions. The model’s ability to reason in open-ended or interactive settings (e.g., planning, dialogue-based decision making, robotic control) is not tested.\n\n- **Lack of Analysis on Visual Feature Usage**: Although the model uses vision encoders at inference time, it is unclear how much the model actually relies on visual input versus prior learned textual knowledge. There is no attention or attribution analysis showing whether the model grounds its reasoning in visual evidence or simply applies text-trained patterns.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns and provided a clear explanation of the paper’s contribution to embodied AI, particularly in addressing limited paired image-text data through text-only training and visual grounding at inference. While I remain cautious about its effectiveness in real-world settings with open-ended action spaces, the proposed idea of disentangling text and image during training presents a novel perspective that may be of value to the community.", "rebuttal": "Thank you for your valuable comments. We appreciate your recognition of our work's innovative method, effectiveness and robustness of the model, and thoughtful analysis. Here we address your concerns and questions:\n\n## W1: Regarding Multiple-Choice QA Format\n\nThis is a good question! We choose multiple-choice question (MCQ) benchmarks for two primary reasons: \n\n1. **Objective and Reproducible Evaluation**: The MCQ format allows for a straightforward and consistent evaluation of decision-making with *accuracy*, which has been widely adopted by prior work on reasoning-based VLMs [1,2]. In our case, it serves as a stable testbed for evaluating final decisions while avoiding the potential bias or ambiguity in scoring open-ended outputs. Evaluating open-ended actions, especially in complex, real-world scenarios, remains a challenging task. Therefore, we focus on MCQ based benchmarks for model evaluation.\n2. **Challenging and Diverse Scenarios**: The benchmarks we used are recently proposed and specifically designed to evaluate decision-making in a wide range of complex, vision-grounded situations. They span human-centered social contexts, embodied agent tasks (e.g., robotics and autonomous driving), and egocentric video understanding, offering a comprehensive evaluation suite for our methods.\n\nWe acknowledge that extending these learned skills to interactive or open-ended environments remains an important and exciting direction for future work. Our current study, however, establishes a critical and reproducible step toward developing models capable of such real-world applications. We will revise our paper to clarify the benchmark choice.\n\n[1] MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning\n\n[2] R1-Onevision: Advancing Generalized Multimodal Reasoning through\nCross-Modal Formalization\n\n---\n\n## Q1 & W2: Regarding Analysis on Visual Feature \n\n> Q1. Model Performance when image descriptions are imperfect\n\nWe first clarify that Praxis-VLM does not use image descriptions during inference. The generated image descriptions are **only used in our preliminary analysis** (Section 2) to explore whether VLMs can perform decision-making when visual contexts are replaced by text.\n\nTo assess the effect of descriptions on VLMs, we conduct an additional experiment where we replace the high-quality image descriptions used in our preliminary analysis (Figure 2, generated by GPT-4o or dataset annotations) with descriptions generated by a weaker model, LLaVA-1.5-7B. We then test the original Qwen2.5-VL-7B model's accuracy.\n\n|     Task       | Original Text Description (high-quality) | LLaVA-1.5-7B Generated Description |\n|-----------------|--------------------------------------|------------------------------------------------|\n|VIVA             | 83.39 | 74.84 |\n|PCA-Bench    | 47.95 | 42.59 |\n\n\nThe results show a significant performance drop when using lower-quality, noisy descriptions, confirming that robust situational understanding is crucial for reliable decision-making to reach a good score for the benchmarks in our experiments. \n\nThese findings indicate that **the vision encoder's quality remains a critical factor in the model’s inference-time reliability**. However, enhancing visual perception typically requires paired image-text training data, which is unavailable in our target scenarios and beyond the scope of our work. Our approach addresses this by disentangling reasoning from perception, allowing reasoning to be trained efficiently on scalable text-only data.\n\n> W2. Does Praxis-VLM Actually Use Visual Features or Relies on Prior Learned Textual Knowledge?\n\nThis question is related to our discussion above. First, the significant performance drop when using noisy image descriptions (LLaVA-generated text) indicates that **accurate visual perception is essential for VLMs to achieve strong performance on these benchmarks**: when situational descriptions degrade, overall performance declines substantially. Praxis-VLM’s strong results on all three benchmarks suggest that the model is not applying patterns learned from text-only training, but instead **actively leveraging visual information during inference**.\n\nSecond, as detailed in our reasoning analysis (Section 5.4), a key component of the model’s reasoning is **Situational Analysis**, where the model explicitly interprets and grounds its reasoning in visual content. This is further illustrated in the output examples (Figures 10–12), where the model consistently begins its reasoning by analyzing the visual situation. These findings collectively suggest that Praxis-VLM grounds its reasoning in visual evidence, rather than relying solely on text-trained patterns. We will highlight this conclusion in our revision.\n\nWe will revise our paper to better clarify this.\n\n---\n\n## Q2. Regarding Synthetic Data Quality and Coverage\n\nOur synthetic dataset pipeline is designed to be **scalable and effective** to create a challenging and diverse dataset that fosters multi-step reasoning: (1) For diversity, we adopt a batch generation strategy, prompting GPT-4o to produce 10 samples at a time. Each batch undergoes deduplication based on word-overlap similarity to remove highly similar situations; (2) For difficulty, we embed explicit complexity instructions in our prompts (Line 649), guiding GPT-4o to generate questions that require non-trivial deliberation.\n\nBeyond this, we avoid manual filtering or heavy curation, enabling fast and domain-agnostic data generation. \n\nTo assess the diversity of the generated scenarios, we prompt GPT-4o to cluster the generated textual situations into topical categories. The results indicate **a broad coverage across varied scenarios**: \n- Workplace Performance and Personal Issues, \n- Resource Allocation\n- Project Management\n- Balancing Competing Interests\n- Policy, Rules & Enforcement\n- Ethical Dilemmas\n- Interpersonal Conflict\n- Emergency Handling\n- Navigating Setbacks\n- Event Planning & Logistics \n- Balancing Inclusivity and Majority Preference\n\n\nNotably, while the generated data cover diverse situations, there can still be a domain gap when compared to certain evaluation benchmarks (e.g., PCA-Bench’s embodied robotics or EgoNormia’s egocentric video contexts). However, Praxis-VLM’s consistent improvements and strong generalization across these benchmarks indicate that the text-based training equips the model with generalized and transferable reasoning skills. Figure 5 also shows that Praxis-VLM learns general reasoning dimensions (situational analysis, consequence evaluation, safety, and norm adherence), enabling effective transfer to multimodal scenarios.\n\nHowever, if the text data lacks variety or introduces bias, we believe the model performance will drop. Unlocking fundamental decision-making skills requires a wide variety of situations and perspectives. Prior work also confirms that dataset diversity is a key factor influencing model performance and alignment [1, 2]. Moving forward, we plan to explore advanced data selection methods to further enhance effective model training.\n \n---\n\n## Q3. Role of Math Pretraining\n\nThis is a great and insightful point! Our motivation for incorporating the math cold-start stage is to provide the model with a foundation in **abstract, generalizable logical reasoning** before introducing task-specific decision-making in Stage 2. The results show that this cold-start initialization measurably enhances the model’s **generalization capabilities**, particularly on the out-of-domain EgoNormia benchmark (as discussed in line 247). This demonstrates that the math training strengthens the model’s underlying reasoning and improves its adaptability to novel, complex decision-making scenarios. \n\nThis finding is also consistent with concurrent work [1], which shows that including mixed-domain training data can improve general reasoning performance. We will revise our paper to better clarify our motivation.\n\n\n> Measurable Improvements in reasoning\n\nThis is a good question! Quantifying reasoning depth and structure for comparison remains a challenge due to the lack of standardized metrics. To approximate this, we utilize GPT-4o as a Rhetorical Structure Theory (RST) discourse parser and convert each generated reasoning into a discourse tree. The average RST tree depths are comparable across models (3.93 vs. 3.90 for models with and without math pretraining), indicating similar surface structure.\n\nTherefore, we further conduct a manual inspection on examples where only the pretrained model produces correct predictions. We find that Praxis-VLM with math pretraining integrates **more relevant symbolic visual cues** into its reasoning, particularly in complex contexts. For instance, in PCA-Bench's autonomous driving scenarios, Praxis-VLM with math pretraining is able to identify and integrate multiple road symbols (e.g., traffic signs, lane markers) into its reasoning, leading to more contextually appropriate action choices, whereas the variant without math pretraining tend to rely on a narrower subset of cues and make less contextually appropriate action choices.\n\nWe hypothesize that the symbolic reasoning rigor inherent in math and geometry tasks foster a more systematic reasoning framework. This structured reasoning appears to transfer effectively to multimodal decision-making by promoting careful observation, multi-step analysis, and integration of fine-grained visual signals.\n\nWe will revise the paper to clarify this motivation and highlight these empirical insights more explicitly.\n\n\n[1] LIMA: Less Is More for Alignment\n\n[2] Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective"}, {"review_id": "6iOUkzqcjY", "reviewer": "Reviewer_J55j", "summary": "This paper presents Praxis-VLM, a framework that enhances the decision-making and reasoning abilities of VLMs by training them on text-only decision-making scenarios. The core idea is to first build reasoning capabilities in a purely textual environment and then transfer these skills to multimodal inference. The method leverages GRPO and introduces an adaptive reward system (focusing on format, accuracy, and reasoning length at different training stages). Notably, Praxis-VLM achieves better performance than standard VLMs and SFT baselines.", "questions": "1. Dataset Requirements: How scalable is the dataset generation process? How much manual intervention is required in crafting seeds, and could this limit transfer to other domains?\n\n2. Visual Context Integration: Why not explore using external tools (e.g., image captioners) to generate text from visual scenes instead of relying solely on synthetic text? Could that have improved the visual-text alignment?\n\n3. Ablation on Visual vs. Text Reasoning: Can you provide results where image-derived text is used during training instead of purely text-generated scenarios? Would this still achieve the same level of reasoning transfer?\n\n4. Role of Rlen: You report that increasing reasoning length sometimes decreases accuracy. Could you provide a detailed ablation isolating the impact of Rlen, and show which reward components ultimately matter most in driving performance gains?\n\n5. Relative Contributions: How much of the final performance improvement comes from text-to-text reasoning versus the GRPO training? Given the small gains, is it possible that most of the improvement is from the RL fine-tuning alone?\n\n6. Adaptive Reward Novelty: How does your adaptive reward meaningfully differ from prior GRPO reward structures? It feels largely similar to other multi-stage RL schemes.\n\n7. SFT Anomalies: Why does SFT sometimes perform worse than the base model? Is this due to overfitting on synthetic text or some limitation in the data quality?", "limitations": "NA", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The idea of using text-only reasoning to bootstrap visual decision-making is nice and data-efficient, especially given the scarcity of image-text reasoning datasets.\n\n2.  The multi-stage GRPO training with adaptive rewards is well-motivated and clearly described, focusing on progressively harder reasoning tasks.\n\n3. Leveraging GPT-4o to synthetically generate a high-quality, diverse text-only dataset reduces the reliance on expensive manual annotations.\n\n4. Praxis-VLM consistently outperforms vanilla VLMs and SFT baselines, even if the margins are modest (~3-4%).\n\nWeaknesses:\n\n1.  Dataset Generation Not Fully Discussed: The process of creating the synthetic dataset relies heavily on GPT-4o with a small set of seed prompts. It’s unclear how representative, diverse, or potentially biased this generated data is. Also, manual effort in seed question crafting might limit scalability.\n\n2. Lack of Ablations on Visual Inputs: The paper does not explore what would happen if image-to-text data (captions or descriptions derived from images) were used instead of purely synthetic text-to-text scenarios.\n\n3. Reward Design: The adaptive reward seems similar to prior GRPO training setups. The novelty here feels incremental, especially since no in-depth reward ablation is provided to justify the significance of each reward component (especially the Rlen reward, whose importance is assumed but not convincingly demonstrated).\n\n4. Limited Performance Gains: The performance improvement over the baselines is modest (3-4%), raising questions about the practical significance versus the added complexity.\n\n5. Results: The cold-start initialization and math-focused training did not show significant advantages in some cases, but the paper does not sufficiently explain why.\n\n6. SFT Underperformance: It’s surprising that SFT and reason with SFT sometimes performs worse than the base model, which raises questions about dataset or training stability but is not addressed.", "was_revised": true, "final_justification": "The paper introduces an interesting idea and Authors addressed some of my questions. However, the improvements presented are not high and experimental section is still weak and hence i will retain my score.", "rebuttal": "Thank you for the insightful feedback. We are encouraged that you find our work to be \"nice and data-efficient,\" and appreciate the motivation and performance of the proposed method. Below, we address your questions:\n\n## W1 & Q1. Data Generation\n\n> Scalability of Data Pipeline\n\nOur dataset pipeline is designed to be **scalable and effective** to create a challenging and diverse dataset that fosters multi-step reasoning: (1) For diversity, we adopt a batch generation strategy, prompting GPT-4o to produce 10 samples at a time. Each batch undergoes deduplication based on word-overlap similarity to remove highly similar situations; (2) For difficulty, we embed explicit complexity instructions in our prompts (Line 649), guiding GPT-4o to generate questions that require non-trivial deliberation.\n\nBeyond these automated steps, we intentionally avoid manual filtering or curation, enabling a **fast and scalable pipeline** for dataset creation.\n\n**The manual seed effort is minimal**: we craft only 10 seed questions (similar to Figure 8) to provide GPT-4o as in-context examples. These seeds serve **only to establish task format and structure rather than domain-specific knowledge**. Consequently, the manual intervention is minimal, enabling easy adaptation to new domains.\n\n> Diversity and Representativeness of Synthetic Data\n\nTo assess the diversity of the generated scenarios, we prompt GPT-4o to cluster the generated textual situations into topical categories. The results indicate a broad coverage across varied scenarios: \n- Workplace Performance and Personal Issues, \n- Resource Allocation\n- Project Management\n- Balancing Competing Interests\n- Policy, Rules & Enforcement\n- Ethical Dilemmas\n- Interpersonal Conflict\n- Emergency Handling\n- Navigating Setbacks\n- Event Planning & Logistics \n- Balancing Inclusivity and Majority Preference\n\n\nWhile the generated situations are more close to the domain of VIVA (human-centered situations), there can be a domain gap compared to other benchmarks (e.g., PCA-Bench’s embodied robotics or EgoNormia’s egocentric video contexts). However, Praxis-VLM demonstrates **consistent improvements and generalization** across all benchmarks, indicating that our text-driven RL training learns generalized reasoning skills rather than overfitting to domain-specific patterns. This is also supported by the analysis in Figure 5, showing that with our text-only RL training, Praxis-VLM successfully acquires fundamental and adaptable reasoning abilities for decision making.\n\nIn future work, we plan to explore advanced data selection approaches to further enhance model training efficiency.\n\n---\n\n## W2 & Q2 & Q3. Regarding Visual Inputs Ablations\n\nWe appreciate the question and would like to clarify a possible misunderstanding. Large-scale multimodal (image-text paired) datasets for vision-grounded decision-making are scarce and, to our knowledge, no such open-source training data are currently available. To address this, our approach deliberately **disentangles reasoning from visual perception**, using text-only training to isolate language as the medium for learning transferable reasoning skills. At multimodal inference, where image-text benchmarks are available, the model applies these reasoning abilities directly to real visual inputs (as shown in Figure 3).\n\n\n> W2 & Q2 & Q3. Why not use image captions?\n\nBecause large-scale multimodal training data is not available for this task, we cannot explore replacing our synthetic textual scenarios with image captions. This limitation is precisely why we investigate whether reasoning skills learned from pure text training can transfer effectively to multimodal settings, bypassing the need for expensive multimodal training data.\n\n\n> Q3. Visual vs. Text Reasoning Ablation\n\nWe cannot test *training* with image captions due to the lack of multimodal training datasets to obtain such captions. However, our method is inherently **compatible**: If we can generate captions, they could serve as additional textual situations to our synthetic data. Our current results already demonstrate that text-only training transfers effectively, as Praxis-VLM shows consistent improvements across multimodal benchmarks without any image-text multimodal training.\n\n---\n\n## W3 & Q4 - Q6. RL Training and Reward\n\n> Q4. Increasing reasoning length does not reduce accuracy\n\nWe would like to clarify that our analysis (Sec 5.2) shows that Praxis-VLM tends to generate **longer reasoning for more challenging samples**. The observed accuracy drop is therefore linked to sample difficulty rather than the reasoning length itself. We will clarify this in the revision.\n\n> W3 & Q4. Ablation of $R_{len}$\n\nBesides accuracy reward, we follow DeepSeek-R1 to include the format-related rewards ($R_{tag}$, $R_{format}$) to ensure outputs follow the explicit reasoning and prediction structure. We incorporate the length reward ($R_{len}$) to encourage more comprehensive analysis and reasoning, which is particularly beneficial for complex decision-making tasks.\n\nTo quantify its impact, we conduct an ablation study on Praxis-VLM-7B by removing $R_{len}$. The results show that $R_{len}$ consistently improves performance across benchmarks:\n\n|     Model       | VIVA   | PCA-Bench  |   EgoNormia  |\n|-----|--------|----|------|\n|Praxis-VLM-7B      | 84.03 | 60.25 |   54.33  |\n|w/o $R_{len}$    | 82.90 | 56.78 |  52.15  |\n\nThese results confirm that longer and more deliberate reasoning chains helps the model perform deeper analysis, leading to accuracy gains.\n\n\n> Q5. Contributions of text-to-text reasoning vs. GRPO training\n\nWe clarify that the text-to-text reasoning and the GRPO training are **fundamentally interdependent**. Praxis-VLM is trained with the GRPO algorithm on synthetic text-to-text data to develop reasoning abilities, which are then transferred to multimodal inference. Thus, we cannot isolate the effects of text-to-text reasoning and GRPO training independently.\n\n\n> W3 & Q6. Novelty of Method\n\nWhile our approach shares the general structure of multi-stage RL, the novelty of our adaptive reward lies in its alignment with stage-specific reasoning skills and its application to text-driven training for multimodal decision-making. \n\nFirst, unlike prior GRPO setups that rely on paired image-text data, we apply this adaptive reward strategy in a text-only RL setup, with the goal of transferring learned reasoning skills to vision-grounded inference. \n\nSecond, our reward function is explicitly tailored to target different cognitive competencies progressively: Stage 1 focuses on format adherence and general logical reasoning; Stage 2 shifts the reward signal toward situational reasoning and decision making. \n\nThis enables data-efficient learning and effectively equips the model with decision-making capabilities, without requiring costly multimodal supervision.\n\n---\n\n## W4. Limited Performance Gains\n\nWe acknowledge your concern and highlight the practical significance of our model  from two perspectives.\n\nFirst, all benchmarks are **recently proposed and designed to challenge current VLMs**. While the gains on in-domain benchmarks such as VIVA are around 3%, it achieves +13% over the baseline on PCA-Bench (7B) and +8% on EgoNormia (7B), demonstrating its ability to handle diverse, out-of-domain scenarios. Moreover, Praxis-VLM is trained in a data-efficient manner, without requirement of image-paired data, reinforcing its practical utility.\n\nSecond, we further compare Praxis-VLM with OpenVLThinker [1], a recent reason-VLM based on Qwen2.5-VL-7B for general visual reasoning trained with multimodal data.\n\n\n|     Model       | VIVA   | PCA-Bench  |   EgoNormia  |\n|----|---------|----------|---------|\n|Praxis-VLM-7B      | 84.03 | 60.25 |   54.33  |\n|OpenVLThinker-7B    | 82.34 | 49.21 |  53.13  |\n\nThese results highlight that with text-only training, Praxis-VLM demonstrates superior performance across all tasks.\n\nAs Praxis-VLM shares the same parameter count as Qwen2.5-VL and is trained solely with text-only data, it introduces minimal added complexity while delivering practically meaningful gains.\n\n---\n\n## W5. Cold-start Initialization\n\nThe math cold-start stage aims to equip the VLM with abstract, generalizable logical reasoning before Stage 2 task-specific training. This initialization proves crucial for **cross-domain generalization**, particularly on the EgoNormia dataset, where Praxis-VLM achieves notable gains.\n\nMeanwhile, the smaller improvements on VIVA are mainly due to domain alignment: our synthetic text corpus in Stage 2 is similar to VIVA’s human-centered tasks (as discussed in W1 & Q1), so the benefits of math pretraining are less pronounced. We will revise our paper (Line 247) to better clarify this.\n\n\n---\n\n## W6 & Q7. SFT Performance\n\nThe degradation of SFT-based models mainly arises from **behavioral cloning and overfitting** (Line 235). As discussed in W1 & Q1, there exists domain gaps between synthetic data and certain benchmarks like PCA-Bench and EgoNormia datasets. SFT training tends to enforce pattern memorization from the training distribution, which may hurt performance when the model is tested **on benchmarks from divergent domains**.\n\nThis observation aligns with prior work [1,2] that reason with SFT primarily establishes reasoning structures **without encouraging exploration or adaptability**. In contrast, RL training allows models to explore beyond static dataset examples, facilitating more **fundamental and transferable** decision-making skills. This difference explains why our GRPO-trained Praxis-VLM consistently generalizes better compared to its SFT counterparts.\n\n\n[1] OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning via Iterative Self-Improvement\n\n[2] SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training"}, {"review_id": "MFeIjeED8D", "reviewer": "Reviewer_x39V", "summary": "The paper proposes the VLM-based reasoning system, Praxis-VLM, that relies on text descriptions of the visual context (an image) in the vision-grounded decision-making process. Praxis-VLM uses reinforcement learning (GRPO method) to learn how to evaluate possible actions and their outcomes. The paper demonstrates that strong visual reasoning capabilities can be accomplished when the visual context (an image) is represented only through a pure textual description.", "questions": "1. It will be interesting to see the results of Praxis-VLM built based on other popular VLMs. It is probable that the results (degree of improvement, if any) might differ from those achieved using Qwen2.5-VL.\n2. How one can be sure, without concept leaking in the prompt formulation, that a textual description of an image generated by GPT- 4o is sufficient and accurate in a given decision-making context?\n3. Do you have any suggestion why there is no difference between „Major” and „Pass@1” scores of Qwen2.5-VL-7B on VIVA dataset?", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The idea of using merely text descriptions of visual scenes to impose effective visual reasoning and consequent decion-making is sound and interesting.\n2. The proposed Praxis-VLM shows its upper-hand over the baselines (base VLMs) in the experimental evaluation.\n3. The proposed approach can be of value in the taks for which the available (image, text) training pairs are scarce.\n\nWeaknesses:\n1. The experimental evaluation is limited to one VLM model in two size versions (3B and 7B parameters). \n2. It is not clear from the paper how a detailed (complete in a semantic sense) descriptions are imposed when using GPT-4o.", "was_revised": true, "final_justification": "My concerns are mostly addressed. The underlying idea of using solely text descriptions of visual scenes to impose effective visual reasoning is interesting, although I still believe the experimental justification should be expanded. I maintain my score.", "rebuttal": "Thank you for the thoughtful feedback. We are encouraged to know that you consider our work sound, interesting, and valuable for scenarios where paired image-text data is scarce. Below we address the weaknesses and questions you raised.\n\n## W1 & Q1. Experiment with Qwen2.5-VL \n\nWe appreciate your suggestion and would like to clarify our choice of Qwen2.5-VL (3B and 7B).\n\nFirst, due to computational constraints, we focus on models at the 3B and 7B scales. Qwen2.5-VL is selected because it is a strong, open-source VLM with competitive performance, offering a reproducible and reliable foundation for our study. Recent work [1,2] suggests that RL mainly elicits and refines knowledge embedded in the base model, rather than learning entirely new capabilities. Therefore, tackling complex decision-making depends critically on the strength of the base model, and Qwen2.5-VL models provide the best open-source option at this scale when our work is conducted.\n\nSecond, we follow established practice in reasoning VLM research, such as OpenVLThinker [3], R1-OneVision [4], and MM-Eureka [5], all of which also build upon Qwen2.5-VL. This ensures comparability and consistency with existing methods.\n\nWe agree that extending Praxis-VLM to larger scales or other architectures is an important future direction. However, our primary goal in this work is to **establish the principle and effectiveness of text-driven reinforcement learning for improving reasoning and decision-making capabilities**. We demonstrate that the reasoning skills learned through language-only training can be **successfully transferred to multimodal inference**. We will clarify this motivation and the rationale behind our current model choice more explicitly in the revision.\n\n\n[1] Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?\n\n[2] S1: Simple test-time scaling\n\n[3] OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning via Iterative Self-Improvement\n\n[4] R1-Onevision: Advancing Generalized Multimodal Reasoning through\nCross-Modal Formalization\n\n[5] MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning\n\n\n---\n\n## W2 & Q2. Regarding descriptions of images \n\nThank you for raising this question. We would like to clarify the following points and address any potential misunderstanding:\n\n\n> W3. How are detailed descriptions imposed using GPT-4o?\n\nPraxis-VLM **does not use generated textual descriptions of images**, either during training or evaluation.\n\nOur approach is motivated by the **scarcity of large-scale image-text paired datasets** of vision-grounded decision-making for VLM training. To address this, we explicitly disentangle reasoning from visual perception and train Praxis-VLM entirely on a **synthetic, text-only dataset**, where situations, questions, and potential decisions, are fully described in natural language, as shown in Figure 3. These synthetic textual training data are generated by GPT-4o **without any reference to benchmark images, ensuring no concept leakage**. \n\nDuring training, only the language model component of the VLM is updated, with the visual encoder disabled. We apply reinforcement learning with adaptive rewards on the synthetic text-only data to develop the model’s multi-step reasoning and decision-making abilities.\n\nAt inference time, the model directly processes the actual visual inputs (images or video frames) from benchmark datasets, without relying on any textual description from GPT-4o. The central idea is that the decision-making skills learned from text-only training transfer effectively to multimodal inference, avoiding the need for paired image-text training data or intermediate descriptions.\n\n\n> Q2. How can one be sure that a textual description of an image generated by GPT-4o is sufficient and accurate?\n\nWe would like to clarify that **GPT-4o generated image descriptions are only used in our preliminary analysis** (Section 2) to explore whether VLMs can perform decision-making when visual contexts are replaced by text descriptions. In these experiments, images are replaced with GPT-4o-generated or human annotated descriptions, and only the language model component of the VLM is used for prediction. These experiments indicates that decision-making and reasoning can be **disentangled** from visual perception, motivating our text-only RL training approach for improving VLM reasoning purely within the language space.\n\nSince Praxis-VLM always uses **actual visual inputs** (i.e., real images) during benchmark evaluations, the accuracy or completeness of GPT-4o-generated descriptions is not relevant to the core method. Our framework remains fully grounded in the visual modality during inference, ensuring valid and reliable evaluation in vision-grounded decision-making contexts.\n\nWe will revise our paper to better clarify this.\n\n\n---\n\n## Q3: \"Major.\" vs. \"Pass@1\" Scores for Qwen2.5-VL-7B \n\nThis is an excellent observation! The \"Major.\" score reflects the accuracy of the most frequent answer across 8 sampled outputs, while \"Pass@1\" measures whether at least one of the 8 outputs is correct. \n\nFor the base Qwen2.5-VL-7B model, these scores are close because the model predicts answers in a **highly deterministic manner**, without explicit reasoning. Its output distribution is sharp and concentrated, and the model tends to repeatedly generate the same answer. As a result, the most frequent answer (\"Major.\") almost always matches the \"Pass@1\" result.\n\nIn contrast, reasoning VLMs (Reason SFT & Praxis-VLM) make predictions with explicit reasoning, enabling them to produce **a broader variety of reasoning trajectories, exploring diverse solution paths, before reaching the final answer**. This diversity increases the likelihood that at least one of the 8 samples is correct, even if the most frequent answer might be wrong, thereby creating a larger gap between \"Major.\" and \"Pass@1\" scores. We will revise our paper to make this clearer.\n\n---\n\n## Regarding Figure 1 Location\n\nThanks for your suggestion. We will revise our paper and relocate the Figure 1 accordingly."}, {"review_id": "LjRPRaJLt8", "reviewer": "Reviewer_rfEF", "summary": "This paper investigates vision-grounded decision-making. The central insight is that the visual scenes can be better understood by first generating a detailed description of the scene, and then reason about the decision based on the description. The proposed Praxis-VLM is trained using a GRPO algorithm with an adaptive reward function", "questions": "1.\tWhile the approach transfers reasoning from vision to texts, it doesn't directly optimize visual grounding during training: Line 126, “during the training phase, visual inputs are replaced by their textual descriptions, and only the language model components of the VLM are updated. This leverages the scalability of text data for knowledge acquisition. Yet, during the inference phase, the entire trained VLM architecture, including the vision encoder, is used to process the image-text input pair”. \n2.\tThe performance of GRPO may rely heavily on the backbone VLM (Qwen2.5-VL as a strong open-source model).", "limitations": "1.\tThe data generation relies heavily on the backbone VLM (GPT-4o). the quality and diversity of synthetic scenarios are not deeply discussed. There's room to explore how scenario variety or GPT hallucination might affect training.\n2.\tGRPO, multi-stage training, and adaptive reward design are used to improve reasoning. Although effective, it is an established approach and may focus more on the engineering perspective.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: see below; Weaknesses: see Questions and Limitation\n\n1.\tThe empirical discovery is interesting (though less surprising), and it also shows the current, even SOTA VLMs, fall short in multimodal understanding. It may provide an alternative approach for a more data-efficient method of reasoning acquisition, bypassing the needs for image-text-reasoning pairs.\n2.\tThe performances seem to be good.\n3.\tThe paper provides thorough analysis, including the role of reasoning length, error cases, clustering of reasoning type, etc.", "was_revised": true, "final_justification": "My concerns are mostly addressed. I like the idea of text-only training to solve visual-language gap. I raised my score to 5: Accept.", "rebuttal": "We sincerely thank you for the valuable feedback and for recognizing the strengths of our work, including the interesting empirical discovery, strong performance, and thorough analysis. Below we address the raised questions and concerns.\n\n## Q1: Not Directly Optimizing Visual Grounding\n\nThis is a good question! We would like to first clarify that our approach focuses on **transferring reasoning skills learned from text-only training to vision-grounded multimodal inference**. Our preliminary analysis (Section 2) shows that reasoning and decision-making can be disentangled from visual perception, allowing us to leverage the scalable text-only data to enhance VLM decision-making abilities. \n\nThis design addresses a critical bottleneck in current VLM development: the lack of accessible, high-quality multimodal data for model training, especially for decision-making tasks. By focusing on text-driven training, our method allows for efficient and cost-effective learning of reasoning skills without relying on multimodal supervision.\n\n\nAs such, **directly optimizing visual grounding during training falls outside the scope of this work**, as it would require a fundamentally different setup involving paired multimodal data, which contrasts with our case where such multimodal training data is unavailable.\n\nAs acknowledged in our Limitation Section (Line 550), we agree that further strengthening visual perception is an important next step for improving vision-grounded decision-making. While beyond the scope of this work, we plan to explore perception-focused training in future research.\n\n---\n\n\n## Q2: Reliance on a Strong Backbone VLM like Qwen2.5-VL\n\nThis is a good question and we would like to clarify our choice of Qwen2.5-VL.\n\nFirst, due to computational constraints, we focus on models at the 3B and 7B scales. Qwen2.5-VL is selected because it is a strong, open-source VLM with competitive performance, offering a reproducible and reliable foundation for our study. Recent work [1,2] suggests that RL mainly elicits and refines knowledge embedded in the base model, rather than learning entirely new capabilities. Therefore, tackling complex decision-making depends critically on the strength of the base model, and Qwen2.5-VL provided the best open-source option at this scale when our work is conducted.\n\nSecond, we follow established practice in reasoning VLM research, including OpenVLThinker [3], R1-OneVision [4], and MM-Eureka [5], all of which also build upon Qwen2.5-VL. This ensures comparability and consistency with existing methods.\n\nWe agree that extending Praxis-VLM to other architectures is an important future direction. However, our primary goal here is to demonstrate the principle and effectiveness of text-driven reinforcement learning for enhancing reasoning and decision making. We will clarify this motivation and model choice more explicitly in the revision.\n\n\n[1] Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?\n\n[2] S1: Simple test-time scaling\n\n[3] OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning via Iterative Self-Improvement\n\n[4] R1-Onevision: Advancing Generalized Multimodal Reasoning through\nCross-Modal Formalization\n\n[5] MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning\n\n\n---\n\n\n## L1: Regarding Data Generation\n\n\n> Scalable data generation pipeline\n\nThis is a good point! While our data generation relies on an off-the-shelf model (GPT-4o), the pipeline is intentionally designed to be both **diverse and scalable**. We use an automated process: GPT-4o is prompted to generate 10 samples per batch, followed by deduplication based on word-overlap similarity to remove highly similar scenario descriptions within each batch. Beyond this step, we **avoid manual filtering or heavy curation**, enabling fast, domain-agnostic dataset creation that can scale to large volumes. \n\nLeveraging LLMs for synthetic generation has been widely adopted in previous research [1]. \nOur approach builds on this by further eliminating the need for costly image-text paired datasets, offering a more practical and data-efficient strategy to enhance VLM reasoning capabilities.\n\n\n\n> Diversity of synthetic scenarios\n\nTo assess the diversity of the generated scenarios, we prompt GPT-4o to cluster the generated textual situations into topical categories. The results indicate a broad coverage across varied scenarios: \n- Workplace Performance and Personal Issues, \n- Resource Allocation\n- Project Management\n- Balancing Competing Interests\n- Policy, Rules & Enforcement\n- Ethical Dilemmas\n- Interpersonal Conflict\n- Emergency Handling\n- Navigating Setbacks\n- Event Planning & Logistics \n- Balancing Inclusivity and Majority Preference\n\n\n> Impact of Synthetic Data on Training\n\nWhile the generated data cover diverse situations, there can still be a domain gap when compared to specific evaluation benchmarks (e.g., PCA-Bench’s embodied robotics or EgoNormia’s egocentric video contexts). However, Praxis-VLM demonstrates **consistent improvements and robust generalization** across all benchmarks, indicating that our text-driven RL training learns generalized, transferable reasoning skills rather than overfitting to domain-specific patterns. This is also supported by the reasoning dimension analysis in Figure 5, showing that even without deliberate domain-specific data curation, Praxis-VLM successfully acquires adaptable reasoning abilities.\n\n\nAs discussed in the Limitation Section (Line 545), we believe exploring advanced data generation and selection methods for more efficient model training is an important direction. We also appreciate your insightful suggestion of how potential GPT hallucinations affect training, which we plan to investigate in future work. We will revise the Limitation Section to further incorporate your suggestions into our discussions. \n\n[1] Self-Instruct: Aligning Language Models with Self-Generated Instructions\n\n---\n\n\n## L2: Regarding Established Approach and focus more on the engineering perspective\n\nThank you for raising this point. While we acknowledge that components such as GRPO, and multi-stage training are established techniques, we respectfully disagree that our work is primarily engineering-focused. The key contribution lies in the novel text-only RL training for VLMs to tackle a fundamental challenge: **enhancing reasoning of VLMs without relying on expensive image-text paired training data**.\n\nWe propose a text-driven reinforcement learning framework that uses language as the primary training medium to develop transferable reasoning capabilities, which are then effectively applied to multimodal decision-making tasks during inference. This framework enables us to decouple reasoning from perception, allowing for scalable and cost-efficient training while preserving strong performance in vision-grounded scenarios.\n\nBy demonstrating that such reasoning skills can be learned purely from text and transferred to complex visual tasks, our work offers a new conceptual perspective on VLM training that goes beyond the use of existing RL methods, and addresses a practical gap in current VLM development."}]

Author rebuttal:

Meta-review: This paper trains multimodal language models to perform better reasoning for visually-grounded decision-making by distilling a RL using text-based reasoning on synthetically generated scenario descriptions. Experiments show that models trained this way outperform existing vision-language models, including ones fine-tuned using SFT To perform reasoning, on several vision-based reasoning benchmarks.
Decision: Accept (poster)