VIVA +: Human-Centered Situational Decision-Making (2025)

Abstract: Multimodal Large Language Models (MLLMs) show promising results for embodied agents in operating meaningfully in complex, human-centered environments. Yet, evaluating their capacity for nuanced, human-like reasoning and decision-making remains challenging. In this work, we introduce VIVA+, a cognitively grounded benchmark for evaluating the reasoning and decision-making of MLLMs in human-centered situations. VIVA+ consists of 1,317 real-world situations paired with 6,373 multiple-choice questions, targeting three core abilities for decision-making: (1) Foundational Situation Comprehension, (2) Context-Driven Action Justification, and (3) Reflective Reasoning. Together, these dimensions provide a systematic framework for assessing a model’s ability to perceive, reason, and act in socially meaningful ways. We evaluate the latest commercial and open-source models on VIVA+, where we reveal distinct performance patterns and highlight significant challenges. We further explore targeted training and multi-step reasoning strategies, which yield consistent performance improvements. Finally, our in-depth analysis highlights current model limitations and provides actionable insights for advancing MLLMs toward more robust, context-aware, and socially adept decision-making in real-world settings.

## introduction the advancement of mllms @xcite @xcite marks a pivotal step toward creating embodied systems that perceive, understand, and interact within complex human environments @xcite . these models are promising for applications ranging from nuanced assistive technologies and collaborative robotics to autonomous systems adept at navigating intricate social spaces @xcite . yet, achieving this potential requires sophisticated reasoning and decision-making capabilities that approximate human cognitive processes. it is particularly critical when confronting dynamic social interactions, practical constraints, and ambiguous situations @xcite @xcite . as such, systematically evaluating the capabilities of mllms in these contexts becomes increasingly vital. recent benchmarks have assessed the decisionmaking capabilities of mllms in areas such as embodied planning @xcite , safety awareness @xcite , and normative action selection @xcite . however, these efforts often target isolated abilities or narrow skill dimensions, such as selecting a proper action or generating a justification. in contrast, human decision-making is inherently integrative and context-sensitive, relying on the dynamic interaction between situation comprehension, contextual reasoning, and social-cognitive inference that extend well beyond surface-level choices @xcite . as a result, existing evaluations fall short of assessing whether mllms can demonstrate the nuanced, adaptive reasoning and decision-making necessary for human-centered contexts, thereby limiting their safe and effective deployment in real-world applications. in this work, we study the task of humancentered situational decision-making in multimodal environments, where mllm-based agents must perceive visual environments, reason contextually, and take actions appropriate to the situation. unlike generic decision-making, this task further requires the agent to understand human norms, interpret social dynamics, and infer implicit needs, ensuring that its decisions are not just contextually relevant but also socially grounded. we address a critical but underexplored question: can mllms perform human-centered decision-making that reflects the integrated cognitive processes humans use in complex environments to make decisions aligned with human expectations? building upon our previous work viva @xcite , we introduce viva+, a novel benchmark to explicitly evaluate human-centered reasoning and decision-making in mllms. our benchmark design is grounded in the theory of naturalistic decision-making (ndm) @xcite , which posits that effective decisions in real-world environments emerge from an iterative interplay of situation assessment, context-sensitive action selection, and socialbehavioral inference, often under uncertainty and constraints. by doing so, viva+ systematically assesses mllms across interconnected layers of cognition involved in decision-making. as shown in figure 1 , viva+ comprises 1,317 real-world images depicting diverse humancentered situations, accompanied by a total of 6,373 multiple-choice questions in spanning seven distinct types mapped to three capability dimensions: (1) foundational situation comprehension @xcite : assesses a model's ability to accurately perceive and interpret the situation by identifying fine-grained visual details and critical contextual information essential for understanding "what is happening." (2) context-driven action justification @xcite : evaluates whether a model can select appropriate actions under the constraints including both social role expectations and physical conditions-i.e., answering "what to do" in a given scenario. (3) reflective reasoning @xcite : captures higher-order reasoning critical for navigating complex and ambiguous situations. this includes inferring implicit roles, analyzing potential misunderstandings, and performing counterintuitive or counterfactual reasoning @xcite . these tasks test whether models can move beyond reactive responses (i.e., system 1 of fast thinking) toward critical and flexible reasoning (i.e., system 2 of slow thinking) necessary for sophisticated decision-making @xcite . by spanning this spectrum, from perceptual understanding to action justification and higherorder reasoning, viva+ offers a holistic framework for evaluating the depth and robustness of model decision-making in realistic, human-centered contexts. we use viva+ to evaluate a suite of state-ofthe-art commercial and open-source mllms and llms, uncovering distinct performance patterns across different cognitive abilities. our in-depth analysis further shows that incorporating targeted training and multi-step reasoning can effectively enhance model performance. we also identify common errors, offering insights into current limitations and directions for future improvements. to the best of our knowledge, viva+ is the first benchmark to systematically evaluate multimodal decision-making in human-centered situations. in conclusion, our primary contributions are: ## conclusion we introduce viva+, a benchmark for evaluating the human-centered reasoning and decisionmaking of mllms. viva+ assesses models across three key cognitive dimensions-situation comprehension, context-sensitive action justification, and reflective reasoning. the experiments and analyses show that current mllms still face challenges in navigating complex, socially grounded scenarios. by offering a comprehensive evaluation, viva+ aims to support the development of more robust and socially aligned ai systems.