D ecision F low: Advancing Large Language Model as Principled Decision Maker (2025)

Abstract: In high-stakes domains such as healthcare and finance, effective decision-making demands not just accurate outcomes but transparent and explainable reasoning. However, current language models often lack the structured deliberation needed for such tasks, instead generating decisions and justifications in a disconnected, post-hoc manner. To address this, we propose DecisionFlow, a novel decision modeling framework that guides models to reason over structured representations of actions, attributes, and constraints. Rather than predicting answers directly from prompts, DecisionFlow builds a semantically grounded decision space and infers a latent utility function to evaluate trade-offs in a transparent, utility-driven manner. This process produces decisions tightly coupled with interpretable rationales reflecting the model’s reasoning. Empirical results on two high-stakes benchmarks show that DecisionFlow not only achieves up to 30% accuracy gains over strong prompting baselines but also enhances alignment in outcomes. Our work is a critical step toward integrating symbolic reasoning with LLMs, enabling more accountable, explainable, and reliable LLM decision support systems. Code and data are at https://github.com/xiusic/DecisionFlow.

## introduction understanding and navigating high-stakes situations is a defining challenge for both human and artificial intelligence @xcite . in critical domains such as healthcare, disaster management, and economic policy, decisions are rarely made by intuition alone @xcite . instead, effective decision-making requires a deliberate and structured process: identifying relevant factors, modeling their interactions, and systematically evaluating * equal contribution. the consequences of possible actions. this structured reasoning ensures that decisions are not only well-informed but also transparent and accountable. such a process is formalized in the concept of decision modeling, which provides a principled framework for navigating complex choices. ## task definition. decision modeling problems typically consist of three key components: (1) a scenario description, (2) an action-based question, and (3) unique constraints. the core challenge lies in systematically analyzing trade-offs between alternative solutions while accounting for strengths and weaknesses under these constraints. we propose a step-by-step reasoning framework that constructs explicit decision structures, enabling more comprehensive and interpretable analysis by language models. formally, we define a decision modeling problem as a triplet: ## conclusion in this work, we present decisionflow, a novel decision modeling framework that equips llms with structured, utility-driven reasoning capabilities for high-stakes domains. by constructing a semantically grounded decision space and inferring latent utility functions, decisionflow enables transparent evaluation of trade-offs and produces decisions tightly coupled with interpretable rationales. empirical results on two challenging benchmarks demonstrate that decisionflow significantly improves decision accuracy while enhancing alignment and fairness compared to strong baselines. these results underscore the value of integrating symbolic structure and utility-based reasoning into llm workflows, offering a promising direction toward more accountable, explainable, and trustworthy ai systems for critical decision support.