STRUX : An LLM for Decision-Making with Structured Explanations (2025)

Abstract: Countless decisions shape our lives, and it is crucial to understand the how and why behind them. In this paper, we introduce a new LLM decision-making framework called STRUX, which enhances LLM decision-making by providing structured explanations. These include favorable and adverse facts related to the decision, along with their respective strengths. STRUX begins by distilling lengthy information into a concise table of key facts. It then employs a series of self-reflection steps to determine which of these facts are pivotal, categorizing them as either favorable or adverse in relation to a specific decision. Lastly, we fine-tune an LLM to identify and prioritize these key facts to optimize decision-making. STRUX has been evaluated on the challenging task of forecasting stock investment decisions based on earnings call transcripts and demonstrated superior performance against strong baselines. It enhances decision transparency by allowing users to understand the impact of different factors, representing a meaningful step towards practical decision-making with LLMs.

## motivation decision-making is complex, as it requires the evaluation of various determinants that can influence outcomes @xcite . this ability is crucial across multiple fields, ranging from healthcare, where decisions can determine patient health outcomes @xcite , to finance, where investment choices can impact financial stability @xcite . for llms to be effective, they must not only identify relevant facts but also weigh the favorable and unfavorable aspects to reach insightful conclusions. to date, it remains unclear whether llms can effectively balance multiple factors in complex scenarios to make rational decisions. llms also produce lengthy, plain text explanations that can sometimes overwhelm users with too much information or ambiguity @xcite @xcite @xcite . as we increasingly rely on those llms for critical decision-making, it is important to prioritize transparency and accountability @xcite . we propose structuring these explanations into a table format, where each fact is listed with a 'strength level' that measures its influence on the decision-making process. this approach not only facilitates review and modification of various facts by humans, but also enhances the transparency of the decisions made. further, a significant advantage of llms is their ability to reason through complex scenarios, which can enhance the decision-making processes @xcite @xcite . notably, dellma @xcite uses classical decision theory to help llms make decisions under uncertainty. it infers a utility function through prompting and optimizes the expected utility using monte carlo estimation. @xcite calculate decision probabilities using a bayesian model and present results on datasets such as common2sense @xcite and plasma @xcite . in contrast, our approach involves fine-tuning an llm with domainspecific knowledge to ensure it prioritizes supporting facts accurately. training instances are generated via a series of reflection steps, without relying on human annotations. our research explores the potential of using earnings call transcripts to forecast stock investment decisions @xcite @xcite . publicly traded companies in the u.s. are mandated by the securities and exchange commission (sec) to regularly report their financial performance, often through earnings calls. these calls include presentations from senior executives, such as the ceo and cfo, followed by a q&a session with financial analysts. the objective is to reassure investors about delta air lines achieved a pre-tax profit of $216 million in the september quarter, marking its first quarterly profit since the pandemic began. revenue recovery in the september quarter reached 66% of 2019 levels, driven by strong consumer demand and an increase in business and international travel. business travel is accelerating with expectations that between 80% and 100% of business travel will return by the end of next year. the airline faces rising fuel prices, projecting a modest loss for the fourth quarter as crude prices have risen nearly 60% year-to-date. ## experimental results we evaluated our strux against strong baselines for forecasting stock investment decisions. this includes dellma @xcite , which incorporates uncertainty into llm decision-making using classical decision theory and has been tested on tasks such as agriculture planning and finance. additionally, we tested gpt-4o-mini-2024-07-18 and llama3-8b-instruct by providing either full transcripts or concise fact tables to elicit investment decisions; see appendix for the prompt. ## conclusion strux marks a notable step in using llms for decision-making. it integrates structured explanations into the decision-making process through a series of reflective steps. strux not only leads to higher accuracy but also improves the transparency of llm decisions, making it a valuable tool for complex decision-making scenarios.