Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in LLM s (2024)

Abstract: Large Language Models (LLMs) have shown remarkable capabilities in tasks such as summarization, arithmetic reasoning, and question answering. However, they encounter significant challenges in the domain of moral reasoning and ethical decision-making, especially in complex scenarios with multiple stakeholders. This paper introduces the Skin-in-the-Game (SKIG) framework, aimed at enhancing moral reasoning in LLMs by exploring decisions’ consequences from multiple stakeholder perspectives. The core components of the framework consist of simulating accountability for decisions, conducting empathy exercises on different stakeholders, and evaluating the risks associated with the impacts of potential actions. We study SKIG’s performance across various moral reasoning benchmarks with proprietary and open-source LLMs, and investigate its crucial components through extensive ablation analyses. Our framework exhibits marked improvements in performance compared to baselines across different language models and benchmarks.

## introduction in recent years, large language models (llms) @xcite @xcite have showcased an unprecedented degree of performance in reasoning @xcite @xcite , optimization @xcite @xcite , education @xcite , and instruction following @xcite . most prior works focused on standard prompting where we expect an answer from the model right away; later work has shown that generating step-by-step reasoning can be superior @xcite @xcite . however, constrained @xcite @xcite @xcite or ethical decision-making in the face * equal contribution of potential risks to society still encounters stumbling blocks @xcite @xcite . moral reasoning, unlike general problemsolving, involves charting the intricate landscape of human values and ethics. this complexity is partly due to the influence of culture and political ideologies on morality @xcite and social biases @xcite . however, there also exist universal moral values that transcend cultural differences @xcite . to address these challenges, most approaches have focused on aligning llms with human values through top-down approaches such as finetuning @xcite or prompting @xcite . recent works have turned to deliberate thinking by counterfactual reasoning to enhance the deduction abilities of llms @xcite . following recent advancements in planning with llms, we argue that the current limitations stem from two main issues: under-exploration of the consequences of probable decisions @xcite @xcite @xcite and a lack of accountability for the llms' choices @xcite . @xcite argue that bearing the outcomes of one's decisions lead to more ethical and responsible choices that minimizes the risky tail events that can be detrimental to every stakeholder affected. inspired by these insights, we present the skin-in-the-game (skig) framework for llms to enhance their moral reasoning capabilities. in our skig framework, we leverage llms to explore different scenarios based on given situations and potential actions. this approach facilitates a deeper understanding of the decision impacts on various stakeholders. we make the language model envision itself as each character in a situation and simulate accountability for its actions as shown in figure 1 . this perspective shift has led to marked improvements, with substantial performance enhancements of up to 70% across a wide array of benchmarks. these improvements are consistent across various types of llms, including both proprietary and open-source models. decision making with llms llms can be adapted to many downstream tasks such as planning and recommendations by prompting @xcite . chain-of-thought @xcite and recent advancements @xcite @xcite improve performance on multi-step reasoning tasks. self-consistency @xcite samples many rationales to help with covering a larger decision landscape. our skig framework is complementary to these approaches but adds the critical dimension of analyzing stakeholder impacts for a given decision under various scenarios. the key is asking llms to "put skin in the game" by explicitly imagining and tracing the impact of any decision or recommendation it makes. from an alignment perspective, we aim to change the intrinsic optimization objective (in mesa-optimization @xcite ) to incorporate multiple stakeholder objectives (see sec. ## method our approach draws inspiration from the skin-inthe-game concept introduced by @xcite . the essence of our method lies in aligning decision-makers with both the potential rewards and risks inherent in their choices. by integrating principles derived from psychology, skin-inthe-game philosophy, and ethical decision-making, our proposed approach not only enhances moral reasoning but also cultivates a more nuanced and conscientious decision-making process. ## experiments we demonstrate that the skin-in-the-game framework outperforms moral reasoning on various baselines standard prompting, zero-shot cot @xcite and the state-of-the-art thought experiment (te) @xcite , across benchmarks mmlu moral scenarios @xcite , moral stories @xcite , ethics commonsense morality @xcite and social chemistry 101 @xcite . this is observed for proprietary models error baselines we perform error analysis and categorize errors into bins representing their root causes: pessimism bias, assumption bias, and binary bias. pessimism bias stems from excessive caution when the model overestimates the likelihood of negative outcomes. assumption bias arises when the model makes decisions based on unsupported assumptions. binary bias occurs when the model defaults to binary judgments for moral gray areas. we instruct human annotators to classify errors into the above bias categories. we additionally evaluate improvements using the error correction rate @xcite and compositionality gap @xcite metrics to assess the performance of our method compared to other baselines. ## discussion in this section, we perform a critical analysis of our framework, using the mmlu moral scenarios bencmark as the primary case study. 4 how accurate is the stakeholder identification in skig? we use the social chemistry 101 dataset to assess stakeholder identification -a crucial step for multi-stakeholder alignment. employing fewshot learning, we prompt a gpt-4 "judge" model with multiple choice questions to evaluate skig stakeholder identification versus social chemistry annotations. skig was correctly able to identify all the primary stakeholders and additional secondary stakeholders with more than 90% accuracy across llms. does skig generate consistent reasoning paths? candidate reasoning paths generated at hightemperature setting for identical questions are presented to a gpt-4 "judge" model to evaluate consistency. we observe high component-wise and overall consistency across different sample rationales. the empathy exercise component shows high consistency of 93%, closely followed by the risk assessment and outcome summary components, which show consistency rates of 92% and 91%, respectively. strong consistency rates within a tight range for all components emphasizes reliable performance of skig. how robust is skig reasoning to different prompts? we conducted an ablation study to assess the robustness of our methodology to variations in linguistic expression. we evaluated ten additional prompt sets with altered lexical choices and syntax versions of the standard prompt. we observe an average accuracy of 70.5% on all the runs. the results show consistent and similar accuracy levels for all the prompt variants. the efficacy of our method lies predominantly in the underlying strategy itself rather than specific prompt wording. does conditioning skig for optimism/pessimism during risk-assessment improve/degrade performance? our analysis of different risk assessment objectives reveals higher accuracy with best-case-only versus worstcase-only goals, both at the aggregated overall level and at individual stakeholder-level. this is due to higher error-correction rates for pessimism bias in best-case. the risk-assessment component has stakeholder level insights from empathy exercise as context, making risk-assessment at overall-level more favorable for the reasoning process. a balanced assessment weighing both best-case and worst-case objectives across stakeholders proves conducive for nuanced risk analysis. does this method necessitate a multi-turn framework or can a single-turn approach suffice? to understand the impact of multi-turn reasoning, we test variants of skig. we observe that single-turn variants performed poorly, with accuracy levels below standard prompting levels at 20% and 22% for all sub-questions in single-turn (st-all) and best performing sub-questions in a single-turn (st-best) respectively. multi-turn variants with shorter reasoning paths resulted in improved accuracy of 59% for best performing sub-questions in multiturn (mt-best) than single-turn variants, baselines, but were not able to match skig (mt-all) accuracy levels. a shorter reasoning path might be chosen when some reduction in accuracy levels are acceptable for multi-stakeholder alignment. ## conclusion we introduced the skin-in-the-game (skig) framework, significantly enhancing llms' moral reasoning by simulating accountability and evaluating impacts from multiple perspectives, particularly emphasizing multi-stakeholder alignment in the decision-making process. key components like empathy exercise and risk assessment reduce common biases, leading to more ethically sound outcomes. our results demonstrate skig's superiority, surpassing previous methods across various benchmarks and llms, and marking a substantial improvement in ethical decision-making.