Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making (2023)

Abstract: Explaining black-box model behavior with natural language has achieved impressive results in various NLP tasks. Recent research has explored the utilization of subsequences from the input text as a rationale, providing users with evidence to support the model decision. Although existing frameworks excel in generating high-quality rationales while achieving high task performance, they neglect to account for the unreliable link between the generated rationale and model decision. In simpler terms, a model may make correct decisions while attributing wrong rationales, or make poor decisions while attributing correct rationales. To mitigate this issue, we propose a unified two-stage framework known as Self-Attribution and Decision-Making (SADM). Through extensive experiments on five reasoning datasets from the ERASER benchmark, we demonstrate that our framework not only establishes a more reliable link between the generated rationale and model decision but also achieves competitive results in task performance and the quality of rationale. Furthermore, we explore the potential of our framework in semi-supervised scenarios.

## introduction large-scale pre-trained models @xcite have achieved state-of-theart results on various tasks @xcite @xcite , but their decisionmaking process is opaque. recent work @xcite @xcite @xcite have revealed that models often rely on superficial clues for predictions, which can make their decisions unconvincing. therefore, it is valuable to motivate models to provide trustworthy rationales to back up their decisions, which facilitates their implementation in real-world applications. figure 1 : with the claim and passage as input, the fid-ex framework generates the model decision, followed by the rationale as output. it is observed that the generated rationale does not convincingly justify the model decision. the underlined part of the passage represents the manually annotated rationale. to mitigate the above issue, we introduce a unified two-stage framework called self-attribution and decision-making (sadm). our sadm framework adopts distinct architectures for training and inference processes. for the training process, we train the model by jointly optimizing both the selfattribution and decision-making objectives. for the inference process, we adopt a two-stage format, which is inspired by the two-stage inference theory of human @xcite . the model is first prompted to extract the rationale from the given input (known as the self-attribution stage), and then, the model is prompted to utilize the extracted rationale to make informed decisions (known as the decision-making stage). moreover, our sadm framework incorporates the fusion-in-decoder (fid) @xcite architecture to address the challenges posed by lengthy texts, and the sentence mark (sm) @xcite strategy to mitigate the issue of random and irrelevant rationale generation during the selfattribution stage. to further enhance the model's comprehension at the decision-making stage, we also introduce a reasoning augment learning (ral) strategy. in our experiments, we introduce the rsq metric to quantitatively assess the reliable link between generated rationales and model decisions. experimental results consistently show significant improvements in the rsq metric for our sadm framework. for task performance and the quality of rationale, our sadm framework also outperforms strong baselines overall. moreover, we conduct ablation experiments to analyze the contribution of each component within our framework. ## sadm in this section, we first introduce our overall sadm framework (sec. 3.1), and then we describe how our sadm framework works in detail from two aspects: training process (sec. 3.2) and inference process (sec. 3.3). ## evaluation metrics consistent with prior work @xcite , we assess task performance using accuracy and evaluate the quality of generated rationales using the intersection-over-union f1 score (iou f1) and token f1 (tf1). we also follow the prior research @xcite to take rationale accuracy (r-acc) to evaluate the quality of generated rationales. additionally, we introduce a novel metric, the reasoning success quotient (rsq), to gauge the extent of the reliable link between the generated rationale and model decision. ## analysis in our analysis, we conduct ablation experiments (in sec. 6.1) to evaluate the effectiveness of each strategy in our sadm framework. we also consider different choices of threshold in the rsq metric (in sec. 6.2) to provide robust results. furthermore, we provide quantitative analysis to evaluate the lack of a reliable link between the rationale and model decision in the competitive fid-ex framework, which will be shown in detail in app. b. ## conclusion our proposed sadm framework establishes a more reliable link between the generated rationale and model decision while improving task performance and rationale quality. furthermore, we observe significant advantages in semi-supervised scenarios. in the future, we will explore how to optimize our framework to attain greater performance improvements.