What to Learn, and How: T oward Effective Learning from Rationales (2022)

Abstract: Learning from rationales seeks to augment model prediction accuracy using human-annotated rationales (i.e. subsets of input tokens) that justify their chosen labels, often in the form of intermediate or multitask supervision. While intuitive, this idea has proven elusive in practice. We make two observations about human rationales via empirical analyses:1) maximizing rationale supervision accuracy is not necessarily the optimal objective for improving model accuracy; 2) human rationales vary in whether they provide sufficient information for the model to exploit for prediction. Building on these insights, we propose several novel loss functions and learning strategies, and evaluate their effectiveness on three datasets with human rationales. Our results demonstrate consistent improvements over baselines in both label and rationale accuracy, including a 3% accuracy improvement on MultiRC. Our work highlights the importance of understanding properties of human explanations and exploiting them accordingly in model training.

## introduction in the past several years, explainability has become a prominent issue in machine learning, addressing concerns about the safety and ethics of using large, opaque models for decision-making. as interest has grown in explanations for understanding model behavior, so has interest grown in soliciting goldstandard explanations from human annotators and using them to inject useful inductive biases into models @xcite . many such explanation datasets have become available recently @xcite . a common format for explanations in nlp is the rationale, a subset of input tokens that are relevant to the decision. a popular architecture for generating such explanations is the rationale model, (a) unsupervised rationale @xcite susan wanted to have a birthday party . she called all of her friends . she has five friends . her mom said that susan can invite them all to the party . her first friend could not go to the party because she was sick . her second friend was going out of town . her third friend was not so sure if her parents would let her . the fourth friend said maybe . the fifth friend could go to the party for sure . susan was a little sad . on the day of the party , all five friends showed up . each friend had a present for susan . susan was happy and sent each friend a thank you card the next week . @xcite how many people did susan call ? | | 5 @xcite prediction: false (b) human rationale @xcite susan wanted to have a birthday party . she called all of her friends . she has five friends . her mom said that susan can invite them all to the party . her first friend could not go to the party because she was sick . her second friend was going out of town . her third friend was not so sure if her parents would let her . the fourth friend said maybe . the fifth friend could go to the party for sure . susan was a little sad . on the day of the party , all five friends showed up . each friend had a present for susan . susan was happy and sent each friend a thank you card the next week . @xcite how many people did susan call ? | | 5 @xcite prediction: true an explain-then-predict architecture which first extracts a rationale from the input and then makes a prediction from the rationale-masked text (that is, only the tokens included in rationale) @xcite . without external supervision on this rationale, we typically pursue parsimony via a sparsity objective. with the benefit of a human-annotated rationale for the true label, we can begin to understand model mistakes in terms of reliance on inappropriate features (and correct them). in the example above, the unsupervised rationale suggests that the model's mistake is due to missing key information about how many friends susan has (i.e., "five"). forcing the model to see these key tokens by only using the human rationale as the input fixes this mistake ( knowing that human rationales contain useful predictive signal, the key question becomes: can we improve model prediction accuracy by incorporating human rationales into training? numerous approaches to using human rationales in training have been tried, including: regularizing the parameters of a (linear) model @xcite ; regularizing model output gradients @xcite ; regularizing internal transformer attention weights @xcite ; and direct supervision on a rationale model @xcite , which serves as our baseline approach in this paper. these approaches have generally failed to significantly improve model prediction accuracy @xcite . a quality these prior approaches have in common is treating human rationales as internally and collectively uniform in predictive utility. that is, any token included in the human rationale is treated as equally important to include in the input representation; vice versa for tokens excluded. furthermore, all human rationales are weighted equally. the reality, we demonstrate empirically via ablation studies in §4, is that the predictive utility of human rationales is distributed unevenly between tokens in a rationale, and unevenly between rationales in a dataset. based on this analysis, we suggest that learning objectives which weight every token equally (accuracy in the case of direct supervision), and every rationale equally, are not optimal for improving downstream model accuracy. we operationalize these hypotheses in four distinct modifications to the baseline rationale model architecture. three of these modify the naive tokenwise accuracy supervision objective, and the fourth implements "selective supervision", ignoring unhelpful human rationales in training. evaluating on three datasets, our proposed methods produce varying levels of improvement over both a baseline bert model and a baseline bertto-bert supervised rationale model, ranging from substantial for multirc (3%) to marginal for e-snli (0.4%). additionally, our methods also improve rationale prediction performance. taken together, our results demonstrate the importance of considering the variance of predictive utility both between and within human rationales as a source of additional training signal. our proposed modifications help pave the way toward truly effective and general learning from rationales. ## data we consider three datasets in this work. all three are document-query text comprehension tasks, where the task is to determine whether the query is true or false given the document. we use the train, development, test splits offered by @xcite . ## analysis to understand properties of human rationales for the purpose of learning from rationales, we analyze the effect of human rationales when they are used as inputs to a trained model. ## methods we propose architecture changes based on these insights. our code is available at https://github. com/chicagohai/learning-from-rationales . ## discussion the analysis in section §4 explores the limits of potential improvement from learning from rationales. it suggests two insights toward improved learning from rationales: 1) that insofar as they boost model accuracy, not all human rationale tokens are equally valuable, e.g., with false positives causing less degradation than false negatives; and 2) we could in principle boost label accuracy with good rationale accuracy on useful (high-sa) rationales and low accuracy on useless (low-sa) ones. we exploit these two insights with four modifications to the baseline architecture. three of these diverge from flat rationale supervision accuracy: rationale supervision class weighting, sentencelevel rationalization, and importance embeddings. the last, selective supervision, pursues utilitydiscriminative weighting during model training. human rationale baseline supervised rationale best model (a) multirc: best model beats supervised baseline [cls] there have been many organisms that have lived in earths past . only a tiny number of them became fossils . still , scientists learn a lot from fossils . fossils are our best clues about the history of life on earth . fossils provide evidence about life on earth . they tell us that life on earth has changed over time . fossils in younger rocks look like animals and plants that are living today . fossils in older rocks are less like living organisms . fossils can tell us about where the organism lived . was it land or marine ? fossils can even tell us if the water was shallow or deep . fossils can even provide clues to ancient climates . [sep] what can we tell about former living organisms from fossils ? | | how they adapted [sep] [cls] there have been many organisms that have lived in earths past . only a tiny number of them became fossils . still , scientists learn a lot from fossils . fossils are our best clues about the history of life on earth . fossils provide evidence about life on earth . they tell us that life on earth has changed over time . fossils in younger rocks look like animals and plants that are living today . fossils in older rocks are less like living organisms . fossils can tell us about where the organism lived . was it land or marine ? fossils can even tell us if the water was shallow or deep . fossils can even provide clues to ancient climates . [sep] what can we tell about former living organisms from fossils ? | | how they adapted [sep] [cls] there have been many organisms that have lived in earths past . only a tiny number of them became fossils . still , scientists learn a lot from fossils . fossils are our best clues about the history of life on earth . fossils provide evidence about life on earth . they tell us that life on earth has changed over time . fossils in younger rocks look like animals and plants that are living today . fossils in older rocks are less like living organisms . fossils can tell us about where the organism lived . was it land or marine ? fossils can even tell us if the water was shallow or deep . fossils can even provide clues to ancient climates . [sep] what can we tell about former living organisms from fossils ? | | how they adapted [sep] prediction: false prediction: true prediction: false (b) fever: human rationale is insufficient [cls] legendary entertainment -lrb -also known as legendary pictures or legendary -rrb -is an american media company based in burbank , california . the company was founded by thomas tull in 2000 and in 2005 , concluded an agreement to co -produce and co -finance films with warner bros . , and began a similar arrangement with universal studios in 2014 . since 2016 , legendary has been a subsidiary of the chinese conglomerate wanda group . [sep] legendary entertainment is a subsidiary of warner bros pictures . [sep] [cls] legendary entertainment -lrb -also known as legendary pictures or legendary -rrb -is an american media company based in burbank , california . the company was founded by thomas tull in 2000 and in 2005 , concluded an agreement to co -produce and co -finance films with warner bros . , and began a similar arrangement with universal studios in 2014 . since 2016 , legendary has been a subsidiary of the chinese conglomerate wanda group . [sep] legendary entertainment is a subsidiary of warner bros pictures . [sep] [cls] legendary entertainment -lrb -also known as legendary pictures or legendary -rrb -is an american media company based in burbank , california . the company was founded by thomas tull in 2000 and in 2005 , concluded an agreement to co -produce and co -finance films with warner bros . , and began a similar arrangement with universal studios in 2014 . since 2016 , legendary has been a subsidiary of the chinese conglomerate wanda group . [sep] legendary entertainment is a subsidiary of warner bros pictures . [sep] prediction: supports prediction: supports prediction: supports (c) e-snli: supervised baseline beats best model [cls] a big dog catches a ball on his nose [sep] a big dog is sitting down while trying to catch a ball [sep] [cls] a big dog catches a ball on his nose [sep] a big dog is sitting down while trying to catch a ball [sep] [cls] a big dog catches a ball on his nose [sep] a big dog is sitting down while trying to catch a ball [sep] prediction: neutral prediction: neutral prediction: contradiction