Unsupervised Selective Rationalization with Noise Injection (2023)

Abstract: A major issue with using deep learning models in sensitive applications is that they provide no explanation for their output. To address this problem, unsupervised selective rationalization produces rationales alongside predictions by chaining two jointly-trained components, a rationale generator and a predictor. Although this architecture guarantees that the prediction relies solely on the rationale, it does not ensure that the rationale contains a plausible explanation for the prediction. We introduce a novel training technique that effectively limits generation of implausible rationales by injecting noise between the generator and the predictor. Furthermore, we propose a new benchmark for evaluating unsupervised selective rationalization models using movie reviews from existing datasets. We achieve sizeable improvements in rationale plausibility and task accuracy over the state-of-the-art across a variety of tasks, including our new benchmark, while maintaining or improving model faithfulness.

## introduction with the advent of large pre-trained language models like gpt-3 @xcite , the size and complexity of deep learning models used for natural language processing has dramatically increased. yet greater performance and complexity can come at the cost of interpretability, masking anything from implementation mistakes to learned bias. a model architecture that justifies its output by providing relevant subsets of input text as a rationale is therefore desirable (see example in figure 1 ). the unsupervised selective rationalization architecture as introduced by @xcite generates rationales alongside predictions by chaining two jointly-trained components, a rationale-generator and a predictor. the generator extracts a rationale: concatenated short and concise spans of the input class: positive ultra low budget but extremely inventive horror film about a group of friends vacationing in a cabin who accidentally awaken an evil force in the woods via the necronomicon, the book of the dead. bruce campbell stars as ash, who eventually becomes the sole survivor and has to battle both the demons from the woods, and his friends who have become demons (including his own girlfriend). the results shown on screen are amazing considering the film's tiny budget, constant location changes, and a filming schedule that was sporadic over two years… ## related work a major challenge with selective rationalization is that discrete selection of rationale tokens is nondifferentiable, making training challenging without additional rationale supervision. @xcite use reinforce-style learning @xcite to propagate the training signal from the predictor to the generator. @xcite propose a differentiable approach leveraging the hard kumaraswamy distribution. @xcite strive to improve rationale comprehensiveness. @xcite focus on avoiding spuriously correlated ra-tionales. @xcite tackle the propensity of selective rationalization models to get stuck in local minima. @xcite use diagnosticsguided training to improve plausibility. our work builds on the previous approaches, since we also frame the generator-predictor interaction as a cooperative game and seek to improve plausibility. the previous approaches have, however, introduced additional training objectives @xcite or involved incorporating a third adversarial @xcite or cooperative @xcite component. this increases model complexity significantly, leading to more resourceintensive and/or complicated training. instead, we demonstrate the effectiveness of online noise injection, a considerably more lightweight approach. an alternative approach is proposed by deyoung et al. (2020) who assemble a series of datasets with labeled rationales; this enables fully supervised rationale learning. given rationale-annotated training sets, @xcite train each model component separately, approaching the accuracy of an entirely black-box model. although this is a compelling direction, requiring supervision reduces the practical usability of this technique, as many applications lack rationale annotations. both unsupervised and supervised selective rationalization approaches generally require a specific token selection strategy to select the output rationale from the generator model @xcite @xcite . no previous work that we are aware of, however, has tried to then modify the output rationale before it is input into the predictor. using online noise injection to enforce prediction stability is therefore a novel approach that adds greater power to the current architectures and can be easily retrofitted. ## implausible rationale generation previous work has conceptualized the interaction between the generator and the predictor as a cooperative game @xcite @xcite @xcite . this repeated sequential game consists of two-round stage games. in the first round, the generator accepts an input sequence x 1:t and outputs a rationale selection as a binary mask m 1:@xmath0:t ⊙ m 1:t satisfies rationale constraints. in the second round, the predictor accepts an input sequence x 1:t ⊙m 1:t and outputs prediction y . the joint objective is to minimize the loss (see equation 2 ) based on the generated mask (see equation 1 ): m 1:t ← gen(x 1:t ; θ gen ), m 1:@xmath1,θpre l(pre(x 1:t ⊙ m 1:t ; θ pre ), ỹ ) (2) for classification, it is customary to minimize the cross-entropy loss l ce . such a system can be shown to maximize mutual information (mmi) of the rationale with respect to the class label provided sufficient generator and predictor capacity as well as a globally optimal generator @xcite : however, this property does not guarantee rationale plausibility. first, mmi does not protect against spurious correlations @xcite . for example, a pleasant taste is not a good explanation for a positive review of a beer's appearance, although the two aspects are strongly correlated. second, mmi does not prevent rationale degeneration if the generator and predictor already contain certain biases, for example from pre-training (jacovi and goldberg, 2021). third, mmi does not prevent rationale degeneration if the generator and predictor are sufficiently powerful to develop a common encoding. @xcite found that providing the generator with a label predicted by a full-input classifier led the generator to develop a communication scheme with the predictor, including a period for positive and a comma for negative examples. jacovi and goldberg (2021) argue that any generator with sufficient capacity to construct a good inner-representation of y can cause rationale degeneration. the key underlying cause is that a sufficiently powerful generator is not disincentivized to produce implausible rationales beyond the assumption that generating a plausible rationale should maximize the expected accuracy of the predictor in the current training iteration. however, since the predictor is treated as a black box, this is not guaranteed. on the i-th training iteration, the generator greedily selects a binary mask m 1:t that minimizes the expected loss: arg min ## online noise injection we propose a strategy that disrupts the generator's learned representation of the predictor pre g,i for all games @xmath2, thereby making it harder for the generator to learn to exploit quirks of the predictor. we use online noise injection, which probabilistically perturbs unimportant words in a rationale sequence x of length t (see algorithm 1). if the generator attempts to generate an implausible rationale during training iteration i, it strategically includes unimportant words from the input text in the generated rationale, relying on the predictor to pick up on the bias. by subtly perturbing the rationale -replacing the unimportant wordsnoise injection disrupts this attempt, and the predictor does not respond to the generator-injected bias favorably as expected by the generator. the generator is therefore forced to unlearn/reset its representation pre g,i of the predictor and reassess its strategy, learning that generating implausible rationales is ineffective. across any two stages i, j of game g, noise injection therefore keeps the learned representations of the predictor more consistent: we use these precomputed probabilities to sample which words to replace as shown in algorithm 1. the words are replaced with random words from the vocabulary v. nonetheless, we also strive to prevent sampling "keywords" from the vocabulary -words that are highly indicative of a label -to avoid confusing the predictor. we compute the sampling probability of w i as its normalized atf*idf, where atf corresponds to term frequency macroaveraged over d: our baseline model builds on the a2r architecture by @xcite who improve training stability by using an auxiliary predictor connected directly to the generator via an attention layer -this allows for gradients to flow. a2r selects top-k 2 bigrams with the highest attention scores from the generator as the rationale and input for the second predictor, with k corresponding to the number of rationale tokens selected as a fraction of the size of the input text. the two components minimize their separate criteria as well as the jensen-shannon divergence of their predictions y a and y r for the attentionbased predictor and the rationale-based predictor, respectively. a2r's generator consists of a fixed glove @xcite embedding layer and a linear token scoring layer. to take full advantage of our noise injection strategy, we replace the limited-capacity generator with bert @xcite . this allows us to use a simpler attention-based predictor than a2r (see figure 2 ). to further manifest the efficacy of noise injection, we opt for a top-k unigram selection strategy which offers less regularization compared to a bigram selection strategy. selecting unigrams is more challenging because it allows the model to select uninformative stopwords like "a" or "the". our architecture is shown in figure 2 . both the selection strategy and the noise injection are modelexternal and untrained. as in @xcite , the attention-based (see equation 10 ) and the rationalebased (see equation 11 ) components are trained using identical objectives -minimizing the sum of the cross-entropy loss and the jensen-shannon divergence of the two predictors: we refer to our model as bert-a2r and add +ni when noise injection is used during training. ## usr movie review dataset previous work on unsupervised selective rationalization used a decorrelated subset of the beeradvocate review dataset @xcite as preprocessed by @xcite . the dataset has recently been removed at the request of beeradvocate and is therefore inaccessible to the scientific community. beeradvocate reviews consists of 80,000 labeled reviews without rationales for training/validation and ∼1,000 labeled reviews with token-level annotated rationales for testing. alternative datasets either include rationale labels for the entire dataset (deyoung et al., 2020) or do not provide rationale labels altogether (e.g. maas et al. (2011)). moreover, large datasets such as multirc or fever tend to provide sentence-level rationales compared to beeradvocate token-level rationales. we thus repurpose existing movie review datasets to recreate a task similar to beer review, enabling new work on unsupervised selective rationalization to evaluate their performance against models designed for beer review. we merge a smaller eraser movie review dataset (deyoung et al., 2020; zaidan and eisner, 2008; pang and lee, 2004) that has full token-level rationale annotations with the lower-cased large movie review dataset (maas et al., 2011) which has no rationale annotations. the movie review task is similar to the binarized beer review task as used in chang et al. (2019); yu et al. (2019); chang et al. (2020); yu et al. (2021); we replaced the generator's fixed glove @xcite embedding layer used in a2r with bert-base. the original a2r uses a fixed glove embedding layer, gru @xcite , and a linear classifier pipeline for each predictor. for the attention-based predictor, we remove the glove-gru pipeline and instead reuse the generator's bert embeddings. for the rationale-based predictor, we replace the glove-gru pipeline with another bert-base. both a2r and bert-a2r feed the masked input text directly into the predictor. to add noise injection during training, we first feed the masked input text into the noise injection component. this component is disabled during evaluation. imdb movie review 1: eraser movie review 1: eraser movie review 2: jim carrey shines in this beautiful movie. this is now one of my favorite movies. i read all about the making and i thought it was incredible how they did it. i can't wait till this comes out on dvd. i saw this in theaters so many times, i can't even count how times i've seen it. class: positive absolute power, the new film produced and directed by clint eastwood, attempts to be a thriller set in the world of hypocritical presidents and their murderous political staff. it is about as thrilling as a lecture on the mating habits of the south american grasshopper. one can only wonder how an utterly absurd script like the one written by william goldman could have ever interested eastwood. not only is the plot unbelievable and contrived, but even the writing itself lacks any consistency or intelligence. class: negative figure 3 : examples from the usr movie review dataset. note that compared to eraser reviews, imdb reviews tend to be shorter; eraser reviews vary in length dramatically. furthermore, eraser rationale annotations are often inconsistent: the rationale for review 1 contains only very short spans, whereas the rationale for review 2 spans a few sentences. the reviews from the eraser dataset were collected and processed by @xcite from the imdb archive of the rec.arts.movies.reviews newsgroup, whereas the large movie review dataset was scraped from the imdb website by @xcite . in order to avoid overlap between the train and test sets, we looked for similarity by searching for matches between lowercased, break-tag-free, stop-word-free, lemmatized sentences which spanned at least 5 tokens to avoid generic matches such as "would not recommend" or "great film !". we discovered no overlap between the datasets. we use 40,000 reviews from the large movie review dataset for training and the remaining 10,000 reviews for validation. we then test our model on the 2,000 annotated examples from eraser movie review. ## experimental setup metrics we evaluate generated rationales across several datasets using different metrics that capture faithfulness and plausibility. faithfulness captures the extent to which the generated rationales truly explain the model's output. for faithfulness, we use comprehensiveness and sufficiency metrics @xcite . a rationale is comprehensive if it extracts all the information contained in the input text that is relevant for prediction and sufficient if it contains enough relevant information to make an accurate prediction. the comprehensiveness score measures the difference between the model's predictions on the entire input text and the input text without the selected rationale (higher is better), whereas the sufficiency score measures the difference between the model's predictions on the entire input text and just on the rationale (lower is better). for plausibility, we use standard alignment metrics in reference to the human-annotated rationales: precision, recall, and f1 score as well as iou-f1 score (referred to as iou in tables) with partial match threshold 0.1 @xcite . we use token-level metrics for movie review which offers token-level annotations and sentence-level metrics for multirc and fever which provide only sentence-level annotations. finally, we report prediction accuracy for the overall classification task. all results are averaged across 5 random seeds and reported as the mean with standard deviation in parentheses. implementation our bert-a2r models are trained for a maximum of 20 epochs for eraser movies and 5 epochs for every other dataset, keeping the checkpoint with the lowest validation loss. all bert-a2r variants use uncased bert-base, a2r closeness parameter λ = 0.1, and the selection strategy of picking the top @xmath3. we compute sentence-level scores by taking sentence-level averages of token scores. for optimization, we used adam @xcite with learning rate 2e-5 and batch size 16. noise injection level p was set to 0.2 for usr and eraser movie review, 0.3 for multirc, and 0.05 for fever. this was determined based on our hyperparameter search. all of the models were trained on a single machine equipped with a 12-core processor, 64 gb of ram, and a gpu with 24 gb of vram. to compare against previous published results, we trained a bert-a2r model on the eraser movie review dataset with and without noise injection and compared our numbers to published results from the best unsupervised selective rationalization systems on this benchmark (see we then train bert-a2r models with and without noise injection on the multirc and fever benchmarks (see task plausibility faithfulness dataset model acc. p r f1 iou com ↑ suf ↓ multirc ba2r 66.1 (1.9) 18.5 (1.6) 21.9 (2.2) 19.3 (1.8) n/a -.01 (.01) -.02 (.02) ba2r+ni 66.4 (0.8) 22.6 (1.2) 26.9 (1.8) 23.8 (1.4) n/a -.01 (.01) -.02 (.02) fever ba2r 82.1 (3.2) 36.3 (0.6) 44.0 (0.3) 36.7 (0.5) n/a .02 (.01) -.01 (.02) ba2r+ni 78.2 (1.9) 39.0 (2.5) 47.2 (2.9) 39.5 (2.5) n/a .02 (.00) .00 (.00) movies ba2r 84.0 (2.9) 36.3 (2.8) 36.5 (2.8) 36.4 (2.8) 30.9 (3.9) .02 (.02) -.04 (.02) ba2r+ni 85.7 (2.7) 38.5 (0.6) 38.7 (0.6) 38.6 (0.6) 34.4 (2.2) .05 (.02) -.02 (.01) we train variants of bert-a2r+ni with different levels of p to examine what noise level is optimal for different datasets (see figure 4 ). we average results across 5 seeds but there is still some noise given that the methodology injects noise into the process. it appears that in all cases noise injection seems to degrade performance once p becomes too high as we would expect since too much noise prevents useful signal from getting through. the optimal p varies depending on the task. rationale alignment performance on fever peaks at just @xmath7.05. the optimum for eraser and usr movie review is at @xmath8.1 and @xmath9.2, respectively. the best performance on multirc was achieved at @xmath10.3. there are numerous factors that might interact with noise injection to cause this behavior: task-specific demands, sentence vs. token-level rationale annotations, and the suitabil-ity of other training parameters. these interactions might be complex, especially with training strategies that dynamically adjust p during training. we leave exploration of these factors for future work.