Are Shortest Rationales the Best Explanations for Human Understanding? (2022)

Abstract: Existing self-explaining models typically favor extracting the shortest possible rationales — snippets of an input text “responsible for” corresponding output — to explain the model prediction, with the assumption that shorter rationales are more intuitive to humans. However, this assumption has yet to be validated. Is the shortest rationale indeed the most human-understandable? To answer this question, we design a self-explaining model, LimitedInk, which allows users to extract rationales at any target length. Compared to existing baselines, LimitedInk achieves compatible end-task performance and human-annotated rationale agreement, making it a suitable representation of the recent class of self-explaining models. We use LimitedInk to conduct a user study on the impact of rationale length, where we ask human judges to predict the sentiment label of documents based only on LimitedInk-generated rationales with different lengths. We show rationales that are too short do not help humans predict labels better than randomly masked text, suggesting the need for more careful design of the best human rationales.

## introduction while neural networks have recently led to large improvements in nlp, most of the models make predictions in a black-box manner, making them indecipherable and untrustworthy to human users. in an attempt to faithfully explain model decisions to humans, various work has looked into extracting rationales from text inputs @xcite , with rationale defined as the "shortest yet sufficient subset of input to predict the same label" @xcite . the underlying assumption is two-fold: (1) by retaining the label, we are extracting the texts used by predictors @xcite ; and (2) short rationales are more readable and intuitive for endusers, and thus preferred for human understanding @xcite . importantly, prior work has knowingly traded off some amount of model performance to achieve the shortest possible rationales. for example, when using less than 50% of text as rationales for predictions, @xcite achieved an accuracy of 84.0% (compared to 91.0% if using the full text). however, the assumption that the shortest rationales have better human interpretability has not been validated by human studies @xcite . moreover, when the rationale is too short, the model has much higher chance of missing the main point in the full text. in figure 1a , although the model can make the correct positive prediction when using only 20% of the text, it relies on a particular adjective, "lifeaffirming," which is seemingly positive but does not reflect the author's sentiment. these rationales may be confusing when presented to end-users. in this work, we ask: are shortest rationales really the best for human understanding? to answer the question, we first design limitedink, a selfexplaining model that flexibly extracts rationales at any target length (figure 1a ). limitedink allows us to control and compare rationales of varying lengths on input documents. besides controls on rationale length, we also design limitedink's sampling process and objective function to be contextaware (i.e., rank words based on surrounding context rather than individually, figure 1b 2 ) and coherent (i.e., prioritize continuous phrases over discrete tokens, figure 1c 2 ). compared to existing baselines (e.g., sparse-ib ), limitedink achieves compatible end-task performance and alignment with human annotations on the eraser (deyoung et al., 2020) benchmark, which means it can represent recent class of self-explaining models. we use limitedink to conduct user studies to investigate the effect of rationale length on human understanding. specifically, we ask mturk participants to predict document sentiment polarities based on only limitedink-extracted rationales. by contrasting rationales at five different length levels, we find that shortest rationales are largely not the best for human understanding. in fact, humans do not perform better prediction accuracy and confidence better than using randomly masked texts when rationales are too short (e.g., 10% of input texts). in summary, this work encourages a rethinking of self-explaining methods to find the right balance between brevity and sufficiency. ## model performance evaluation we first validate limitedink on two common rationale evaluation metrics, including end-task performance and human annotation agreement. ## human evaluation equipped with limitedink, we next carry out human studies to investigate the effect of rationale length on human understanding. ## discussion by examining human prediction performance on five levels of rationale lengths, we demonstrate that the shortest rationales are largely not the best for human understanding. we are aware that this work has limitations. the findings are limited to movie reviews dataset, and we only evaluate human performance with rationales generated by the proposed limitedink. still, our findings challenge the "shorter is better" assumption commonly adopted in existing self-explaining methods. as a result, we encourage future work to more cautiously define the best rationales for human understanding, and trade off between model accuracy and rationale length. more concretely, we consider that rationale models should find the right balance between brevity and sufficiency. one promising direction could be to clearly define the optimal human interpretability in a measurable way and then learn to adaptively select rationales with appropriate length. ## conclusion to investigate if the shortest rationales are best understandable for humans, this work presents a selfexplaining model, limitedink, that achieves comparable performance with current self-explaining baselines in terms of end-task performance and human annotation agreement. we further use lim-itedink to generate rationales for human studies to examine how rationale length can affect human understanding. our results show that the shortest rationales are largely not the best for human understanding. this would encourage a rethinking of rationale methods to find the right balance between brevity and sufficiency.