On the Impact of Temporal Concept Drift on Model Explanations (2022)

Abstract: Explanation faithfulness of model predictions in natural language processing is typically evaluated on held-out data from the same temporal distribution as the training data (i.e. synchronous settings). While model performance often deteriorates due to temporal variation (i.e. temporal concept drift), it is currently unknown how explanation faithfulness is impacted when the time span of the target data is different from the data used to train the model (i.e. asynchronous settings). For this purpose, we examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks. Our experiments show that (i) faithfulness is not consistent under temporal variations across feature attribution methods (e.g. it decreases or increases depending on the method), with an attention-based method demonstrating the most robust faithfulness scores across datasets; and (ii) select-then-predict models are mostly robust in asynchronous settings with only small degradation in predictive performance. Finally, feature attribution methods show conflicting behavior when used in FRESH (i.e. a select-and-predict model) and for measuring sufficiency/comprehensiveness (i.e. as post-hoc methods), suggesting that we need more robust metrics to evaluate post-hoc explanation faithfulness. Code will be made publicly available.

## introduction one way of improving the transparency of deep learning models in natural language processing (nlp) is by extracting explanations that justify model predictions @xcite ). an explanation (i.e. rationale) consists of a subset of the input and is considered faithful when it accurately shows the reasoning behind a model's prediction @xcite @xcite . for example, removing a faithful rationale from the input should result into a prediction change. two widely used methods for extracting rationales are (i) feature attribution methods that produce a distribution over the input tokens, indicating their contribution (i.e. importance) to the model's prediction @xcite ; and (ii) select-then-predict models that consist of two components, a rationale extractor and a predictor. the rationale extractor extracts rationales, and the predictor is trained on extracted rationales so that its predictions are inherently faithful @xcite . previous work has focused on evaluating explanation faithfulness in synchronous settings where the training and testing data come from the same temporal distribution @xcite @xcite , or out-of-domain settings @xcite where the training and testing data come from a different domain regardless of temporal drifts in the testing data. however, human languages evolve @xcite @xcite as manifested by novel usages developed for existing words (e.g. mouse is a mammal or a computer accessory) and new words and topics (e.g. covidiot during the covid-19 pandemic) that appear over time. language evolution leads to temporal concept drifts and a diachronic degradation of model performance in many nlp tasks when these are evaluated in asynchronous settings, i.e. training and testing data come from different time periods @xcite @xcite @xcite . in this paper, for the first time, we extensively analyze the impact of temporal concept drift on model explanations. we evaluate the faithfulness of rationales extracted using eight feature attribution approaches and three select-then-predict models over six text classification tasks with chronological data splits. our contributions are as follows: ## extracting explanations we extract explanations using two standard approaches: (i) post-hoc methods; and (ii) selectthen-predict models. ## qualitative analysis we also conduct a qualitative analysis on the rationales extracted by α∇α, to find possible reasons that justify its stability and robustness when moving from synchronous to asynchronous settings (i.e. invariance to concept drift), when using fresh. figure 4 shows wordclouds (larger words appear more frequently in the rationales) for factcheck (a, b) and yelp (c, d), on the synchronous split syn and the asynchronous split asy2. starting with factcheck, we observe that salient words change when moving to asynchronous settings. for example, in syn, the extracted rationales contain words like "web", "site" and "published". in contrast, rationales from asy2 contain words like "video", "social" and "post". these indicate a shift in how misinformation is spread across time (i.e. different types of media), which surprisingly is picked up by the rationales when moving to asynchronous settings. similarly in yelp, whilst the majority of most frequent words remains similar across chronological splits (e.g. "delicious", "nice", "amazing"), we still observe some concept drift that is again picked up by the rationales. for example, it appears that more recent restaurant reviews are concerned about the experience and appearance of the restaurant, as picked up in asy2. this is highlighted by the fact that asy2 contains frequently the words "experience" and "pretty" (to describe the place and food, we found several examples through a manual analysis that refer to these two concepts). on the other hand, we note that syn does not contain words relevant to the experience and appearance of the place. ## to trust sufficiency or not? the contradictory patterns observed between posthoc explanations and fresh (see section 5), questions the efficacy of using sufficiency to measure faithfulness in asynchronous settings. inspired by the explanation-game @xcite , we use the classifier from fresh as a layperson and measure its ability to generate the same predictions as the full-text trained model. our hypothesis is that if a feature attribution produces highly suffi-cient rationales, the layperson should also have a high predictive performance (when using the fulltext model's predictions as gold labels) and vice versa. if sufficiency is reliable as a metric, we therefore expect that the most sufficient rationales to be obtained using α∇α, followed by x∇x and the least sufficient to be dl. figure 5 shows the performance of the layperson (i.e. fresh), in predicting the original predictions of the full-text trained model (i.e. a higher score denotes a higher agreement). we first observe that α∇α outperforms in most datasets both dl and x∇x. for example, in factcheck, α∇α shows almost perfect agreement (approximately 100%) with the full-text trained model in syn and both asy splits, highlighting the efficacy of this feature attribution in extracting faithful explanations. contrary to our expectations and similar to observations with fresh, dl consistently outperforms x∇x, even outperforming α∇α in certain cases. for example in yelp across both asynchronous settings, using dl the layperson is able to reach the same predictions as the full-text model, in approximately 80% of the instances. in comparison, using x∇x, the layperson reaches the same predictions in only 55% of the instances. our findings suggest that sufficiency, as a metric for measuring faithfulness, cannot be trusted in asynchronous settings and also raises concerns for synchronous settings. ## conclusion we conducted an extensive empirical study to shed light on the impact of temporal drift on model explanations in asynchronous settings, including post-hoc methods and select-then-predict models. we demonstrate that faithfulness is not consistent under temporal variations across feature attribution methods, while select-then-predict models are mostly robust with negligible drops in predictive performance. in the future, we plan to extend our study into more tasks and data from different languages. we also plan to explore whether instance specific feature attribution improves faithfulness in asynchronous settings @xcite .