Lifelong Explainer for Lifelong Learners (2021)

Abstract: Lifelong Learning (LL) black-box models are dynamic in that they keep learning from new tasks and constantly update their parameters. Owing to the need to utilize information from previously seen tasks, and capture commonalities in potentially diverse data, it is hard for automatic explanation methods to explain the outcomes of these models. In addition, existing explanation methods, e.g., LIME, which are computationally expensive when explaining a static black-box model, are even more inefficient in the LL setting. In this paper, we propose a novel Lifelong Explanation (LLE) approach that continuously trains a student explainer under the supervision of a teacher – an arbitrary explanation algorithm – on different tasks undertaken in LL. We also leverage the Experience Replay (ER) mechanism to prevent catastrophic forgetting in the student explainer. Our experiments comparing LLE to three baselines on text classification tasks show that LLE can enhance the stability of the explanations for all seen tasks and maintain the same level of faithfulness to the black-box model as the teacher, while being up to 10ˆ2 times faster at test time. Our ablation study shows that the ER mechanism in our LLE approach enhances the learning capabilities of the student explainer. Our code is available at https://github.com/situsnow/LLE .

## introduction explaining a model's predictions to practitioners and end users, especially in the case of a blackboxmodel, is non-trivial. recent research on explainable artificial intelligence usually considers feature attribution as a local explanation, i.e., how much each feature contributes to the outcome of the model. related works include backpropagationbased methods, where the influence of model outcome is backpropagated according to gradients or layer-wise rules @xcite @xcite ; perturbation-based methods, which observe changes in model performance after feature perturbation @xcite , or approximate the local decision boundary through perturbed samples @xcite ; and model-based methods, which train an explainer model by optimizing an explanation-meritorious objective, 1 such as robustness/stability @xcite that requires similar examples to have similar explanations. all these methods aim to explain static black-box models, whereas explaining dynamic ones, as in the lifelong learning (ll) @xcite setting, is under-explored. we propose a lifelong explanation (lle) approach that learns to explain the outcome of a ll black-box under the supervision of a teacher explanation algorithm. the key challenge in ll is to prevent catastrophic forgetting @xcite ) of knowledge learnt from preceding tasks while learning from a new task. to prevent this, an experience replay (er) mechanism @xcite is exploited to replay a small amount of past data in order to maintain performance on all seen tasks. however, the dynamicallychanging black-box model may make the er of previously generated explanations sub-optimal. we investigate an er mechanism that replays previously seen examples together with explanations from the teacher produced in the current step. specifically, we incorporate the er mechanism into the training of the student explainer, which focuses on the faithfulness of the generated explanations, i.e., how well an explanation aligns with the ll black-box model outcome. our empirical results show that the lle explainer (i) enhances the stability of explanations, (ii) is as faithful to the black-box model as the teacher, and (iii) is faster than the teacher at test 1 an objective that maximizes the quality of an explanation. time. our ablation study on er shows that regenerating the teacher's explanations for past examples significantly improves the faithfulness and stability of the student explanations. ## problem definition in this paper, we consider a lifelong learning (ll) setting comprising a sequence of text classification tasks {t 1 , t 2 , ..., t t }. each task t t has its own train/validation/test sets (d t tr , d t va , d t ts ), each of which contains a set of paired examples {(x x x t i , y t i )} nt @xmath0, where x x x t is the input (e.g., a document), y @xmath1.g., a topic label), y t denotes the label set in task t t , and n t is the total number of examples in the set. the goal is to train a classifier f θ θ θ which continuously learns and accumulates knowledge from the data in each task t t . specifically, at an arbitrary step t, f θ θ θ optimizes a loss function: nt @xmath2, y t i ), on the training data d t tr . in addition, we require f θ θ θ to remember the preceding knowledge at each step t, so as to maintain its performance on all previous tasks, i.e., t 1 , t 2 , ..., t t-1 . in order to achieve this goal, the classifier f θ θ θ is usually allowed to access a memory that stores a limited number of samples from the previous tasks. the performance measure of f θ θ θt at each step t is: 1 t t @xmath3,j , where acc f,j denotes the accuracy of f θ θ θt on task t j . ## lifelong explanation (lle) we now present the training and testing phase of our lle algorithm (figure 1 ) to explain a dynamically-changing black-box classifier. at time step t in the training phase, we are given a task t t , its training set d t tr and the black-box model f θ θ θt (figure 1a ). we first collect the explanations r r r t i for each input x x x t i in d t tr from a teacher algorithm a. here, r r r t i contains the features (words) in the input x x x t i that are important for the prediction made by f θ θ θt (x x x t i ). we then train our lle explainer g φ φ φt with the set of explanations {r r r t i } nt @xmath4. training the lle differs from training the generic ll classifier. firstly, unlike ll, which predefines task boundaries to determine the memory saving strategy, lle can simply reuse the same set of memorized examples in ll, and thus is insensitive to this setting; we use sparse experience replay @xcite , which replays examples from the memory randomly. secondly, the generic ll algorithm saves the fixed ground-truth label y in the memory. however, in lle, for an input in the memory m, the ground-truth explanation at time step t -1 may differ from the one algorithm 1 lifelong explanation (lle) 1: f θ θ θ : underlying ll black-box classifier 2: g φ φ φ : explainer model 3: a: teacher explanation method 4: k: numbers of randomly selected examples 5: m: training memory 6: procedure explainermodel(f θ θ θ ) 7: m ← m ∪ randomsubset(d t tr , kt) 13: explain f θ θ θ t (d 1 ts ), ..., f θ θ θ t (d t ts ) using g φ φ φ t 14: in the testing phase (figure 1b ), we no longer require the teacher algorithm a as the lle explainer g φ φ φ has already learnt how to produce explanations for unseen examples at each time step t. we randomly select ten tasks from the amazon customer review dataset 2 and fine-tune a pretrained distilled bert @xcite on these tasks, achieving a 97% test accuracy. details of the dataset, training of f θ θ θ and accuracies appear in appendices b.1 and b.2. 2 we use the datasets provided by huggingface datasets api https://huggingface.co/datasets/amazon_us_reviews . ## conclusion and future work we have proposed a lifelong explanation (lle) method that learns from a teacher and leverages an er mechanism to explain a constantly-changing black-box. our experimental results show that lle can improve the stability of a teacher's explanation, and maintain a comparable level of faithfulness, while performing up to two order of magnitudes faster. our ablation study has shown the effectiveness of er using most recently generated explanations. the performance of lle in ll settings consisting of problems other than classification, e.g., relation extraction, is still under-explored. the evaluation of lle based on other merits of explanations, such as simulatability @xcite , can also be an interesting research direction.