DARE : Towards Robust Text Explanations in Biomedical and Healthcare Applications (2023)

Abstract: Along with the successful deployment of deep neural networks in several application domains, the need to unravel the black-box nature of these networks has seen a significant increase recently. Several methods have been introduced to provide insight into the inference process of deep neural networks. However, most of these explainability methods have been shown to be brittle in the face of adversarial perturbations of their inputs in the image and generic textual domain. In this work we show that this phenomenon extends to specific and important high stakes domains like biomedical datasets. In particular, we observe that the robustness of explanations should be characterized in terms of the accuracy of the explanation in linking a model’s inputs and its decisions - faithfulness - and its relevance from the perspective of domain experts - plausibility. This is crucial to prevent explanations that are inaccurate but still look convincing in the context of the domain at hand. To this end, we show how to adapt current attribution robustness estimation methods to a given domain, so as to take into account domain-specific plausibility. This results in our DomainAdaptiveAREstimator (DARE) attribution robustness estimator, allowing us to properly characterize the domain-specific robustness of faithful explanations. Next, we provide two methods, adversarial training and FAR training, to mitigate the brittleness characterized by DARE, allowing us to train networks that display robust attributions. Finally, we empirically validate our methods with extensive experiments on three established biomedical benchmarks.

## introduction research in explainable ai (xai) has seen a surge in recent years. xai methods aim to provide insight into the inference process and the causal links between inputs and outputs of deep neural networks (dnns). this is pivotal in addressing many aspects of dnns, such as fairness, potential biases and scopes of safe deployment. especially in safetycritical domains, such as healthcare, faithful and robust explanations (jacovi and goldberg, 2020) accompanying the predictions of dnns are key to enable their deployment and understand potential false predictions and risks. beside faithfulness, which quantifies the accuracy by which the explanations characterize the true decision-making process of the model, a second property of explanations that has been highlighted as important is plausibility @xcite @xcite . plausibility quantifies the "degree to which some explanation is aligned with the user's understanding" (jacovi and goldberg, 2021). as such, plausibility tells us whether an explanation is found convincing and informative for domain experts. these independent but related properties are therefore crucial ingredients to provide explanations that are accurate and robust, as well as domain-relevant and convincing @xcite . attribution methods such as saliency maps @xcite , @xcite or integrated gradients @xcite highlight the input features that are deemed important in the decision process as heat maps. these are especially useful, as such maps are easy to interpret and no specific domain knowledge is needed to provide them. moreover, methods like integrated gradients fulfill several desiderata of faithful explanations (jacovi and goldberg, 2020), which makes them an even more attractive option to explain dnns. however, recent work has shown that attribution maps do not necessarily fulfill the robustness aspect of faithful explanations. in particular, it has been shown in the vision domain that small input alterations can be crafted so as to change the attribution maps drastically, while leaving the prediction unchanged @xcite . very recently, the same phenomenon has been confirmed in the textual domain as well @xcite . but what about the plausibility of the explanations? this paper starts by first pointing out that the importance of plausibility has been overlooked in favor of exclusively focusing on faithfulness, in particular in the textual domain. this is important because, when trying to protect a system from adversarial attacks against explanations, it is not only crucial to quantify their faithfulness, but also the plausibility of the possible adversarial samples. in fact, unfaithful but plausible explanation attacksconvincing lies -have been pointed out to be particularly pernicious, since they are more difficult for domain experts to spot than equally unfaithful and implausible perturbations and explanationsunconvincing lies - @xcite . these observations are crucial for the use of ai explainability in high stakes scenarios, as in automated medical diagnosis, ehr classification or triage @xcite , a medical professional might overlook some critical areas in a cancer cell image or disregard certain important words because they do not appear relevant according to an otherwise plausible explanation. in this work, we focus on the robustness aspect of faithful and plausible attributions in biomedical text classification problems. specifically, we investigate how to extract in-context adversarial perturbations which are plausible in each specific task domain under consideration. then, we propose our attribution robustness (ar) estimator that quantifies ar in a domain-specific way. finally, we explore methods to mitigate the domain-specific fragility of explanation methods in order to train text classifiers that can safely be deployed in safetycritical use cases like healthcare. we summarize our contributions as follows: ## related work recently, attribution methods like saliency maps @xcite , integrated gradients @xcite , @xcite or shapley values (lundberg and lee, 2017) have been widely deployed in the medical domain for deep learning @xcite . these methods aim to provide insight into the inference process in dnns. they highlight features in the input that are deemed relevant in the decision process, without requiring any domain-specific knowledge or heavy computation resources. thus, they have been widely adopted in areas where predictions need to be accompanied by explanations, such as analysis of medical images @xcite or important symptoms that contribute to or against a given diagnosis @xcite . the authors tjoa and guan (2020) provide an extensive survey on such methods applied in several medical problem setups. the work of @xcite explore the robustness of such attribution methods and find that they are susceptible to adversarial perturbations, both in the image domain @xcite , and the text domain @xcite @xcite . however, these works operate on general, non-domainspecific text. none investigate domain-specific text, such as healthcare, where most datasets possess unique vocabularies and semantics. we aim to provide insight into how current methods can be adapted to such specific technical domains. in order to mitigate the highlighted fragility of attributions in dnns, several methods have been developed. the authors @xcite ; @xcite ; @xcite propose ## background and motivation in this chapter, we introduce the background and motivation of our ar estimation. we define a text classifier f as a function that maps a text sample s to a label l from a given set of labels l. in discrete input spaces like text, f is a function composition of a non-differentiable embedding e that maps the discrete inputs into a continuous domain r h×p , and a differentiable classifier f mapping the embeddings to the output logits r |l| . we denote @xmath0, l) |@xmath1...|s|-1} , w @xmath2, @xmath3, |s| = n } as the set of n text samples s with a label @xmath4, each containing a sequence of words w i drawn from the vocabulary w, h the embedding dimension and p the maximum sequence length. attributions are functions @xmath5, f, l) that assign a value to each word w i in a text sample s, indicating its importance in the dnn inference process. we sum up the attribution values of each w i -s em-bedding, resulting in a single value for each word. attribution robustness is defined as the lipschitz attribution robustness constant @xcite , given in the following equation: here, r(s) denotes the robustness of attribution method a computed for text sample s with label l, drawn from l, and classifier f . the function d[a(s, f, l), a(s, f, l)] denotes the distance between original and adversarial attribution maps a(s, f, l) and a(s, f, l), n (s) is a predefined neighborhood of text sample s. the term d s (s, s) indicates the distance of adversarial and original input texts. the robustness of an attribution method on a test dataset s then becomes the average of r(s) over the dataset. note that the robustness of an attribution method on a classifier is inversely proportional to the constant computed in equation (1), as large attribution distances and small input distances result in large constants, indicating low robustness. this reflects the definition of the robustness property of faithful explanations (jacovi and goldberg, 2020). our first contribution is conceptual and is motivated by the observation that plausibility is a criterion that is rooted in the specific domain and the semantic conventions within it. thus, methods to guard against adversarial attacks on explanations need to be domain adaptive to conform to the threat model that prioritizes unfaithful explanations which are semantically plausible in the domain under consideration, thus are particularly misleading and potentially dangerous. the strategy we propose to control domainadaptive plausibility is based on the observation that while the numerator in equation (1) characterizes faithfulness by quantifying the effect of adversarial attacks on attributions, the denominator can be adapted to capture plausibility by promoting adversarial attacks that remain close to the original input in a semantically meaningful way, in the domain under consideration. in particular, while @xcite utilize the cosine distance of sentence embeddings obtained from domainagnostic encoders like universal sentence encoder @xcite and @xcite , we can obtain a domain-specific measure of distance by using embeddings trained on the domain of interest. this will control the plausibility of the adversarial samples by making sure that their domain-dependent semantic distance remains close to the original inputs. ## medical attribution robustness current ar estimation algorithms @xcite were designed to operate in the general text domain, such as news articles @xcite , movie reviews @xcite or product reviews @xcite and make use of the generously available labeled data in these domains. this section describes our proposed methods to adapt these algorithms to the biomedical and healthcare domains where data is sparse and the vocabularies are domain-specific. we describe our datasets and models, we observe that current estimators can be made domain-adaptive by abstracting the candidate extractor and finally, inspired by the works of @xcite , we introduce our estimation algorithm domainadaptivearestima-tor (dare), which can effectively be used to estimate ar in the domain of biomedical text. ## ar in multilabel datasets many text classification datasets in healthcare do not only have one label per sample. in hoc, multiple hallmarks can be associated with an abstract, and mimic contains hardly any discharge summary with only one associated icd-9 code. however, current ar estimation definitions only focus on the single label case. therefore, we make the following modifications to make ar work in the multilabel case. first, we modify the prediction constraint from equation (1) to reflect multilabel predictions. the label l becomes a set of predicted labels, and the prediction constraint in equation ( 2 ) holds as long as the predicted set of labels from the original sample is equal to the one from the adversarial sample. we denote this constraint as p in our estimation algorithm. second, attribution methods compute maps on a per-class basis, where the overall attribution @xmath6, f, l) equals the attribution of the single predicted class l. in a multilabel case, we extend this notion to the sum of attributions for each predicted class, thus the overall attribution map becomes @xmath7, f, l i ). ## experiments in this section, we report our experiments and setup to estimate attribution robustness in the biomedical domain. we compare the robustness of four attribution methods on three text classifiers trained naively and with robust optimization objectives (adversarial training and far). our results show that the naively trained models are heavily sensitive to imperceptible word substitution attacks, while the two robust training methods significantly increase attribution robustness, with far outperforming adversarial training. ## conclusion in this work, we explored the attribution robustness of biomedical text classification. we extended current robustness estimators to introduce dare, a domain-adaptive ar estimator. then, we showed on three different biomedical datasets that classifiers trained without robust objectives lack robustness to small input perturbations in this domain as well. in order to mitigate this, we proposed two training methods, adversarial training and far to train neural networks that yield robust attributions even in the presence of carefully crafted input perturbations. with our experiments, we show that adversarial training and far are able to increase the attribution robustness significantly, with far giving the best results. our work is a key milestone for the deployment of dnns in the biomedical domain, as such a safety-critical application area requires sound and faithful explanations. in the future, we plan to extend our investigation from text classification to other nlp problems in the biomedical domain. moreover, investigating the robustness of other types explanation methods is an important future research direction.