Utilizing Multimodal Feature Consistency to Detect Adversarial Examples on Clinical Summaries (2020)

Abstract: Recent studies have shown that adversarial examples can be generated by applying small perturbations to the inputs such that the well- trained deep learning models will misclassify. With the increasing number of safety and security-sensitive applications of deep learn- ing models, the robustness of deep learning models has become a crucial topic. The robustness of deep learning models for health- care applications is especially critical because the unique characteristics and the high financial interests of the medical domain make it more sensitive to adversarial attacks. Among the modalities of medical data, the clinical summaries have higher risks to be attacked because they are generated by third-party companies. As few works studied adversarial threats on clinical summaries, in this work we first apply adversarial attack to clinical summaries of electronic health records (EHR) to show the text-based deep learning systems are vulnerable to adversarial examples. Secondly, benefiting from the multi-modality of the EHR dataset, we propose a novel defense method, MATCH (Multimodal feATure Consistency cHeck), which leverages the consistency between multiple modalities in the data to defend against adversarial examples on a single modality. Our experiments demonstrate the effectiveness of MATCH on a hospital readmission prediction task comparing with baseline methods.

## introduction deep learning has been shown to be effective in a variety of real-world applications such as computer vision, natural language processing, and speech recognition @xcite @xcite . it also has shown great potentials in clinical informatics such as medical diagnosis and regulatory decisions @xcite , including learning representations of patient records, supporting disease phenotyping, and conducting predictions @xcite . however, recent studies show that these models are vulnerable to adversarial examples @xcite . in image classification, researchers have demonstrated that imperceptible changes in input can mislead the classifier @xcite . in the text domain, synonym substitution or character/word level modification on a few words can also cause the model to misclassify @xcite . these perturbations are mostly imperceptible to human but can easily fool a high-performance deep learning model. adversarial examples have received much attention in image and text-domain, yet very few work has been done on electronic health records (ehr). most existing works on adversarial examples in medical domains have been focused on medical images @xcite . a few works have studied adversarial examples in numerical ehr data @xcite @xcite . despite these attempts, there is no work on evaluating the adversarial robustness of clinical natural language processing (nlp) systems, as well as the potential defense techniques. although there are some existing defense techniques in the text domain, these methods cannot be directly applied to clinical texts due to the special characteristics of clinical notes. on one hand, for ordinary texts, spelling or syntax checks can easily detect adversarial examples generated by introducing misspelled words. however, there are originally plenty of misspelling words or abbreviations in clinical notes, which places challenges to distinguish whether a misspelled word is under attack. one the other hand, data augmentation is another strategy of some adversarial defense techniques in text domain. for example, synonyms encoding method (sem) @xcite is a data preprocessing method that inserts a synonym encoder before the input layers to eliminate adversarial perturbations. however, for clinical notes, a large number of words are proper nouns which makes it difficult to generate synonym set thus challenging to apply such defense. adversarial training @xcite has also been applied to increase the generalization ability of textual deep learning models. however, no research has studied the effectiveness of applying adversarial training in the training of text-based clinical deep learning systems. we note that most existing defense mechanisms have focused on a single modality of the data. however, ehr data always comes in multiple modalities including diagnoses, medications, physician summaries and medical image, which presents both challenges and opportunities for building more robust defense systems. this is because some modalities are particularly susceptible to adversarial attacks and still lack effective defense mechanisms. for example, the clinical summary is often generated by a third-party dictation system and has a higher risk to be attacked. we believe that the correlations between different modalities for the same entity can be exploited to defend against such attacks, as it is not realistic for an adversary to attack all modalities. in this work, we propose a novel defense method, multimodal feature consistency check (match), against adversarial attacks by utilizing the multimodal properties in the data. we assume that one modality has been compromised, and the match system detects whether an input is adversarial by measuring the consistency between the compromised modality and another uncompromised modality. to validate our idea, we conduct a case study on predicting the 30-day readmission risk using an ehr dataset. we craft adversarial examples on clinical summary and use the sequential numerical records as another un-attacked modality to detect the adversarial examples. figure 1 depicts the highlevel flow of our system. the main contributions of this paper include: the results validate that they outperform existing state-of-the-art defense methods in the medical domain. ## related work there have been many adversarial works on single modality adversarial tasks. qiu et al. (2019) provided a comprehensive summary of the latest progress on adversarial attack and defense technology, categorized by applications including computer vision, natural language processing, cyberspace security, and physical world. esmaeilpour et al. (2019) reviewed the existing adversarial attacks in audio classification. since our case study focuses on attack and defense of text modality, we mainly review the text-based attacks and defenses in this section. 2.1 attack methods for text data kuleshov et al. (2018) proposed a greedy search algorithm (gsa), which iteratively changes one word in a sentence and substitute the word with one of the synonymous that improves the objective function the most. alzantot et al. (2018) introduced a genetic algorithm (ga) which is a populationbased synonym replacement algorithm including processing, sampling and crossover. gong et al. (2018) proposed to search for adversarial examples in the embedding space by applying gradientbased methods on text embedding (text-fgm) and then reconstructed the adversarial texts by the nearest neighbor search. gao et al. (2018) presented the deepwordbug algorithm to generate small perturbations in the character-level. this algorithm does not require the gradient. ren et al. (2019) proposed a new synonym substitution method, probability weighted word saliency (pwws), which considered the word saliency as well as the classification probability. jin et al. (2019) proposed textfooler, an adversarial approach by identifying the important words and then prioritize to replace them with the most semantically similar and grammatically correct words. this is the first attempt to attack the emerging bert model on text classification. we compare these algorithms from the following aspects: document level vs. word level. text-fgm and ga are document level attacks, which apply an attack on the whole text. deepwordbug, gsa, psww, and textfooler are word level attacks that perturb individual words. deepwordbug, psww and textfooler use heuristics to measure the importance of each word and select words to perturb. continuous vs. discrete. text-fgm is a continuous attack, because the gradient-based perturbation is applied on the embedding of the words. all other attacks are discrete attacks, which are applied directly on words. semantic vs. syntactic. gsa, psww, text-fgm and textfooler can be categorized as a semantic attack since their strategies are to replace words or text with synonyms, while deepwordbug is a syntactic attack because it is based on character-level modification. ga can generate both semantically and syntactically similar adversarial examples. back-box vs. white-box. gsa, ga and text-fgm are white-box attacks, because attackers need to access the model structure and model pa-rameters to calculate the gradient. deepwordbug, textfooler and psww are black-box attacks. in this paper, we evaluate our detection method against text-fgm and deepwordbug, which represent all the categories mentioned above. text-fgm. in text-fgm, any gradient based attacks, such as deepfool @xcite , fast gradient method (fgm) @xcite (both fgsm and fgvm) can be applied. applying fgvm on text is defined as follows. given a classifier f and a word sequence in the following work, in order to minimize the number of words that need to be perturbed, we iteratively perform perturbation on one word at a time based on the importance score of the words, instead of applying perturbation on the entire sequence. in this way, we can maximize the overall semantic similarity between clean and adversarial sentences. deepwordbug. deepwordbug first computes the word importance to the target sequence classifier. at each step, it selects the most important word and constructs an adversarial word applying a character level swap, substitution or deletion. it iterates until the label is flipped or the number of words changed is larger than a threshold. ## method in this section, we will explain our high-level idea and intuitions behind match. system overview. the main idea of match is to reject adversarial examples if the features from one modality are far away from another un-attacked modality's features. in match, we assume that there is duplicate information in multiple modalities (e.g., 'gray cat' in an image caption and a gray cat in image) and manipulating information can be harder in one modality than another modality. thus, it is difficult for an attacker to make coherent perturbations across all modalities. in other words, using the gradient to find the steepest change in the decision surface is a common attack strategy, but such a gradient can be drastically different from modality to modality. moreover, for a certain modality, even if the adversarial and clean examples are close in the input space, their differences would be amplified in the feature space. therefore, if another un-attacked modality is introduced, the difference between the two modalities can be a criteria to distinguish adversarial and clean examples. figure 2 shows our detection pipeline using text and numerical features. note that, while we use text and numerical modalities for the experiments, our framework works for any modalities. ## conclusion in this work, we proposed match, a novel defense method by taking advantage of another modal's properties to detect adversarial examples on clinical notes. we evaluated our approaches with two different attack strategies: text-fgm and deep-wordbug. we conducted experiments on the 30day readmission prediction task by detecting adversarial examples in text modalities and use numerical modality to do the multi-modal consistency check. our experiments showed the effectiveness of match compared to the baseline methods. although we only evaluated match on clinical deep learning system and only attack on the clinial text modality, we believe match would be a general framework that could work on any multi-modality dataset. in the future, it would be interesting to extending and evaluating the frame-