Stabilized Medical Image Attacks

1 INTRODUCTION . Computer Aided Diagnosis ( CADx ) has been widely applied in the medical screening process . The automatic diagnosis benefits doctors to efficiently obtain health status to avoid disease exacerbation . Recently , Convolutional Neural Networks ( CNNs ) have been utilized in CADx to improve the diagnosis accuracy . The discriminative representations improve the performance of medical image analysis including lesion localization , segmentation and disease classification . However , recent advances in adversarial examples have revealed that the deployed CADx systems are usually fragile to adversarial attacks ( Finlayson et al. , 2019 ) , e.g. , small perturbations applied to the input images can deceive CNNs to have opposite conclusions . As mentioned in Ma et al . ( 2020 ) , the vast amount of money in the healthcare economy may attract attackers to commit insurance fraud or false claims of medical reimbursement by manipulating medical reports . Moreover , image noise is a common issue during the data collection process and sometimes these noise perturbations could implicitly form adversarial attacks . For example , particle contamination of optical lens in dermoscopy and endoscopy and metal/respiratory artifacts of CT scans frequently deteriorate the quality of collected images . Therefore , there is a growing interest to investigate how medical diagnosis systems respond to adversarial attacks and what we can do to improve the robustness of the deployed systems . While recent studies of adversarial attacks mainly focus on natural images , the research of adversarial attacks in the medical image domain is desired as there are significant differences between ∗L.Gong and Y . Song are corresponding authors . The code is available at https : //github.com/ imogenqi/SMA two domains . Beyond regular RGB cameras , there are various types of medical imaging equipments ( e.g. , Computed Tomography ( CT ) scanners , ultrasound transducers and fundus cameras ) to generate dramatically different images . Fig . 1 shows three examples where an image captured from fundus camera is in ( a ) , an image captured from the CT scanner is in ( e ) and an endoscopic video frame is in ( i ) . As can be seen in the figure that these three images have little in common . The huge data variance across different modalities of medical images brings more challenges to develop a technology that works for all the modalities . In addition , existing investigations on medical adversarial attacks are limited . In Finlayson et al . ( 2019 ) , adversarial examples are shown to deteriorate the diagnosis accuracy of deep learning based medical systems . These medical attack methods are mainly based on those from natural images ( e.g. , Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2014 ) and Project Gradient Descent ( PGD ) ( Madry et al. , 2017 ) , which are insufficiently developed for different types of medical data . As shown in Fig . 1 , the adversarial examples generated by FGSM and PGD do not consistently decrease the network ’ s performance in ( b ) , ( c ) , ( f ) , ( g ) , ( j ) and ( k ) . The data variance in ( a ) and ( e ) leads to the inconsistent attack results by existing methods . In this paper , we propose a medical image attack method to consistently produce adversarial perturbations that can fool deep medical diagnosis systems working with different medical data modalities . The perturbations are iteratively generated via taking partial derivatives of a well-defined objective function that is composed of a deviation loss term and a stabilized loss term with respect to the input . By maximizing the deviation loss term , the adversarial attack system enlarges the divergence between CNN predictions and the ground truth to have effective attack samples . To handle the aforementioned ubiquitous data noise issue in medical images , we propose a novel stabilization loss term as an extra regularization , which ensures a consistent deviation trajectory for the crafted attack samples . Meanwhile , the stabilization term avoids the local optima in the optimization process caused by the image noise . The proposed stabilization loss term is designed to measure the difference between two CNN predictions , where the first prediction is from the crafted adversarial sample and the second one is from the same sample processed with a Gaussian smoothing . Given an adversarial example A and its Gaussian smoothed result Ã , the loss stabilization term constrains the corresponding CNN predictions ( i.e. , f ( A ) and f ( Ã ) ) to be similar via a minimization process . The intuition from the scale space optimization ( Lindeberg , 1992 ) indicates that the minimization of f ( A ) and f ( Ã ) will exhaustively search the perturbation space to smooth the single spot for local optimum escape . We further analyze this stabilized loss term via KL-divergence and find that the CNN predictions are steered towards a fixed objective spot during iterations . This stabilization improves the attack effectiveness on different types of medical data including CT , fundus , and endoscopic images . We evaluate the proposed Stablized Medical Image Attack ( SMIA ) on several medical datasets ( APT , 2019 ; EAD , 2019 ; Kag , 2015 ) , including the recent COVID-19 ( COV , 2019 ) lung CT . Thorough evaluations demonstrate that the proposed method is effective to produce perturbations that decrease the prediction accuracy of different medical diagnosis systems . Our investigation provides a guidance for strengthening the robustness of these medical systems towards adversarial attacks . 2 RELATED WORK . In this section , we literately review existing adversarial attack methods on both natural and medical images . Meanwhile , we survey relevant medical image analysis tasks where SMIA is deployed . 2.1 ADVERSARIAL ATTACK . There are extensive investigations on adversarial attacks for natural image classifications . In Goodfellow et al . ( 2014 ) , FGSM was proposed to generate adversarial examples based on the CNN gradients . A DeepFool method was proposed in Moosavi-Dezfooli et al . ( 2016 ) to compute minimal perturbations based on classifier ’ s linearization . In Moosavi-Dezfooli et al . ( 2017 ) , an iterative algorithm was proposed to generate perturbations and showed the existence of a universal ( imageagnostic ) adversarial perturbations . In Baluja & Fischer ( 2017 ) , a Transformation Network ( ATN ) was trained to generate adversarial examples without gradient involvement . The adversarial training and provable defense were proposed in Balunovic & Vechev ( 2020 ) to achieve both attack robustness and high accuracy . The capsule based reconstructive attack was proposed in Qin et al . ( 2020 ) to cause both misclassifications and reconstructive errors . Besides image classification , several attack methods were proposed for semantic segmentation , object detection and object tracking Jia et al . ( 2020 ) . In Fischer et al . ( 2017 ) ; Dong et al . ( 2019 ) , the classification based attacks were shown transferable to attack deep image segmentation results . The universal perturbations were demonstrated existing in Moosavi-Dezfooli et al . ( 2017 ) . Moreover , a Dense Adversary Generation ( DAG ) method was proposed Xie et al . ( 2017 ) for both semantic segmentation and object detection attacks . The general idea of natural image attacks was to iteratively generate perturbations based on the CNN gradients to maximize the network predictions of adversarial examples and the ground truth labels . This idea was also reflected in Finlayson et al . ( 2019 ) to show the medical attacks . Different from existing methods , we propose a stabilized regularization term to ensure the consistent generation of adversarial perturbations , which are effective for different types of medical image datasets . 2.2 DEEP MEDICAL IMAGE ANALYSIS . The deep Convolutional Neural Networks ( CNNs ) have been shown effective to automatically analyze medical images ( Litjens et al. , 2017 ; Razzak et al. , 2018 ) . The common CADx applications include classifying the stage of disease , the detection and segmentation of organs and lesions . Disease classification . Most medical systems formulate disease diagnosis as an image classification task . The types of diseases are predefined and each type corresponds to one category . During the classification process , there are single or multiple images as input for disease diagnosis . In Shen et al . ( 2015 ) , a multi-scale CNN was proposed to capture the feature representation of lung nodule patches for accurate classification . A multi-instance layer and a multi-scale layer were proposed in Li et al . ( 2019 ) to diagnose diabetic macular edema and myopic macular degeneration . Besides diagnosing the types of disease , existing medical systems were also able to predict the disease status ( Gulshan et al. , 2016 ) by empirical status categorization . Organ and lesion detection . The detection of organ and lesion is inspired by the object detection framework for natural images ( e.g. , Faster-RCNN ( Ren et al. , 2015 ) , FPN ( Lin et al. , 2017 ) , and Yolo ( Redmon et al. , 2016 ) ) . Besides , 3D spatial information of the medical data is explored in the 3D detection framework . In Ding et al . ( 2017 ) , a 3D-CNN classification approach was proposed to classify lung nodule candidates that were previously detected by the Faster-RCNN detector . The RPN ( Ren et al. , 2015 ) was extended in Liao et al . ( 2019 ) to become 3D-RPN for 3D proposal generation to detect lung nodules . Different from the 3D detection framework , a multi-scale booster was proposed in Shao et al . ( 2019 ) with channel and spatial attentions that were integrated into FPN for suspicious lesion detection in 2D CT slices . The detection methods based on 2D image input reduced heavy computational cost brought in the 3D inputs for 3D detection methods . Organ and lesion segmentation . The medical segmentation was significantly advanced via deep encoder-decoder structures ( e.g. , U-Net ( Ronneberger et al. , 2015 ) ) . This architecture contains a contracting path ( i.e. , encoder ) to capture global context and a symmetric expanding path ( i.e. , decoder ) to obtain precise localization . There were several works built upon U-Net . In Brosch et al . ( 2016 ) , skip connections were utilized in the first and last convolutional layers to segment lesions in brain . A V-Net was proposed in Milletari et al . ( 2016 ) to segment brain ’ s anatomical structures . It followed 3D U-Net structure consisting of 3D convolutional layers . A dual pathway and multi-scale 3D-CNN architecture was proposed in Kamnitsas et al . ( 2017 ) to generate both global and local lesion representations in the CNN for brain lesion segmentation . Existing medical methods mainly utilized deep encoder-decoder architectures for end-to-end segmentation of organs and lesions . As illustrated above , deep medical diagnosis systems differ much from the CNN architectures that are developed for natural images . Moreover , the variance of different modalities of medical images is significantly larger than that of natural images . Therefore , the adversarial attacks designed for natural images are not often effective in the medical domain . Nevertheless , the limitations that are brought by huge network and data variance are effectively solved via our stabilized medical attack . 3 PROPOSED METHOD . In this section , we illustrate the details of medical image attacks . We first show the objective function of SMIA that consists of a loss deviation term and a loss stabilization term . The loss deviation term produces the perturbation to decrease the image analysis performance , while the loss stabilization term is consistently updated during iterations and constrains these perturbations to low variance . Then , we analyze how SMIA affects the generated perturbations during iterative optimization from the perspective of KL-divergence . The analysis is followed by a visualization showing the variance and cosine distance of perturbations by utilizing the loss stabilization term .

Summary: The authors proposed to introduce a combination of a loss deviation term and a loss stabilization term to generate more consistent adversarial perturbations on medical images. The loss deviation term increases the divergence between the CNN prediction of an adversarial example and its ground truth label. At the same time, the loss stabilization term ensures similar CNN predictions of this example and its smoothed input. The authors tested against 3 different medical image datasets obtained by different modalities. The proposed strategy seems straightforward and the benefits are clearly demonstrated with these three datasets.