XAI-Based Detection of Adversarial Attacks on Deepfake Detectors
Authors: Ben Pinhasov
Raz Lapid
Rony Ohayon
Moshe Sipper
Yehudit Aperstein
Venue: Accepted by TMLR ()

Abstract: We introduce a novel methodology for identifying adversarial attacks on deepfake detectors using eXplainable Artificial Intelligence (XAI). In an era characterized by digital advancement, deepfakes have emerged as a potent tool, creating a demand for efficient detection systems. However, these systems are frequently targeted by adversarial attacks that inhibit their performance. We address this gap, developing a defensible deepfake detector by leveraging the power of XAI. The proposed methodology uses XAI to generate interpretability maps for a given method, providing explicit visualizations of decision-making factors within the AI models. We subsequently employ a pretrained feature extractor that processes both the input image and its corresponding XAI image. The feature embeddings extracted from this process are then used for training a simple yet effective classifier. Our approach contributes not only to the detection of deepfakes but also enhances the understanding of possible adversarial attacks, pinpointing potential vulnerabilities. Furthermore, this approach does not change the performance of the deepfake detector. The paper demonstrates promising results suggesting a potential pathway for future deepfake detection mechanisms. We believe this study will serve as a valuable contribution to the community, sparking much-needed discourse on safeguarding deepfake detectors.

Published in Transactions on Machine Learning Research (08/2024)
XAI-Based Detection of Adversarial Attacks on Deepfake Detectors
Ben Pinhasov*
cen.pinnasson08..feeao..cc
Afeka, The Academic College of Engineering in Tel Avic
Raz Lapid*
ruzla@ppot..guu.cc
Ben-Gurion University of the Negeu, Beer-Sheva, 8410501, Isruel & DeepKeep, Tel-Aviv Israel
Rony Ohayon
rony@deepkeep.cdt
DeepKeep, Tel-Avio, Isroel
Moshe Sipper
sipper@bgu.ac.
Ben-Gurion University of the Negeo, Beer-Sheva, 8410501, Israel
Yehudit Aperstein
apersteiny@afebo.cc.
Afeka, The Academic College of Engineering Tel Auic
Reviewed on OpenReview: https: // openreview. net/ forum? id- 7pBKren111
Abstract
We introduce a novel methodology for identifying adversarial attacks on deepfake detectors using eXplainable Artificial Intelligence (XAI). In characterized by digital advance- ment, deepfakes have emerged a potent tool, creating demand for efficient detection systems. However, these systems are frequently targeted by adversarial attacks that inhibit their performance. We address this gap, developing defensible deepfake detector by lever- aging the power of XAI. The proposed methodology uses XAI to generate interpretability maps for a given method, providing explicit visualizations decision decision-making factors within the AI models. We subsequently employ a pretrained feature extractor that processes both the input image and its corresponding XAI image. The feature embeddings extracted from this process then used for training a simple yet effective classifier. Our approach tributes not only to the detection of deepfakes but also enhances the understanding of possible adversarial attacks, pinpointing potential vulnerabilities. Furthermore, this ap- proach does not change the performance of the deepfake detector. The paper demonstrates promising results suggesting a potential pathway for future deepfake detection mechanisms. We believe this study will serve as a valuable contribution to the community, sparking much-needed discourse on safeguarding deepfake detectors.
1 Introduction
Deepfake technology, which involves generating realistic media content, has advanced significantly in recent years, leading to the creation of increasingly sophisticated and convincing fake videos, images, and audio recordings (Westerlund, 2019; Khanjani et al., 2021; Guarnera et al., 2020b). In response, there has been a growing interest in developing deepfake detectors capable identifying and flagging these manipulated media (Le & Woo, 2023; Hou al., 2023; Lapid al., 2024a; Rana al., 2022; Lyu, 2020; Dolhansky al., 2020; Guarnera al., 2020a). However, with any security system, these detectors vulnerable to
"Equal contribution
Coode available attps://githul www.//owo////eeeeeee Laaazla/AAI-a draaed-Detection-ode watarraoo-drrrrrraa---taaaee htw.aaaall--ttttaaeeeeeeeeeeeeee Ihis research was supported by the Israeli Innovation Authority through the Trust.AT consortium

---

Published in Transactions on Machine Learning Research (08/2024)
adversarial attacks, which aim to deceive manipulate detector outputs by making subtle changes to the input, highlighting the need for robust detection methods.
Adversarial examples can significantly compromise the reliability of deepfake detection systems and have the potential to causee harm, especially given the difficulty of detecting deepfakes in general. Therefore, there is a critical need address the vulnerability of deepfake detectors to adversarial inputs and incorporate defenses against such attacks in the training of detection systems (Hussain al., 2022).
Concomitantly, the field of eXplainable Artificial Intelligence (XAI) has gained momentum, its aim being to render machine learning (MI.) and deep learning (DL.) techniques more transparent, providing clear interpretations of model decisions to humans.
Previous research has focused on the development deepfake detection models, with some studies incorporat- ing XAT tools to evaluate the vulnerability of deepfake detectors (Gowrisankar & Thing, 2024). Additionally, there have been efforts to detect deopfake audio using XAI models such as LIME, SHAP, and GradCAM (Govindu et al., 2023).
Despite these advancements, there remains gap in the research concerning the detection of adversarial attacks on deepfake detectors using XAI-based approaches. The problem area of interest for this paper is the potential vulnerability of deepfake detectors to adversarial attacks and the need for effective adversarial detection mechanisms.
In this paper, we address the aforementioned gap by proposing an XAI-based adversarial detector (Baniecki & Biecek, 2024) for adversarial attacks on deepfake detectors. We conduct experiments to test the effectiveness of the proposed detector in identifying and mitigating adversarial attacks existing deepfake detection models. By leveraging the insights provided by XAI maps, we aim to improve the ability of deepfake detector systems to accurately distinguish between real and fake content, even if they were adversarially attacked.
Our paper's premise is that deepfake detectors are vulnerable to adversarial attacks and require an addit ional which we obtain through XAI maps enhance their reliability. We seek to examine
layer of protection whichh the following rescarch question:
(Q) How can XAI used to detect adversarial attacks on deepfake detectors?
To answer (Q) we will explore the various XAI methods for detecting such attacks on deepfake detectors.
Our contributions are as follows:
Introduction of an innovative approach to identifying adversarial attacks on deepfake detection systems. The proposed method offers significant potential for bolstering the security of deepfake detectors and other machine-learning frameworks.
Integration of XAI techniques to enhance transparency and interpretability detecting adversarial attacks on deepfake detectors. This addition not only detects attacks but also might provide insights into the deeiiion-making process, fostering trust in the detection outcomes, crucial for real-world applications.
Empirical evidence demonstrating our method's robustness, through its successful defense against both familiar and previously unseen adversarial attacks, highlighting its resilience and versatility in variety of adversarial contexts.
The next scction describes the previous work done this area. Section 3 describes the methodology used the experiments. Section 4 describes our experimental setup and Section 5 presents the results. Our findings are discussed in Section 6, followed by conclusions in Section 7.
2

---

Published in Transactions on Machine Learning Research (08/2024)
2 Previous Work
In the domain of deepfake detection, the burgeoning threat of AT-generated manipulations has spurred the development of various detection methodologies. These methodologies broadly fall into two categories: con- ventional (Le et al., 2022) and end-to-end approaches. Conventional methods, as demonstrated in Figure 1, primarily focus on classifying cropped face images as real or fake. However, their efficacy is contingent upon accurate face detection, leading to vulnerabilities when face localization is improcise. Notably, recent studies have highlighted the susceptibility of these methods to adversarial manipulations, showcasing the need for enhanced robustness (Hussain et al., 2020).
Conventional ap- proach to deepfake detection: External face detection de- tects the face crops and each crop passes through the deep- lake detector. Each row rep- resents different adversarial attack.
Figure 1:
Process Facecrop
Detection
Process Facecrop
Conversely, end-to-end approaches offer a more comprehensive analysis by not only discerning the authen- ticity of faces but also localizing manipulated regions at varying levels of gramularity, namely detection and segmentation (Le al., 2022). Despite their advancements, these techniques are not immunc adversarial subversions, underscoring the criticality of bolstering their resilience.
Transitioning to adversarial attacks, the field of machine learning security confronts the persistent challenge posed by adversarial examples. These intentionally crafted inputs exploit vulnerabilities in deep neural network models across diverse domains, compelling erroneous predictions (Szegedy et al., 2014). Noteworthy are the gradient-based attacks, extensively explored in the literature, which exploit gradients to generate adversarial perturbations (Papernot et al., 2015; Lapid & Sipper, 2023a; Papernot et al., 2017; Li & Lyu, 2019; Moosavi-Dezzinn Dezfooli et al., 2017: Tamam et al., 2023; Carlini & Wagner, 2017a; Lapid & Sipper, 2023b; Eykholt et al., 2018; Shi et al., 2019; Lapid et al., 2022; Carlini & Wagner, 2018; Qin et al., 2019; Neckhara et al., 2019; Ebrahimi et al., 2018; Lapid et al., 2024b; Belinkov & Bisk, 2018; Hagiwara al., 2019).
Recently, Gowrisankar & Thing (2024) presented a novel attack on deepfake detection systems, involving the addition of noise to manipulated images in visual concepts that significantly influence the classification of authentic images, as indicated by XAI map of the authentic image. In our study we investigated attacks that perturb all pixels in the image, rather than confining the perturbation to a limited region.
In practical terms, adversarial attacks on deepfake detectors can lead to dire ramifications, such the proliferation of misinformation, erosion of trust in media sources, and potentially catastrophic social and political consequences. The surreptitious nature of deepfakes, compounded by their ability to evade detection through adversarial attacks, underscores the urgent need for robust defense mechanisms.
To mitigate such threats, the integration of eXplainable Artificial Intelligence (XAI) emerges as a promising avenue (Angelov et al., 2021; Confalonieri et al., 2021; Došilovié et al., 2018; Samek & Müller, 2019). As complex ML models incrcasingly obfuscate their decision-making processes, XAI endcavors demystify these black boxes, furnishing interpretable explanations for model predictions (Adadi & Berrada, 2018). By bridging the chasm between AI performance and human comprehension, XAI not only engenders user trust but also enhances accountability and decision-making.
In the domain of deepfake detector defense, existing strategies primarily revolve around fortilying model training to bolster adversarial resilience. Techniques such as adversarial training entail augmenting the training process with adversarial examples to enhance model robustness (Madry et al., 2019). While this approach shows promise, it can inadvertently alter the model's decision boundaries, potentially leading overfitting decreased performance clean data.
3

---

Published in Transactions on Machine Learning Research (08/2024)
Similarly, methodologies like smart watermarking introduce additional complexity to the training pipeline, which may hinder model generalization and scalability (Tv, 2021). Randomized smoothing (Cohen et al., 2019), another prevalent defense mechanism, seeks to mitigate adversarial vulnerabilities by adding random noise to the dataset during training. While effective in certain sccnarios, this technique can obscure genuine Teatures in the data, impairing the model's discriminative capabilities. Moreover, the increased computat overhead associated with randomized smoothing may impede real-time deployment, limiting its practical utility.
Chen et al. (2021) proposed system designed to combat deepfakes, using two-step process. First, it identified manipulated arcas of video by analyzing inconsistencies and distortions. Then, it employed this information to reconstruct the original, authentic content. This approach effectively defends against deepfakes, even when the attacker's methods are unknown. This deepfake defense might struggle with complex reconstructions and may be computationally expensive.
Despite these drawbacks, these defense mechanisms represent significant strides in fortifying deepfake detec- tors against adversarial subversion. However, careful consideration of their trade-offs is imperative ensure judicious balance between robustness and performance. This paper aims to navigate these nuances by leveraging XAI methodologies to optimize detector performance, particularly in identifying and thwarting adversarial incursions, while mitigating the adverse effects of conventional defense strategies. Importantly, our approach does not alter the underlying model architecture; instead, it involves training a new adversarial detector on top of the existing framework, ensuring compatibility and scalability across different deepfake detection systems.
3 Methodology
Our methodology involves a structured approach to improve adversarial-attack detection on deepfake detec- tors, integrating benchmark dataset, established detection models, creation of an attacked dataset, incorpo- ration of XAT techniques, and our novel detection model. Figure 3 depicts the flow for classifying a face crop as real or attacked.
We start by outlining the dataset preparation used for deepfake detection evaluation (Section 3.1), followed by review Section 3.2 of the doepfake detection models applied in our study. We then describe Section 3.3 the creation of an attacked dataset using four established attack methods, enhancing the testing scope for our approaches. The use of XAT techniques is detailed in Section 3.4, aimed at increasing the interpretability of the adversarial detection process. Lastly, in Section 3.5 we introduce our adversarial- attack detection model, emphasizing its unique contribution to detecting and mitigating adversarial thrcats, thereby ad advancing the capability of deepfake detection systems.
3.1 Dataset Preparation
To evaluate the robustnoss of our methodology, we attack the FF+ dataset (sec Section 4) using different attacks: Projected Gradient Descent (PGD) (Madry et al., 2017), Fast Gradient Sign Method (Goodfellow et al., 2014), Auto Projected Gradient Descent (APGD) (Croce & Hein, 2020), Natural Evolution Strategies (NES) (Qiu et al., 2021), and Square Attack (Andriushchenko constraint. The adversarial detector was trained using PGD only; the other attacks were used for testing only. For training our model the atlack process ileratively applied PGD with fixed maximum perturbation = 16/255 to generate manipulated instances, resulting in a dataset twice the original size. Subsequently, interpretability maps were generated across this augmented dataset using XAI techniques outlined in Section 3.4, facilitating the assessment of our approach's 's sensitivity to manipulated features for discriminating between authentic and manipulated videos.
al., 2020). Herein we focus on the -
3.2 Deepfake Detection Models
The XceptionNet architecture (Chollet, 2017) replaces Inception modules (Szegedy et al., 2017) in con- volutional neural networks with depth-wise separable convolutions. Depth-wise separable convolutions first
4

---

Published in Transactions on Machine Learning Research (08/2024)
perform spatial convolution independently over each channel of the imput, then perform a 1x1 convolution to project the output channels onto new channel space.
Xception assumess that mapping cross-channel correlations and spatial correlations in convolutional neural networks can be entirely decoupled. Chollet (2017) showed that there is a spectrum between regular convo- lutions and depth-wise separable convolutions, with Inception modules being an intermediate point. They demonstrated that Xception, built entirely from depth-wise separable convolutions, achieves slightly better classification performance over the ImageNet dataset as compared to Inception V3 with similar number of parameters. Chollet (2017) argued that depth-wise separable convolutions offer similar representational power as Inception modules while being casier to use, like regular convolution layers.
In our research we use a pretrained model, as presented by Hussain et al. (2020).
Eft is one model of an ensemble of several models for detecting facial manipulation in videos, presented by Bonettini et al. (2021). EfficienNNetBSST trained using Siamese strategy with a triplet margin loss function. This extracts deep features that achieve good separation between real and fake faces in the encoding space. Siamese training produces feature descriptor that favors similarity between samples of the same class. Bonettini et al. (2021) showed that EfficientNetBAST complements the other models in the ensemble, demonstrating improved detection accuracy and quality over individual models on the FF-+ and DFDC datasets. The fusion of FfffiiientNetBBST with other diverse models helps the overall ensemble system outperform the baseline for facial manipulation detection.
In our research, we use a single pre-trained model from the ensemble, which exhibited good performance.
3.3 Adversarial Attacks
In this section we describe 5 atlack methods that we use for creating the attacked-videos dataset. Three are white-box attacks: PGD (Madry et al., 2017), FGSM (Goodfellow et al., 2014), and Auto Projected Gradient Descent (APGD) (Croce & Hein, 2020). Two are black-box attacks: Natural Evolution Strategies (NES) (Qiu et al., 2021) and Square Attack (Andrinshchenko et al., 2020).
White-box attacks involve full access to the target model's architecture and parameters, enabling adversaries to craft adversarial examples more efficiently. In contrast, a black- attack means an adversary has limited access to the target model and its parameters, making it challenging to craft adversarial examples directly, In the context of image classifiers, an adversarial attack aims to generate perturbed inputs that can mislead the classifier, without explicit knowledge of the model's internal parameters. Black-box attacks have received increased allention in recent years (Andriushchenko et al., 2020; Qiu et al., 2021; Lapid et al., 2022; Chen et al., 2020; Tamam et al., 2023; Lapid & Sipper, 2023a).
We tested our algorithm on various attack algorithms, to assess the robustness and generalization of our method. The allack algorithm PGD was used for creating a training dataset and a test dataset. The allack algorithms APGD, NES, and Square were used for creating a test dataset to assess the robustness of our method.
PGD. We assume that the attacker has complete access to the deepfake detector, including the face- extraction pipeline and the architecture and parameters of the classification model. To construct adversarial examples we used PGD (Madry et al., 2017) to optimize the following loss function:
f= ===aa(7(rreen - ((x)take 0).
(1)
Here, Z(x)y is the logit of label y, where y E (real, fake). Minimizing the above loss function maximizes the score for our target label Real. The loss function adopted in this study, as advocated by Carlini & Wagner (2017b), was specifically chosen for its empirical effectiveness in generating adversarial samples with reduced distortion, and its demonstratod resilience to defensive distillation strategics. We used PGD to optimize the above objective while constraining the magnitude of the perturbation as follows:
Ti+1 = I; 11. (w.ssig(V...... C(0, xi. v)),
(2)
where II, is a projection of the step to e such that || xllp S c, Vi, e is the allowed perturbation, a is the step size, L is the loss function, 0 is the model's weights, and y is the target prediction.
5

---

Published in Transactions on Machine Learning Research (08/2024)
FGSM. The FGSM method, pioncered by Goodfellow et al. (2014), operates by perturbing imput data based the sign of the gradient of the loss function with respect to the input. This perturbation is scaled by a small constant, e, to ensure that changes are imperceptible but effective in fooling the deepfake detector. FGSM is simple and efficient, making it a popular choice for crafting adversarial examples. Despite its simplicity, FGSM can achieve significant perturbations, underscoring its effectiveness in bypassing deepfake detection systems.
APGD. The APGD approach, described by Croce & Hein (2020), addresses problematic issues by partition- ing N iterations into initial exploration and exploitation phase. The transition between these phases involves gradual step-size reduction. A larger step size allows swift exploration of the parameter space (S), while a smaller one optimizes the objective function locally. Step-size reduction depends on the optimiza- tion trend, ensuring it aligns with objective-function growth. Unlike traditional PGD, the APGD algorithm adapts stop sizes basod on the budget and optimization progress. After step-size reduction, optimization restarts from the best-known point.
Natural Evolution Strategies (NES) (Qiu et al., 2021) is an optimization algorithm inspired by the process of natural selection. It operates by iteratively perturbing candidate solutions and selecting those that result in improved fitness. NES does not require gradient information from the black-box model.
Square Attack (Andriushchenko et al., 2020) is a black-box attack method specifically designed for image classifiers. formulates the adversarial example generation as an optimization problem. Square Attack uses random square perturbations because they were proved successfully fool convolution- based image classifiers, with a smart initialization.
3.4 XAI Techniques
Integrated Gradients method (Sundararajan et al., 2017) is an altribution technique that aims to identify the input features that have the most significant impact on a deep network's output. To apply this method, the gradient of the output with respect to the input features integrated along a straight-line path from a baseline input to the actual input. The result provides an attribution value that reflects the contribution of each input feature to the output.
Saliency. Simonyan et al. (2013) introduced the concept of saliency as one of the earliest pixel-attribution techniques. This approach involves computing the gradient of the loss function for specific class of interest with respect to the input pixels. The resulting map shows the relative importance of input features, with negative and positive values indicating their contribution towards or against the class prediction.
Input Gradient technique (Shrikumar al., 2016) calculates the contribution of input features to a model's output by computing the gradient of the output with respect to each input feature. Specifically, the absolute value of each gradient is multiplied by the corresponding input value to measure the feature's impact on the output. This approach is based on the intuition that higher absolute gradient values indicate more significant contributions to the model's prediction. Overall, the Input x Gradient method provides a means of attributing model outputs to their underlying input features, aiding in model interpretability and identifying potential sources of bias or error.
Guided Backpropagation (Springenberg et al., 2014) is a modified version of the backpropagation algo- rithm that restricts the flow of gradients to only positive values during the backpropagation process. This is achieved by zeroing out the gradients for all negative values. The underlying concept is that positive gradients correspond to input features that positively contribute to the output, while negative gradients indicate features that negatively affect the output. By limiting the flow back to the input to only positive gradients the method aims to highlight the relevant input features that play a vital role in the model's prediction. Consequently, this approach can aid in the interpretation a network's decision maaiinnn--inngggg process by providing insight into the features that contribute to its output.
Examples of the above XAT techniques can be seen in Figure 2.
6

---

Published in Transactions on Machine Learning Research (08/2024)
Figure 2: FF+ dataset im- ages. Leftmost image: image in pixel space. Four images to the right are XAI maps produced by: Guided Back- propagation (Guid. Back.), Input x Gradient (Inp. x Grad.), Integrated Gradients (Int. Grad.), and Saliency. Top row shows the XAI meth- ods with attacks, the other rows show the XAI methods for the various attack algo- rithmss We can clearly see that the XAIs act differently for each given example.
Image
Guid. Back.
Tup. x Grad.
Iut. Grad.
Saliency
Original
PGD
APGD
Square
NES
3.5 Adversarial Attack Detection
For the creation of our datasets we employed both real and fake videos from the FF+ dataset, creating sets of real or attacked videos and XAI maps. We then formed pairs of umattacked/attacle images and their corresponding XAT maps. Leveraging a pretrained backbone model, e.g., ResNet50 (He et al., 2016), we generated embeddings (feature vectors) for these images. To classify them into unattacked or attacked categorics, we appliod two layers of lincar transformation followed by an activation function. This composite architecture is herein referred to the Ne55c0 model.
Our detection system operates in the manner illustrated in Figure 3. First, delineate three pivotal compo- nents: face detector f, deepfake detector 9, and adversarial detector d. Let us define f Rd + Rd as a face delector extracting the face from a given input image. Subsequently, let Rdd + R2 demote a binary classifier that classifies an image as deepfake or authentic, based on a face crop. Lastly, d: R2d x IR" - R represents a binary adversarial detector, evaluating whether a given face crop is under attack, using both the face crop and its corresponding XAI map.
Note that now have two "axes" of interest, as it were: real fake images, and attacked vs. unattacked images.
Given an image x E Rd, we begin by passing it through the face detector (f), yielding a face crop x' e Rdd, Subsequently, this face crop undergocs assessment by the deepfake detector 9, producing a classification outcome. case y classifies x' as authentic, we extract its associated XAI map, E Rdd, from g. We then submit both x' and h to the adversarial detector which in turn determines whether x' has been subjected to an adversarial attack or not.
4 Experiments
We conducted experiments assess the performance of the deepfake detoctors based on XceptionNet and EfficientNetBAST. following the methods deseribed by Chollet (2017) and Bonettini et al. (2021). We used the + (FF++) dataset (Rossler et al., 2019) to assess the eveecsiveness of our proposed face-

---

Published in Transactions on Machine Learning Research (08/2024)
Input Frame
Frame Classification Fake
Facecrop
Fake
XAI Map
Face Detector
Deepfake Detector
Real
XAI
Creator
TCH
Concatenation
Facecrop Embedding
Backbone Model
Facecrop Embedding
Embedding
Embedding
Classificattin Head
Real/Attacked
Figure 3: Frame analysis in suspected deepfake videos involves face extraction and classification using deepfake detectors. If classified fake, the frame is labeled fake. If classified as real, the face undergoes XAI map creation. The resulting XAI map and face are processed through a backbone model to generate embeddings, which are then input into the classification head to determine 'unattacked' or 'attacked' status.
forgery detection method. This dataset comprises over 5000 videos manipulated using techniques such DeepFakes, Face2Face, FaceShifter, FaceSwap, and NeuralTextures.
The FF++ dataset consists of 1000 real videos and 1000 fake videos for each manipulation technique. For our experiments, we randomly picked 60 videos from this dataset, with 50 allocated for training (including validation) and 10 for testing, resulting in 20,000 images for training and 5,000 images for testing, per each model and XAI configuration. These videos were subjected to h.254 compression at a compression rate of 23, preparing them for deepfake detection using the models outlined in Section 3.2. The real and fake videos were obtained from the original dataset, while the attacked videos were generated from the fake videos using the techniques presented in Section 3.3.
Herein we focus on the || ol norm constraint. For all attacks conducted we use a perturbation constraint of e 16/255. In the case of white-box attacks, the hyperparameters used were as follows: for the PGD allack a maximum of 100 (lerations; for a maximum of 100 iterations and restarts. For black-box attacks the hyperparameters were: for the NES attack-a step size of 1 1/255, a maximum of 100 iterations, and 5 restarts; for the Square attack-a maximum of 5000 iterations, and the campling-distribuun
default configuration (1 iteration); for the APGD atlack
parameter was set to 0.8.
We evaluated deepfake detection accuracy using two metrics. The videos metric, denoted Vid, computes the majority vote of all fake and real frames in each vidco and assigns label of fake if more than 50% of the frames were fake. The frame-by- frame metric, denoted as F2F, measures the average precision of all individual frames.
Table summarizes the pre-attack delection results, i.e., before performing adversarial altacks.
Table 1: Precision of deepfake detectors on FF+ dataset. Precision was calculated by 2 metrics; video and frame- framo-by-frane by-frame (see text).
Real
Fake
Vid
F2F
F2F
Neeption.et
100.00%
99.36%
100.00%
99.82%
EffCCientNetB4ST
96.00%
88.59%
98.00%
96.55%
Table summarizes the post-attack detection results on deepfake images. The latter shows that attacked images cause the deepfake detector to lose its ability to ascertain real from not.
8

---

Published in Transactions on Machine Learning Research (08/2024)
Table 2: Precision deepfake detectors on the attacked FF+ dataset. Herein, we focus only on attacked deepfake images that are supposed be classified as deepfake. When attacked, the performant deepfake detectors of Table fail completely. This shows that attacked images cause the deepfake detectors to lose their ability to differentiate real from not.
NeticientNeeBBSTT
Xception
Vid
F2F
Vid
F2F
PGD
0.00%
0.00%
0.00%
1.00%
FGSM
20.00%
20.66%
0.00%
0.00%
APGD
0.00%
0.10%
0.00%
0.10%
Square 0.00%
0.20%
0.00%
0.10%
NES
0.00%
5.70%
0.00%
1.00%
The
model underwent training using two distinct configurations. First, the finetuning process vndolved both the ResNet50 backbone model and the associated classification head. In the second configuration, the backbone model remained in frozen state, and training was administered only to the classification head. The schematic of the Potect-ResNettiy model is presented in Figure 4.
Detect- ResNet50 architecture. The backbone model contains pretrained ResNet50, and the clas- sification head contains linear layers.
Figure 4:
ResNet50 Model
Classification Layers
Input
Zero Padding
Cow
Batch Norm
ReLu
Max Pool
Conv Block
ID Block
Conv GonvBlock Block
ID Block
Cony Block
ID Block
Conv Block
ID Block
Aug Pool
Flattening
Linear 512
Rel.
Linear2
Real/Attacked
Stage
Stage 2
Stege
Stage
Stage
Detect-ResNeet model was trained with a learning rate of 0.001, minibatch size of 16, and 100 iterations. We used the cross-cntropy loss function, and the Adam optimizer (Kingma & Ba, 2014) hyperparam- eters le 8, B1 0.9, and B3 0.999.
Furthermore, conducted comparison of the dataset's accuracy under two conditions: 1) when using XAI maps, and 2) when these maps are rendered black (resulting in zero tensors), referred to as PGD-B. This latter comparison aims to show the importance of XAI maps on accuracy and to establish a baseline for testing. Specifically, we employed trained models and tested them without the presence of XAIs.
In addition, our study incorporates adaptive attack strategics into the evaluation framework. Importantly, our proposed detection methodology assumes the adversary does not possesses simultaneous access both to deepfake detection and to adversarial detection mechanisms. We believe this assumption accords with real- life scenarios such those encountered in social media platforms, where attacker's actions are limited to uploading a video without knowledge of the underlying modules operating within the application. We advocate for the deployment of our approach in real-world settings, emphasizing its accessibility exclusively through API calls. Herein, we used a simple loss function:
Lstandard-sdaptive = LBCE(f(x real) - LBCE(g(æ + 8), unattacked),
(3)
where BCE the Binary Cross- Cross-Entropy Loss, f is the deepfake detector, real is the 'real' class, g is the adversarial detector, and ben is the 'benign' class. Thus the adversary's 's goal is to minimize Lstandard-adaptiie
Moreover, we added another loss function, which tries to minimize the distance of the resultant XAI map:
LxAI-adaptive CBCE(f(x 0), real)= [|(AAl= + +6; XAI((; fl)
(4)
where XAI is function that takes an input and model, and outputs a XAI map. Again, the adversary's goal is to minimize this loss function.
9

---

Published in Transactions on Machine Learning Research (08/2024)
Finally, incorporated generalization assessments into study, wherein evaluated the efficacy of an adversarial detection model initially trained XceptionNet against the alternative architecture of Efficient- NetB4ST, and, conversely, we examined the performance of the model trained on EfficientNetB4ST when confronted with XceptionNet.
4.1 Computational Overhead Experiments
To comprehensively assess the computational overhead introduced by integrating XAT techniques, de- signed an experiment specifically focusing measuring the additional computational costs. The experimen- tal sctup included a high-performance computing environment with an Intel Xeon Gold 6136 @ 3Ghz CPU, NVIDIA GRID P40-24Q, and 64GB of RAM. The software environment consisted of Windows 10, Python 3.9, Pytorch 2.0.1, and the Captum library for implementing XAT techniques (Kokhlikyan et al., 2019) We measured the processing time and resource usage (CPU/GP) utilization and memory consumption) of our adversarial detection model, both with and without the integrated XAI methods.
The experiment structured follows:
Baseline Measurement: Run the adversarial--detetioo model without any XAI techniques, record- ing the processing time and memory consumption.
XAI Integration Measurement: Integrate the selected XAI techniques into the pipeline and repeat the measurements.
Comparison: Compare the baseline and XAI integration measurements quantify the overhead.
5 Results
Table 3 and Table 4 assess the efficacy of our approach against adversarial attacks on deepfake detectors. We focus on two distinct scenarios: one with finetuned ResNet50 model and the other with a frozen pretrained ResNet50 model, finetuning the classifier head only. We set herein a classification threshold of 0.5; Appendix A presents statistics about different thresholds.
Table 3: Accuracy (Real/Attacked in Figure 3) for various attacks, on the complete test set of a finetuned ResNet50. The asterisk (*) associated with the PGD attack denotes its use in training the model. The best average result reached is shown in boldface. Mean does not include PGD-B experiments.
Deepfake Detector
Attack
G-Backprop
Inp x Grad
Int-Grad Saliency
(PGD-B*)
(85.04%)
(84.61%)
(85.00%)
(89.16%)
PGD*
99.60%
97.54%
94.48%
99.60%
FGSM
91.90%
89.93%
85.62%
93.11%
NeIS
APGD
98.86%
97.10%
93.80%
94.40%
Square
61.36%
63.13%
62.13%
62.30%
NES
99.80%
97.66%
95.21%
99.57%
Average
90.30%
89.07%
86.24%
89.80%
(PGD-B*)
(61.43%)
(8183%)
(81.81%)
(53.135)
PGD
95.92%
86.63%
82.93%
90.57%
FGSM
50.88%
73.67%
65.22%
85.73%
XceptionNet
APGD
91.06%
54.83%
77.47%
67.34%
Square
52.11%
81.98%
80.75%
82.20%
NES
88.54%
86.75%
85.80%
89.78%
Average
75.76%
76.77%
78.43%
85.12%
Total Average
83.03%
82.92%
82.33%
87.46%
For the finetuned ResNet50 model (Table 3), we observe that the average accuracy across all evaluated attacks ranges from 75.76% 90.30%. Notably, Saliency exhibits the highest success rate at detecting
10

---

Published in Transactions on Machine Learning Research (08/2024)
Table 4: Accuracy Attacked in Figure 3) for various attacks, on the complete test set, when finetuning only the classification head of a pretrained ResNet50. The asterisk (*) associated with the PGD attack denotes its use in training the model. The best average result reached shown in boldface. Mean does not include PGD-B experiments.
Deepfake Detector
Attack
G-Backprop
Inp Grad
Int-Grad
Saliency
(PGD-B*)
(69.00%)
(74.65%)
(78.26%)
(78.08%)
PGD
93.46%
92.36%
91.58%
98.02%
FGSM
76.65%
82.26%
83.70%
88.02%
NelBAST
APGD
96.04%
95.14%
92.94%
98.83%
Square
53.67%
61.22%
60.25%
65.60%
NES
93.57%
91.33%
88.60%
96.60%
Average
82.68%
84.16%
83.11%
89.41%
(PGD-B*)
(50.32%)
(63.13%)
(53.14%)
(65.70%)
PGD*
90.07%
69.44%
70.33%
72.80%
FGSM
51.28%
60.70%
65.16%
73.55%
NceptionNet
APGD
93.57%
73.01%
69.65%
82.72%
Square
46.03%
57.53%
53.76%
59.68%
NES
81.13%
76.70%
76.05%
82.85%
Average
72.11%
67.17%
67.00%
74.32%
Total Average
77.51%
75.96%
75.21%
81.85%
the attacks, achieving an average accuracy of 87.46%. Conversely, G-Backprop, Inp Grad and Int-Grad demonstrate relatively lower average accuracies of 83. 83.03%, 82.92% and 82.33%, respectively.
Performance slightly declined when confronted with the Square attack, while using EfficientNetB4STT, with accuracies ranging from 61.36% to 13% across all the XAI techniques. This decrease in performance be attributed to the model's training using PGD only, indicating potential limitation in generalization to other adversarial attack types. On the other hand, the performance of our approach against Square Attack on NceptionNet is better, reaching a best score 82.20% with Saliency.
These results underscore the vulnerability of the XAI techniques to various adversarial attacks, with notable variations in attack success rates.
Turning our attention to the second configuration of the ResNet50 model (Table 4), i.e., finetuning the classification head only, we note similar trends in attack effectiveness. Across all evaluated attacks, the average accuracy ranges from 67.00% to 89.41%. Once again, Saliency emerges as the most successful, achieving an average accuracy of 81.85%. This is followed by G-Backprop, with an average accuracy of 77.54%. Conversely, Inp Grad and Int-Grad exhibit lower average accuracies of 75.96% and 75.21%, respectively.
We note the role of PGD-B in elucidating the significance of XAI in approach. PGD-B represents scenario where XAI contributions are nullified, essentially simulating scenario devoid of interpretability. Tn this context, PGD-B results underscore the pivotal role of XAI techniques in our methodology. When XAI is absent, the effectiveness of our approach dwindles, (up to ~ 10% difference in the finetuning experiments and up to ~ 40% in finetuning the classifier head only), highlighting the dependency on XAI methodologies for robust detection of adversarial attacks on deepfake detectors.
The experimentation underscores the criticality of leveraging XAI not only to enhance model interpretability but also to fortify model resilience against adversarial manipulations, thereby underscoring the synergy between XAI and traditional input features in bolstering deepfake detection mechanisms.
Our findings underscore the balance between finetuning the entire model versus specific layers, adding another consideration. Finetuning the entire model allows for thorough adaptation to the target domain, enhancing task alignment and performance against adversarial attacks. Yet, requires significant computational re- sources and may not be viable in resource-limited settings. On the other hand, finetuning only certain layers provides more practical option, reducing computational burden while enabling some domain adaptation.
11

---

Published in Transactions on Machine Learning Research (08/2024)
However, it restricts model adaptation and may result in lower performance, especially against advanced adversarial attacks. On average, the detection rate on FeficientNedBBSTT is better than the detection rate on XceptionNet.
Table and Table 7 show results of the adaptive attacks, which are aware of both the deepfake detector and the adversarial detector, while using Equation and Equation respectively. We emphasize again that we do not think it is realistic scenario and our detector was not trained against these kinds of attacks. In the first scenario, using Equation 3, the attacks successfully fooled both detectors, with EfficientNetB4ST showsing more resillience than XceptionNet. Using Equation 4, Eeft shows excellent performance, while XeeptionNet still fails to resist these attacks.
Table 5: Comparison of Adaptive Allacks
Table 6: Adaptive attacks using Equation
Table 7: Adaptive attacks using Equation 4.
NeeptionNet
Net
Xception Net
Efffiient.VeBBBST
Vid
F2F
Vid
F2F
Vid
F2F
Vid
F2F
PGD
0.00%
0.00%
0.00%
0.16%
PGD
0.00%
0.24%
100.00%
98.60%
FGSM
0.00%
0.49%
80.00%
88.50%
FGSM
0.00%
0.25%
100.00%
98.30%
In addition we conducted a generalization study where we evaluated the generalization of our approach: training on one model and evaluating on another. The results are delineated in Table 8, where we see gener- alization performance ranging from 66.00% 82.97%. We note that on average training on XceptionNet's attacked images and evaluating FfffiiietNNetBSTT yields the best results, ranging from 75.05%-82..97%
Table 8: Transferability results for various attacks. A pretrained adversarial detector trained on a backbone deepfake detector is tested on attacks that were optimized on another backbone detector.
Source Model Transferred Model
Attack
G-Backprop
Inp x Grad
Int Grad
Saliency
PGD"
67.33%
03%
80.50%
71.68%
FGSM
67.38%
08.022
80.67%
71.67%
Nef B4ST + XceptionNet
APGD
70.30%
63.37%
82.80%
85.30%
Square
47.10%
66.56%
82.93%
76.45%
NES
77.90%
68 08%
81.64%
99.60%
Average
66.00%
06.81%
81.71%
$0.9.%%
PGD*
95.90%
86,84%
83.21%
90.8%%
FCSM
50.88%
73.67%
65.22%
85.73%
NceptionNet NefteeenNeBBBST
APGD
89.80%
64.83%
77.85%
66%
Square
52.41%
81.98%
80.73%
82.18%
NES
86.35%
87.25%
83.32%
90.16%
Áverage
75.07%
76.51%
78.06%
82.97%
Total Average
70.54%
71.86%
79.88%
81.95%
Overall, we have assessed the performance different XAI methods, with varying degrees of success, depend- ing on the attack. Training an adversarial detector on NeficiennNetBBST yields moro-resilient detector than XceptionNet, on average. Our findings emphasize the importance of developing robust defense strategies to safeguard against the proliferation of synthetic manipulations of digital media.
5.1 Computational Overhead Results
The results of our computational overhead experiments are presented in Table 9, which summarizes the processing- metrics for both the baseline model and the pipeline with integrated XAI techniques.
These results indicate that while XAI techniques enhance explainability, they also introduce varying degrees of computational overhead. The Integrated Gradients method, in particular, incurs substantial overhead, making it less practical for real-time applications, compared to other XAI techniques. Conversely, methods like G-Backprop and Input Gradient offer more balanced trade-off between computational cost. and
12

---

Published in Transactions on Machine Learning Research (08/2024)
Table 9: Computational overhead of our proposed pipeline.
Average Processing Time (ms)
Model
Baseline
G-Backprop
Inp x Grad
Int-Grad
Salience
XceptionNet
222.0
338.1 (+52%)
269.4 (+21%)
3640.3 (+1539%)
264.6 (+19%)
EfficientNct
247.1
361.5 (+46%)
335.0 (+36%)
8440.3 (+33116%
358.1 (+-45%
explainability, exhibiting relatively lower processing times while still providing valuable insights into model decisions.
In addition to processing time, we also measured the memory overhead introduced by the XAI techniques. The baseline models have a memory usage of approximately 1.1GB. Integrating the XAI methods increases the memory usage by approximatcly 1GB, resulting in a total memory consumption of around 2.1GB. This additional memory overhead is primarily due to the storage requirements for the gradients and intermediate computations used by the XAI techniques.
The increased memory footprint is crucial consideration for deploying these models in real-world settings, especially on resource-constrainned devices. While the overhead is significant, it remains manageable within the context of modern computing environments equipped with sufficient memory resources. This analysis un- derscores the importance of balancing the need for explainability with the available computational resources, particularly applications where memory constraints are a critical factor.
6 Discussion
Effectiveness of XAI in adversarial detection. The use of XAI techniques in the context of detoct- ing adversarial attacks on deepfake detectors has demonstrated significant promise. The interpretability provided by XAT methods, such as feature attribution and saliency maps, enhances our understanding of model decision-making processes. In our experiments the integration of XAI facilitated the identification of subtle adversarial manipulations that might have otherwise gone unnoticed, as shown in the black XAI image experiments (Table 3 and Table 4). This highlights the importance of incorporating explainability mechanisms in deepfake detection systems to improve their robustness against adversarial attacks. Addition- ally, our findings reveal nuanced insights into the effectiveness of XAI techniques in miligating adversarial attacks. Specifically, we observed that Saliency and G-Backprop outperform other methods in scenarios where the entire network is fino-tuned, while Saliency demonstrates superior performance when only the classification head is fine-tuned. Furthermore, on average, using EfficientNedB4STT XAIs yield better re- sults than XceptionNet. This distinction underscores the importance of considering the architectural and training constraints when integrating XAI methods for robust deepfake detection.
Ethical Use of XAL While this paper emphasizes the benefits of using XAI to enhance deepfake detectors, it is crucial to consider potential ethical concerns. XAT could potentially be misused by adversaries to reverse-engineer Al systems and exploit vulnerabilities, thereby compromising the very robustness these techniques aim to enhance. To mitigate such risks, it is essential to implement strict security protocols governing the dissemination and application of XAT insights. This includes controlling access to sensitive interpretability data, collaborating with cybersccurity experts, and developing XAI methods that balance transparency with security. Continuous ethical deliberation within the research community also necessary to ensure responsible AI usage.
Socictal Implications. Enhanced dcepfake detectors have significant socictal benefits, such as preventing misinformation ion and protecting digital content integrity. However, their dual-use nature raises concerns about privacy and surveillance. These technologies could be misused to monitor individuals' digital expressions, infringing on privacy rights. To address these concerns, comprehensive guidelines for the responsible deploy- ment of deepfake detectors must be proposed and adhered to. These guidelines should ensure transparency, legal and ethical consistency, and protection of privacy rights. Multi-stakeholler dialogue involving policy-
13

---

Published in Transactions on Machine Learning Research (08/2024)
makers, technologists, civil society, and the public is essential to define and enforce ethical boundaries for deploying these technologies.
Vulnerabilities of deepfake detectors. Despite advances in deepfake detection models, our study reveals inherent vulnerabilities that adversarial actors can exploit. Adversarial attacks, particularly those crafted with the intent. to deceive deepfake detectors, pose a formidable challenge. Traditional evasion techniques, such as input perturbations and gradient-based attacks, were successful in deceiving state-of-the-art deepfake detectors. This underscores the need for ongoing research and development in enhancing the robustness of deepfake detection models against adversarial manipulations.
Importance of explainability for trustworthiness. Explainability not only contributes to the detection of adversarial attacks but also plays a crucial role in establishing trustworthiness in Al systems. The ability to provide clear and interpretable justifications for model predictions instills confidence in end-users and facilitates a better understanding potential vulncrabilities. In applications where the consequences of false positives or false negatives can be severe, the transparency afforded by XAI methods becomes indispensable. Thus, KAI-based methods should be further employed and analyzed.
Generalization and transfcrability. Our experiments considered diverse set of adversarial attacks and deepfake detection models to evaluate the generalization and transferability of adversarial attacks. Our findings suggest that certain adversarial techniques remain effective across different datasets and models, emphasizing the need for standardized evaluation protocols and robust defenses that can withstand a variety of adversarial strategies.
Limitations. While our proposed adversarial detector for attacks on deepfake detectors demonstrates promising results, its efficacy may be limited by the diversity of adversarial attacks it has been trained on, and its generalization capability across different models and datasets. Additionally, exploring a wider range of e values in crafting adversarial examples could provide deeper insights into the robustness of the detector. Morcover, the possibility of adversarics targeting our detector itself and the ethical implications of its deployment underscore the need for ongoing research to address these challenges and ensure its practical applicability and societal impact. However, it is reasonable to assume that the attacker lacks access to the adversarial detector.
Future directions. The dynamic landscape of adversarial attacks on deepfake detectors calls for continuous research efforts. Future directions may include the exploration of novel XAI techniques, the development of adversarially robust deepfake detection models, and the investigation of real-time detection strategies. Ad- ditionally, interdisciplinary collaborations involving experts in computer vision, machine learning, and ethics can contribute to a holistic approach in addressing the evolving challenges posed by deepfake technology.
7 Conclusions
The use of XAI techniques affords significant potential in identifying adversarial attacks on deepfake detoc- tors, with Guided Backpropagation proving notably accurate in this regard. Enhancing model interpretability is crucial in bolstering detection capabilities.
Deepfake detectors are vulnerable to adversarial attacks, facilitated by methods like PGD, APGD, NES, and Square, emphasizing the need for robust defense mechanisms to maintain reliability and efficacy. While fine- tuning the entire adversarial detection model enhances adaptability and resilience, entails computational overhead, necessitating careful consideration.
XAI maps offer valuable cues for discerning adversarial perturbations, aiding in the differentiation between authentic and manipulated inputs, thereby reinforcing defenses. Despite promising generalization capabili- ties, diversification in attack modalities during training is essential for improved elficacy and robustness.
Acknowledging inherent constraints esperially concerning attack diversity and evaluation across models underscores the nccessity for continued rescarch to enhance robustness and generalizability Further explo- ration into scenarios where attackers have knowledge of detection systems is crucial for developing counter- measures against sophisticated attacks.
14

---

Published in Transactions on Machine Learning Research (08/2024)
In conclusion, the synergy between XAI and robust learning strategies shows promise in safeguarding deep- fake detectors. Continued research is vital to address limitations and broaden the applicability of these methodologies, emphasizing the importance of collaborative efforts in fortifying Al systems' reliability and integrity.
15

---

Published in Transactions on Machine Learning Research (08/2024)
References
Amina Adadi and Mohammed Berrada. Pocking inside the black-box: A survey on explainable artificial intelligence (xai). IEEE Access, 6:52138 52160, 2018. doi: 10.109/ACCESS..01... 200.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query- efficient black-box adversarial attack via random search. In European conference on computer vision, pp. 484 501. Springer, 2020.
Plamen Angelov, Eduardo Soares, Richard Jiang, Nicholas 1 Arnold, and Peter M Atkinson. Explainable artificial intelligence: analytical review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11(5):e1424 2021.
Hubert Baniecki and Przemyslaw Biecek. Adversarial attacks and defenses in explainable artificial intelli- gence: A survey, Information Fusion, pp. 102303, 2024.
Yonatan Belinkov and Yonatan Bisk. Synthotic and natural noise both break neural machine translation, 2018.
Nicolo Bonettini, Edoardo Daniele Cannas, Sara Mandelli, Luca Bondi, Paolo Bestagini, and Stelano Tubaro. Video face manipulation detection through ensemble of cnns. In 2020 25th international conference on pattern recognition (ICPR), pp. 5012 5019. IEEE, 2021.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, 2017a.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium security and privacy (sp), pp. 39-57. Icce, 2017b.
Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech-to-text, 2018.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjumpattack: query-efficient decision- based attack. In 2020 ieee symposium on security and privacy (sp), pp. 1277-1294 IEEE, 2020.
Zhikai Chen, Lingxi Xie, Shanmin Pang, Yong He, and Bo Zhang. Magdr: Mask-guided detection and roconstruction for defending deepfakes. In Proceedings of the IEEE/CVF Conference Computer Vision and Pallern Recognilion, pp. 9014 9014-9023, 2021.
François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1251-1258, 2017.
Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing, 2019.
Roberto Confalonieri, Ludovik Coba, Benedikt Wagner, and Tarck R Besold. A historical perspective of explainable artificial intelligence. Wiley Interdisciplinary Reviews: Dala Mining and Knowleegge Discovery, 11(1)=e1391, 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMIR, 2020.
Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The deepfake detection challenge (dfde) dataset, 2020.
Filip Karlo Došilovié, Mario Brčić, and Nikica Hlupić. Explainable artificial intelligence: A survey. In 2018 4 1st International convention on information and communication technology, electronics and microelec- tronies (MIPRO), pp. 0210 0215. IEEE, 2018.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White- Thxite-box adversarial examples for text classification, 2018.
16

---

Published in Transactions on Machine Learning Research (08/2024)
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Physical adversarial examples for object detectors, 2018.
Tan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXio:1412.6577 2014.
Aditi Govindu, Preeti Kale, Aamir Hullur, Atharva Gurav, and Parth Godse. Deepfake audio detection and justification with explainable artificial intelligence (xai). 2023.
Balachandar Gowrisankar and Vrizlynn L.L. Thing. An adversarial attack approach for explainable ai evaluation on deepfake detection models. Computers and Security, 139:103684, 2024. ISSN 0167-4048. doi: http:///00oi..og11111111////// 2023. 103684. URL p11//8016740422005941.
com/science/articlle
Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. Deepfake dctection by analyzing convolutional traces. In Proceedings of the IEEE/CV conference on compuler vision and pollern recognition workshops, pp. 666-667, 2020a.
Luca Guarnera, Oliver Giudice, Cristina Nastasi, and Sebastiano Battiato. Preliminary forensics analysis of deepfake images. In 2020 AEIT international anneual conference (AEIT), pp. 1 6. IEEE, 2020b.
Masato Hagiwara, Takumi Ito, Tatsuki Kuribayashi, Jun Suzuki, and Kentaro Inui. TEASPN: Frame- work and protocol for integrated writing assistance environments. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLPP) System Demonstrations, pp. 229 234, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D1s DDD19-30933 URL www.a/l/oooooooooo0////// org/D19-3039.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, and Jianjum Zhao. Evading deepfake detectors via adversarial statistical consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12271-12280, 2023.
Shehzeen Hussain, Paarth Neckhara, Malhar Jere, Farinaz Koushanfar, and Julian McAuley. Adversarial deepfakes: Evaluating vulnerability of deepfake detectors to adversarial examples, 2020.
Shehzeen Hussain, Paarth Neekhara, Brian Dolhansky, Joanna Bitton, Cristian Canton Ferrer, Julian McAuley, and Farinaz Koushanfar. Exposing vulnerabilities of deepfake detection systems with robust attacks. Digital Threats: Research and Practice (DTRAP), 3(3):1 2022.
Zahra Khanjani, Gabrielle Watson, and Vandana P Janeja. How deep are the fakes? focusing on audio doepfake: survey. arXiv preprint arXiv:2111.14208, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic oplimization. arXio preprinl arXiv:1412.6960 69800 2014.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Jonathan Reynolds, Alexander Melnikov, Natalia Lunova, and Orion 2019.
Pytorch captum. https: com/ wwt.l/ctooooooeeeeeee pttpp///tcttttmm
Raz Lapid and Moshe Sipper. Patch of invisibility: Naturalistic black-box adversarial attacks on object detectors. Xiviv preprint ar 230::2303.04238, 2023a.
Raz Lapid and Moshe Sipper. see dead people:
Gray-box adversarial attack on image-to-text models.
5th Workshop Machine Learning for Cyhersecurity, part of FCMLPKDD 2023, 2023b.
17

---

Published in Transactions on Machine Learning Research (08/2024)
Raz Lapid, Zvika Haramaty, and Moshe Sipper. An evolutionary, gradient-free, query-effiiient, black-box algorithm for generating adversarial instances in deep convolutional neural networks. Algorithmss, 15(11):
407, 2022.
Raz Lapid, Almog Dubin, and Moshe Sipper. Fortify the guardian, not the treasure: Resilient adversarial detectors. arXiv preprint 12112000,. 2024a.
Raz Lapid, Ron Langberg, and Moshe Sipper. Open sesame! universal black-box jailbreaking of large language models. In ICLR 2024 Workshop on Secure and Trushworthy Large Language Models, 2024b.
URL https:: /ttp://:////eeeiie///ee/ wttprreewwwweeeeettteenneeeeeeee htt./forummidd-Ouyy0nnaa..
Binh M Le and Simon S Woo. Quality-agnostic deepfake detection with intra-model collaborative learning. In Proceedings of the LEEE/CVE International Conference on Computer Vision, pp. 22378 22389, 2023.
Bngna-Nghia Le, Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. Robust deepfake on unrestricted media: Generation and detection, 2022.
Yuezun Li and Siwei Lyu. Exposing deepfake videos by detecting face warping artifacts, 2019.
Luochen Lv. Smart watermark to defend against deepfake image manipulation. In 2021 IEEE 6th In- ternational Conference on Computer and Communication Systems (ICCCS), pp. 380 384, 2021. doi: 10.1109 2DCCCC5522..2021.994422
Siwei Lyu. Deeplake detection: Current challenges and next steps. In 2020 IEEE international conference on multimedia & expo workshops (ICMEW), pp. 1-6. IEEE, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks. arXiv preprint ar Xiv: 176: 060806 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, 2019.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations, 2017.
Paarth Neekhara, Shehzeen Hussain, Prakhar Pandey, Shlomo Dubnov, Julian McAuley, and Farinaz Koushanfar. Universal adversarial perturbations for speech recognition systems, 2019.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings, 2015.
Nicolas Papernot, Patrick McDaniel, Tan Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning, 2017.
Yao Qin, Nicholas Carlini, Ian Goodfellow, Garrison Cottrell, and Colin Raffel. Imperceptible, robust, and targeted adversarial examples for automatic speech recognition, 2019.
Hao Qiu, Leonardo Lucio Custode, and Giovanni lacca. Black-box adversarial attacks using evolution strategies. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1827-1833, 2021.
Md Shohel Rana, Mohammad Nur Nobi, Beddhu Murali, and Andrew H Sung. Deepfake detection: A systematic literature review. IEEE access, 10:25494 -25113, 2022.
Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Niess- ner. Faceforensics+#:: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), October 2019.
Wojeiech Samek and Claus-Robers Müller. Towards explainable artificial intelligence. Explainable AI: interpreling, explaining and visualizing deep learning, pp. 5-22, 2019.
18

---

Published in Transactions on Machine Learning Research (08/2024)
Yucheng Shi, Siyu Wang, and Yahong Han. Curls and whey: Boosting black-box adversarial attacks, 2019.
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just a black box: Learning important features through propagating activation differences. orXiv preprint arXiv:1605.017 000000113, 2016.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint 1Kio:331.6003 2013.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXie preprint orXiv: 44 11000800, 2014.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks, 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2014.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander Alemi. Inception-v4, inception- and the impact of residual connections on learning. In Proceedings of the AAAI conference artificial intelligence, volume 31, 2017.
Snir Vitrack Tamam, Raz Lapid, and Moshe Sipper. Foiling explanations in deep neural networks. Trans- actions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openrevae/ http://pnrrrree...ee/trrrrr id=wvLQKHtyLK.
Mika. Westerlund. The emergence of deepfake technology: A review. Technology innovation management review, 9(11), 2019.
19

---

Published in Transactions on Machine Learning Research (08/2024)
A
Appendix
We present ROC curves for our experiments on detecting adversarial attacks on deepfake detectors. These curves illustrate the performance (rade-ofls of our models under different attack scenarios.
The ROC curves provide a visual representation of our models' sensitivity to deepfake detection amidst adversarial manipulation. By examining these curves, readers can assess the effectiveness of our proposed techniques in mitigating adversarial attacks.
Tables Table 10 and Table 11 show the results for fully finetuned and classification-eadd only finetuned ResNet50 on EfficientNetBBSTT. respectively. Tables Table 12 and Table 13 show the same results while using XceptionNet as the deepfake detector.
20

---

Published in Transactions on Machine Learning Research (08/2024)
Table 10: RoC curves of a fully finetuned ResNet50 on adversarial attacks that were optimized on Efficient- NetB4ST.
Fully finetuned ResNet50 on EfficientNNeBBST
ElliciertNe:B4ST PGD
ElficenNNNeBBST
10-
10-
0.8
0.8
Tiue Forlvve Rate
true rostive eoze
2.6
0.6
0.1
0.4
0.
Guid. Backgrap Grad.
02-
Gud, Bactpro Grad
0.0
Saliency
Safency
EfficientNetB4ST. NES
Efficientle:B45T: Square
Tus Postree fleee
Irue 115t12 Pare
Beckprop
Grad.
Int. Gradients
Int Grediants
Saliency
Salency
0.0
0.2
11.4
0.5
06
022
0.4
U.
0.5
False Positive Rate
False Positive hate
ElficentNeBBSST FGSM
Tiue Foblive Pate
3.8
3.4
0.2
Guid. Backgrop
Inc. x Grod. Gradients
211
Salienay
0.3
Talse Posiltue
21

---

Published in Transactions on Machine Learning Research (08/2024)
Table 11: RoC curves of classification head- frad-fint ResNet50 on adversarial attacks that were optimized BeS.
Classification head-finetuned ResNet50 on EfficientNNeB4ST
EllicienNN::4STTT
ElliceniNNeB4ST
1.-
10-
0.8
0.8
Tiue Forlive Rate
Irve rostive eaze
2.6
0.6
0.1
0.4
0.2
Guid. Backgrap Grad.
02-
Gud, Bactprop
X Grad
Gradiants Saliency
0.0
00-
Safency
EfficientNetB4ST NEES
Efficiontle:B45T: Square
Tus Postree fleee
irue 115th2 Pare
Backgrap
Beckprap x
Ing. Grad. Int. Gradients Sallency
Int Grediants
Salency
0.0
0.2
01.4
0.5
UL
022
U.4
U.b
0.5
False Positive Rate
False Positive hate
ElTicentNeIBSST FGSM
1.2
Thue Rootive Pate
3.8
0.4
0.2
Guid. Backgrop
Inc. x Grod.
Gradients
21.3
Saliensy
0.3
Talse Posiltve Rate
22

---

Published in Transactions on Machine Learning Research (08/2024)
Table 12: RoC curves of fully finetuned ResNet50 on adversarial attacks that were optimized on Xception- Net.
Fully finetuned ResNet50 on XceptionNet
KceptionNet- PGD
XceptionNet APGO
1.-
100
0.8
0.8
Tiue Forlive Rate
Irve Postive Hoze
2.6
0.6
0.4
0.4
0.2
Guid. Backgrap Grad.
02-
Gud, Bactprop
X Grad
Gradiants Saliency
0.0
00-
Salency
0.5
XcaptionNet- NES
XceptionNet- - Square
Tus Postree flare
Irue re5th2 Pare
0.6
0.6
Guid. Beckgrap
Beckprop
Ing. Grad.
x
Int. Gradients Saliency
Int Orediants
Salency
0.0
0.2
01.4
0.5
U6
022
U.4
U.b
u.5
False Positive Rate
False Positvve hate
XceptionNee FGSM
1.1
Thue Footive Pate
3.8
0.4
0.2
Guid. Backgrop
Inc. x Grod.
Gradients
21.3
Saliensy
02
0.3
Talse Posilive Rate
23

---

Published in Transactions on Machine Learning Research (08/2024)
Table 13: RoC curves of classification head fead-finttune ResNet50 on adversarial attacks that were optimized Nee.
Classification head-finetuned ResNet50 on XceptionNet
KceptionNet- PGD
XceptionNet- APGD
10-
10-
0.8
0.8
Tiue Forlive Rate
Irue Postive eoze
2.6
0.6
0.4
0.1
0.2
Guid. Backgrap Grad.
02-
Gud, Bactprop
X Grad
Gradiants
0.0
Saliency
00-
Safency
XcaptionNet- NES
XceptionNet- - Square
Guid. Backprop Grad Irt Fowdeets
Safercy
Tus Postnee f181e
Irue resth2 Part
0.6
Guid Beckgrap
Ing. Grad.
Int. Gradients Saliency
0.0
0.2
U1.4
0.5
U6
0.2
0.4
U.
0.5
False Positive Rate
False Positive hate
XceptionNee FGSM
1.1
Thue Rootive Pate
3.8
0.4
0.2
Guid. Backgrop
Inc. x Grod.
Gradients
21.3
Saliensy
02
0.3
Talse Posilive Rate
24


Reviews:
[{"review_id": "fp8ubF2lyT", "reviewer": "Reviewer_uJpt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Innovative Approach: Integrating XAI techniques to detect adversarial attacks on deepfake detectors is a novel approach that enhances transparency and interpretability. This is crucial for building trust in AI systems, especially in sensitive applications.\n- Empirical Validation: The paper provides robust empirical evidence demonstrating the proposed method's effectiveness in defending against familiar and previously unseen adversarial attacks. This showcases the method's resilience and versatility.\n- Minimal Performance Impact: The proposed approach does not negatively impact the performance of the deepfake detectors. This is a significant advantage as it ensures that the original detection capabilities are retained while adding an extra layer of security.\n- Comprehensive Evaluation: The study includes a thorough evaluation using various attack methods and datasets, which strengthens the validity of the results.\n\nWeaknesses:\n- Focus on White-Box Attacks: The study predominantly focuses on white-box attacks, where the attacker has full access to the model. Real-world scenarios often involve black-box attacks, which are not as extensively covered.\n- Assumptions About Attackers: The study assumes that attackers do not have simultaneous access to deepfake and adversarial detection mechanisms. This assumption may not always hold true in real-world settings, potentially limiting the applicability of the results.\n- Dependence on XAI Maps: The proposed method's effectiveness relies heavily on the quality and reliability of the XAI maps, which may vary across different datasets and types of attacks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a novel methodology for detecting adversarial attacks on deepfake detectors using eXplainable Artificial Intelligence (XAI). The primary contribution is leveraging XAI to generate interpretability maps, enhancing transparency and understanding of AI model decisions. These maps are combined with a pre-trained feature extractor to develop a robust adversarial detection model. The approach maintains the performance of existing deepfake detectors while improving their resistance to adversarial attacks. The study demonstrates promising results, suggesting potential pathways for future deepfake detection mechanisms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Detailed Computational Analysis:\n- Provide a comprehensive analysis of the computational overhead introduced by integrating XAI techniques, particularly in real-time applications. This will help assess the practicality of deploying the proposed method in real-world settings.", "broader_impact_concerns": "- Ethical Use of XAI: While the paper highlights the benefits of using XAI to improve model interpretability and robustness, it should also discuss potential ethical concerns related to the misuse of XAI in adversarial contexts. For example, XAI techniques could be used to reverse-engineer and exploit vulnerabilities in AI systems.\n\n- Societal Implications: Deploying enhanced deepfake detectors can have significant societal implications, such as preventing misinformation and protecting digital content's integrity. However, there is also a potential for surveillance and privacy invasion misuse. The paper should address these dual-use concerns and propose guidelines for responsible deployment.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fQGw3DG0vA", "reviewer": "Reviewer_Xifk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The perspective of using XAI maps for adversarial attack detection is interesting.\n\n2. The experimental results are decent, and the evaluation on XAI techniques is comprehensive.\n\nWeaknesses\n\n1. I appreciate the discussion of limitations in the paper, but I think more experiments are needed to justify the effectiveness of the approach. First, it is important to include white-box attack results where the attacker has access to both the deepfake detector and the attack detector. Even if such attacks are again successful, it would be helpful to see whether adding the adversarial detector makes it harder to attack.\n\n2. Is it required to train different attack detectors for different Deepfake detectors? Have the authors evaluated the generalizability across different backbone models?\n\n3. All adversarial examples in the evaluation share the same perturbation constraint. Have the authors evaluated the transferability across different $\\epsilon$? What is the norm of the constraint, is it $L_\\infty$ or $L_2$? Also, if the model is trained on other attack types, does it affect the transferability?\n\n4. Note that while G-backprop performs better than other XAI techniques on PGD attacks, it performs much worse on Square. It would be good to add more discussion and explanation on this comparison. Also, it would be good to add results on more diverse white-box and black-box attacks, so we can see whether G-backprop is still overall better than other XAI techniques.\n\n5. The terminology in Section 5 is confusing. G-Backprop and other XAI techniques are not attacks, they are defenses against adversarial attacks. However, Section 5 frequently refers to them as attacks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes to leverage XAI techniques for detecting adversarial attacks against Deepfake detectors. Specifically, they train an adversarial attack detector with PGD adversarial examples, and the model input is the concatenation of the face crop and the associated XAI map. The authors evaluate their approach on multiple white-box and black-box attacks, and examine several XAI techniques. The results show that leveraging XAI maps significantly improves the adversarial attack detection accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please add white-box attack results where the attacker has access to both the deepfake detector and the attack detector.\n\n2. Please explain whether the attack detector can transfer across different backbone models.\n\n3. Please add discussion on the transferability across different $\\epsilon$, and explain what is the norm of the constraint. Also, please add discussion on the transferability with different attack types used for training.\n\n4. Please add more discussion and explanation on the comparison across different XAI techniques. Also, it would be good to add results on more diverse white-box and black-box attacks, so we can see whether G-backprop is still overall better than other XAI techniques.\n\n5. Please revise Section 5 so that XAI techniques such as G-Backprop are not referred to as attacks.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "toDJ5Mw2qB", "reviewer": "Reviewer_FZic", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\nThis paper considers a good range of (non-adaptive) attacks and achieves improved detection results. The difference in XAI features before and after the attacks are also clearly presented with a nice figure. Good job!\n\n**Weakness**\n\nThe main weakness, which is also a critical one, is the lack of analysis against adaptive attacks. As the field of adversarial machine learning have advanced over years, it is imperative to have some basic checks against adaptive attacks, i.e., the attacker knows the defense mechanism in place and considers the defense in its objective. (For example, See https://proceedings.nips.cc/paper/2020/file/11f38f8ecd71867b42433548d1078e38-Paper.pdf).\n\nAs an example, I wonder\n1) Are the changes in XAI features differentiable with respect to changes in the video, and\n2) If yes, can you include that change in the attackers' objective function? E.g., in PGD, the attacker can introduce a negative norm of change in XAI feature in the objective as a regularization. The regularizer can also have a hyperparameter of weight: larger weight on the regularizer -> the attacker wants to keep XAI closer to the original.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Detecting deepfake vs. real videos is a challenge to ML. Existing deepfake detectors are prone to adversarial examples, i.e., carefully calibrated deepfake examples can evade the detection. This paper proposes a defense mechanism by introducing XAI features in the detection process. The mechanism performs well on selected attacks as the XAI features before and after the attacks are significantly different.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please list two possible adaptive attacks to the current defense mechanism and report the robust accuracy against these stronger attacks. Thanks!", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers pointed out that:
- The proposed methodology is innovative and shows significant promise in addressing a critical issue in deepfake detection.
- The new results and discussion after the initial review period are appreciated and they address the technical questions raised. 
- This work provides an interesting perspective for Deepfake attack detection.

The overall sentiment of reviewers is positive.
Decision: Accept (poster)