Amnesia as a Catalyst for Enhancing Black Box Pixel Attacks in Image Classification and Object Detection
Authors: Dongsu Song
Daehwa Ko
Jay Hoon Jung
Venue: NeurIPS 2024 poster (2024)
Keywords: Adversarial attack
Query-based pixel attack
Image classification
Object detection
RL
Primary area: machine_vision

Abstract: It is well known that query-based attacks tend to have relatively higher success
rates in adversarial black-box attacks. While research on black-box attacks is actively
being conducted, relatively few studies have focused on pixel attacks that
target only a limited number of pixels. In image classification, query-based pixel
attacks often rely on patches, which heavily depend on randomness and neglect
the fact that scattered pixels are more suitable for adversarial attacks. Moreover, to
the best of our knowledge, query-based pixel attacks have not been explored in the
field of object detection. To address these issues, we propose a novel pixel-based
black-box attack called Remember and Forget Pixel Attack using Reinforcement
Learning(RFPAR), consisting of two main components: the Remember and Forget
processes. RFPAR mitigates randomness and avoids patch dependency by
leveraging rewards generated through a one-step RL algorithm to perturb pixels.
RFPAR effectively creates perturbed images that minimize the confidence scores
while adhering to limited pixel constraints. Furthermore, we advance our proposed
attack beyond image classification to object detection, where RFPAR reduces
the confidence scores of detected objects to avoid detection. Experiments
on the ImageNet-1K dataset for classification show that RFPAR outperformed
state-of-the-art query-based pixel attacks. For object detection, using the MSCOCO
dataset with YOLOv8 and DDQ, RFPAR demonstrates comparable mAP
reduction to state-of-the-art query-based attack while requiring fewer query. Further
experiments on the Argoverse dataset using YOLOv8 confirm that RFPAR
effectively removed objects on a larger scale dataset. Our code is available at
https://github.com/KAU-QuantumAILab/RFPAR.

Amnesia as a Catalyst for Enhancing Black Box Pixel Attacks in Image Classification and Object Detection
Dongsu Song
Daehwa Ko
Department of Artificial Intelligence
Department of Software Korea Aerospace University daehwa00 1 22100mma11lm11m1m com
Korea Aerospace University raister28730gmail. com
Jay Hoon Jung' Department of Artificial Intelligence Korea Aerospace University jujungekau. ac.kr kr
Abstract
It is well known that query-based attacks tend have relatively higher success rates in adversarial black-box attacks. While research on black-box attacks is ac- tively being conducted, relatively few studies have focused on pixel attacks that target only limited number of pixels. In image classification, query-based pixel attacks often rely on patches, which heavily depend on randomness and neglect the fact that scattered pixels are more suitable for udversarial attacks. Moreover, to the best of our knowledge, query-based pixel attacks have not been explored in the field of object detection. To address these issues, we propose novel pixel-based black-box attack called Remember and Forget Pixel Attack using Reinforcement Learning(FPPAR) consisting of two main components: the Remember and For- get processes. REPAR mitigates randomness and avoids patch dependency by leveraging rewards generated through one-step RL algorithm to perturb pixels. RFPAR effectively creates perturbed images that minimize the confidence scores while adhering to limited pixel constraints. Furthermore, we advance our pro- posed attack beyond image classification to object detection, where RFPAR re- duces the confidence scores of detected objects to avoid detection. Experiments on the ImageNet-IK 1K dataset for classification show that RFPAR outperformed state-of-the-art query-based pixel attacks. For object detection, using the MS- COCO dataset with YOLOv8 and DDQ, REPAR demonstrates comparable mAP reduction state-of-the of-thh--rtrrr art query-based attack while requiring fewer query. Fur- ther experiments on the Argoverse dataset using YOLOv8 confirm that RFPAR effectively removed objects on larger scale dataset. Our code available
Introduction
Deep learning models susceptible adversarial attacks, which involve subtle modifications of input data that are imperceptible to humans but lead to incorrect predictions by the model[[]]. As deep learning technologies become commercialized in the real world, the issue of adversarial attacks has garnered increasing attention.
Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

ImageNet
MS-COCO
Argoverse
Clean
cpr du
person 0L7
Cock
2 Objects
Objects
Perturbation Adversarial
Coil
No Objects
No Objects
0.004
%
Figure 1: Adversarial examples generated by RFPAR. The first column represents images from ImageNet (image classification), the second column from MS-COCO (object detection), and the third column from Argoverse (object detection). Each row represents different condition: the first row shows clean images, the second row shows adversarially perturbed images, and the third row shows the perturbation levels with the ratio of attacked pixels to total pixels. Labels in the images indicate detected objects or classifications, such as "Cock" in ImageNet, "2 Objects" in MS- COCO, and "5 Objects" in Argoverse. In the adversarial row, labels are altered due to perturbations, resulting in misclassification or undetected objects, such as "Coil" instead of "Cock" ImageNet and objects detected in MS-COCO and Argoverse. The perturbation indicates the percentage of pixels attacked in the image. The percentages were 0.004% for ImageNet, 0.027% for MS-COCO, and 0.1 14% for Argoverse.
Adversarial attacks can be broadly categorized into white-box attacks and black-box attacks[[]. In white-box attacks/[3 E. S], attackers devise attack strategies based on internal information about deep learning models, such as training data, gradients of the outputs with respect to the weights, and other details about the learning process for given samples. Conversely, in black-box attacks[Ki, M. attackers can access only limited information such as the probability of the correct prediction for given sample. Given that real-world attackers typically only possess limited information about the model, black-box attacks are more realistic than white-box attacks. In other words, research on black-box attacks and their defenses is crucial in order to develop robust and secure machine learning systems.
Black-box attacks are also categorized into query-based methods [E M. and transfer-based methods[[2).. Query-based attacks are generating adversarial examples by repeatedly querying the victim model with modified images[[3].. Transfer-based attacks involve generating adversarial ex- amples for a surrogate model that successfully deceive another model[[a]]. Transfer-based attacks are highly efficient since they do not require knowledge of the victim model. However, the dis- crepancies in model architecture, training data, and training methodologies between the surrogate and victim models often result in a lower success rate for these attacks compared to query-based attacks[L3).. Conversely, although query-based attacks achieve higher success rates, they require significant number of queries to the victim model. Therefore, reducing the number of queries in query-base based attacks is critical issue.

---

Forget Process
Remember Process
Memory
If Converge
Best Reward maxtr",
Perturbated Image
Training (not Converge)
Best Rewarded
image
Agent, Memory
Reward
Initialization
Write
Fooled
A
Victim
Clean
Adversarial Agent
Erase
Model
Adversarial Image
Image
Figure 2: The model architecture of RFPAR: the Remember and Forget process. During the Re- member process, the RL model generates perturbed images and corresponding rewards. Memory compares these with previously stored values and retains only the highest reward and its associated image. Once the rewards converge to a certain value, the Forget process starts and resets the RL. agent and memory, then reintroduces the perturbed images that gained the highest reward to the Remember process. The process continues until an adversarial image is generated or a predefined number cycles reached, at which point it terminates.
The pioneering pixel attack method, OnePixel[8], employed Differential Evolution (DE) to generate adversarial images. An advanced approach, ScratchThat[E], used DE create curves and applied parametric model to perturbations, reducing parameters and improving performance. A more recent study, PIXLE[[I]) enhanced query efficiency and attack success rate by using simple algorithm instead of DE. Briefly, PIXLE generates adversarial images by selecting arbitrary patches in a clean image and applying the brightness of these pixels to others. Although this method improved perfor- mance, it ignored the fact that pixcls independent of cach other due to its reliance on patches and exhibited inefficiencies stemming from randomness brightness mapping. The previous study, PatchAttack[IISS utilized RL model embed textures in specifie regions of the clean image, dis- covering vulnerable patches and reducing randomness, which significantly decreased the number of queries and improved attack success rates. Unfortunately, this method still depended patches, requiring at least 3% of the image area to be attacked.
Query- based attacks in object detection are more challenging than those in image classification. The first query-based attack in object detection, PRFA[[6]) generated adversarial images using parallel rectangle flipping strategy. Recent research, GARSDC[LZ[] employed genetic algorithm create adversarial images, improving optimization efficiency by using adversarial examples generated from transfer-basee based attacks the initial population. Query-based attacks black-box models inher- ently challenging, and targeting only few pixels is even more difficult to study. To the best of our knowledge, pixel attacks have been limited white-box transfer attack methods[ITR, I19, 20, 21]]. In this study, we extend our proposed attack from image classification to object detection, introduc- ing the first query- based pixel attack. Experiments show that method achieves comparable mAP reduction on YOLO to state-of-the-at methods while significantly reducing the number of queries, demonstrating its effectiveness in object detection.
In this study, we introduce a novel method called the Remember and Forget Pixel Attack using Reinforcement Learning (RFPAR). Briefly, in the Remember process, the clean image is initially taken as input by the RL agent, and the loss function is optimized. During this optimization, the highest reward and its corresponding perturbed image are stored in memory. If the highest rewards do not change for while, we define this as the convergence rewards. Once the rewards converge, the Forget process is initiated, resetting the RL agent and memory to forget previous information. After resetting, the stored image is fed as input to the RL agent, and the Remember process begins again. Extensive experiments demonstrate that our attack is effective for image classification and successfully extends to object detection.
In summary, our main contributions are:
3

---

We propose a novel query-based black-box box pixel attack consisting of the Remember and Forget processes. Our approach outperformed state-of-the-ee attacks on the ImageNet-IK classification task, achieving average attack success rate improvement of 12. %2 while reducing the number of queries by 26.0% and the modified L.p norm by 41.1%.
We advance query-based pixel attacks from image classification to object detection, in- troducing the query-based pixel attack. Our experiments demonstrate that proposed method effectively compromises object detection systems. It achieves an average mean Average Precision (mAP) reduction of 0.29 in YOLO, comparable to state-of-the-art query- based attacks, while reducing the number of queries by 52.8%. To the best of our edge, the proposed method is the first black-box query-based pixel attack for object detec- tion.
To evaluate performance on larger scale dataset, we conducted experiments using YOLOv8 as the victim model on the Argoverse-I. 1.1 validation dataset. We also achieved high removal rate of detected objects above 0.9 in Argoverse, similar to our results in MS- COCO. The results demonstrate that our proposed method effectively reduces the number of detected objects in images with a resolution of x 1200. Additionally, examining the adversarial perturbation results of RFPAR on Argoverse and MS-COCO, we observe that successfully reduces the number of detected objects while attacking only very small areas 0. % and 0.02%, respectively.
2 Remember and Forget Pixel Attack Using Reinforcement Learning
In this section, we introduce our proposed method. In Section 2. we define the problem math- ematically. Section 2 details the Remember process, which is the internal iterative structure of our algorithm, including agent, environment, and memory. Finally, we explain the Forget process, which serves as the external iterative structure in Section M
2.1 The Problem Formalization
We consider image classifier as f: CXHH + R. where C. H, and W represent the channel, height, and width a given sample x, respectively, and c denotes the number of classes. The classifier f computes probabilities for each class for sample the probability of a sample x being predicted as the I-th class. The prediction of the classifier can be expressed arg fixx))(xx) For an image classification attack, the objective is to minimize pixel perturbation ó such that the classifier's prediction for an input æ diverges from its true label y. This problem can be formalized
Furthermore, ft(x:) is defined as
min 5
arg max f1(x + 56 # y
s.t. 7 æ|1o
|10100 Se e EE
Here, E (1. and Il-Ilo denotes the attack level and the Lo norm. E is the perturbed image, defined as x x--d 8. Notably, the attack levels are positive integers, which implies that the perturbations at the pixel level.
Similarly, the object detector is defined by the function f C - BB6 where represents the maximum number of objects that can be detected by the model. The information about each bounding box location, object's class, and confidence score (indexed by 4)) is encapsulated in dimensional vectors. To prevent the attack from targeting objects that are incorrectly predicted, establish a confidence threshold of 0.5. If objects surpass this threshold from among candidates, then only these objects are classified as detected. Furthermore, f% (x) indicates the confidence score that the model identifies the o-th object as belonging to the predicted class from a sample x. In this context, the number nxx) of detected objects from sample is Σ. 1 fg (a) 0.5), where signifies the indicator function that takes the value one if the inequality holds and the value zero otherwise.
The goal of the object detector attack to reduce the number of detected objects by modifying the minimum number of pixels, which is formally expressed as:

---

min max (n(x) - n(())
6
(2)
s.t. 10010 S, E, E Z+.
where e denotes the attack level and denotes the perturbed image. Therefore, our objective is to eliminate bounding boxes from the detection model by iteratively accumulating pixel attacks. We address this problem by combining one-step REINFORCEJ23] with our approach.
2.2 Remember Process
Agent. We construct an environment where an attacking agent interacts to generate adversarial images. The agent's policy utilizes CNN- architecture, where given sample R xxW the agent observes the image and takes actions to determine the location (X, coordinates) and brightness (R, G, B) modify, then generates perturbed images. We define two types of actions for brightness: "Write" and "Erase." The "Write" " action overwrites the pixel with the maximum brightness, while the "Erase" action sets the brightness to zero. This configuration chosen because, based on our experience, the attack success rate is higher when applying maximum changes to the pixels. Figure shows the adversarial images generated by the agent. The agent generates the actions through random sampling of normal distributions, where the means and standard deviations trained by the neural networks. The of actions Ar, where represents the training epoch of RL, contains N subsets corresponding to the number of attack pixels in each Remember process. These subsets are composed of the X, Y coordinates, and brightness values for each channel. The set A, is defined as {aj a22, ag. ... .0042 ... ai 02 ag ......... ..V , where a₁ and represent the X
and Y coordinates, respectively, and to ac+2 represent the brightness values for each channel. For the "Write" action, the brightness values set to the maximum value, whereas for the "Erase" action, they are set to O. For each pixel, the perturbed image T is generated as follows:
(al+2
if --j=d of and k = az
F.s.A
[T otherwise
where i, j. and k are indices for channel, height, and width, respectively. In other words, Fij.k represents the brightness at position (j, k). Adding a pixel to the image is repeated for d from 1 W. Hence, the image is perturbed by / pixels. The equation describes the generation of т by repeatedly altering the brightness of each channel at the position (a1, 02) in the given sample r. The agent trains by using the gradient of the reward and the log probability of the sampled actions.
The environment evaluates the image generated by the agent and assigns a reward. The reward is defined as:
Environment.
Σ (z) (T, !(Z).
where n represents the number of detected objects in the image for object detection, while ((z) (n(x)) - nT)) signifies the number of removed objects after the adversarial attack. lo is the index for the confidence score of the detected object. Hence, the reward is defined as the sum of the differences in confidence scores for each object plus the number of objects removed.
For classification, lo is the index for the correct class, and is set to 1. ?(I)) is set to 1 if the adversarial image generation is successful and 0 otherwise. In essence, the reward is calculated as the sum of the differences in the probability of the correct class between the perturbed and original image, along with an additional component indicating whether the model successfully created an adversarial example.
Memory. The role of memory is to save the best reward value and its corresponding perturbed im- ages. The stored information is also used to determine when the Forget process should start. Without memory, RL models tend to identify universally shared vulnerabilities in the clean images provided to the victim model. In contrast, common vulnerabilities. To minimize unnecessary queries that converge on such vulnerabilities, we have incorporated memory concepts into the RL approach.
objective is generate adversarial attacks regardless of these
In our approach, memory stores the maximum reward values y* and their corresponding perturbed images F" by selectively saving the higher reward r max(r", rr), where denotes the rewards
5

---

Table The results of adversarial attacks on the ImageNet dataset. Each score represents the mean success rate of the attack, mean Lo norn and mean the number of queries. In terms of the success rate, a higher value signifies better performance, whereas for the Lo norm and the number of queries, lower values are indicative of superior performance. The best method is highlighted in bold.
Model
Test accuracy
Attack
Succes rate t
Lo +
Query
OnePixel
9.3
1453
ScratchThat(B)
40.9 %
420
9418
V[T-B|24]
81.07
PiXIe[[LI]
51.4
286
728
RFPAR(Ours)
64.1 %
211
613
OnePixel
8.1 %
15
5100
ScratchThat(8]
38.1 %
95
1400
ResNeXt550(25)
77.62
Pixle[[LI]
89.1 %
538
663
RFPAR (Ours)
95.3 %
138
442
OnePixel [[]]
12.3 %
15
1358
ScratchThat((]
60.6
427
8653
RegNerX-32GHIP
80.62
Pixle[[[I]
73.7 %
276
705
REPAR(Ours)
88.4 %
164
484
OnePixel
14.1 %
15
1248
ScratchThat[]1
60.6
425
8367
DenseNet16l 27
77.14 %
Pixle[[LI]
82.3 %
243
625
RFPAR(Ours)
91.7 %
152
464
OnePixel
14.2
1128
ScratchThat(A
65.3 %
425
8828
MNASNet [28]
73.46 %
Pixle
83.7
240
607
REPAR(Ourg)
95.0 %
150
442
OnePixel
8.1
15
1461
ScratchThat[B]
51.8 %
420
9293
MobileNet-V31l [3
74.04
Pixle
69.6
306
769
RFPAR(Ours)
86.6 %
213
596
given by the environments during the t-th training epoch. After training each epoch of data, the algorithm checks whether the reward values have bounded. We define the rewards bounded if the following condition is satisfied:
Ft
(5)
rz
where 1) signifies the bound threshold. This equation indicates that the rate of increase in the reward stored in memory less than n, The convergence of rewards is defined as the rewards being bounded for certain period, denoted as T. Both 7) and T are hyperparameters. If the reward converges, the Remember process ceases and the Forget process starts.
2.3 Forget Process
The goal of the Forget process reset the trained RL model and its memory, and feed the image It as a new input for the reset RL model. Additionally, the maximum Lo increase for the reset RL model, is determined by the number reward convergences, attack pixels, and channels. This process is implemented to prevent the agent from overfitting, which can hinder effective exploration of new inputs. The impact of memory and initialization discussed in Section
Experiments
Section B.I details the dataset, evaluation metrics, victim models, and hyperparameters used in our experiments. In Scction 32 evaluate our proposed attack image classification by comparing

---

it with previous attack methods. Section + compares the performance of our method on object detection, varying the attack dimension (a 0.01 to 0.05), and compares the results with other query-based attacks. In Section B.4. we conduct experiments on the Argoverse dataset, which has larger image dimensions, and discuss the findings. Finally, Section (.5 presents an ablation study on the memory and initialization components we introduced. Additional experimental results can be found in Appendix and D
3.1 Experimental Details
Datasets, Metrics and Hardware. For image classification, we use the validation dataset from ImageNet- [[BO]. To reduce computational costs, we extract one correctly classified image per category from the victim model, resulting in total dataset of 1000 images for adversarial attack attempts. We evaluate our methods with respect to different victim models by calculating the success rate, Lo norm, and the number of queries. The success rate represents the percentage of successful adversarial attacks out of the 1000 images, with higher values indicating better performance. The Lo norm refers to the number of non-zero elements in perturbation 6, with lower values indicating better performance. The number of queries indicates how often the victim model is queried to generate an adversarial example, with fewer queries indicating better performance. The ATA (ATtacked Area) refers to the proportion of pixels in the image that were attacked, a lower value indicates fewer changes. For object detection, 1 I validation set[32]]. To facilitate comparison with PRFA [T6]] and GARSDC [[Z]], we use mAP to evaluate the attacks. The mAP is calculated as the average over thresholds ranging from IOU 0.5 to 0.95, Additionally, RM indicates the average percentage of objects removed from the clean image, while RD refers the decrease in mAp. Both a lower mAP and higher RM indicate greater success. Lastly, we used an AMD Ryzen 9 5900X, RTX 3090TI, and 64.0GB of RAM, running on Windows 11 with CUDA version 12.1.
the 2017 validation set from the MS-COCO [3taset[33]] and
Victim Models. For image classification, we select six pre-trained models on the PyTorch platform as victim models: VIT[24], ResNeXt50(255), RegNetX-32GFIDR DenseNet161 [27], MNASNe((28), and MobileNNet-33[25
We compare the performance of our attack with OnePixel [8], ScratchThat(5], and Pixle[[[]]. For object detection, we use the pre-trained YOLOv8n model from the YOLOv8 [22) platform and the pre-trained DDQ DETR-4scale model from the MMDetection platform.
Hyperparameter. Our attack method utilizes four hyperparameters: the maximum number of iterations, the pixel attack rate a, the bound threshold 7, and the duration T for maintaining the convergence condition. In the Remember process, a is hyperparameter that determines the number of pixels to attack, proportional to the image size. The number of pixels A to be attacked is defined as (H + W)/2 a. By default, we set the maximum number of iterations to 100 and 17 to 0.05. For image classification, we use T = 3 and a = 0.01. For object detection, we experiment with T = 20 and a values ranging from 0.01 to 0.05.
3.2 Evaluation of Classification Attacks
Table M presents a performance comparison of various adversarial attack methods on different victim models for image classification. RFPAR consistently achieves the highest success rate, significantly outperforming the other three attack methods. For instance, for the VIT model, RFPAR achieves success rate of 64.1%, compared to OnePixel's 9.3%, SeratchThat's 40.9%, and Pixle's 51.4%. The trend is similar for other models, with RFPAR showing substantial improvements in success rate. Regarding the Lo norm, which measures the sparsity of the perturbations, RFPAR generally achieves a lower Lo norm than ScratchThat and Pixle but higher than OnePixel. For example, in the case of ResNeXt50, RFPAR has an Lg norm of 138, compared to OnePixel's 15, ScratchThat's 95, and Pixle's s 538. While OnePixel has the lowest Lo norm, its success rate is significantly lower than RFPAR's, s, indicating a offe-ooff between perturbation sparsity and attack effectiveness. In terms of the number of queries, RFPAR requires fewer queries than the other methods, except for OnePixel in some cases. This demonstrates that REPAR more efficient in terms of query cost, which is crucial for practical adversarial attacks. Overall, RFPAR exhibits superior performance across all victim models in terms of success rate while maintaining competitive To norms and requiring fewer queries compared to other methods, making it an effective and efficient approach.
7

---

Table Attack Results on Object Detection Models. The subseripts after RFPAR denote pixel attack rate, a. RM indicates the average percentage of objects removed from the clean image. Lo represents the average ||o||o- Query denotes the average number of queries made to the victim model. Higher RMM. lower mAP, lower Lo. and lower Query values indicate better performance.
YOLOv8
DDQ
Attacks
RM 1
mAP
Lo
Query &
RM 1
mAP J
Lo 1
Query +
clean
0.398
0.376
RFPAR0.01
0.65
0.218
521
1403
0.60
0.125
391
1450
RFPAR0.02
0.70
0.187
955
1427
0.73
0.103
787
1690
RFPARo.00 0.03
0.75
0.151
1459
1374
0.76
0.075
1074
1512
RFPAR
0.76
0.150
1814
1348
0.80
0.061
1429
1457
RFPAR Ro.05
0.91
0.111
2043
1254
0.83
0.054
1780
1528
3.3 Evaluation of Object Detection Attacks
Attacking object detection models is more challenging than attacking image classification models because there are more objects to consider in the object detection task. More pixels need to be modified, adjusted by a from 0.01 0.05, to deceive the victim models. 'abble E compares the per- formance different a values of the REPAR method on two object detection models, YOLOv8 and DDQ. The RM rate for YOLOv8 increases from 0.65 (RFPARo.00 to 0.91 (RFPARo.05) and for DDQ from 0.60 to 0.83, indicating that stronger attacks remove more detected objects. The mAp also decreases from 0.218 to 0.111 for YOLOv8 and from 0.125 to 0.054 for DDQ. At a = 0.05, our attack success- fully reduced the mAP by average of 0.301 and achieved a RM of 0.87. The number of queries remains relatively stable, ranging from 1254 to 1427 for YOLOv8 and from 1450 to 1690 for DDQ, suggesting consistent query despite increasing perturbation intensity. Overall, the results indi- cate that the REPAR method is highly effective in generating adversarial attacks on object detection models, balancing perturbation sparsity, and attack effectiveness while maintaining query efficiency.
Table 3: Comparison to other meth- ods. RD means reduction in mAP.
YOLO
Attacks
RD f
Query
PREA
0.21
2949
GARSDC
0.29
2691
RFPAR
0.29
1270
To demonstrate the effectiveness of our method, we compared it with other query-based black-box attacks. Table P shows the performance of three different attack methods PRFA, GARSDC, and RFPAR - on the YOLO object detection model. this table, RD refers to the decreased mAP value, and Query indicates the average number of queries. The RFPAR method shows strong performance by achieving the highest RD (tied with GARSDC) and requiring the fewest queries. This indicates that RFPAR is not only effective in reducing the YOLO model's performance but also efficient in terms of the number of queries needed to achieve this reduction. GARSDC also demonstrates high effectiveness with the same reduction RFPAR but requires more than twice the number of queries. Overall, REPAR stands out as the most balanced and efficient attack method in this comparison.
3.4 Experiments on a Larger Scale Data
To verify the effectiveness of our proposed method on larger dimensions 1920x 1200, we randomly selected one video sample from the Argoverse dataset and conducted experiments using YOLOv8. The experi- mental results are presented in Table El The RM achieved 0.94, indicating successful reduction in the number of detected objects. Argoverse achieved a RM of 0.94, similar to the RM observed for MS-COCO. The ATA for these datasets was 0. 1% and 0.02%, re-
Table 4: Comparison on dataset. ATA means the ratio of altered pixels to the image size.
YOLO
Datasets
RM f
RD
ATA 1
Query +
MS-COCO Argoverse
0.91
0.29
0.02 %
1270
0.94
0.05
0.10 %
1906
spectively, indicating that only a very small portion of the image area was attacked. However, the

---

mAP did not decrease significantly as in previous experiments. This discrepancy can be explained by considering that RFPAR primarily reduces the number of objects detected. a particular class has many objects, reducing their number may not significantly impact the overall mAP due to the presence of other classes. summary, while RFPAR successfully removes objects the larger Argoverse dataset, its effectiveness in reducing mAP is limited in datasets with high density of objects specific classes.
3.5
A Alltion study
100
Success Rate [96]
RSPAR4
AFPAR
60-
40-
20-
ResNeXt
RegNetX
DenseNet
MNASNet
Mobile-V3
Figure 3: Ablation study. The x and y axes show different victim models and the attack success rate, respectively. The notation signifies the inclusion of the initialization step in the Forget process, and M denotes that the Remember process incorporates memory.
In this section, we analyze the impact of Initialization (I) and Memory (M) on our model's perfor- mance. If Initialization is ablated in the Forget process, the Agent is not reinitialized and retains information from the previous Remember process. On the other hand, if Memory is ablated, the Agent's reward, instead of the reward stored in Memory, serves lation experiments under similar query conditions and present the results in Appendix G As shown in Figure E, REPAR denotes the baseline state without and M, while RFPAR. indicates the inclu- sion of specific processes. Comparing RFPAR and RFPAR M. it is evident that the introduction of memory significantly enhances the attack success
bound condition. We conduct ab-
This result suggests that the RL method ben- cfits from storing the highest-reward images of restricted pixels during each Forget process. When comparing RFPAR, and RFPAR we observe that initialization prevents RL model from overfit- ting to specific patterns and escaping local optima, thereby improving performance. Conversely, the comparison between RFPAR and RFPAR, indicates that Initialization alone, without memory, has a negligible impact. This finding implies that RL model without memory fails to generate meaning- ful adversarial attacks. In summary, memory supports RL model in generating effective adversarial attacks, while Initialization prevents overfitting and enhances overall performance.
4 Conclusion
In this paper, propose the Remember and Forget Pixel Attack using Reinforcement I Learning (RF- PAR) for attacking neural network models with limited pixels. Traditional pixel- attacks have been confined image classification, but method extends this approach to include object detec- tion as well. For image classification, we compared the performance of REPAR against OnePixel, ScratchThat, and PIXLE across six victim models using the ImageNet- 1K dataset, and RFPAR demonstrated superior performance.
object detection, we evaluated RFPAR on the MS-COCO dataset using YOLOv8 and DDQ models, comparing it with PRFA and GARSDC attacks. RFPAR achieved performance comparable to the ofate-of-the-eee query-based attack GARSDC, while reduc- showed that RFPAR
ing the number of queries by 52.8%, proving its efficiency. Additionally, is capable of performing pixel attacks on larger datasets, specifically the Argoverse dataset with di- mensions, surpassing the sizes of the ImageNet and MS-COCO datasets. Our findings may enable malicious individuals to compromise real-world Al systems. Consequently, research on defenses against adversarial attacks is becoming increasingly important.
Broader Impacts. Defects in camera sensors, such hot pixels or dead pixels, can impact im- age quality and degrade the performance of neural network models. Our approach mimics these camera defects. In this paper, RFPAR simulates real-world issues by replacing specific pixels with

---

values of either zero or one, inducing incorrect predictions by the neural network. Since these types of perturbations can occur in practice, is crucial for neural networks to be robust against them. However, research on pixel-based T- attacks is limited compared to other types of attacks. Our approach helps analyze model vulnerabilities with respect to both adversarial attacks and real-world scenarios, contributing to the development of more robust neural networks that can withstand such defects. Additionally, the phenomenon where the prediction changes with only a small number of pixel modifications that do not alter the overall meaning can be considered an anomaly in artificial neural networks. This type of attack provides important insights into understanding the limitations of neural networks.
Limitations. In this work, the pixel values are either zero or one. While the meaning remains unchanged, this can still make it noticeable in certain cases. Additionally, the Forget process is quite simple. The time complexity of RFPAR is worse than that of other pixel attacks. However, at the ImageNet scale, RFPAR outperforms others in speed. This result is presented in Appendix HL In future work, we will apply meta-learning to the Forget process and aim to reduce not only Lo. but also Loc.
Negative Impacts. In applications like defective product detection [G_]] and disease prediction systems [35]], adversarial attacks could degrade product quality or lead to incorrect diagnoses, which may have serious, or even fatal, consequences. Our proposed approach increases the effectiveness of query-based black attacks, making them more applicable to real-world scenarios. As result, vision Al systems may face significant threats to their functionality and reliability. Therefore, it crucial for these systems to proactively identify potential vulnerabilities and implement robust defenses.
Mitigation of Risks. Our method requires an average of over 1000 queries to successfully deceive an object detection model. Similarly, shown in Table M in the Appendix IEL transformer- based models also require an average of over 1000 queries to achieve a high success rate. If we limit the number of queries around 1000 in short period of time, our method can easily defend the model. For CNN-based models, since fewer queries are needed, limiting the queries to 400 can effectively defend the model. Additionally, according the attack results adversarially trained models shown in Table X in the Appendix EL adversarial training effectively reduces the attack success rate and increases the number of queries needed. Therefore, by adversarially training the models and appropriately limiting the queries, this attack can be defended against.
Acknowledgments and Disclosure of Funding
This research was supported by the Korea Institute for Advancement of Technology (KIAT) through the Ministry of Trade, Industry, and Energy (MOTIE) of the Korean government (Grant No. P0017124, HRD Program for Industrial Innovation), the Korea Agency for Infrastructure Technol- ogy Advancement (KAIA) under the Ministry of Land, Infrastructure, and Transport (Grant No. 563644)) and the Ministry of Education and the National Research Foundation of Korea as part of the "Convergence and Open Sharing System (NCCOSS)" Project.
References
[1] Zaremba W. Sutskever I. Bruna J. Erhan Goodfellow J. Szegedy, C. and Fergus. Intrigu- ing properties of neural networks. In International Conference on Learning Representations, ICLR, 2014.
[2] Muku S. Tulasi A. Bhambri, and B. Buduru, A. A study of black box adversarial attacks in computer vision. CoRR, abs/1912.01667, 2019. http://arrv..nn//b//9911.00011:
[3] Shiens J. Goodfellow, I. and C. Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, ICLR, 2015.
[4] Rice L. Wong, and Z. Kolter. Fast better than free: Revisiting adversarial training. In International Conference Learning Representations, ICLR, 2020.
10

---

[5] Makelov A. Schmidt L. Tsipras D. Madry, A. and A. Vladu. Towards deep learning models re- sistant adversarial attacks. In International Conference on Learning Representations, ICLR. 2018.
[6] Dong Y. Pang T. Su H. Cheng, S. and J. Zhu. Improving black-box adversarial attacks with transfer-based prior. In Advances in Neural Information Processing Systems, NeurIPS, 2019.
[7] Wang S. Shi, Y. and Y. Han. Curls and whey: Boosting black-box adversarial attacks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2019.
[8] Vargas D. V. Su, J. and K. Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828-811, 2019.
[9] Hitaj B. Ciocarlie G. F. Jere, M. and F. Koushanfar. Scratch that! an evolution-based adversarial attack against neural networks. CoRR, abs/1912.0231e 2019. http://arriv.or//abb/911..0000
[10] Babu A. R. Mousavi S. Ghorbanpour S. Gundecha V. Gutierrez R. L. Guillen A. Sarkar, and A. Naug. Reinforcement learning based black-box adversarial attack for robustness im- provement. In IEEE International Conference on Automation Science and Engineering. CASE, 2023.
[1]] Dántoni D. Nicolosi A. Pomponi, and S. Scardapane. Rearranging pixels is a powerful black-bou attack for RGB and infrared deep learning models. IEEE Access, 11:1 11:11298-111 11306, 2023.
[12] Wu W. Zhang J. Deng, Y. and Z. Zheng. Blurred-dilated method for adversarial attacks. In Advances in Neural Information Processing Systems, NeurlPS, 2023.
[13] Wu B. Fan Y. Liu L. Li Z. Feng, Y. and S.-T. Xia. Boosting black-box attack with partially transferred conditional adversarial distribution. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2022.
[14] Chen X. Liu C. Liu, Y. and D. Song. Delving into transferable adversarial examples and black- box attacks. In International Conference on Learning Representations, ICLR, 2017.
[15] Kortylewski A. Xie C. Cao Y. Yang, C. and A. L. Yuille. Patchattack: A black-box texture- based attack with reinforcement learning. European Conference on Computer Vision, ECCV, 2020.
[16] Wu B. Fan Y. Wei X. Liang, S. and X. Cao. Parallel rectangle flip attack: A query- black-box attack against object detection. In /EEEE/CV International Conference on Computer Vision, ICCV, 2021.
[17] Li L. Fan Y. Jia X. J. Wu B. Liang, S. and X. Cao. A large-scale multiple-objective method for black-box attack against object detection. In European Conference on Computer Vision, ECCV, 2022.
[18] Zolf A. Demetrio L. Biggio B. Shapira, A. and A. Shabtai. Phantom sponges: Exploiting non-maximum suppression to attack deep object detectors. In /EEE/CVF Winter Conference on Applications of Computer Vision, WACV, 2023.
[19] Li C. Wen S. Han Q.-L. Nepal S. Zhang X. Wang, D. and Y. Xiang. Daedalus: Breaking nonmaximum suppression in object detection via adversarial examples. IEEE Transactions Cybernetics, 52(8):7427-7440, 2022.
[20] He F. Huang X. Chen, S. and K. Zhang. Relevance attack on detectors. Pattern Recognition, 124: 10849, 2022.
[21] Zhang Li, D. and K. Huang. Universal adversarial perturbations against object detection. Pattern Recognition, 110:107584, 2021.
[22] Córdova Esparza D. M. Terven, J. R. and J.-A. Ronero-González. A comprehensive review of YOLO architectures in computer vision: From yolovl to yolov8 and YOLO-NAS. Machine Learning and Knowledge Extraction, 5(4):1680-1716, 2023.

---

[23] McAllester D. A. Singh S. Sutton, R. S. and Y. Mansour. Policy gradient methods for reinforce- ment learning with function approximation. In Advances in Neural Information Processing Systems, NeurlPS, 1999.
[24] Beyer L. Kolesnikov A. Weissenborn D.-Zhai X. Unterthiner T. Dehghani M. Minderer M. Heigold G. Gelly S. Uszkoreit J. Dosovitskiy, A. and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, ICLR, 2021.
[25] Girshick R. B. Dollár P. Tu-7.. Xie, S. and K. He. Aggregated residual transformations for deep neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2017.
[26] Kosaraju R. P. Girshick R. B.-He K. Radosavovic, I. and P. Dollár. Designing network design spaces. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, CVPR, 2020.
[27] Liu Z. van der Maaten I.. Huang, G. and K. Q Weinberger. Densely connected convolutional networks. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, CVPR, 2017.
[28] Chen B. Pang R. Vasudevan V.-Sandler M. Howard A. Tan, M. and Q. V. Le. Mnasnet: Platform- neural architecture search for mobile. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2019.
[29] Pang R. Adam H. Le Q. V. Sandler M. Chen B. Wang W. Chen L.-C. Tan M. Chu G. Va- sudevan V. Howard, A. and Y. Zhu. Searching for mobilenetv3. In IEEE/CVF International Conference Computer Vision, ICCV, 2019.
[30] Dong W. Socher R. Li L.-J. Li Deng. and Fei-Fei. Imagenet: large-scale hierarchi- cal image database. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2009.
[31] Maire M. Belongie S. J.-Hays J. Perona P. Ramanan D. Dollár P. Lin, T.-Y. and C. L. Zitnick. Microsoft COCO: common objects in context. In European Conference on Computer Vision, ECCV, 2014.
[32] Wang Y.-X. Li, M. and D. Ramanan. Towards streaming perception. In European Conference on Computer Vision, ECCV, 2020.
[33] Wang X. X.-Wang Pang J.-Lyu C. Zhang W. Luo P. Zhang, S. and Chen. Dense distinct query for end-to-end object detection. In IEEE/CVF Conference Computer Vision and Puttern Recognition, CVPR, 2023.
[34] Ciuti G.-Milazzo M. Chiurazzi M.-Roccella S. Oddo C. M. Czimmermann, T. and P. Dario. Visual-based defect detection and classification approaches for industrial applications A SUR- VEY. Sensors, 20(5):145) 2020.
[35] Linda Wang. Zhong Qiu Lin, and Alexander Wong. Covid-net: tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images. Scientific Reports, 10:19549, 2020.
[36] Yu Y.JJao J. Xing E. P. El Ghaoui L. Zhang, H. and M. I. Jordan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the International Conference on Machine Learning, ICML, 2019.
[37] Hu H.-Lin Y. Yao Z. Xie Z. Wei Y. Ning J. Cao Y. Zhang-Z. Dong L. Wei F. Liu, Z. and B. Guo. Swin transformer V2: scaling up capacity and resolution. In IEFF/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2022.
[38] Cord M.-Douze M. Massa F. Sablayrolles A. Touvron, H. and H. Jégou. Training data-efficient image transformers distillation through attention. In Proceedings of the International Con- ference Machine Learning, ICML, 2021.
12

---

[39] Chi Y.- Y. Lei Z. Zhang, S. and S. Z. Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2020.
[40] Su W.-Lu L. Li B. Wang X. Zhu, X. and J. Dai. Deformable DETR:
deformable transformers for end-to-end object detection. In International Conference on Learning Representations, ICLR, 2021.
[41] Wu D.-Wang Y. Guo Y. Mo, Y. and Y. Wang. When adversarial training meets vision trans- formers: Recipes from training to architecture. In Advances in Neural Information Processing Systems, NeurIPS, 2022.
[42] Basart S.-Mu N. Kadavath S. Wang Dorundo E. Desai R. Zhu T. Parajuli-S. Guo M. Song D. Steinhardt J. Hendrycks, D. and Gilmer. The many faces of robustness: A critical analysis of cito-f-distribution generalization. In /EEE/CVF International Conference on Computer Vision, ICCV, 2021.
13

---

A Related Work
Adversarial Attack. Adversarial attacks manipulate clean images with imperceptible modifica- tions to fool Deep Neural Networks (DNNs) into making incorrect predictions. These attacks are broadly divided into black-box and white-box attacks. White-box attacks leverage detailed informa- tion about the target model, including training data and gradients, to craft adversarial images. In contrast, black-box attacks, which do not rely on any internal information about the victim model, are divided into transfer-based and query-based strategies. Transfer-based attacks create adversarial images using a surrogate model, aiming for these examples to also be effective against the target model. Query-based attacks iteratively modify clean images and query the victim model, using the resulting confidence scores to refine the attack. Typically, attack strategies are evaluated using the Lp norm to restrict the perturbation to remain imperceptible to humans| [3 9 1. [6].
Black-box Pixel Attack in Image Classification. Unlike other metrics, the Lo norm, also known as pixel norm, targets only a small subset of pixels in clean image rather than attacking all of them. The pioneering pixel attack method, OnePixel[8], employed Differential Evolution (DE) to generate adversarial images. An advanced approach, ScratchThat[]], used DE to create curves and applied parametric model to perturbations, reducing parameters and improving performance. A more recent study, PIXLE([]]], enhanced query efficiency and attack success rate by using simple algorithm instead of DE. Briefly, PIXLE generates adversarial images by selecting arbitrary patches in a clean image and applying the brightness of these pixels to others. Although this method improved per- formance, it ignored pixel independence due to its reliance on patches and exhibited inefficiencies stemming from randomness in brightness mapping. The previous study, PatchAttack([IS), utilized RL to embed textures in specific regions of the clean image, discovering vulnerable patches and reducing randomness, which significantly decreased the number of queries and improved attack suc- cess rates. Unfortunately, this method still depended on patches, requiring at least 3% of the image area to be attacked. Our research focuses on eliminating patch dependency by attacking individual pixels and reducing randomness through RL. Extensive experiments demonstrate that our proposed attack outperforms the state-of-the -frrt methods in both query efficiency and attack success rate.
Query-based Adversarial Attack in Object Detection. Adversarial attacks in object detection are more challenging than those image classification. The first query-based attack in object de- tection, PRFA [I6]] generates adversarial images using a parallel rectangle flipping strategy. Recent research, GARSDCILZZ), employs genetic algorithm to create adversarial images, improving opti- mization efficiency by using adversarial examples generated from mansfer-based attacks as the initial population. We extend our proposed attack from image classification to object detection. Experi- ments show that our method achieves a comparable mAP reduction on YOLO to ofate-of-the-art methods while significantly reducing the number of queries, demonstrating its effectiveness in object detection.
B Theoretical Insight.
We initially used a multi-step REINFORCE approach but identified issues, leading us to propose the Forget and Remember processes using one-step REINFORCE. Generating adversarial exam- ples with multi-step REINFORCE involves the objective function U = E |> >"- R[s,, axl mo]] where 7 is the discount factor, 8, is the image step I, 0g is the action 8t, and the reward is R[st. og [Ta] = (e.) ($1+1), where fe.,, is the confidence score of the true label y. Here. a single pixel perturbation. We find that significant oscillations can be observed in the objective function. Let 7" be the minimum number of steps to create an adversarial example. The sequence of pixels does not matter, leading to variations in the value of the objective function due to different orderings of at. Thus, for is E f0. 1,2...... rT} and i; A ig. the optimal objective function value .....
U = E 1>x1 R a;, Tel]. with r"! permutations. This complicates training and increases the queries and La. To address this, we proposed the Forget and Remember process using one-step REINFORCE. Pixel perturbations at 7" are defined as A,- E. at. By the intermediate value theorem, there exists C' in [r. + A..] such that fo.)(x) > (o) > (2 A,.). We propose a Forget and Remember process using one-step REINFORCE to iteratively find this C, assuming C' E {i + op, I + a1, x + a. This one-step approach avoids the fluctuations of multi-step methods, offering better query efficiency and lower Ly-
such
14

---

C Experimental Results on Image Classification
In this section, we present experimental results that could not be included in the main The
results of attacking the ResNeXt50 model on the ImageNet-1K KK dataset are shown in Fig P. The parameters for the attack were set as follows: the maximum number of iterations was 100, a was 0.01, was 0.05, and the duration 7 was 3.
Original Image
Adversorial Image
Original Image
Adversaria Image
Bock Goon
fixes
tarling
611
PISt
Figure 4: Adversarial examples generated by RFPAR on the ImageNet dataset. The "Original Image" is the original unaltered image, the "Delta" represents the difference between the Original Image and the Adversarial Image, and the "Adversarial Image" is the image with the altered predic- tion. The predicted labels are shown below the Original Image and the Adversarial Image.
15

---

D Experimental Results on Object Detection
In this section, we present the experimental results for Object Detection from the main text. The results of attacking the YOLOv&n model on the MS-COCO dataset are shown, with the following parameters: the maximum number of iterations was set 100, ranged from 0.01 to 0.05, 7 was 0.05, and the duration T was 20. These results can be reproduced using the provided code.
Figure 5: Adversarial examples generated by RFPAR on the MS-COCO dataset. The Original Image represents the unaltered image, and the Delta shows the difference between the Original Image and the Adversarial Image. The parameter is hyperparameter that determines the attack level; a higher value of attacks more pixels. We conducted experiments with ranging from 0.01 to 0.05. The Delta Image resulting from values of 0.01 to 0.05 is presented in columns 2 to 6, and the Adversarial Image generated from the same a values shown in columns Image typically indicates image with changed prediction, but in this context, it also includes unsuccessful attacks. We present the results of Delta and Adversarial Images according to different values of a.
11. The Adversarial
16

---

E Additional Experiments
In this section, we present additional experiments that were omitted due page limitations. Table B shows the results on various transformer-based models (ViT-L [23]], Swin- [3Z]], and Deit-B [[ These results demonstrate that RFPAR is effective transformer-bast the results for object detection models, ATSS [ and Deformable DETR | showing that RFPAR is also effective for object detection. Finally, Table @ presents results for transforee--basee models (ViT-B, L, H. Swin-V2, and Deit-B) with iteration limits of 100 and 200. These results indicate that RFPAR requires more queries achieve comparable attack success rate transformer-basee models compared to CNN-based models.
models. Table f provides
Table 5: The results of transformer-based classifiers.
ViT-L
Swin-V. V2
Deit
Attacks
SR f
Lo 4
Query
SR f
Lg
Query 4
SR f
Lo
Query +
OnePixel
8.9%
15
1654
5.0%
15
1686
8.4%
15
1137
Pixle
66.4%
531
1396
66.8%
1052
1509
71.0%
551
1473
REPAR
78.0%
355
1042
69.4%
608
1096
84.3%
412
1161
Table 6:
The results of object detection models.
Atss
Deformable DETR
Attacks
RM f
mAP L
Lo &
Query +
RM f
mAP +
Lo +
Query J
clean
0.227
0.339
RFPAR 0.01
0.74
0.048
491
1530
0.61
0.170
333
1466
RFPAR0.02
0.88
0.026
1025
1633
0.69
0.134
512
1502
RFPARo.03
0.90
0.026
1357
1504
0.72
0.135
869
1488
RFPARo.04
0.91
0.008
1666
1243
0.76
0.110
1200
1488
RFPAR0.03
0.92
0.006
2074
1288
0.78
0.073
1274
1335
Table 7: The performance of RFPAR transformer-bssed models with different iteration limits
maximum of Iteration =
100
maximum of Iteration 200
Model
Succes rate t
Lo +
Query +
Succes rate 1
Lo
Query J
VIT-B 64.1%
211
613
83.4 %
352
995
ViT-L 59.9%
209
618
78.0 %
355
1042
ViT-H 62.2%
166
582
73.5%
229
917
Swin-V2
46.2%
352
611
69.4 %
608
1096
Deit-B
60.2%
249
676
84.3 %
412
1161
F Experiments on Adversarially Trained Models
In this section, we present experiments on adversarially trained models (Adv. VIT [+]] and Adv. ResNeXt101 [2)). Table & shows that RFPAR is effective on these models, although its success rate is lower compared to generally trained models. Proportional calculations indicate that RFPAR reduced ViT's performance from 69.10% to 37.11%, which, according to Appendix D of the Adv. ViT paper [II]], is more effective than CW20 (38.5 92%,,,%, PGD-20 (37.96%) ) and PGD-100 (37.52%) but slightly less effective than AutoAttack (34.62%). This demonstrates that our black-box attack, RFPAR, is nearly effective as white-box attacks, despite having access to only limited informa- tion.
17

---

Table 8:
The results of adversarial trained models.
Adv. VIT
Adv. ResNeXt101
Attacks
Succes rate f Lo +
Query |
Succes rate t
Lo +
Query
OnePixel
2.9% 34.0%
2083
4.4%
15
1102
Pixle
780
1912
42.5%
302
769
RFPAR
46.3%
547
1452
57.4%
243
626
G Query in Ablation study
In this section, present Query regarding the ablation study and conduct experiments under similar conditions to ensure a fair comparison of each process.
Table 9: Query for ablation study.
ViT-B
ResNeXt
RegNetX
DenseNet
MNASNet
MobileNet-V?.
RFPAR
614
529
623
534
461
548
RFPAR
662
404
444
404
364
348
RFPARM
712
889
820
723
726
659
RFPARM/1
613
442
484
464
442
596
H
Time complexity
Given the input dimension size N and constants K₃: OnePixel has O(K)) complexity, ScratchThat has O(N2), Pixle has (K2)), REPAR has O(N), PREAR has O(3)), and GARSDC has O(N). For image classification tasks, RFPAR's linear increase in time complexity with image size is more favorable than ScratchThat's s exponential increase but less so than OnePixel and Pixle. In object detection tasks, both REPAR and GARSDC linear increases in time complexity with larger images, making them less advantageous than PRFAR.
The higher time complexity compared most other studies is limitation of our research. However, RFPAR generates attacks using neural networks, similar to GARSDC, and benefits from the high performance of GPUs, allowing for faster computations despite the increased time complexity. We present the experimental times in Table IIC
To improve efficiency, we propose integrating our method with meta-learning. RFPAR involves the agent learning afresh on image multiple times, which can mitigate overfitting but also results in unnecessary queries. Meta-learning could enable the agent to quickly adapt to new tasks, enhancing efficiency by learning more rapidly.
Table 10: The experimental times in Table MI of the main paper
ViT
RegNeIX-32GF
MNASNet
DenseNet161
MobileNet V3
OnePixel
3h 2m
4h 36m
50m
2h 52m
50m
ScratchThat
5d 12h 39m
11d 11h 27m
3d 19h Im
7d 6h 43m
6d ll 8m
Pixle
4h 48m
8h 16m
3h 3m
13h 33m
5h 14m
RFPAR
1h 20m
1h 20m
20m
47m
29m
18

---

NeurIPS Paper Checklist
The checklist designed to encourage best practices for responsible machine learning research, ad- dressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (12 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evalu- ation. While "[Yes] " generally preferable to "[No] is perfectly acceptable answer "[No] provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased binary way. acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] question, in the justification please point to the section(s) where related material for the question can be found.
used"). In
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: we have summarized our contributions and specified the category to which our method belongs in Section II
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
19

---

Justification: In Section PL we also describe the limitations of the proposed method.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: We present only experimental results without providing any theoretical re- sults.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have detailed our proposed method in Sections and reproducibility.
D
ensure
20

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), relcasing of a model checkpoint, or other means that are appropriate to the research performed.
also be provided via
While NeurlPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have included the code and some of the data used in the experiments in the supplemental material to ensure reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ mnaps..cc/ubblcc/uuddeee www.//ooooeossiiiooooooio Posiooooiii))) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
21

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Section B.I. we have specified the key hyperparameters used in the experi- ments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars not provided due to the high computational cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Although we did not report the time exccution, the environment we used is specified in Section B.I.
Guidelines:
22

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Yes, we conducted in the paper conforms, in every respect. Specifically, we briefly discussed the potential negative impacts on society in Section !!
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We briefly discuss the negative societal impacts in Section g
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications,
authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
23

---

Answer: [NA]
Justification: It is not yet applicable to real-world scenarios as it is still challenging to cause significant disruption.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
This is explained in Section 6
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license. copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswi thcode wwt.ccmmccc...c datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
Justification:
This is shown in Fig e and Section D
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moddl as part of their
submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects

---

Question: For crowdsoureing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification:
We did not conduct any research involving human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper deseribe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We did not conduct any research involving human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva-
lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "slSY8WIeKj", "reviewer": "Reviewer_p4SQ", "summary": "The paper proposes a pixel-based black box attack called RFPAR that uses RL to perturb pixels. The paper has a remember and forgetting mechanism. In the remember step, they memorize the perturb images to reduce the number of queries and in forgetting, they try to remove them from the memory. Although their main target are object detection algorithms, they are showing good accuracy drop, aka attack effectiveness on image classification task such as ImageNet-1K.", "questions": "I like the idea and the methodology of the paper. However, the drop of accuracy in ViT is the lowest and the other object detection/image classification methods are all CNN-Based. Therefore, the effectiveness of the attack on transformer based model is vague in my opinion. It would be great if the authors provide more experiments on transformers or limit the scope of their method to to convolutional networks. \n\nAlternatively, an experiment that shows the effect of higher number of queries can also be helpful. Despite its success, the rate of the attack succession on ViT is significantly lower than CNN based methods. Perhaps increasing the number of queries could fix this ?", "limitations": "Please refer to weaknesses and questions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The experiments of the paper are thorough and comprehensive. \nThey are proving RFPAR's effectiveness. The numbers and the effectiveness of the method is perfectly proven.", "weaknesses": "In my opinion, paper doesn't have that much of flaw. However, most of the models that are used in the paper are CNN-based model. The only transformer model used is the VIT which paper didn't specified which variant. Given the numbers, assuming the authors used VIT-Based, I'd like to know the effect of the attack on larger models like ViT-Huge, ViT-large, DeIT, etc. \n\nBesides, unlike the claim of the paper that is main target is object detection, most of the experiments are targeting image classification methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer p4SQ, \n\nThank you for reading our paper and providing comments to help improve it. Below, we address your concerns.\n\n---\n\n**C1:“The only transformer model used is the VIT which paper didn't specified which variant. Given the numbers, assuming the authors used VIT-Based, I'd like to know the effect of the attack on larger models like ViT-Huge, ViT-large, DeIT, etc.\"**\n\nThank you for your valuable suggestions regarding the transformer-based models. We have conducted experiments on transformer-based models, specifically ViT-large, Swin-V2-t and Deit-B, and the results are presented in Table 1 of the (PDF). Following your advice, we increased the number of queries for Pixle and RFPAR in our experiments. Our method, RFPAR, continues to demonstrate superior effectiveness compared to other pixel attack methods, even on different transformer-based models. This result implies that our method is effective even for transformer-based models. \n\nFurthermore, while the attack success rates on transformer-based models are slightly lower than on CNN-based models, they require more than twice the number of queries and $L0$. This outcome indicates that transformer-based models exhibit greater robustness against pixel attacks compared to CNN-based models. \n\n---\n\n**C2: “Besides, unlike the claim of the paper that is main target is object detection, most of the experiments are targeting image classification methods.”**\n\nIn response to your suggestions, we conducted additional experiments on two new models. We tested Atss with a \"Resnet101\" backbone and two-stage Deformable DETR with a \"Resnet50\" backbone using the MS-COCO dataset. The results are presented in Table 2 of the (PDF). Our experimental results show that our method can remove more than 78\\% of detected objects, resulting in a reduction of mAP by 97\\% for Atss and 78\\% for Deformable DETR, demonstrating the success of our attack. Notably, the attack on Atss reduced the number of queries by 30\\% to 1288, compared to 1837 queries presented by the state-of-the-art query-based attack, GARSDC, showing that RFPAR is query-efficient, similar to the results on Yolo. The Attack Area (ATA) is minimal, at 0.04\\% for Atss and 0.01\\% for Deformable DETR, making the attacks difficult to detect by human observers and preserving the overall semantics of the images. This result demonstrates that our method is an efficient and effective approach for queries.\n\n\n---\n\n**C3: “The effectiveness of the attack on transformer based model is vague in my opinion. Despite its success, the rate of the attack succession on ViT is significantly lower than CNN based methods. Perhaps increasing the number of queries could fix this ?”**\n\nThank you for your suggestion. We have incorporated your feedback and increased the number of queries for our experiments on transformer-based models. We conducted experiments with a maximum of 100 and 200 iterations, and the results are presented in Table 3 of the (PDF). Increasing the queries demonstrated that RFPAR achieved an average success rate of approximately 80\\% on transformer-based models. This result indicates that our method remains effective even with transformer-based models.\nHowever, as mentioned earlier, it required more than twice the number of queries and $L0$ compared to CNN-based models, indicating that transformer-based models are more robust against pixel attacks. Nevertheless, given sufficient queries, pixel attacks can still achieve high success rates on transformer-based models. These results underscore the necessity of implementing defense mechanisms when applying most models in real-world scenarios.\n\n---\n\nThank you once again for your thoughtful comments aimed at improving our paper. If you have no remaining concerns, we would greatly appreciate it if you could increase your score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WOiZIQblMP", "reviewer": "Reviewer_RX8p", "summary": "This paper proposes a new adversarial attack method called Remember and Forget Pixel Attack using Reinforcement Learning (RFPAR) for image classification and object detection models. The key contributions are:\n- A novel pixel-based black-box attack using reinforcement learning with \"Remember\" and \"Forget\" processes\n- Extension of query-based pixel attacks from image classification to object detection\n- Experimental results showing improved performance over existing methods on ImageNet and MS-COCO datasets", "questions": "1. Can you provide more theoretical insight into why the Remember and Forget processes lead to improved performance? Perhaps some analysis of the optimization landscape?\n1. How does the computational cost of RFPAR compare to other methods, especially for larger images? Are there ways to improve efficiency further?\n1. Have you tested RFPAR against any adversarial defenses? How do you expect it to perform against adversarially trained models?", "limitations": "The ethics and potential negative impacts of this work are not sufficiently discussed. Developing more effective adversarial attacks could have serious consequences. This paper should discuss potential negative impacts and how they might be mitigated.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper's approach is original in using a dual-stage reinforcement learning process to manage adversarial attacks, which is a unique combination in the context of pixel-based black-box attacks.\n\nThe paper is well-written with a clear structure.", "weaknesses": "Lack of Theoretical Foundation: The paper primarily focuses on empirical results without providing theoretical insights or proofs to back the efficacy of the Remember and Forget processes in adversarial contexts.\n\nLimited Evaluation: The robustness of the method to various defenses is not evaluated. It's unclear how RFPAR would perform against adversarially trained models or other defense techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer RX8p,\n\nThank you for reading our paper and providing comments to help improve it. Below, we address your concerns.\n\n---\n\n**C1) \"Can you provide more theoretical insight into why the Remember and Forget processes lead to improved performance?\"**\n\nWe initially used a multi-step REINFORCE approach but identified issues, leading us to propose the Forget and Remember processes using one-step REINFORCE. Generating adversarial examples with multi-step REINFORCE involves the objective function $U = E \\left[ \\sum_{0}^{\\mathcal{\\tau}} \\gamma^{\\tau-t} R[s_t,a_t|\\pi_\\theta] \\right]$, where $\\gamma$ is the discount factor, $s_t$ is the image at step $t$, $a_t$ is the action at $s_t$, and the reward is $R[s_t,a_t|\\pi_\\theta] = f_{\\theta,y}(s_{0}) - f_{\\theta,y}(s_{t+1}), f_{\\theta,y}$ is the confidence score of the true label $y$. Here, $a_t$ is a single pixel perturbation. We find that significant oscillations can be observed in the objective function. Let $\\tau^*$ be the minimum number of steps(pixels) to create an adversarial example. The sequence of pixels does not matter, leading to variations in the value of the objective function due to different orderings of $a_t$. Thus, for $i_t \\in \\\\{0,1,2,\\cdots,\\tau^*\\\\} \\text{ and } i_j \\neq i_k$, the optimal objective function value is $U^* = E \\left[ \\sum_{0}^{\\mathcal{\\tau^*}} \\gamma^{\\tau-t} R[s_t,a_{i_t}|\\pi_\\theta] \\right]$, with $\\tau^*!$ permutations. This complicates training and increases the queries and $L_0$. To address this, we proposed the Forget and Remember process using one-step REINFORCE.\n\nPixel perturbations at $\\tau^*$ are defined as $A_{\\tau^*} = \\sum_0^{\\tau^*} a_t$. By the intermediate value theorem, there exists a $C$ in $[x,x+A_{\\tau^*}]$ such that $f_{\\theta,y}(x) > N > f_{\\theta,y}(x+A_{\\tau^*})$ for some $N$. We propose a Forget and Remember process using one-step REINFORCE to iteratively find this $C$, assuming $C \\in \\\\{x + a_0, x + a_1, \\cdots, x + a_{\\tau^*}\\\\}$. This one-step approach avoids the oscillations of multi-step REINFORCE, offering better query efficiency and lower $L_0$.\n\n\n---\n\n**C2) “How does the computational cost of RFPAR compare to other methods, especially for larger images? Are there ways to improve efficiency further?”**\n\nGiven the input dimension size $N$ and constants $K_i$: OnePixel has $O(K_1)$ complexity, ScratchThat has $O(N^2)$, Pixle has $O(K_2)$, RFPAR has $O(N)$, PRFAR has $O(K_3)$, and GARSDC has $O(N)$. In image classification tasks, RFPAR's time complexity increases linearly with larger image sizes, which is more favorable than ScratchThat's exponential increase but less advantageous compared to OnePixel and Pixle. In object detection tasks, both RFPAR and GARSDC exhibit linear increases in time complexity with larger images, making them less advantageous than PRFAR. This can be seen as a limitation of our method.\n\nHowever, RFPAR generates attacks using neural networks, similar to GARSDC, and benefits from the high performance of GPUs, allowing for faster computations despite the increased time complexity. We present the experimental times in Table 4 in the (PDF).\n\nTo improve efficiency, we propose integrating our method with meta-learning. RFPAR involves the agent learning afresh on the image multiple times, which can mitigate overfitting but also results in unnecessary queries. Meta-learning could enable the agent to quickly adapt to new tasks, enhancing efficiency by learning more rapidly.\n\n---\n\n**C3) “Have you tested RFPAR against any adversarial defenses? How do you expect it to perform against adversarially trained models?”**\n\nWe conducted experiments on adversarially pre-trained VIT  and RexNeXt101 , with results presented in Table 5 of the (PDF). Compared to other pixel attacks, our method achieves the highest success rate, effective queries, and ranks second in $L_0$. Proportional calculations indicate that RFPAR reduced VIT's performance from 69.10% to 37.11%. According to Appendix D of [6], this reduction is more effective than those achieved by CW20 (38.92%), PGD-20 (37.96%), and PGD-100 (37.52%), but less effective than AutoAttack (34.62%). This demonstrates that our Black-box attack, RFPAR, is as effective as White-box attacks, even though it uses only limited information.\n\n---\n\n**C4) “The ethics and potential negative impacts of this work are not sufficiently discussed. This paper should discuss potential negative impacts and how they might be mitigated.“**\n\nOur proposed method causes models to malfunction with fewer queries and higher success rates than previous pixel attacks. In the \"Argoverse Attack Video.pptx\" (0:30 to 0:40) from our supplementary material, our attack successfully misclassifies a nearby person detected with high confidence, posing a safety risk for vision-based systems. However, pixel attacks currently require over 1000 queries, making real-time application impractical.\n\nIn image classification tasks, adversarially trained models halved the success rate of pixel attacks and increased the required queries, showing that adversarial training and query limitations can effectively mitigate these attacks. According to Table 1 in the (PDF), effective attacks on transformer-based models require about 1000 queries, while effective attacks on CNN-based models can be achieved with approximately 500 queries. This result shows that CNN models are relatively easy to deceive, highlighting the necessity of applying defensive techniques to CNN models in real-world scenarios. We will add the discussed content into the final version. \n\n---\n\nThank you once again for your thoughtful comments aimed at improving our paper. If you have no remaining concerns, we would greatly appreciate it if you could increase your score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gxWRGa3qVw", "reviewer": "Reviewer_R24Y", "summary": "The authors present a few-pixel querying blackbox attack for image classification and object detection models, called Remember and Forget Pixel Attack using Reinforcement Learning (RFPAR). As this is a blackbox attack, the setting assumes the attacker does not have access to the target model weights, though as a querying attack, they assume they can query many test inputs to the victim model to optimize the attack. The resulting attacks edit only a small number of pixels and are effective at changing the classification target or at suppressing all objects in detection models. The authors demonstrate their attack on several models and dataset settings. They additionally present ablations of their attack optimization method.", "questions": "Details of the Ablations (Section 3.5) could be presented more clearly. If M is ablated, how does the optimization change? Is it equivalent to only running the inner loop once? If I is ablated, does this mean the RL model is not reset between outer iterations? Is the optimization still run for a similar number of steps?", "limitations": "The authors do not extensively discuss the limitations of the work. In Section 4 they briefly discuss the potential negative impact that adversarial attacks can have.\n\nThe authors do not present any defense testing for their attack. It would be good to see some defense testing, to know if this attack is easily defended against or not.\n\nAs this is a work focused on presenting an adversarial attack, there is a potential risk of negative impact.", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The authors present an effective attack that achieves high ASR with few pixel edits. Moreover, the authors demonstrate their attack not only on image classification but on object detection as well.\n\nThe authors also demonstrate their attack on a diverse collection of models for both the classification and detection settings.\n\nCompared to the baseline methods, the proposed RFPAR achieves high ASR and lower queries. In terms of the image edit distance (L0), theirs is second only to OnePixel, though their ASR is significantly higher than OnePixel.", "weaknesses": "In terms of presentation, it seems like the “Forget Process” is very simple, as it is only resetting the RL agent and passing the output of the previous optimization round. I recognize that the authors have demonstrated the importance of this step in their ablations. I just feel like they present the Remember Process and Forget Process as if both are equally major steps, when Forget is only a minor step compared to Remember.\n\nWhile the percentage of pixels edited is small, the attacks can still be noticeable to the human eye, especially when you have dark pixels on a bright background region like the sky. This is less noticeable when not zoomed in on the images. The author’s method optimizes pixels with an all-or-nothing approach, setting each pixel channel to either 0 or 255, which makes the edits more visible.\n\nThe results on Argoverse (Section 3.4) are somewhat limited. If I understand correctly, the analysis is only applied to one image that was attacked.\n\nThe paper lacks a dedicated section to discuss related works, though some parts of the introduction discuss related works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer R24Y,\n\nThank you for reading our paper and providing comments to help improve it. Below, we address your concerns.\n\n---\n\n**C1: \"Details of the Ablations could be presented more clearly. \"**\n1) **M ablation:**\nM stands for Memory in the Remember process, and I represents Initialization in the Forget process. If Memory is ablated, the Agent's reward is used for the bound condition. Convergence occurs when the Agent's reward meets this condition, and the final image is passed to the Forget process. Without Memory, only the Agent is reinitialized, repeating until the maximum iterations are reached.\n\nQ) If M is ablated, how does the optimization change?\n\nNo, the optimization process proceeds as before. However, the bound condition that halts this process now depends on the Agent's reward.\n\nQ) Is it equivalent to only running the inner loop once?\n\nNo, the Forget and Remember processes are conducted without memory for the maximum number of iterations we have specified. We used the same maximum number of iterations (100) for all ablation tests.\n\n2) **I Ablation:**\nIf Initialization is ablated in the Forget process, the Agent is not reinitialized and retains information from the previous Remember process. This can cause the search to focus on specific pixels with strong rewards, potentially missing vulnerabilities in other regions.\n\nQ) If I is ablated, does this mean the RL model is not reset between outer iterations?\n\nYes, the Agent is not reinitialized and retains previously learned information, leading to a focus on specific areas.\n\nQ) Is the optimization still run for a similar number of steps?\n\nYes, we used the same hyperparameters for all experiments. We will present the ablation study query results.\n\n| Ablation      | ViT  | ResNeXt| RegNetX| DenseNet| MNASNet | MobileNet-V3|\n|----------------|------|--------|--------|---------|---------|-------------|\n| RFPAR         | 614  | 529    | 623| 534         | 461     | 548         |\n| RFPAR$_{I}$   | 662  | 404    | 444| 404         | 364     | 348         |\n| RFPAR$_{M}$   | 712  | 889    | 820| 723         | 726     | 659         |\n| RFPAR$_{M+I}$ | 613  | 442    | 484| 464         | 442     | 596         |\n\n---\n\n**C2: \"It seems like the “Forget Process” is very simple. Forget is only a minor step compared to Remember.\"**\n\nYes, the Forget process is simple and mainly reinitializes the Agent. However, our ablation study showed that both the Forget and Remember processes are essential for effective adversarial attacks. If either process is omitted, performance degrades significantly. Although the Forget process is simple now, it has the potential for enhancement with meta-learning. Therefore, we consider the Forget process a major step along with the Remember process.\n\n---\n\n**C3: “The paper lacks a dedicated section to discuss related works.”**\n\nDue to page limitations, the essential related works have been included in the introduction. We will add a more detailed discussion on related works in the Appendix.\n\n---\n\n**C4: \"While the percentage of pixels edited is small, the attacks can still be noticeable to the human eye\"**\n\nAs you mentioned, people might notice pixel changes when zoomed in or in specific situations. Finding perturbations within the [0,255] range using the Agent is very difficult, so we chose an all-or-nothing approach. Fortunately, this method attacks fewer pixels and targets the entire area of the image, preserving its meaning for viewers. This is demonstrated in the \"Argoverse Attack Video.pptx\" in our supplementary materials. While some pixel irregularities might be noticeable, the objects in the video remain recognizable, although the detection model malfunctions.\n\n---\n\n**C5: \"The results on Argoverse are somewhat limited.\"**\n\nWe apologize for any confusion. We conducted experiments on a single video sequence from the Argoverse dataset, specifically \"e9a96218-365b-3ecd-a800-ed2c4c306c78\" from the Argoverse-1.1 validation set, which includes 469 images. you can see the \"Argoverse video attack\" in the supplementary material.\n\n---\n\n**C6: “The authors do not present any defense testing for their attack.”**\n\nWe conduct experiments on adversarially pre-trained VIT and RexNeXt101, with results presented in Table 5 of the (PDF). Our method achieves the highest success rate, effective queries, and ranks second in $L_0$. Proportional calculations indicate that RFPAR reduced VIT's performance from 69.10% to 37.11%. According to Appendix D of [6], this reduction is more effective than those achieved by CW20 (38.92%), PGD-20 (37.96%), and PGD-100 (37.52%), but less effective than AutoAttack (34.62%). This demonstrates that our Black-box attack, RFPAR, is as effective as White-box attacks, even though it uses only limited information.\n\n---\n\n**C7: “The authors do not extensively discuss the limitations of the work. In Section 4 they briefly discuss the potential negative impact that adversarial attacks can have.”**\n\nOur attack effectively deceives the model by targeting only a small number of pixels with effective queries. This highlights the risk of deploying models solely based on their training performance in the real world. However, in object detection, while the query efficiency is maintained, the attack still requires over 1000 queries, which can be easily thwarted by simply limiting the number of queries. In image classification tasks, adversarial training can increase the required number of queries and reduce the attack success rate by approximately half. This implies that applying adversarial training and query limitations to classification models can successfully defend against pixel attacks. Therefore, it is essential to implement defense techniques such as adversarial training or query limitations when deploying models in real-world applications.\n\n---\n\nWe will add the discussed points into the final version.  If you have no remaining concerns, we would greatly appreciate it if you could increase your score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

Thank you once again for reading our paper and providing insightful comments. We have made every effort to address your concerns. If our responses adequately address the reviewers’ concerns, we kindly request that the reviewers consider increasing your scores.

Due to space constraints, we are unable to present all tables within each response. So, we have submitted them as a PDF here. 
In the image classification task, we conducted experiments on a dataset consisting of one accurately predicted image per label from ImageNet-1K, similar to main paper. For the object detection task, we utilized the same dataset employed in the main paper.

- Table 1: Experimental results on various transformer-based models, VIT-L[1], Swin-V2[2], and Deit[3]. SR means Succes rate.

- Table 2: Experimental results on object detection models, Atss[4] and Deformable DETR[5].

- Table 3: Experimental results on transformer-based models (ViT-B, L, H, Swin-V2, Deit-B) with iteration limits of 100 and 200.

- Table 4: Experimental times for the experiments presented in Table 1 of the main paper.

- Table 5: Experimental results on adversarially trained Adv. ViT[6] and ResNeXt101[7].



Reference

[1] A. Dosovitskiy el al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale," in International Conference on Learning Representations, 2021.

[2] Z. Liu et al., "Swin Transformer V2: Scaling Up Capacity and Resolution," in Conference on Computer Vision and Pattern Recognition, 2022.

[3] H. Touvron el al., "Training data-efficient image transformers \& distillation through attention" in International Conference on Machine Learning, 2021.

[4] S. Zhang et al., "Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection," in Conference on Computer Vision and Pattern Recognition, 2020.

[5] X. Zhu et al., "Deformable DETR: Deformable Transformers for End-to-End Object Detection," in International Conference on Learning Representations, 2021.

[6] Y. Mo et al., "When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture," in Advances in Neural Information Processing Systems, 2022.

[7] D. Hendrycks et al., "The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization," in International Conference on Computer Vision, 2021.

Meta-review: The rebuttal addressed most of the concerns raised by the reviewers and now all reviewers support accepting the paper. Hence, it will be accepted. Unfortunately, some reviewers did not engage well with the authors. Anyways, the authors's response (e.g., more results on larger ViT models) was satisfactory in my view.
Decision: Accept (poster)