\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{array}
\usepackage{float}
\usepackage{multirow}
\usepackage{amstext}
\usepackage{amssymb}
\usepackage{graphicx}

\makeatletter

\providecommand{\tabularnewline}{\\}
\floatstyle{ruled}
\newfloat{algorithm}{tbp}{loa}
\providecommand{\algorithmname}{Algorithm}
\floatname{algorithm}{\protect\algorithmname}

\def\year{2022}\relax
\usepackage{aaai22}  %
\usepackage{times}  %
\usepackage{helvet}  %
\usepackage{courier}  %
\usepackage[hyphens]{url}  %
\usepackage{graphicx} %
\urlstyle{rm} %
\def\UrlFont{\rm}  %
\usepackage{natbib}  %
\usepackage{caption} %
\DeclareCaptionStyle{ruled}{labelfont=normalfont,labelsep=colon,strut=off} %
\frenchspacing  %
\setlength{\pdfpagewidth}{8.5in}  %
\setlength{\pdfpageheight}{11in}  %
\usepackage[algo2e, ruled, linesnumbered]{algorithm2e}

\usepackage{newfloat}
\usepackage{listings}
\lstset{%
	basicstyle={\footnotesize\ttfamily},%
	numbers=left,numberstyle=\footnotesize,xleftmargin=2em,%
	aboveskip=0pt,belowskip=0pt,%
	showstringspaces=false,tabsize=2,breaklines=true}
\floatstyle{ruled}
\newfloat{listing}{tb}{lst}{}
\floatname{listing}{Listing}
\pdfinfo{
/Title (Meta Adversarial Perturbations)
/Author (Chia-Hung Yuan, Pin-Yu Chen, Chia-Mu Yu)
/TemplateVersion (2022.1)
}
\setcounter{secnumdepth}{2} %
\title{Meta Adversarial Perturbations}
\author {
    Chia-Hung Yuan\textsuperscript{\rm 1,2},
    Pin-Yu Chen\textsuperscript{\rm 1,3},
	Chia-Mu Yu\textsuperscript{\rm 2}
}
\affiliations {
    \textsuperscript{\rm 1}MIT-IBM Watson AI Lab\\
    \textsuperscript{\rm 2}National Yang Ming Chiao Tung University\\
	\textsuperscript{\rm 3}IBM Research\\
    jimmy.chyuan@gmail.com, pin-yu.chen@ibm.com, chiamuyu@nycu.edu.tw
}
\makeatother

\usepackage{babel}
\begin{document}
\maketitle
\begin{abstract}
A plethora of attack methods have been proposed to generate adversarial examples, among which the iterative methods have been demonstrated the ability to find a strong attack. However, the computation of an adversarial perturbation for a new data point requires solving a time-consuming optimization problem from scratch. To generate a stronger attack, it normally requires updating a data point with more iterations. In this paper, we show the existence of a \textit{meta adversarial perturbation} (MAP), a better initialization that causes natural images to be misclassified with high probability after being updated through only a one-step gradient ascent update, and propose an algorithm for computing such perturbations. We conduct extensive experiments, and the empirical results demonstrate that state-of-the-art deep neural networks are vulnerable to meta perturbations. We further show that these perturbations are not only image-agnostic, but also model-agnostic, as a single perturbation generalizes well across unseen data points and different neural network architectures.
\end{abstract}

\section{Introduction}
Deep neural networks (DNNs) have achieved remarkable performance in many applications, including computer vision, natural language processing, speech, and robotics, etc. However, DNNs are shown to be vulnerable to adversarial examples \cite{szegedy2013intriguing,goodfellow2014explaining}, i.e. examples that are intentionally designed to be misclassified by the models but nearly imperceptible to human eyes. In recent years, many methods have been proposed to craft such malicious examples \cite{szegedy2013intriguing,goodfellow2014explaining,moosavi2016deepfool,kurakin2016adversarial,madry2017towards,carlini2017towards,chen2017ead}, among which the iterative methods, such as PGD \cite{madry2017towards}, BIM \cite{kurakin2016adversarial}, and MIM \cite{dong2018boosting}, have been demonstrated to be effective to craft adversarial attacks with a high success rate. Nevertheless, to craft a stronger attack with iterative methods, it usually requires updating a data point through more gradient ascent steps. This time-consuming process gives rise to a question: is it possible to find a \textit{single} perturbation, which can be served as a good meta initialization, such that after a few updates, it can become an effective attack for different data points?

Inspired by the philosophy of meta-learning \cite{schmidhuber1987evolutionary,bengio1990learning,andrychowicz2016learning,li2016learning,finn2017model}, we show the existence of a quasi-imperceptible \textit{meta adversarial perturbation} (MAP) that leads natural images to be misclassified with high probability after \textbf{being updated through only one-step gradient ascent update}. In meta-learning, the goal of the trained model is to quickly adapt to a new task with a small amount of data. On the contrary, the goal of the meta perturbations is to rapidly adapt to a new data point within a few iterations. The key idea underlying our method is to train an initial perturbation such that it has maximal performance on new data after the perturbation has been updated through one or a few gradient steps. We then propose a simple algorithm, which is plug-and-play and is compatible with any gradient-based iterative adversarial attack method, for seeking such perturbations. By adding a meta perturbation at initialization, we can craft a more effective adversarial example without multi-step updates.

We summarize our main contributions as follows:
\begin{itemize}
\item We show the existence of image-agnostic learnable meta adversarial perturbations for efficient robustness evaluation of state-of-the-art deep neural networks.
\item We propose an algorithm (MAP) to find meta perturbations, such that a small number of gradient ascent updates will suffice to be a strong attack on a new data point.
\item We show that our meta perturbations have remarkable generalizability, as a perturbation computed from a small number of training data is able to adapt and fool the unseen data with high probability.
\item We demonstrate that meta perturbations are not only image-agnostic, but also model-agnostic. Such perturbations generalize well across a wide range of deep neural networks.
\end{itemize}

\section{Related Works}
There is a large body of works on adversarial attacks. Please refer to \cite{chakraborty2018adversarial,akhtar2018threat,biggio2018wild} for comprehensive surveys. Here, we discuss the works most closely related to ours.

\subsection{Data-dependent Adversarial Perturbations}
Despite the impressive performance of deep neural networks on many domains, these classifiers are shown to be vulnerable to adversarial perturbations \cite{szegedy2013intriguing,goodfellow2014explaining}. Generating an adversarial example requires solving an optimization problem \cite{moosavi2016deepfool,carlini2017towards} or through multiple steps of gradient ascent \cite{goodfellow2014explaining,kurakin2016adversarial,madry2017towards,chen2017ead} for each data point independently, among which the iterative methods have been shown to be able to craft an attack with a high success rate. Given a data point $x$, a corresponding label $y$, and a classifier $f$ parametrized by $\theta$. Let $L$ denote the loss function for the classification task, which is usually the cross-entropy loss. FGSM \cite{goodfellow2014explaining} utilizes gradient information to compute the adversarial perturbation in one step that maximizes the loss:

\begin{equation}
x'=x+\epsilon\,\text{sign}(\nabla_{x}L(f_{\theta},x,y)),\label{eq:fgsm}
\end{equation}
where $x'$ is the adversarial example and $\epsilon$ is the maximum allowable perturbation measured by $l_{\infty}$ distance. This simple one-step method is extended by several follow-up works \cite{kurakin2016adversarial,madry2017towards,dong2018boosting,xie2019improving}, which propose iterative methods to improve the success rate of the adversarial attack. More specifically, those methods generate adversarial examples through multi-step updates, which can be described as:

\begin{equation}
x^{t+1}=\Pi_{\epsilon}\big(x^{t}+\gamma\,\text{sign}(\nabla_{x}L(f_{\theta},x,y))\big),\label{eq:pgd}
\end{equation}
where $\Pi_{\epsilon}$ projects the updated perturbations onto the feasible set if they exceed the maximum allowable amount indicated by $\epsilon$. $x^{0}=x$ and $\gamma=\epsilon/T$, where $T$ is the number of iterations. To generate a malicious example that has a high probability to be misclassified by the model, the perturbation sample needs to be updated with more iterations. The computational time has a linear relationship with the number of iterations, thus it takes more time to craft a strong attack.

\subsection{Universal Adversarial Perturbations\label{subsec:uap}}
Instead of solving a data-dependent optimization problem to craft adversarial examples, \cite{moosavi2017universal} shows the existence of a universal adversarial perturbation (UAP). Such a perturbation is image-agnostic and quasi-imperceptible, as a single perturbation can fool the classifier $f$ on most data points sampled from a distribution over data distribution $\mu$. That is, they seek a perturbation $v$ such that

\begin{equation}
f(x+v)\neq f(x)\text{ for "most" }x\sim\mu.\label{eq:uap}
\end{equation}

In other words, the perturbation process for a new data point involves merely the addition of precomputed UAP to it without solving a data-dependent optimization problem or gradient computation from scratch. However, its effectiveness is proportional to the amount of data used for computing a universal adversarial perturbation. It requires a large amount of data to achieve a high fooling ratio. In addition, although UAP demonstrates a certain degree of transferability, the fooling ratios on different networks, which are normally lower than 50\%, may not be high enough for an attacker. This problem is particularly obvious when the architecture of the target model is very different from the surrogate model \cite{moosavi2017universal}.

Although there are some works \cite{yang2021model,yuan2021meta} that seem similar to our method, our goal is completely different. \cite{yuan2021meta} proposes to use a meta-learning-like architecture to improve the cross-model transferability of the adversarial examples, while \cite{yang2021model} devise an approach to learn the optimizer parameterized by a recurrent neural network to generate adversarial attacks. Both works are distinct from the meta adversarial perturbations considered in this paper, as we seek a single perturbation that is able to efficiently adapt to a new data point and fool the classifier with high probability.

\begin{algorithm}

\SetArgSty{textnormal}
\SetKw{KwAll}{all}

\KwIn{$\mathbb{D}$, $\alpha$, $\beta$, $f_{\theta}$, $L$, $\Pi_{\epsilon}$}

\KwOut{Meta adversarial perturbations $v$}

\BlankLine

Randomly initialize $v$

\While{not done}{

\For{minibatch $\mathbb{B}=\{x^{(i)},y^{(i)}\}\sim\mathbb{D}$}{

Evaluate $\nabla_{v}L(f_{\theta})$ using minibatch $\mathbb{B}$
with perturbation $v$

Compute adapted perturbations with gradient ascent: $v'=v+\alpha\nabla_{v}L(f_{\theta},\mathbb{B}+v)$

Sample a batch of data $\mathbb{B}'$ from $\mathbb{D}$

Evaluate $\nabla_{v}L(f_{\theta})$ using minibatch $\mathbb{B}'$
with adapted perturbation $v'$

Update $v\leftarrow v+\beta\nabla_{v}L(f_{\theta},\mathbb{B}'+v')$

Project $v\leftarrow\Pi_{\epsilon}(v)$

}

}

\Return{$v$}

\caption{\label{alg:map}Meta Adversarial Perturbation (MAP)}
\end{algorithm}

\begin{table*}
\begin{centering}
\begin{tabular}{|c|c|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|}
\hline 
\multicolumn{2}{|c|}{{\small{}Attack\textbackslash Model}} & {\small{}VGG11} & {\small{}VGG19} & {\small{}ResNet18} & {\small{}ResNet50} & {\small{}DenseNet121} & {\small{}SENet} & {\small{}MobileNetV2}\tabularnewline
\hline 
\hline 
\multirow{2}{*}{{\small{}Clean}} & {\small{}$\mathbb{D}$} & {\small{}100.0\%} & {\small{}100.0\%} & {\small{}100.0\%} & {\small{}100.0\%} & {\small{}100.0\%} & {\small{}100.0\%} & {\small{}100.0\%}\tabularnewline
 & {\small{}$\mathbb{T}$} & {\small{}92.6\%} & {\small{}93.7\%} & {\small{}95.3\%} & {\small{}95.4\%} & {\small{}95.4\%} & {\small{}95.8\%} & {\small{}94.1\%}\tabularnewline
\hline 
\multirow{2}{*}{{\small{}FGSM}} & {\small{}$\mathbb{D}$} & {\small{}28.0\%} & {\small{}53.0\%} & {\small{}47.0\%} & {\small{}29.0\%} & {\small{}41.0\%} & {\small{}40.0\%} & {\small{}30.0\%}\tabularnewline
 & {\small{}$\mathbb{T}$} & {\small{}29.3\%} & {\small{}49.4\%} & {\small{}41.4\%} & {\small{}35.7\%} & {\small{}35.5\%} & {\small{}38.2\%} & {\small{}32.8\%}\tabularnewline
\hline 
\multirow{2}{*}{{\small{}UAP}} & {\small{}$\mathbb{D}$} & {\small{}99.0\%} & {\small{}98.0\%} & {\small{}58.0\%} & {\small{}32.0\%} & {\small{}33.0\%} & {\small{}42.0\%} & {\small{}42.0\%}\tabularnewline
 & {\small{}$\mathbb{T}$} & {\small{}88.9\%} & {\small{}83.3\%} & {\small{}45.8\%} & {\small{}33.5\%} & {\small{}25.5\%} & {\small{}32.5\%} & {\small{}45.8\%}\tabularnewline
\hline 
\multirow{2}{*}{{\small{}MAP}} & {\small{}$\mathbb{D}$} & {\small{}22.0\%} & {\small{}31.0\%} & {\small{}21.0\%} & {\small{}14.0\%} & {\small{}12.0\%} & {\small{}18.0\%} & {\small{}13.0\%}\tabularnewline
 & {\small{}$\mathbb{T}$} & {\small{}22.0\%} & {\small{}36.1\%} & {\small{}20.3\%} & {\small{}17.4\%} & {\small{}20.8\%} & {\small{}17.6\%} & {\small{}16.3\%}\tabularnewline
\hline 
\end{tabular}
\par\end{centering}
\caption{\label{tab:untarget-attack}The accuracy against different attacks on the set $\mathbb{D}$, and the test set $\mathbb{T}$ (lower means better attacks).}
\end{table*}
\section{Meta Adversarial Perturbations}

We formalize in this section the notion of meta adversarial perturbations (MAPs) and propose an algorithm for computing such perturbations. Our goal is to train a perturbation that can become more effective attacks on new data points within one- or few-step updates. How can we find such a perturbation that can achieve fast adaptation? Inspired by the model-agnostic meta-learning (MAML) \cite{finn2017model}, we formulate this problem analogously. Since the perturbation will be updated using a gradient-based iterative method on new data, we will aim to learn a perturbation in such a way that this iterative method can rapidly adapt the perturbation to new data within one or a few iterations.

Formally, we consider a meta adversarial perturbation $v$, which is randomly initialized, and a trained model $f$ parameterized by $\theta$. $L$ denotes a cross-entropy loss and $\mathbb{D}$ denotes the dataset used for generating a MAP. When adapting to a batch of data points $\mathbb{B}=\{x^{(i)},y^{(i)}\}\sim\mathbb{D}$, the perturbation $v$ becomes $v'$. Our method aims to seek a single meta perturbation $v$ such that after adapting to new data points within a few iterations it can fool the model on almost all data points with high probability. That is, we look for a perturbation $v$ such that

\begin{equation}
f(x+v')\neq f(x)\text{ for "most" }x\sim\mu.\label{eq:map-high-level}
\end{equation}
We describe such a perturbation \textit{meta} since it can quickly adapt to new data points sampled from the data distribution $\mu$ and cause those data to be misclassified by the model with high probability. Notice that a MAP is image-agnostic, as a single perturbation can adapt to all the new data.

In our method, we use one- or multi-step gradient ascent to compute the updated perturbation $v'$ on new data points. For instance, using one-step gradient ascent to update the perturbation is as follows:

\begin{equation}
v'=v+\alpha\nabla_{v}L(f_{\theta},\mathbb{B}+v),\label{eq:map-inner-update}
\end{equation}
where the step size $\alpha$ is a hyperparameter, which can be seen as $\gamma$ in Eq. (\ref{eq:pgd}). For simplicity of notation, we will consider a one-step update for the rest of this section, but it is straightforward to extend our method to multi-step updates.

The meta perturbation is updated by maximizing the loss with respect to $v$ evaluated on a batch of new data points $\mathbb{B}'$ with the addition of the updated perturbation $v'$. More precisely, the meta-objective can be described as:

\begin{equation}
\begin{array}{l}
\max_{v}\sum_{\mathbb{B}\sim\mathbb{D}}L(f_{\theta},\mathbb{B}'+v')\\
=\max_{v}\sum_{\mathbb{B\sim\mathbb{D}}}L\big(f_{\theta},\mathbb{B}'+(v+\alpha\nabla_{v}L(f_{\theta},\mathbb{B}+v))\big).
\end{array}\label{eq:map-meta-obj}
\end{equation}

Note that the meta-optimization is performed over the perturbation $v$, whereas the objective is computed using the adapted perturbation $v'$. In effect, our proposed method aims to optimize the meta adversarial perturbation such that after one or a small number of gradient ascent updates on new data points, it will produce maximally effective adversarial perturbations, i.e. attacks with a high success rate.

We use stochastic gradient ascent to optimize the meta-objective:

\begin{equation}
v\leftarrow v+\beta\nabla_{v}L(f_{\theta},\mathbb{B}'+v'),\label{eq:map-outer-update}
\end{equation}
where $\beta$ is the meta step size. Algorithm \ref{alg:map} outlines the key steps of MAP. At line 9, MAP projects the updated perturbations onto the feasible set if they exceed the maximum allowable amount indicated by $\epsilon$. A smaller $\epsilon$ makes an attack less visible to humans.

The meta-gradient update involves a gradient through a gradient. This requires computing Hessian-vector products with an additional backward pass through $v$. Since backpropagating through many inner gradient steps is computation and memory intensive, there are a plethora of works \cite{li2017meta,nichol2018first,zhou2018deep,behl2019alpha,raghu2019rapid,rajeswaran2019meta,zintgraf2019fast} try to solve this problem after MAML \cite{finn2017model} was proposed. We believe that the computation efficiency of MAP can benefit from those advanced methods.

\begin{table*}
\begin{centering}
\begin{tabular}{|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|>{\centering}p{1.75cm}|}
\cline{2-8} \cline{3-8} \cline{4-8} \cline{5-8} \cline{6-8} \cline{7-8} \cline{8-8} 
\multicolumn{1}{>{\centering}p{1.75cm}|}{} & {\small{}VGG11} & {\small{}VGG19} & {\small{}ResNet18} & {\small{}ResNet50} & {\small{}DenseNet121} & {\small{}SENet} & {\small{}MobileNetV2}\tabularnewline
\hline 
{\small{}VGG11} & \textbf{\small{}22.0\%} & {\small{}37.2\%} & {\small{}24.9\%} & {\small{}19.6\%} & {\small{}24.2\%} & {\small{}20.5\%} & {\small{}20.2\%}\tabularnewline
\hline 
{\small{}VGG19} & {\small{}22.9\%} & {\small{}36.1\%} & {\small{}24.5\%} & {\small{}18.3\%} & {\small{}22.0\%} & {\small{}19.2\%} & {\small{}18.3\%}\tabularnewline
\hline 
{\small{}ResNet18} & {\small{}22.7\%} & {\small{}33.6\%} & \textbf{\small{}20.3\%} & {\small{}17.1\%} & {\small{}21.6\%} & {\small{}18.3\%} & {\small{}17.8\%}\tabularnewline
\hline 
{\small{}ResNet50} & {\small{}23.6\%} & {\small{}35.6\%} & {\small{}23.0\%} & {\small{}17.4\%} & {\small{}20.8\%} & {\small{}19.3\%} & {\small{}18.1\%}\tabularnewline
\hline 
{\small{}DenseNet121} & {\small{}23.1\%} & \textbf{\small{}32.7\%} & {\small{}21.3\%} & \textbf{\small{}16.1\%} & {\small{}20.8\%} & {\small{}18.1\%} & {\small{}16.9\%}\tabularnewline
\hline 
{\small{}SENet} & {\small{}22.5\%} & {\small{}34.9\%} & {\small{}23.7\%} & {\small{}17.5\%} & {\small{}20.8\%} & \textbf{\small{}17.6\%} & {\small{}17.5\%}\tabularnewline
\hline 
{\small{}MobileNetV2} & {\small{}23.7\%} & {\small{}35.3\%} & {\small{}22.2\%} & {\small{}16.7\%} & \textbf{\small{}20.7\%} & {\small{}18.0\%} & \textbf{\small{}16.3\%}\tabularnewline
\hline 
\hline 
{\small{}FGSM} & {\small{}29.3\%} & {\small{}49.4\%} & {\small{}41.4\%} & {\small{}35.7\%} & {\small{}35.5\%} & {\small{}38.2\%} & {\small{}32.8\%}\tabularnewline
\hline 
\end{tabular}
\par\end{centering}
\caption{\label{tab:transferability}Transferability of the meta adversarial perturbations across different networks (with one-step update on the target model). The percentage indicates the accuracy on the test set $\mathbb{T}$. The row headers indicate the architectures where the meta perturbations are generated (source), and the column headers represent the models where the accuracies are reported (target). The bottom row shows the accuracies of FGSM on the target models without using meta perturbation at initialization.}
\end{table*}
\section{Experiments}
We conduct experiments to evaluate the performance of MAP using the
following default settings.

We assess the MAP on the CIFAR-10 \cite{krizhevsky2009learning} test set $\mathbb{T}$, which contains 10,000 images. We follow the experimental protocol proposed by \cite{moosavi2017universal}, where a set $\mathbb{D}$ used to compute the perturbation contains 100 images from the training set, i.e. on average 10 images per class. The maximum allowable perturbation $\epsilon$ is set to $8/255$ measured by $l_{\infty}$ distance. When computing a MAP, we use one gradient update for Eq. (\ref{eq:map-inner-update}) with a fixed step size $\alpha=\epsilon=8/255$, and use the fast gradient sign method (FGSM) in Eq. (\ref{eq:fgsm}) as the optimizer. We use seven trained models to measure the effectiveness of MAP, including VGG11, VGG19 \cite{simonyan2014very}, ResNet18, ResNet50 \cite{he2016deep}, DenseNet121 \cite{huang2017densely}, SENet \cite{hu2018squeeze}, and MobileNetV2 \cite{sandler2018mobilenetv2}. We consider FGSM \cite{goodfellow2014explaining} and universal adversarial perturbation (UAP) \cite{moosavi2017universal} as our baselines. We implement baselines using the same hyperparameters when they are applicable.

\subsection{Non-targeted Attacks}
First, we evaluate the performance of different attacks on various models. For the FGSM and MAP, we compute the data-dependent perturbation for each image by using a one-step gradient ascent (see Eq. (\ref{eq:fgsm})) to create non-targeted attacks. For the UAP, we follow the original setting as \cite{moosavi2017universal}, where we add the UAP on the test set $\mathbb{T}$ without any adaptation. 

The results are shown in Table \ref{tab:untarget-attack}. Each result is reported on the set $\mathbb{D}$, which is used to compute the MAP and UAP, as well as on the test set $\mathbb{T}$. Note that the test set is not used in the process of the computation of both perturbations. As we can see, MAP significantly outperforms the baselines. For all networks, the MAP achieves roughly 10-20\% improvement. These results have an element of surprise, as they show that by merely using a MAP as an initial perturbation for generating adversarial examples, the one-step attack can lead to much lower robustness, compared with the naive FGSM. Moreover, such a perturbation is \textit{image-agnostic}, i.e. a single MAP works well on all test data. We notice that for some models, the UAP performs poorly when only using 100 data for generating the perturbation. These results are consistent with the earlier finding that the UAP requires a large amount of data to achieve a high fooling ratio \cite{moosavi2017universal}.

\subsection{Transferability in Meta Perturbations}

We take a step further to investigate the transferability of MAP. That is, whether the meta perturbations computed from a specific architecture are also effective for another architecture. Table \ref{tab:transferability} shows a matrix summarizing the transferability of MAP across seven models. For each architecture, we compute a meta perturbation and show the accuracy on all other architectures, with one-step update on the target model. We show the accuracies without using MAP at initialization in the bottom row. As shown in Table \ref{tab:transferability}, the MAP generalizes very well across other models. For instance, the meta perturbation generated from the DenseNet121 achieves comparable performance to those perturbations computed specifically for other models. In practice, when crafting an adversarial example for some other neural networks, using the meta perturbation computed on the DenseNet121 at initialization can lead to a stronger attack, compared with the from-scratch method. The results show that the meta perturbations are therefore not only image-agnostic, but also \textit{model-agnostic}. Such perturbations are generalizable to a wide range of deep neural networks.

\subsection{Ablation Study}

While the above meta perturbations are computed for a set $\mathbb{D}$ containing 100 images from the training set, we now examine the influence of the size $|\mathbb{D}|$ on the effectiveness of the MAP. Here we use the ResNet18 for computing the MAP. The results, which are shown in Fig. \ref{fig:different-size}, indicate that a larger size of $\mathbb{D}$ leads to better performance. Surprisingly, even using only 10 images for computing a meta perturbation, such a perturbation still causes the robustness to drop by around 15\%, compared with the naive FGSM. This verifies that meta perturbations have a marvelous generalization ability over unseen data points, and can be computed on a very small set of training data.

\begin{figure}
\centering{}\includegraphics[width=0.9\columnwidth]{figures/map-vs-fgsm_size}\caption{\label{fig:different-size}Accuracy on the test set $\mathbb{T}$ versus the number of images in $\mathbb{D}$ for learning MAP.}
\end{figure}
\section{Conclusion and Future Work}

In this work, we show the existence and realization of a meta adversarial perturbation (MAP), an initial perturbation that can be added to the data for generating more effective adversarial attacks through a one-step gradient ascent. We then propose an algorithm to find such perturbations and conduct
extensive experiments to demonstrate their superior performance. For future work, we plan to extend this idea to time-efficient adversarial training \cite{shafahi2019adversarial,wong2019fast,zhang2019you,zheng2020efficient}. Also, evaluating our attack on robust pre-trained models or different data modalities is another research direction.

\bibliographystyle{aaai22}
\bibliography{aaai22}

\end{document}


Peer reviews:

content: confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct
rating: 6: Marginally above acceptance threshold
review: This paper proposes a meta adversarial perturbation (MAP) and obtains a better initialization that causes natural images to be misclassified with high probability, which is only updated through a one-step gradient ascent update.

Experimental performance can be demonstrated that the method can better mislead adversarial example classifiers and achieve better performance. 

The weakness is also listed as follows:

1.	The authors are encouraged to evaluate the performance on more datasets, such as ImageNet.

2.	More baselines should be introduced for comparison, such as DIM[1] and TIM[2].


[1] Xie et al. Improving transferability of adversarial examples with input diversity

[2] Dong et al. Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks


title: Meta Adversarial Perturbations