G ENERATING A DVERSARIAL E XAMPLES WITH T ASK O RIENTED M ULTI -O BJECTIVE O PTIMIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Deep learning models, even the-state-of-the-art ones, are highly vulnerable to adversarial examples.
Adversarial training is one of the most efﬁcient methods to improve the model’s robustness.
The key factor for the success of adversarial training is the capability to generate qualiﬁed and divergent adversarial examples which satisfy some objectives/goals (e.g., ﬁnding adversarial examples that maximize the model losses for simultaneously attacking multiple models).
Therefore, multi-objective optimization (MOO) is a natural tool for adversarial example generation to achieve multiple objectives/goals simultaneously.
However, we observe that a naive application of MOO tends to maximize all objectives/goals equally, without caring if an objective/goal has been achieved yet.
This leads to useless effort to further improve the goal-achieved tasks, while putting less focus on the goal-unachieved tasks.
In this paper, we propose Task Oriented MOO to address this issue, in the context where we can explicitly deﬁne the goal achievement for a task.
Our principle is to only maintain the goal-achieved tasks, while letting the optimizer spend more effort on improving the goal-unachieved tasks.
We conduct comprehensive experiments for our Task Oriented MOO on various adversarial example generation schemes.
The experimental results ﬁrmly demonstrate the merit of our proposed approach.
I NTRODUCTION
Deep neural networks are powerful models that achieve impressive performance across various domains such as bioinformatics (Spencer et al., 2015), speech recognition (Hinton et al., 2012), computer vision (He et al., 2016), and natural language processing (Vaswani et al., 2017).
Despite achieving state-of-the-art performance, these models are extremely fragile, as one can easily craft small and imperceptible adversarial perturbations of input data to fool them, hence resulting in high misclassiﬁcations (Szegedy et al., 2014; Goodfellow et al., 2015).
Accordingly, adversarial training (AT) (Madry et al., 2018; Zhang et al., 2019) has been proven to be one of the most efﬁcient approaches to strengthen model robustness (Athalye et al., 2018).
AT requires challenging models with divergent and qualiﬁed adversarial examples (Madry et al., 2018;
Zhang et al., 2019; Bui et al., 2021)
so that the robustiﬁed models can defend against adversarial examples.
Therefore, generating adversarial examples is an important research topic in Adversarial Machine Learning (AML).
Several perturbation based attacks have been proposed, notably PGD (Madry et al., 2018), CW (Carlini & Wagner, 2017), and AutoAttack (Croce & Hein, 2020).
Most of them aim to optimize a single objective/goal, e.g., maximizing the cross-entropy (CE) loss w.r.t.
the ground-truth label (Goodfellow et al., 2015; Madry et al., 2018), maximizing the Kullback-Leibler (KL) divergence w.r.t.
the predicted probabilities of a benign example (Zhang et al., 2019), or minimizing a combination of perturbation size and predicted loss to a targeted class as in Carlini & Wagner (2017).
However, in many contexts, we need to ﬁnd qualiﬁed adversarial examples satisfying multiple objectives/goals, e.g., ﬁnding an adversarial example that can attack simultaneously multiple models in an ensemble model (Pang et al., 2019; Bui et al., 2021), ﬁnding an universal perturbation that can attack simultaneously multiple benign examples (Moosavi-Dezfooli et al., 2017).
Obviously, these adversarial generations have a nature of multi-objective problem rather than a single-objective one.
Consequently, using single-objective adversarial examples leads to a much less adversarial robustness in ensemble learning as discussed in Section 4.
Multi-Objective Optimization (MOO) (Désidéri, 2012) is an optimization problem to ﬁnd a Pareto optimality that aims to optimize multiple objective functions.
In a nutshell, MOO is a natural tool for the aforementioned multi-objective adversarial generations.
However, a direct and naive application of MOO to generating robust adversarial examples for multiple models or ensemble of transformations does not work satisfactorily (cf.
Appendix E).
Concretely, it can be observed that the tasks are not optimized equally.
The optimizing process focuses too much on one dominating task and can be trapped easily by it, hence leading to downgraded attack performances.
Intuitively, for multi-objective adversarial generations, we can explicitly investigate if an objective or a task achieves or fails to achieve its goal (e.g., the current adversarial example can fool a model successfully or unsuccessfully in multiple models).
To avoid some tasks dominating others during the optimization process, we can favour more the tasks that are failing and pay less attention to the tasks that are performing well.
For example, in the context of attacking multiple models, we update an adversarial example x a to favor the models that x a has not attacked successfully yet, while trying to maintain the attack capability of x a on the already successful models.
In this way, we expect that no task really dominates others and all tasks can be updated equally to fulﬁll their goals.
Bearing this in mind, we propose a new framework named TA sk Oriented M
ulti-
O bjective
O ptimization (TA-MOO) with multi-objective adversarial generations as the demonstrating applications.
Speciﬁcally, we learn a weight vector (i.e., each dimension is the weight for a task) lying on a simplex corresponding to all tasks.
To favor the unsuccessful tasks while maintaining the success of the successful ones, we propose a geometry-based regularization term that represents the distance between the original simplex and a reduced simplex which involves the weight vectors for the currently unsuccessful tasks only.
Furthermore, along with the original quadratic term of the standard MOO helping to improve all tasks, minimizing our geometry-based regularization term encourages the weights of the goal-achieved tasks to be as small as possible, while inspiring those for the goal-unachieved ones to have a sum close to 1.
By doing so, we aim to focus more on improving the goal-unachieved tasks, while still maintain the performance of goal-achieved tasks.
Most related work to ours is Wang et al.
(2021), which considers the worst-case performance across all tasks.
However, this original principle reduces the generalizability to other tasks.
To mitigate this issue, a speciﬁc regularization was proposed to balance all tasks’ weights.
Our work, which casts an adversarial generation task as a multi-objective optimization problem, is conceptually different from that work, although both methods can be applied to similar tasks.
Further discussion about relate work can be found in Appendix A.
To summarize, our contributions in this work include: (C1)
Ours is among the ﬁrst works that views an adversarial generation task as a multi-objective optimization problem and comprehensively study the issues when applying multi-objective optimization to tackle this adversarial generation task.
(C2)
To mitigate the shortcomings of the original MOO (Désidéri, 2012;
Sener & Koltun, 2018) when applying to multi-objective adversarial generation, we propose the TA-MOO framework which employs a geometry-based regularization term to favor the unsuccessful tasks, while trying to maintain the performance of the already successful tasks.
(C3)
We conduct comprehensive experiments for three adversarial generation tasks and one adversarial training task including: (i) attacking multiple models, (ii) learning universal perturbation, (iii) attacking over many data transformations, and (iv) adversarial training on ensemble learning setting.
The experimental results show that our TA-MOO outperforms the baselines by a wide margin on the three aforementioned adversarial generation tasks.
More importantly, our adversary brings a great beneﬁt on improving adversarial robustness.
B ACKGROUND
We revisit the background of multi-objective optimization (MOO), which lays the foundation for our task-oriented MOO in the sequel.
Given multiple objective functions f ( δ ) :
=
[ f 1 ( δ ) , ..., f m ( δ )] where each f i : R d → R , we aim to ﬁnd the Pareto optimal solution that simultaneously maximizes all objective functions:
[Equation]
While there are a variety of MOO solvers (Miettinen, 2012; Ehrgott, 2005), in this paper, we adapt from the multi-gradient descent algorithm (MGDA) that was proposed suitably for end-to-end learning by Désidéri (2012).
Speciﬁcally, MGDA combines the gradients of individual objectives to a single optimal direction that increases all objectives simultaneously.
The optimal direction corresponds to the minimum-norm point that can be found by solving the quadratic programming problem:
[Equation]
) where ∆ m = (cid:8) π ∈ R m + : (cid:107) π (cid:107) 1 = 1 (cid:9) is the m -simplex and Q ∈ R m × m is the matrix with Q ij = ∇ δ f i ( δ ) T ∇ δ f j ( δ ) .
Finally, the solution of the problem 1 can be found iteratively with each update step δ =
δ + ηg where g is the combined gradient g = (cid:80) mi =1 w ∗ i ∇ δ f i ( δ ) and η > 0 is a sufﬁciently small learning rate.
Furthermore, Désidéri (2012) also proved that by using an appropriate learning rate at each step, we reach the Pareto optimality point δ ∗ at which there exist w ∈ ∆ m such that (cid:80)
[Equation]
O UR P ROPOSED M ETHOD 3. T ASK O RIENTED M ULTI -O BJECTIVE O PTIMIZATION
We now present our TA sk Oriented M
ulti-
O bjective O ptimization (TA-MOO).
We consider the
MOO problem in (1) where each task T i ( i = 1 , ..., m ) corresponds to the objective function f i ( δ ) ( i = 1 , ..., m ) .
Additionally, assume that given a task T i , we can explicitly observe if this task has currently achieved its goal (e.g., the current adversarial example x can fool successfully the model f i ), which is named a goal-achieved task.
We also name a task that has not achieved its goal a goal-unachieved task.
Different from the standard MOO, which equally pays equal attention to all tasks, our TA-MOO focuses on improving the currently goal-unachieved tasks, while trying to maintain the performance of the goal-achieved tasks.
By this principle, we expect all tasks would be equally improved to simultaneously achieve their goals.
To be more precise, we depart from δ 0 and consecutively update in L steps to obtain the sequence δ 1 , δ 2 , ..., δ L that approaches the optimal solution.
Considering the t -th step (i.e., 1 ≤ t ≤ L ), we currently have δ t and need to update it to obtain δ t +1 .
We examine the tasks that have achieved their goals already and denote them as T 1 , T 2 , ..., T s without the loss of generalization.
Here we note that the list of goal-achieved tasks is empty if s = 0 and the list of goal-unachieved tasks is empty if s = m .
Speciﬁcally, to ﬁnd δ t +1 , we ﬁrst solve the following optimization problem (OP):
[Equation]
where Q ∈ R m × m with Q ij = ∇ δ f i ( δ t ) T ∇ δ f j ( δ t ) , λ > 0 is a trade-off parameter, and Ω ( w ) is a regularization term to let the weights focus more on the goal-unachieved tasks.
We next compute the combined gradient g t and update δ t as:
[Equation]
The OP in (3) consists of two terms.
The ﬁrst term w T Qw ensures that all tasks are improving, while the second term Ω ( w ) serves as the regularization to restrict the goal-achieved tasks T 1 , ..., T s by setting the corresponding weights w 1 , ..., w s as small as possible.
Before getting into the details of the regularization, we emphasize that to impose the constraint w ∈ ∆ m , we parameterize w = softmax ( α ) with α ∈ R m and solve the OP in (3) using gradient descent.
In what follows, we discuss our proposed geometry-based regularization term Ω ( w ) .
Simplex-based regularization.
Let S u
= (cid:8) [Equation]
=
[Equation]
[Equation]
[Equation]
s
[Equation]
=
[Equation]
be a simplex w.r.t.
the goal-unachieved tasks and S = { 0 s } × S u be the extended simplex, where 0 s is the s -dimensional vector of all zeros.
We deﬁne the regularization term Ω ( w ) as the distance from w to the extended simplex S :
[Equation]
Because S is a compact and convex set and (cid:107) w − π (cid:107) 22 is a differentiable and convex function, the optimization problem in (4) has a unique global minimizer Ω ( w ) =
(cid:107) w − proj S w [Equation] 22 , where the projection proj S ( w ) is deﬁned as
[Equation]
The following lemma shows us how to ﬁnd the projection proj S ( w ) and evaluate Ω ( w ) .
Lemma 1.
Sorting w s +1: m into u s +1: m such that u s +1 ≥
u s +2 ≥ ... ≥ u m .
Deﬁning ρ = max (cid:110)
[Equation]
≤
[Equation]
m : u i
[Equation]
i
[Equation]
s
[Equation]
− (cid:80) ij = s +1 u j (cid:17) > 0 (cid:111) .
Denoting γ = 1 ρ (cid:0) 1 − (cid:80) ρi
=
s +1 u
i (cid:1) , the projection proj S ( w ) can be computed as
[Equation]
otherwise Furthermore, the regularization Ω ( w ) has the form:
[Equation]
With further algebraic manipulations, Ω ( w ) can be signiﬁcantly simpliﬁed as shown in Theorem 1.
Theorem 1.
The regularization Ω ( w ) has the following closed-form:
[Equation]
The proof of Lemma 1 and Theorem 1 can be found in Appendix B.1.
Evidently, the regularization term in Eq.
(6) in Theorem 1 encourages the weights w 1: s associated with the goal-achieved tasks to be as small as possible and the weights w s +1: m associated with the goal-unachieved tasks to move closer to the simplex S u (i.e., (cid:80) mi
= s +1 w
i is closer to 1 ).
Parameterized TA-MOO.
Algorithm 1 summarizes the key steps of our TA-MOO.
We use gradient descent to ﬁnd solution δ for the OP 1 in L steps and at each iteration we solve the OP in 3 in K steps using gradient descent solver with the parameterization w = softmax ( α ) .
To reduce computational cost, at each iteration we reuse the previous solution α and use a few steps K (i.e., K ≤ 10 ) to get new solution.
We then compute the combined gradient g t and ﬁnally update δ t to δ t +1 using the combined gradient g t (or sign ( g t ) in the case of L ∞ norm).
The projecting operation in step 13 is to project δ to a valid space specifying to applications that we introduce hereon. λ t + η δ g t .
Project δ t +1 to a valid space (speciﬁc to domain, e.g., (cid:107) δ (cid:107) ≤ (cid:15) ).
14: end for 15: Output δ ∗ = L .
3. A PPLICATIONS IN A DVERSARIAL G ENERATION
Although TA-MOO is a general framework, we in this paper focus on its applications in adversarial generation.
Following Wang et al.
(2021), we consider three tasks of generating adversarial examples.
Generating adversarial examples for an ensemble model.
Considering an ensemble classiﬁer with multiple classiﬁcation models h 1 , h 2 , ..., h m , where h i ( x ) ∈ ∆ M = (cid:8) π ∈ R M + : (cid:107) π (cid:107) 1 = 1 (cid:9) with the number of classes M .
Given a data sample x , our aim is to ﬁnd an adversarial example x a = x + δ that can successfully attack all the models.
Speciﬁcally, we consider a set of tasks each of which, T i , is about whether x + δ can successfully attack model
h i , deﬁned as:
[Equation]
where y is the ground truth label of x , I is the indicator function and h i ( x, k ) returns the probability to predict x to the class k .
To ﬁnd a perturbation δ that can attack successfully all models, we solve the following multi-objective optimization problem:
[Equation]
where f i ( δ ) = (cid:96) ( h i ( x + δ ) , y ) with the loss function (cid:96) which could be the cross-entropy (CE) loss (Madry et al., 2018), the Kullback-Leibler (KL) loss (Zhang et al., 2019), or the Carlini-Wagner (CW) loss (Carlini & Wagner, 2017).
Generating universal perturbations.
Considering a single classiﬁcation model h with h ( x ) ∈ ∆ M and a batch of data samples x 1 , x 2 , ..., x B , we would like to ﬁnd a perturbation δ with (cid:107) δ (cid:107) ≤ (cid:15) such that x ai = x
i + δ, i = 1 , ..., B , are adversarial examples.
We deﬁne the task T i as ﬁnding the adversarial example x ai = x
i + δ for data sample x i .
For each task T i , we can deﬁne its goal as ﬁnding successfully the adversarial example x ai :
[Equation]
To ﬁnd the perturbation δ , we solve the following multi-objective optimization problem:
[Equation]
where f i ( δ ) = (cid:96) ( h ( x ai ) , y i ) =
(cid:96) ( h ( x i + δ ) , y i ) with y i the ground-truth label of x i .
Generating adversarial examples against transformations.
Considering a single classiﬁcation model h and m categories of data transformation P 1: m (e.g., rotation, lighting, and translation).
Our goal is to ﬁnd an adversarial attack that is robust to these data transformations.
Speciﬁcally, given a benign example x , we would like to learn a perturbation δ with (cid:107) δ (cid:107) ≤ (cid:15) that can successfully attack the model after any transformation t i ∼ P i is applied.
To formulate as an MOO problem, we consider the task T i as ﬁnding the adversarial example x ai =
t
i ( x + δ ) with t i ∼ P i .
For each task T i , we can deﬁne the goal as ﬁnding successfully the adversarial example x ai :
[Equation]
To ﬁnd the perturbation δ , we solve the following multi-objective optimization problem:
[Equation]
( δ )] , where f i ( δ ) =
E t
i ∼P i [ (cid:96) ( h ( t i ( x + δ )) , y )] with y the ground-truth label of x .
E XPERIMENTS
In this section, we provide extensive experiments across four settings: (i) generating adversarial examples for ensemble of models (ENS, Sec 4.1), (ii) generating universal perturbation (UNI, Sec 4.3) , (iii) generating robust adversarial examples against Ensemble of Transformations (EoT, Sec 4.4), and (iv) adversarial training for ensemble of models (AT, Sec 4.2).
The details of each setting can be found in Appendix C.
General settings.
Through our experiments, we use six common architectures for the classiﬁer including ResNet18 (He et al., 2016), VGG16 (Simonyan & Zisserman, 2014), GoogLeNet (Szegedy et al., 2015), EfﬁcientNet (Tan & Le, 2019), MobileNet Howard et al.
(2017), and WideResNet
Zagoruyko & Komodakis (2016) with the implementation 1 .
We evaluate on the full testing set of two benchmark datasets which are CIFAR10 and CIFAR100 (Krizhevsky et al., 2009).
We observed that the attack performance is saturated with standard training models.
Therefore, to make the job of adversaries more challenging, we use Adversarial Training with PGD-AT (Madry et al., 2018) to robustify the models and use these robust models as the victim models in our experiments.
Evaluation metrics.
We use three metrics to evaluate the attack performance including (i) A-All: the
Attack Success Rate (ASR) when an adversarial example can achieve goals in all tasks.
This is considered as the most important metric to indicate how well one method can achieve in all tasks; (ii)A-Avg: the average Attack Success Rate over all tasks which indicate the average attacking performance; (iii) { A-i } Ki =1 : Attack Success Rate in each individual task.
For reading comprehension purposes, if necessary the highest/second highest performance in each experimental setting is highlighted in
Bold /Underline and the most important metric(s) is emphasized in blue color.
Baseline methods.
We compare our method with the Uniform strategy which assigns the same weight for all tasks and the MinMax method (Wang et al., 2021) which examines only the worstcase performance across all tasks.
To increase the generality to other tasks, MinMax requires a regularization to balance between the average and the worst-case performance.
We use the same attack setting for all methods: the attack is the L ∞ untargeted attack with 100 steps, step size η δ = 2 /and perturbation limitation (cid:15) = 8 / 255 .
The GD solver in TA-MOO uses 10 steps with learning rate η w = 0 .
Further detail can be found in Appendix C.
4. A DVERSARIAL E XAMPLES FOR E NSEMBLE OF M ODELS (ENS)
Experimental setting.
In our experiment, we use an ensemble of four adversarially trained models:
ResNet18, VGG16, GoogLeNet, and EfﬁcientNet.
The architecture is the same for both the CIFAR and CIFAR100 datasets except for the last layer which corresponds with the number of classes in each dataset.
The ﬁnal output of the ensemble is an average of the probability outputs (i.e., output of the softmax layer).
We use three different losses as an object for generating adversarial examples including CE (Madry et al., 2018), KL (Zhang et al., 2019), and CW (Carlini & Wagner, 2017).
Results 1: TA-MOO achieves the best performance.
Table 1 shows the results of attacking the ensemble model on the CIFAR10 and CIFAR100 datasets.
It can be seen that TA-MOO signiﬁcantly outperforms the baselines and achieves the best performance in all the settings.
For example, the improvement over the Uniform strategy is around 10% on both datasets with the CE loss.
Comparing to the MinMax method, the biggest improvement is around 3% for CIFAR10 with CW loss and the lowest one is around 0.6% with the KL loss.
The improvement can be observed in all the settings, showing the generality of the proposed method.
Results 2: When does not MOO work?
It can be observed that MOO falls behind all other methods, even compared with the Uniform strategy.
Our hypothesis for the failure of MOO is that in the original setting with an ensemble of 4 diverse architectures (i.e., ResNet18, VGG16, GoogLeNet, and EfﬁcientNet) there is one task that dominates the others and makes MOO become trapped (i.e., focusing on improving the dominant task).
To verify our hypothesis, we measure the gradient norm (cid:107)∇ δ f i ( δ ) (cid:107) corresponding to each model and the ﬁnal weight w of 1000 samples and report the results in Table 2.
It can be seen that the EfﬁcientNet has a much lower gradient strength, therefore, it has a much higher weight.
This explains the highest ASR observed in EfﬁcientNet and the large gap of 19% (56.11% in EfﬁcientNet and 37.05% in GoogLeNet).
To further conﬁrm our hypothesis, we provide an additional experiment on a non-diverse ensemble model which consists of 4 individual
ResNet18 models.
It can be observed that in the non-diverse setting, the gradient strengths are more balanced across models, indicating that no task dominates others.
As a result, MOO shows its effectiveness by outperforming the Uniform strategy by 4.3% in A-All.
Results 3: The importance of the Task-Oriented regularization.
It can be observed from Tablethat in the diverse setting, TA-MOO has a much lower gap (4%) between the highest ASR (53.4% at EfﬁcientNet) and the lowest one (49.29% at GoogLeNet) compared to MOO ( 19%).
Moreover, while the ASR of EfﬁcientNet is lower by 2.7%, the ASRs of all other architectures have been improved considerably (i.e., 12% in GoogLeNet).
This improvement shows the importance of the Task-Oriented regularization, which helps to avoid being trapped by one dominating task, as happened in MOO.
For the non-diverse setting, when no task dominates others, TA-MOO still shows its effectiveness when improving the ASR in all tasks by around 5%.
The signiﬁcant improvement can be observed in all settings (except the setting on EfﬁcientNet with the CIFAR10 dataset) as shown in Table 1, and demonstrates the generality of the Task-Oriented regularization.
[Table] [Table]
Table 3 reports the SAR-All metric of transferred adversarial examples crafted from a source ensemble (RME) on attacking target ensembles (e.g., RMEVW is an ensemble of 5 models).
A higher number indicates a higher success rate of attacking a target model, therefore, also implies a higher transferability of adversarial examples.
It can be seen that our TA-MOO adversary achieves the highest attacking performance on the whitebox attack setting, with a huge gap of 9.24% success rate over the Uniform strategy.
Our method also achieves the highest transferability regardless diversity of a target ensemble.
More speciﬁcally, on target models such as REV, MEV, and RMEV, where members in the source ensemble (RME) are also in the target ensemble, our TA-MOO signiﬁcantly outperforms the Uniform strategy, with the highest improvement is 5.19% observed on target model
RMEV.
On the target models EVW and MVW which are less similar to the source model, our method still outperforms the Uniform strategy by 1.46% and 1.65%.
The superior performance of our adversary on the transferability shows another beneﬁt of using multi-objective optimization in generating adversarial examples.
By reaching the intersection of all members’ adversarial regions, our adversary is capable to generate a common vulnerable pattern on an input image shared across architectures, therefore, increasing the transferability of adversarial examples.
More discussion can be found in Appendix D.1.
4. A DVERSARIAL T RAINING WITH TA-MOO FOR E NSEMBLE OF M ODELS (ENS)
We conduct adversarial training with adversarial examples generated by MOO and TA-MOO attacks to verify the quality of these adversarial examples and report results on Table 4.
The detailed setting and more experimental results can be found in Appendix D.2.
Result 1: Reducing transferability.
It can be seen that the SAR-All of MOO-AT and TA-MOO-AT are much lower than that on other methods.
More speciﬁcally, the gap of SAR-All between PGD-AT and TA-MOO-AT is (5.33%) 6.13% on the (non) diverse setting.
The lower SAR-All indicating that adversarial examples are harder to transfer among ensemble members on the TA-MOO-AT model than on the PGD-AT model.
[Table] [Table]
MinMax-AT 79.
MOO-AT 79. TA-MOO-AT 79.
Result 2: Producing more robust single members.
The comparison of average SAR shows that adversarial training with TA-MOO produces more robust single models than PGD-AT does.
More speciﬁcally, the average robust accuracy (measured by 100% - A-Avg ) of TA-MOO-AT is 32.17%, an improvement of 6.06% over PGD-AT in the non-diverse setting, while there is an improvement of 4.66% in the diverse setting.
Result 3: Adversarial training with TA-MOO achieves the best robustness.
More speciﬁcally, on the non-divese setting, TA-MOO-AT achives 38.22% robust accuracy, an improvement of 1% over MinMax-AT and 5.44% over standard PGD-AT.
On the diverse setting, the improvement over MinMax-AT and PGD-AT are 0.9% and 4%, respectively.
The root of the improvement is the ability to generate stronger adversarial examples in the the sense that they can challenge not only the entire ensemble model but also all single members.
These adversarial examples lie in the joint insecure region of members (i.e., the low conﬁdence region of multiple classes), therefore, making the decision boundaries more separate.
As a result, adversarial training with TA-MOO produces more robust single models (i.e., lower SAR-Avg) and signiﬁcantly reduces the transferability of adversarial examples among members (i.e., lower SAR-All).
These two conditions explain the best ensemble adversarial robustness achieved by TA-MOO.
4. U NIVERSAL P ERTURBATION (UNI)
Experimental setting.
We follow the experimental setup in Wang et al.
(2021), where the full test set (10k images) is randomly divided into equal-size groups (K images per group).
The comparison has been conducted on the CIFAR10 and CIFAR100 datasets, with an adversarially trained ResNet model and CW loss.
We observed that the ASR-All was mostly zero, indicating that it is difﬁcult to generate a general perturbation for all data points.
Therefore, in Table 5 we use ASR-Avg to compare the performances of the methods.
More experiments on VGG16 and EfﬁcientNet models can be found in Appendix D.3.
Results.
Table 5 shows the evaluation of generating universal perturbations on the CIFAR10 and
CIFAR100 datasets, respectively.
K represents the number of images that are using the same perturbation.
The larger the value of K , the harder it is to generate a universal perturbation that can be applied successfully to all images.
It can be seen that with a small number of tasks (i.e.,
K =4), MOO and TA-MOO achieve lower performance than the MinMax method.
However, with a large number of tasks (i.e, K ≥ 8 ), MOO and TA-MOO show their effectiveness and achieve the [Table] [Table] MinMax 59.
MOO 53.
TA-
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
best performance.
More speciﬁcally, on the CIFAR10 dataset, the improvements of MOO over the
Uniform strategy are 5.6%, 4%, 3.2%, and 2.5% with K = 8 , K = 12 , K = 16 , and K = 20 , respectively.
On the same setting, TA-MOO signiﬁcantly improves MOO by around 4% in all the K settings and consistently achieves the best performance.
Unlike the ENS setting, in the UNI setting,
MOO consistently achieves better performance than the Uniform strategy .
This improvement can be explained by the fact that in the UNI setting with the same architecture and data transformation, no task dominates the others.
There will be a case (a group) when one sample is extremely close to/far from the decision boundary, and hence easier/harder to fool.
However, in the entire test set with a large number of groups, the issue of dominating tasks is lessened.
4. R OBUST A DVERSARIAL E XAMPLES AGAINST T RANSFORMATIONS (E O T)
Results.
Table 6 shows the evaluation on the CIFAR10 and CIFAR100 datasets with 7 common data transformations.
It can be observed that (i) MOO has a lower performance than the baselines, (ii)
the
Task Oriented regularization signiﬁcantly boosts the performance, and (iii) our TA-MOO method achieves the best performance on both settings and outperforms the MinMax method 0.6% and 1.1% in the CIFAR10 and the CIFAR100 experiments, respectively.
The low performance of MOO in observation (i) is again caused by the issue of one task dominating others.
In the EoT setting, it is because of the V-vertical ﬂip transformation as shown in Table 6.
Observation (ii) provides another piece of evidence to support the effectiveness of the Task-Oriented regularization for MOO.
This regularization boosts the ASRs in all the tasks (except V - the dominant one), increases the average
ASR by 5.45% and 2.5% in the CIFAR10 and CIFAR100 experiments, respectively.
C ONCLUSION
In this paper, we propose Task Oriented Multi-Objective Optimization (TA-MOO), with speciﬁc applications to adversarial generation tasks.
We develop a geometry-based regularization term to favor the goal-unachieved tasks, while trying to maintain the the goal-achieved tasks.
We conduct comprehensive experiments to showcase the merit of our proposed approach on generating adversarial examples and adversarial training.
On the other hand, there are acknowledged limitations of our method such as weaknesses of the gradient-based solver and lacking theory on algorithm’s convergence which might be target for future works.
APPENDIX
The Appendix provides technical and experimental details as well as auxiliary aspects to complement the main paper.
Brieﬂy, it contains the following:
• Appendix A: Discussion on related work.
• Appendix B: Detailed proof and an illustration of our methods.
• Appendix C: Detailed description of experimental settings.
• Appendix D.1: Additional experiments on transferability of adversarial examples in the ENS setting.
• Appendix D.2:
Additional experiments on adversarial training with our methods.
• Appendix D.3: Additional experiments on the UNI setting.
• Appendix D.4: Additional experiments on the EoT setting.
• Appendix D.5: Additional comparison on speed of generating adversarial examples.
• Appendix D.6: Additional experiments on sensitivity to hyper-parameters.
• Appendix D.7: Additional comparison with standard attacks on attacking performance.
• Appendix D.8: Additional experiments on attacking the ImageNet dataset.
• Appendix E.1: Additional discussions on the dominating issue and when MOO can work.
• Appendix E.2: A summarary on the importance of Task-Oriented regularization.
• Appendix E.3: Discussion on the limitation of MOO solver.
• Appendix E.4: Discussion on correlation between the objective loss and attack performance.
• Appendix E.5: Additional experiments with multi-task learning methods.
• Appendix E.6: Discussion on the conﬂicting between gradients in the adversarial generation task.
• Appendix E.7: Discussion on the convergence of our methods.
• Appendix E.8: Additional experiments with MOO with different initializations.
A R ELATED W ORK
Multi-Objective Optimization for multi-task learning.
(Désidéri, 2012) proposed a multigradient descent algorithm for multi-objective optimization (MOO) which opens the door for the applications of MOO in machine learning and deep learning.
Inspired by Désidéri (2012), MOO has been applied in multi-task learning (MTL) (Sener & Koltun, 2018; Mahapatra & Rajan, 2020), few-shot learning (Ye et al., 2021), and knowledge distillation (Du et al., 2020).
Speciﬁcally, the work of Sener & Koltun (2018) viewed multi-task learning as a multi-objective optimization problem, where a task network consists of a shared feature extractor and a task-speciﬁc predictor.
The work of Mahapatra & Rajan (2020) developed a gradient-based multi-objective MTL algorithm to ﬁnd a solution that satisﬁes the user preferences.
The work of Lin et al.
(2019) proposed a Pareto MTL to ﬁnd a set of well-distributed Pareto solutions which can represent different trade-offs among different tasks.
Recently, the work of Liu et al.
(2021b) leveraged MOO with Stein Variational Gradient Descent (Liu & Wang, 2016) to diversify the solutions of MOO.
Additionally, the work of Ye et al.
(2021) proposed a bi-level MOO which can be applied to few-shot learning.
Finally, the work of Du et al.
(2020) applied MOO to enable knowledge distillation from multiple teachers.
Generating adversarial examples with single-objective and multi-objective optimizations.
Generating qualiﬁed adversarial examples is crucial for adversarial training (Madry et al., 2018; Zhang et al., 2019).
Many perturbation based attacks have been proposed, notably FGSM (Goodfellow et al., 2015), PGD (Madry et al., 2018), TRADES (Zhang et al., 2019), CW (Carlini & Wagner, 2017),
BIM (Kurakin et al., 2018), and AutoAttack (Croce & Hein, 2020).
Most adversarial attacks aim to maximize a single objective, e.g., maximizing the cross-entropy (CE) loss w.r.t.
the ground-truth label (Madry et al., 2018), maximizing the Kullback-Leibler (KL) divergence w.r.t.
the predicted probabilities of a benign example (Zhang et al., 2019), or maximizing the CW loss (Carlini & Wagner, 2017).
However, in some contexts, we need to generate adversarial examples maximizing multiple objectives or goals, e.g., attacking multiple models (Pang et al., 2019; Bui et al., 2020) or ﬁnding universal perturbations (Moosavi-Dezfooli et al., 2017).
To this end, the work of Wang et al.
(2021) examined the worst-case scenario by casting the problem of interest as a min-max problem for ﬁnding the weight of each task.
However, this principle leads to a problem of lacking generality in other tasks.
To mitigate the issue, Wang et al.
(2021) proposed a regularization to strike a balance between the average and the worst-case performance.
The ﬁnal optimization was formulated as follow:
[Equation]
Where f i ( v ) is the victim model’s loss (i.e., cross entropy loss or KL divergence) and γ > 0 is the regularization parameter.
The authors used the bisection method (Boyd et al., 2004) with project gradient descent for the inner minimization and project gradient ascent for the outer maximization.
There are several major differences in comparison to MOO and TA-MOO methods: (i)
In principle,
MinMax considers the worst-case performance only while our methods improve performance of all tasks simultaneously.
(ii) MinMax weighs the tasks’ losses to ﬁnd the minimal weighted sum loss in its inner minimization, however, as discussed in Section E.
the higher weighted loss does not directly imply the higher success rate in attacking multi-tasks simultaneously.
In contrast, our methods use multi-gradient descent algorithm (Deb, 2011) in order to increase losses of all tasks simultaneously.
(iii)
The original principle of MinMax leads to the biasing problem to the worst-case task.
The above regularization has been used to mitigate the issue, however, it considers all tasks equally.
In contrast, our TA-MOO takes goal-achievement status of each task into account and focuses more on the goal-unachieved tasks.
Recently, Guo et al.
(2020) proposed a multi-task adversarial attack and demonstrated on the universal perturbation problem.
However, while Wang et al.
and ours can be classiﬁed as an iterative optimization-based attack, Guo et al.
(2020) requires a generative model in order to generate adversarial examples.
While this line of attack is faster than optimization-based attacks at the inference phase, it requires to train a generator on several tasks beforehand.
Due to the difference in setting, we do not compare with that work in this paper.
More recently, Qiu et al.
(2022) proposed a framework to attack a generative Deepfake model using the multi-gradient descent algorithm in their backpropagation step.
While their method also use the multi-objective optimization for generating adversarial examples, there are several major differences to ours.
Firstly, their method aims for a generative Deepfake model while our method aims for the standard classiﬁcation problem which is the most common and important setting in AML.
Secondly, we conduct comprehensive experiments to show that a direct and naive application of MOO to adversarial generation tasks does not work satisfactorily because of the gradient dominating problem.
Most importantly, we propose the TA-MOO method which employs a geometry-based regularization term to favor the unsuccessful tasks, while trying to maintain the performance of the already successful tasks. We have conducted extensive experiments to show that our TA-MOO consistently achieves the best attacking performance across different settings. We also conducted additional experiments with SOTA multi-task learning methods which are PCGrad (Yu et al., 2020) and CAGrad (Liu et al., 2021a) in Appendix E.5. Compared to these methods, our
TA-MOO still achieves the best attack performance thanks to the Task Oriented regularization.
B F URTHER D ETAILS OF THE P ROPOSED M ETHOD B. P ROOFS
Lemma 1.
Sorting w s +1: m into u s +1: m such that u s +1 ≥
u s +2 ≥ ... ≥ u m .
Deﬁning ρ = max (cid:110) s
+ 1 ≤ i ≤ m : u i +
i − s (cid:16) 1 − (cid:80) ij =
s +1 u j (cid:17) > 0 (cid:111) .
Denoting γ = 1 ρ (cid:0) 1 − (cid:80) ρi
=
s +1 u
i (cid:1) , the projection proj S ( w ) can be computed as proj S ( w ) i = (cid:26)1 ≤ i ≤ s max { w i + γ, 0 } otherwise
Furthermore, the regularization Ω ( w ) has the form:
[Equation]
Proof.
The proof is based on Wang & Carreira-Perpinán (2013) with modiﬁcations.
We need to solve the following OP:
[Equation]
We note that π 1 = ...
=
π s = 0 .
The OP of interest reduces to
[Equation]
Using the Karush-Kuhn-Tucker (KKT) theorem, we construct the following Lagrange function:
[Equation]
Setting the derivative w.r.t.
π i to zeros and using the KKT conditions, we obtain:
[Equation]
β [Equation] If π i > 0 , β i = 0 , hence π i = w i + γ > 0 .
Otherwise, if π i = 0 , w i + γ = − β i ≤ 0 .
Therefore, w s +1: m has the same order as π s +1: m and we can arrange them as:
[Equation]
It appears that 1 = (cid:80) mi
= s +
π
i = (cid:80) ρi
=
s +
π
i = (cid:80) ρi
=
s +1 ( w i + γ ) =
(cid:80) ρi
=
s +1 w
i + ( ρ − s ) γ .
Hence, we gain γ = 1 ρ − s (cid:2) 1 − (cid:80) ρi
= s +1 w i (cid:3) = ρ − s (cid:2) 1 − (cid:80) ρi =
s +1 u i (cid:3) .
We now prove that ρ = max (cid:110) s
+ 1 ≤ i ≤ m : u i +
i − s (cid:16) 1 − (cid:80) ij =
s +1 u j (cid:17) > 0 (cid:111) .
• For i
= ρ , we have
[Equation]
• For i < ρ , we have
u i
+ i [Equation] i [Equation]
j = [Equation] u j
[Equation]
i [Equation] i [Equation]
[Equation] i
[Equation]
1 − i (cid:88)
j = s + u j 

=
i − s  
( i − s ) w i + ρ − 1 (cid:88)
j = s + π j − i (cid:88)
j = s + w j  
=
i − s  
( i − s ) w i + ρ − 1 (cid:88)
j = i + π j + i (cid:88)
j = s + ( π j − w j ) 
 =
i − s  
( i − s ) ( w i + γ ) + ρ − 1 (cid:88)
j = i + π j   =
i − s  
( i − s )
π i + ρ − 1 (cid:88)
j = i + π j   [Equation]
• For i > ρ , we have u i + i
[Equation]

[Equation]
i
[Equation]
j
[Equation]
s
[Equation]
[Equation]
j
[Equation]
i − s  
( i − s )
u i + 1 − i (cid:88)
j = s + u j 

=
i − s
 
( i − s )
w i + ρ − 1 (cid:88)
j = s + π j − i (cid:88)
j = s + w j 
 = i − s  
( i − s )
w i + ρ − 1 (cid:88)
j = s + ( π j − w j ) − i (cid:88)
j = ρ w j 

=
i − s
 
( i − s )
w i + ( ρ − s − 1) γ − i (cid:88)
j = ρ w j 

=
i
[Equation]
s   ( ρ −
s −
[Equation]
( w i + γ ) + i (cid:88)
j = ρ ( w i − w j ) 
 ≤ 0 . Therefore, ρ = max (cid:110) s
+ 1 ≤ i ≤ m : u i +
i − s (cid:16) 1 − (cid:80) ij =
s +1 u j (cid:17) > 0 (cid:111) .
Finally, we also have π i = max { w i + γ, 0 } , i = s + 1 , ..., m and π i = 0 , i = 1 , ..., s .
Theorem 1.
The regularization Ω ( w ) has the following closed-form:
[Equation]
Proof.
Recall ρ = max (cid:110) s
+ 1 ≤ i ≤ m : u i +
i − s (cid:16) 1 − (cid:80) ij =
s +1 u j (cid:17) > 0 (cid:111) .
Therefore, ρ = m because we have u m
[Equation]
It follows that γ
[Equation]
(cid:32)
[Equation] − m (cid:88) i = s + [Equation]
[-+ γ 
-]u
[Equation] (cid:33)
= m [Equation]
(cid:32) 1 − m (cid:88) i = s + w i (cid:33) ≥ 0 . [Equation]
B. I LLUSTRATIONS OF H OW MOO AND TA-MOO W ORK
Figure 1 illustrates solutions of MOO and TA-MOO in a scenario of 2 goal-achieved tasks (with corresponding gradients ∇ f s 1 , 2 ) and 2 goal-unachieved tasks (with corresponding gradients ∇ f u 1 , 2 ).
As illustrated in the left ﬁgure, with standard MOO method, where all the tasks’ gradients have been considered regardless their status and the solution associated with the minimal norm is the perpendicular vector as suggested by geometry (Sener & Koltun, 2018).
If considering the goalunachieved tasks only as in the middle case, the MOO solution is the edge case.
However, this extreme strategy ignores all the goal-achieved tasks which might lead to the instability.
The Task Oriented regularization strikes a balance between the two aforementioned strategies as illustrated in the right ﬁgure.
The method focuses more on improving the goal-unachieved tasks while spend less effort to maintain the goal-achieved tasks.
With λ = 0 the TA-MOO optimal solution is equivalent to the standard MOO optimal solution while it becomes the MOO solution in the case of the goal-unachieved tasks only when λ → ∞ .
C E XPERIMENTAL SETTINGS
General settings.
Through our experiments, we use six common architectures including ResNet (He et al., 2016), VGG16 (Simonyan & Zisserman, 2014), GoogLeNet (Szegedy et al., 2015),
EfﬁcientNet (B0) (Tan & Le, 2019), MobileNet Howard et al.
(2017), and WideResNet (with depth 34 and widen factor 10) Zagoruyko & Komodakis (2016) with the implementation of https:
//github.com/kuangliu/pytorch-cifar .
We evaluate on the full testing set (10k) of two benchmark datasets which are CIFAR10 and CIFAR100 (Krizhevsky et al., 2009).
More speciﬁcally, the two datasets have 50k training images and 10k testing images, respectively, with the same image resolution of 32 × 32 × 3 .
However, while the CIFAR10 dataset has 10 classes, the CIFAR dataset has 100 classes and fewer images per class.
Therefore, in general, an adversary is easier to attack a CIFAR100 model than a CIFAR10 one as shown in Table 7.
We observed that the attack performance is saturated with standard training models.
Therefore, to make the job of adversaries more challenging, we use Adversarial Training with PGD-AT (Madry et al., 2018) to robustify the models and use these robust models as victim models in our experiments.
Speciﬁcally, we use the
[Table] SGD optimizer (momentum 0.9 and weight decay 5 × 10 − 4 ) and Cosine Annealing Scheduler to adjust the learning rate with an initial value of 0.1 and train a model in 200 epochs as suggested in the implementation above.
We use PGD-AT L ∞ (Madry et al., 2018) with the same setting for both
CIFAR10 and CIFAR100 datasets, i.e., perturbation limitation (cid:15) = 8 / 255 , k = 20 steps, and step size η = 2 / 255 .
Method settings.
In this work, we evaluate all the methods in the untargeted attack setting with
L ∞ norm.
The attack parameters are the same among methods, i.e., number of attack steps 100, attack budget (cid:15) = 8 / 255 and step size η δ = 2 / 255 .
In our method, we use K =10 to update the weight in each step with learning rate η w = 0 .
Tradeoff parameter λ
= 100 in all experiments.
In MinMax (Wang et al., 2021), we use the same γ = 3 for all settings and use the authors’ implementation 2 .
Attacking ensemble model settings.
In our experiment, we use an ensemble of four adversarially trained models: ResNet18, VGG16, GoogLeNet, and EfﬁcientNet.
The architecture is the same for both the CIFAR10 and CIFAR100 datasets except for the last layer which corresponds with the number of classes in each dataset.
The ﬁnal output of the ensemble is an average of the probability outputs (i.e., output of the softmax layer).
We use three different losses as an object for generating adversarial examples including Cross Entropy (CE) (Madry et al., 2018), Kullback-Leibler divergence (KL) (Zhang et al., 2019), and CW loss (Carlini & Wagner, 2017).
Universal perturbation settings.
We follow the experimental setup in Wang et al.
(2021), such that the full test set (10k images) is randomly divided into equal-size groups (K images per group).
The comparison has been conducted on the CIFAR10 and CIFAR100 datasets, and CW loss.
We use adversarial trained ResNet18, VGG16 and EfﬁcientNet as base models.
We observed that the
ASR-All was mostly zero, indicating that it is difﬁcult to generate a general perturbation for all data points.
Therefore, we use ASR-Avg to compare the performances of the methods.
Robust adversarial examples against transformations settings.
In our experiment, we usecommon data transformations including I-Identity, H-Horizontal ﬂip, V-Vertical ﬂip, C-Center crop,
B-Adjust brightness, R-Rotation, and G-Adjust gamma.
The parameter setting for each transformation has been shown in Table 8.
In the deterministic setting, a transformation has been ﬁxed with one speciﬁc parameter, e.g., center cropping with a scale of 0.6 or adjusting brightness with a factor of 1.3.
While in the stochastic setting, a transformation has been uniformly sampled from its family, e.g., center cropping with a random scale in range (0.6, 1.0) or adjusting brightness with a random factor in range (1.0, 1.3).
The experiment has been conducted on adversarially trained ResNet18 model with the CW loss. [Table] gamma = [Table]
gamma =
U (0 . 7 ,
[Table] [Table] D A DDITIONAL E XPERIMENTS D. T RANSFERABILITY OF ADVERSARIAL EXAMPLES IN THE ENS SETTING
We conduct an additional experiment to evaluate the transferability of our adversarial examples.
We use an ensemble (RME) of three models: ResNet18, MobileNet, and EfﬁcientNet as a source model and apply different adversaries to generate adversarial examples to this ensemble.
We then use these adversarial examples to attack other ensemble architectures (target models), for example,
RMEVW is an ensemble of 5 models including ResNet18, MobileNet, EfﬁcientNet, VGG16 and
WideResNet.
Table 9 reports the SAR-All metric of transferred adversarial examples, where a higher number indicates a higher success rate of attacking a target model, therefore, also implies a higher transferability of adversarial examples.
The ﬁrst column (heading RME) shows SAR-All when adversarial examples attack the source model (i.e., the whitebox attack setting).
The Uniform strategy achieves the lowest transferability.
It can be observed from Table 9 that the Uniform strategy achieves the lowest SAR in the whitebox attack setting.
This strategy also has the lowest transferability in attacking other ensembles (except an ensemble RVW).
MinMax’s transferability drops on dissimilar target models.
While MinMax achieves the second-best performance in the whitebox attack setting, its adversarial examples have a low transferability when target models are different from the source model.
For example, in the target model RVW where there is only one member of the target model from the source model (RME) (i.e., R or ResNet18), MinMax achieves a 23.75% success rate which is lower than the Uniform strategy by 1.28%.
Similar observation can be observed on target models EVW and MVW, where MinMax outperforms the Uniform strategy by just 0.2% and 0.6%, respectively.
TA-MOO achieves the highest transferability on a diverse set of ensembles .
Our TA-MOO adversary achieves the highest attacking performance on the whitebox attack setting, with a huge gap of 9.24% success rate over the Uniform strategy.
Our method also achieves the highest transferability regardless diversity of a target ensemble.
More speciﬁcally, on target models such as REV, MEV, and
RMEV, where members in the source ensemble (RME) are also in the target ensemble, our TA-MOO signiﬁcantly outperforms the Uniform strategy, with the highest improvement is 5.19% observed on target model RMEV.
On the target models EVW and MVW which are less similar to the source model, our method still outperforms the Uniform strategy by 1.46% and 1.65%.
The superior performance of our adversary on the transferability shows another beneﬁt of using multi-objective optimization in generating adversarial examples.
By reaching the intersection of all members’ adversarial regions, our adversary is capable to generate a common vulnerable pattern on an input image shared across architectures, therefore, increasing the transferability of adversarial examples.
D. A DVERSARIAL T RAINING WITH TA-MOO
Setting.
We conduct adversarial training with adversarial examples generated by MOO and TAMOO attacks to verify the quality of these adversarial examples.
We choose an ensemble of 3 MobileNet architectures (non-diverse set) and ensemble of 3 different architectures including [Table]
ResNet18, MobileNet and EfﬁcientNet ( we We measure
SAR of examples by
We use and with η = 2 / 255 for robustness evaluation.
We use SGD optimizer with momentum 0.9 and weight decay 5e-4.
Initial learning rate is 0.1 with
Cosine Annealing scheduler and train on 100 epochs.
Result 1.
Reducing transferability.
It can be seen that the SAR-All of MOO-AT and TA-MOO-AT are much lower than that on other methods.
More speciﬁcally, the gap of SAR-All between PGD-AT and TA-MOO-AT is (5.33%) 6.13% on the (non) diverse setting.
The lower SAR-All indicating that adversarial examples are harder to transfer among ensemble members on the TA-MOO-AT model than on the PGD-AT model.
Result 2.
Producing more robust single members.
The comparison of average SAR shows that adversarial training with TA-MOO produces more robust single models than PGD-AT does.
More speciﬁcally, the average robust accuracy (measured by 100% - A-Avg ) of TA-MOO-AT is 32.17%, an improvement of 6.06% over PGD-AT in the non-diverse setting, while there is an improvement of 4.66% in the diverse setting.
Result 3.
Adversarial training with TA-MOO achieves the best robustness.
More speciﬁcally, on the non-divese setting, TA-MOO-AT achives 38.22% robust accuracy, an improvement of 1% over MinMax-AT and 5.44% over standard PGD-AT.
On the diverse setting, the improvement over
MinMax-AT and PGD-AT are 0.9% and 4%, respectively.
The root of the improvement is the ability to generate stronger adversarial examples in the the sense that they can challenge not only the entire ensemble model but also all single members.
These adversarial examples lie in the joint insecure region of members (i.e., the low conﬁdence region of multiple classes), therefore, making the decision boundaries more separate.
As a result, adversarial training with TA-MOO produces more robust single models (i.e., lower SAR-Avg) and signiﬁcantly reduces the transferability of adversarial examples among members (i.e., lower SAR-All).
These two conditions explain the best ensemble adversarial robustness achieved by TA-MOO.
D. U NIVERSAL P ERTURBATION (UNI)
Additional experimental results.
In addition to the experiments on ResNet18 as reported in Table 5, we would like to provide additional experimental results on two other adversarial trained models
VGG16 and EfﬁcientNet as shown in Table 11.
It can be seen that, TA-MOO consistently achieves the best attacking performance on ResNet18 and VGG16, on both CIFAR10 and CIFAR100 datasets, with K ≥ 8 .
[Table] [Figure]
[Table] [Figure]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
Why does MOO work?
As shown in Table 11, MOO consistently achieves better performance than the Uniform strategy (except for the setting with EfﬁcientNet on the CIFAR100 dataset).
To ﬁnd out the reason for the improvement, we investigate the gradient norm (cid:107)∇ δ f ( δ ) (cid:107) and weight w for the ﬁrst, and second groups (as an example) and the average over 100 groups of the testset as shown in
Table 12.
It can be seen that in the ﬁrst and second groups, there are some tasks that have signiﬁcantly low gradient strengths than other tasks.
The gap of the strongest/weakest gradient strength can be a magnitude of 10 6 indicating the domination of one task over others.
While this issue can cause the failure as in the ENS setting, however, in the UNI setting, the lowest gradient strengths in each group correspond to unsuccessful tasks (unsuccessful adversarial examples) and vice versa.
Recall that we use the multi-gradient descent algorithm to solve MOO, which in principle assigns a higher weight for a weaker gradient vector.
Therefore, in the UNI setting, while the dominating issue still exists, fortunately, the result still ﬁts our desired weighting strategy (i.e., higher weight for an unsuccessful task and vice versa).
Moreover, when there are a large number of groups (i.e., 100 groups), the issue of dominating tasks is alleviated.
The average gradient strength is more balanced as shown in Table 12.
This explains the improvement of MOO over the Uniform strategy in the UNI setting.
D. R OBUST A DVERSARIAL E XAMPLES AGAINST T RANSFORMATIONS (E O T)
We observed that in EoT with the stochastic setting, adjusting gamma sometimes has the overﬂow issue resulting in an inﬁnite gradient.
Recall that our method using MGDA to solve MOO which relies on the stability of gradient strengths.
Therefore, in the case of having inﬁnite gradients, learning weight w is unstable, resulting to lower performance in both MOO and TA-MOO.
To overcome the overﬂow issue, we allocate memory to cache the valid gradient of each task in the previous iteration and replace the inﬁnite value in the current iteration with the valid one in the memory.
The storage only requires a tensor with the same shape as the gradient (i.e., as the exact size of the input), therefore, it does not increase the computation resource signiﬁcantly.
As shown in Table 13, this simple technique helps to improve performance of TA-MOO by 5.3% on both the
CIFAR10 and CIFAR100 datasets.
It also helps to improve performance of MOO by 0.8% and 4.8%, respectively.
Finally, after overcoming the gradient issue, the TA-MOO achieves the best performance on the CIFAR100 dataset and the second best performance on the CIFAR10 dataset (0.4% lower in
ASR-All but 0.8% higher in ASR-Avg when comparing to MinMax).
This result provides additional evidence of the advantage of our method.
[Table] [Table]
MinMax 30.
MOO 21.
TA-MOO 31.
D-C
Uniform 56.
MinMax 59.
MOO 53.
TA-MOO 60.
S-C
Uniform 31.
MinMax 33.
MOO 26.
TA-MOO 28.
MOO (cid:63) 27.
TA-MOO (cid:63) 32.
S-C
Uniform 59.
MinMax 61.
MOO 54.
TA-MOO 56.
MOO (cid:63) 58. TA-MOO
[Table] [Table] [Table]
G ENERATING S PEED C OMPARISON AND E XPERIMENTS ’ S TABILITY
Generating Speed Comparison.
Table 15 shows the average time to generate one adversarial example in each setting.
The results are measured on the CIFAR10 dataset with ResNet18 architecture in the Ensemble of Transformations (EoT) and Universal Perturbation (Uni) settings.
We use 1 Titan
RTX 24GB for the EoT experiment and 4 Tesla V100 16GB each for the other experiments.
It is worth mentioning that our primary focus in this paper is showing the advantage of MOO and the Task-Oriented regularization in generating adversarial examples.
Therefore, we did not try to optimize our implementation in terms of generating time.
Experiments’ Stability.
We conduct an experiment with 5 different random seeds to generate adversarial examples for the ENS setting to evaluate the stability of experimental results on choosing of random seed.
The experiment is on the CIFAR10 dataset, with an ensemble of 4 architectures including ResNet18, VGG16, GoogLeNet, and EfﬁcientNet.
We report mean and variation values in
Table 16.
It can be observed that there is a slight variation in attack performances across methods.
The variation is small enough compared to the gap between methods (i.e., the biggest variation is 0.32% in SAR-All while the smallest gap is 2.51% between MOO and the Uniform approach), therefore, making the comparison still reliable.
D. S ENSITIVITY TO H YPER - PARAMETERS
In this section we provide an analytical experiment on the sensitivity of our TA-MOO method to the tradeoff λ .
The study has been conducted with the ENS setting with CE loss and the EoT setting with deterministic transformations using ResNet18 architecture.
All experiments are on the CIFAR dataset.
The value of λ is changed from 1 to 1000.
It can be observed from Figure 3a (the ENS setting) that (i) increasing λ reduces the performance of dominated task (i.e., ASR on the EfﬁcientNet decreases from 54.49% at λ = 1 to 53.40% at λ = 100 ) while increases performances of other tasks.
In overall, it signiﬁcantly increases the ASR-All performance of the entire ensemble from 29.14% at λ = 1 to 38.01% at λ = 100 .
(ii)
However, over-high λ (i.e., λ > 200 ) leads to the drop of performance in all tasks, resulting in a lower overall performance.
A similar observation can be seen in the EoT setting in Figure 3b.
The attack performance on the dominated task (V-Vertical ﬂipping) decreases from 86.11% at λ = 50 to 83.67% at λ = 200 .
In contract, in the same range of λ the overall performance increases from 32.85% to 34.36%.
The performances of all tasks decrease when using too large λ (i.e., λ > 200 ).
Based on the result of this study, we choose λ = 100 in all the other experiments.
D. C OMPARISON WITH S TANDARD A TTACKS
We conducted an additional comparison on the ENS setting to further conﬁrm the effectiveness of our method over standard adversarial attacks (which consider an entire ensemble as a single model).
More speciﬁcally, we compare with AutoAttack (Croce & Hein, 2020), Brendel-Bethge attack (BB)
[Figure]
[Table]
(Brendel et al. [Table] [Table] Carlini
[Table]
attack
[Table]
(Carlini
[Table]
Wagner [Table] [Table] attack (Madry et al., 2018).
For AutoAttack, we use the standard version which includes 4 different attacks.
For BB attack, we initialized with the PGD attack with 20 steps.
For CW attack, we set the conﬁdence factor to 1.0.
We evaluate these attacks on 2 ensemble settings, a diverse (D) ensemble set with 4 different architectures (ResNet18, VGG16, GoogLeNet, and EfﬁcientNet) and a non-diverse (ND) ensemble set with 4 ResNet18 architectures.
It can be seen from the Table 17 that our TA-MOO attack consistently achieves the best attack performance, with a signiﬁcant gap compared to the best standard attack.
More speciﬁcally, our
TA-MOO method achieves 38.01% (SAR-All metric) on the diverse ensemble set, while the second best attack is AutoAttack with 30.71% (a gap of 7.3%).
On the non-diverse set, the gap between our TA-MOO and AutoAttack is still notably large at 4%.
These standard attacks consider an entire ensemble as a single model, i.e., aim to optimize a single objective given a single ensemble output.
Therefore, they cannot guarantee a successful attack on each member.
[Table]
D. A TTACKING THE I MAGE N ET DATASET
Experimental Setting.
We conduct experiments on the ENS setting using the adversarial pretrained models on the RobustBench (Croce et al., 2021).
We use two sets of an ensemble to verify the importance of our task-oriented strategy.
The ﬁrst set is the robust ensemble (RE) set including 3 robust models: ResNet18 (model ID: Salman2020Do_R18 (Salman et al., 2020), robust accuracy 25.32%), ResNet50 (model ID: Salman2020Do_R50 (Salman et al., 2020), robust accuracy 34.96%) and ResNet50 (model ID: Wong2020Fast (Wong et al., 2019), robust accuracy 26.24%).
The second set is the less-robust ensemble (LE) which includes 3 models: ResNet (model ID: Salman2020Do_R18), ResNet50 (model ID: Salman2020Do_R50) and the standard training ResNet50 (model ID: Standard_R50, robust accuracy 0%).
We use both targeted attack and untargeted attack settings, with (cid:15) = 4 / 255 , and η = 1 / 255 with 20 steps.
We use 5000 images of the validation set to evaluate.
Experimental Results.
We report experimental results with different settings in Table 18, where
RE/LE/TAR/UNTAR represents Robust Ensemble/Less-Robust Ensemble/Targeted Attack/Untargeted Attack, respectively.
It can be seen that, in the robust ensemble setting (RE-TAR and REUNTAR), our MOO achieves a similar performance compared to the baseline, while TA-MOO has a further improvement over MOO.
The gap of SAR-All between TA-MOO and the uniform weighting strategy is 0.1% in the targeted attack setting (RE-TAR), while that in the untargeted attack setting is 1.2%.
In the less-robust ensemble setting (LE-TAR and LE-UNTAR), the improvement of our methods over the baseline is higher than in the robust ensemble setting.
With the gap of SAR-All between TA-MOO and the uniform strategy is 0.38% with the targeted attack setting (LE-TAR), while the gap in the untargeted setting (LE-UNTAR) is 15.22% a signiﬁcantly higher.
While it is acknowledged that the targeted attack is a more common protocol in attacking the ImageNet dataset (Athalye et al., 2018), however, we believe that our signiﬁcant improvement on the untargeted attack is still worth noting.
We conduct an additional experiment on the EoT setting with the ImageNet dataset and report result in Table 19.
In this experiment, we use the robust pretrained ResNet18 model (model ID:
Salman2020Do_R18) as the victim model.
We use the standard attack setting, i.e., targeted attack with (cid:15) = 4 / 255 , η = 1 / 255 with 20 steps.
It can be seen that both MOO and TA-MOO could obtain a better attack performance than the uniform strategy.
It is a worth noting that, in the experiment on the CIFAR10/CIFAR100 datasets (i.e., Table 6 in the main paper) the dominating issue of the vertical ﬁlliping exists and prevents MOO to obtain a better performance.
In the ImageNet dataset, the dominating issue is less serious, therefore, explains the improvement of MOO and corroborates our hypothesis on the issue of dominating task.
[Table]
[Table]
E A DDITIONAL D ISCUSSIONS E. W HEN DOES MOO W ORK ?
The dominating issue.
On one hand, there is the dominating issue that happens in all the three settings.
The issue can be recognized by the gap of attack performance among tasks.
For example, in Table 21 (i
[Table]
e., the ENS setting with the diverse ensemble and MOO method), the gap between highest ASR (at EfﬁcientNet) and lowest ASR (at GoogLeNet) is 19%.
In the EoT setting, the problem is even worse: The largest gap observed is 53.6% as shown in Table 14 (the highest ASR is 88.19% with Vertical ﬂipping and the lowest ASR is 34.54% with Horizontal ﬂipping in with MOO - D-C10 setting).
The dominating issue is also be recognized by the observation that a signiﬁcant small gradient strength of one task on comparison with other tasks’ strength.
For example, in Table
[Table]
21 it can be seen that the gradient strength corresponding to the EfﬁcientNet architecture (mean value is 0.98) is much lower than those of other architectures (mean values are at least 4.29).
As the result, the weight corresponding to the EfﬁcientNet architecture is much higher than those of others.
The root of the dominating issue can be the natural of the setting (i.e., as shown in Table 14 with the
EoT setting, when the domination of the Vertical ﬂipping task can be observed in all methods) or because of the MOO solver which is discussed in Section E.
Overcoming the dominating issue.
On the other hand, if overcoming this issue, MOO can outperform the Uniform strategy.
For example, on attacking the non-diverse ensemble model (i.e.,
ResNets) MOO surpasses the Uniform strategy by 4.3% and 3.5% in the ASR-All and ASR-Avg metrics, respectively.
On generating universal perturbations, MOO outperforms the Uniform strategy in most of the settings.
As discussed in Section D.4, a simple memory caching trick can helps to overcome the inﬁnite gradient issue and signiﬁcantly boosts the performance of MOO or TA-MOO.
Therefore, we believe that developing a technique to lessen the dominating issue might be a potential extension to further improve the performance.
Balancing among goal-unachived tasks.
We observed in the EoT setting, the dominating issue is strictly serious when gradients of some tasks are much weaker/stronger than others.
It is because of the natural of the transformation operations, therefore, this issue happens regardless status of the tasks.
In the set of goal-unachieved tasks’ gradients can exist a dominated one, resulting to a much higher weight of the dominated task.
Therefore, in order to strike a more balance among goal-unachieved tasks, we apply an additional regularization which minimizes the entropy of goal-unachieved weights
H ( w ) =
(cid:80) mi
= s +1 − w i log w i .
If all tasks have been achieved (i.e., s = m ) then the additional regularization will be ignored.
This additional regularization helps to improve further 2% in the EoT setting.
E. I MPORTANCE OF THE T ASK -O RIENTED R EGULARIZATION .
In this discussion, we would like to provide more experimental results in the ENS and EoT settings to further emphasize the contribution of the Task-Oriented regularization.
Figure 4 shows the ASR of each individual task in the ENS setting with three losses and the EoT setting with ResNet architecture and deterministic transformations.
As shown in Figure 4a, in the ENS setting, the MOO adversary produces a much higher ASR on the EfﬁcientNet architecture than other architectures with any losses.
In contrast, the TA-MOO adversary has a lower ASR on the EfﬁcientNet architecture but a much higher ASR on other architectures.
Similar observation can be seen in Figure 4b such that the
ASR corresponding to the V-ﬂipping of MOO is slightly higher than that of TA-MOO, however, the
ASR on other transformations of MOO is much lower than those of TA-MOO.
E. M ORE E FFICIENT MOO S OLVERS
Discussions on the weighted-sum method.
One of the most common approaches to solve the
MOO problem is the scalarizing method, which formulates a single-objective optimization (SOO) such that the optimal solutions to the SOO problem are Pareto optimal solutions to the MOO problem.
While this line of approach (e.g., weighted-sum method) is suitable for end-to-end learning such as deep learning, there are several acknowledged weaknesses: (i) the choice of utility function has a large impact on the computational complexity of the resulted SOO problem (Bjornson et al., 2014;
Björnson & Jorswieck, 2013); (ii) a small change in weights may results in big changes in the combined objective (Caballero et al., 1997), and vice versa, a huge different weights may produce nearly similar result (Coello Coello, 1999); (iii) it does not work well in the case of a non-convex objective space (Deb, 2011).
One of the most common replacement for the weighted-sum method is the (cid:15) constraint method which is applicable to either convex or non-convex problem.
Applying a more efﬁcient MOO solver might be one of the potential extensions of this work.
Discussions on the gradient descent solver.
Inspired by Sener & Koltun (2018), in this paper we use multi-gradient descent algorithm (Deb, 2011) as an MOO solver which casts the multi-objective problem to a single-objective problem.
While Sener & Koltun (2018) used Frank-Wolfe algorithm to [Figure] project the weight into the desired simplex, we use parameterization with softmax instead.
Although this technique is much faster than Frank-Wolfe algorithm, it has some weaknesses that will be addressed in our future work.
More speciﬁcally, the GD solver with softmax parameterization cannot handle well the edge case which is the root of the dominating issue.
The snippet code E.3 provides a minimal example of quadratic optimization problem as similar in MGDA, where the goal is to ﬁnd w ∗ = argmin w ∈ ∆ w (cid:80)
i = (cid:107)
w i g i (cid:107) 22 .
The solver is the Gradient Solver with softmax parameterization.
With input 1 where none of elements dominates others, the solver works quite reasonable with the weights corresponding to 4 ﬁrst elements are equal and less than the last one (corresponding to bigger strength).
With input 2 where g 5 (cid:29) g 1 , the solver still works well where w 1 = 1 corresponding to the minimal strength g 1 = 0 .
However, with input 3 , the solver fails to ﬁnd a good solution (which should be w =
[1 , 0 , 0 , 0] given that input).
It is a worth noting that the main goal of this paper is to show the application of Multi-objective Optimization for generating adversarial examples and the impact of the Task-Oriented regularization.
Therefore, while the issue of the gradient descent solver is well recognized, we did not take effort to try with a better solver
21 22 # Result
[Table]
input
[Table]
Listing 1: Python example of the Gradient Solver with softmax parameterization E. C ORRELATION BETWEEN THE O BJECTIVE L OSS AND A TTACK P ERFORMANCE .
It is broadly accepted that to fool a model, a feasible approach is maximizing the objective loss (i.e.,
CE, KL, or CW loss), and the higher the loss, the higher the attack success rate.
While it is true with the same architecture, we found that it does not hold when comparing different architectures.
Figure 5 shows the adversarial loss and the attack success rate for each model in the ENS setting.
With the
CW loss as the adversarial objective, it can be observed that there is a positive correlation between the loss value and the ASR, i.e., the higher the loss, the higher the ASR.
For example, with the same adversarial examples, the adversarial loss on EfﬁcientNet is the highest and so is ASR.
However, there is no clear correlation observed when using CE and KL losses.
Therefore, the higher weighted loss does not directly imply a higher success rate for attacking an ensemble of different architectures.
The MinMax method (Wang et al., 2021) which solely weighs the tasks’ losses, therefore, does not always achieve a good performance in all the tasks.
E. A DDITIONAL E XPERIMENTS WITH M ULTI -T ASK L EARNING M ETHODS
It is worth noting again that our work is the ﬁrst to apply multi-objective optimization to the adversarial generation task for the classiﬁcation problem. Therefore, the MOO approach should be considered our naive approach instead of a baseline to compare with. However, it is a worth research question that: Might better multi-objective optimizations with a speciﬁc constraint to favor diverse Pareto solutions can improve performance in our setting? Therefore, in this section we would like to provide additional experiments with recent multi-task learning methods to explore how better constrained approaches can improve over the naive MOO.
We applied two recent multi-task learning methods
PCGrad Yu et al.
(2020) and CAGrad Liu et al.
(2021a)
with implementation from their ofﬁcial repositories into our adversarial generation task.
For PCGrad we use the mean as the reduction mode.
For CAGrad we use parameter α = 0 .
5 and rescale = 1 as in their default setting.
We experiment on attacking ensemble of models setting with two settings, a diverse set D with 4 different architectures including R-ResNet18, V-VGG16, G-GoogLeNet, E-EfﬁcientNet and a non-diverse set ND with
ResNet18 models.
It can be seen from the Table 22 that in the diverse ensemble setting, the two additional methods signiﬁcantly outperform the standard MOO method with the improvement gaps of SAR-All around 3% and 5%, respectively.
In the non-diverse ensemble setting, while PCGrad achieves a lower performance than the standard MOO method, CAGrad can outperform the MOO method with a 2.7% improvement.
The improvement on the diverse set of PCGrad and CAGrad over the standard
MOO method is more noticeable than on the non-diverse set.
It can be explained by the fact that on the diverse set of model architectures, there is a huge difference in gradients among architectures, therefore, requires a better multi-task learning method to handle the constraint between tasks.
On the other hand, on both ensemble settings, our TA-MOO still achieves the best performance, with a huge gap of (5.8%) 7.8% compared to the second best method on the (non) diverse setting.
It is because our method can leverage a supervised signal from knowing whether a task is achieved or not to focus on improving unsuccessful tasks.
It is a huge advantage compared to unsupervised multi-task learning methods as MOO, PCGrad, and CAGrad.
E. C ONFLICTING BETWEEN GRADIENTS IN THE ADVERSARIAL GENERATION TASK
In multi-task learning setting, conﬂicting between gradient is the common issue to tackle with.
More speciﬁcally, the gradients with respect to the (shared) model parameter of task f
i and task f
j can have a negative correlation (i.e., cosine similarity between ∇ θ f i ( θ, δ ) and ∇ θ f j ( θ, δ ) is negative).
However, in the adversarial generation task, we consider the gradient with respect to the input (e.g., ∇ δ f ( θ, δ ) ) to update the adversarial examples.
As we explore through empirical experiments,
[Figure]
[Table]
[Table]
the issue that we to deal with is not the gradient conﬂiction problem but the gradient domination problem.
These gradients with respect to the inputs can have a positive correlation but also have a huge difference in their strengths.
In this speciﬁc challenge, the standard MOO which solely relies on the gradient strengths to calculated the weight for each task is strongly sensitive to the gradient domination problem and in some cases cannot lead to a good solution as discussed in Appendix E.
To further support our hypothesis, we would like to provide a measurement on the cosine similarity between gradients on different ensemble members on the ENS setting in Table 23.
Each cell (row-ith, column-jth) of the Table reports the cosine similarity between gradient ∇ δ f i ( δ ) of model ith and gradient ∇ δ f j ( δ ) of model jth (w.r.t.
the same input δ ).
It can be seen that the gradients between different architectures has the positive correlation instead of negative correlation.
On the other hand, as shown in the last row, the gradient norm (cid:107)∇ δ f i ( δ ) (cid:107) varies widely among architectures.
While this observation is in line with the widely accepted phenomenon about the transferability of adversarial examples, it also does support our motivation to derive the TA-MOO method to improve the standard
MOO.
E.
D ISCUSSION
ON THE C ONVERGENCE OF OUR METHODS
In multi-task learning, the gradient of each task is calculated with respect to the (shared) model parameter (e.g., ∇ θ f ( θ, δ ) ).
Therefore, to quantify the convergence of a multi-task learning method, we can measure the gradient norm of the comment gradient direction to quantify the convergence of the model.
The gradient norm is expected to be a very small value when the model reaches to the
Pareto optimality points.
However, in adversarial generation problem, the gradient of each task is calculated with respect to the input (e.g., ∇ δ f ( θ, δ ) ).
Therefore, unlike in the multi-task learning, there is a different behavior of gradient in the adversarial generation task.
To verify our hypothesis, we measure the gradient norm of the gradient over all attack iterations and visualize in Figure 6.
It can be seen that the gradient norm of all attacks tends to converge to a large value.
It is a worth noting that we use projected gradient descent with l ∞ in all attacks.
Therefore, in each attack iteration, the amount to update is not the gradient ∇ δ f ( θ, δ ) but the sign of it scaling with a step size η δ .
However, there is still an interesting observation such that MOO and TA-MOO attack have a much lower gradient norm than other attacks.
We would like to propose a simple alternative approach to quantify the convergence of our method in the adversarial generation setting.
More speciﬁcally, we leverage the advantage of the adversarial generation task such that we can access to the label to audit whether the task is successful or not.
Therefore, we simply measure the loss and the success attack rate over all attack iterations as shown in Figure 7.
First, we would like to recall the deﬁnition of the Pareto optimality.
Given m objective function f ( δ ) (cid:44) [ f 1 ( δ ) , ..., f m ( δ )] , the Pareto optimality δ ∗ of the multi-objective optimization δ ∗ = argmax δ f ( δ ) if there is no feasible solution δ (cid:48)
such that is strictly better than δ ∗ in some tasks (i.e., f i ( δ (cid:48) )
>
f i ( δ ∗ ) for some i ) while equally good as δ ∗ in all other tasks (i.e., f j ( δ (cid:48) ) =
f j ( δ ∗ ) , j (cid:54) = i ).
Bear this deﬁnition in mind, it can be seen from the loss progress of MOO attack in Figure 7a that (i) from iteration 1st to around iteration 10th
all the losses are increased quickly showing that the method optimize efﬁciently; (ii) after iteration 10th, the loss w.r.t.
the EfﬁcientNet model (i.e., model3 in the legend) continually increases while other losses continually decrease.
Therefore, any solution after iteration 10th do not dominate each other indicating that the method reaches the Pareto front.
On the other hand, it can be seen from Figure 7b that the loss progress of our TA-MOO is more stable.
TA-MOO also can optimize to the optimal point efﬁciently as MOO does, however, after reaching the peak, the losses in all tasks are more stable than those in MOO.
This observation indicates that the solutions after the peak point are also in the Pareto front but are more concentrated than those in MOO.
It can explain the stability of the success attack rate in TA-MOO in Figure 7b.
Comparing across both MOO and TA-MOO at their last iteration shows that while the loss w.r.t.
the EfﬁcientNet model (model3) in MOO is a bit higher than that in TA-MOO, these other losses w.r.t.
V/G/E models in
MOO is lower than those in TA-MOO.
This observation indicates that in term of losses, the solutions of MOO and TA-MOO do not dominate each other.
However, the solution of TA-MOO is more stable and leads better ﬁnal attacking performance.
E. A DDITIONAL E XPERIMENTS WITH D IFFERENT INITIALIZATIONS FOR MOO
In our method, the default initialization for the weight w is 1 /m equally for all tasks.
Therefore, one raising valid concern is that Might better initialization can help to boost the performance? .
To answer
[Figure]
[Table]
this question, we ﬁrst ﬁnd the optimal initital weight by using the weight at the last iteration when running MOO and TA-MOO attacks with the default initialization.
For example, as shown in Figure 8a for the ENS setting with diverse architectures, the average weight that MOO assigns for model
R/V/G/E converging to 0.15/0.17/0.15/0.53 ( set A ), respectively.
The average weights’ distribution learned by TA-MOO is 0.19/0.25/0.19/0.37 ( set B ), respectively.
It is a worth noting that, we consider each set of weights for each data sample separately, and the above weights are just the average over entire testing set (e.g., 10K sample), while the full statistic (mean ± std) of weights can be seen in
Table 2.
In order to make the experiment to be more comprehensive with diverse initializations, we use two additional sets including set C=[0.22, 0.23, 0.22, 0.33] and set D=[0.24, 0.25, 0.24, 0.27].
Given these above four weights sets A/B/C/D, we then init the standard MOO with one of these above sets and adjust the learning rate η w with three options 5e-3, 5e-5, 1e-8 and report results in
Table 24.
The complete attacking progress can be seen in Figure 8.
It can be seen from Table 24 that better initialization does help to improve the performance of the standard MOO.
The best setting is the initialization with set D and η w = 5e-3 achieves 29.53% in A-All metric, a 4.37% improvement over the default MOO initialization.
It can be seen from the evolution of the weights in Figure 8c that even initializing with the converged weights (i.e., set A) from the pre-running attack, the weight of each task does not stand still but converges to a different value.
It is another different behavior in adversarial generation task compared to the multi-task learning problem.
On the other hand, despite of the extensive tuning, the performance of MOO is still far below the TA-MOO approach, with the gap of 8.48% in A-All metric.
[Figure]