On the Adversarial Robustness of Mixture of Experts
Authors: Joan Puigcerver
Rodolphe Jenatton
Carlos Riquelme Ruiz
Pranjal Awasthi
Srinadh Bhojanapalli
Venue: NeurIPS 2022 Accept (2022)
Keywords: mixture of experts
moe
adversarial
robustness

Abstract: Adversarial robustness is a key desirable property of neural networks. It has been empirically shown to be affected by their sizes, with larger networks being typically more robust. Recently, \citet{bubeck2021universal} proved a lower bound on the Lipschitz constant of functions that fit the training data in terms of their number of parameters. This raises an interesting open question, do---and can---functions with more parameters, but not necessarily more computational cost, have better robustness? We study this question for sparse Mixture of Expert models (MoEs), that make it possible to scale up the model size for a roughly constant computational cost. We theoretically show that under certain conditions on the routing and the structure of the data, MoEs can have significantly smaller Lipschitz constants than their dense counterparts. The robustness of MoEs can suffer when the highest weighted experts for an input implement sufficiently different functions. We next empirically evaluate the robustness of MoEs on ImageNet using adversarial attacks and show they are indeed more robust than dense models with the same computational cost. We make key observations showing the robustness of MoEs to the choice of experts, highlighting the redundancy of experts in models trained in practice.

On the Adversarial Robustness of Mixture of Experts
Joan Puigcerver Google Research
Rodolphe Jenatton Google Research
Carlos Riquelme Google Research
Pranjal Awasthi
Srinadh Bhojanapalli
Google Research
Google Research
Abstract
Adversarial robustness is a key desirable property of neural networks. It has been empirically shown to be affected by their sizes, with larger networks being typically more robust. Recently, Bubeck and Sellke [3] proved a lower bound on the Lipschitz constant of functions that fit the training data in terms of their number of param- eters. This raises an interesting open question, do-and can can-functions with more parameters, but not necessarily more computational cost, have better robustness? We study this question for sparse Mixture of Expert models (MoEs), that make it possible to scale up the model size for a roughly constant computational cost. We theoretically show that under certain conditions on the routing and the structure the data, MoEs can have significantly smaller Lipschitz constants than their dense counterparts. The robustness of MoFs can suffer when the highest weighted experts for an input implement sufficiently different functions. We next empirically evaluate the robustness of MoEs on ImageNet using ad adersariaa attacks and show they are indeed more robust than dense models with the same computational cost. We make key observations showing the robustness of MoEs to the choice of experts, highlighting the redundancy of experts in models trained in practice.
Introduction
Adversarial robustness refers prediction robustness of given machine learning model to adversarial, but bounded, changes to the input. Neural networks trained with standard classification objectives have been shown to have poor adversarial robustness [25]. a property attributed to their overparametrization. Conversely, in practice, larger models, that have more parameters and higher computation cost, have shown better robustness [20, 26, 1, 12].
In a recent work, Bubeck and Sellke [3] studied this phenomenon from theoretical perspective, by analyzing the relationship between model size and its Lipschitz constant which measures the sensitivity of a function to changes in the input. In particular the authors proved that any function with P parameters, that memorizes N input data points in D dimensions, has Lipschitz constant
of at least 0(( (N This shows that, on given dataset, larger models can have better robustness (smaller Lipschitz constant). Note that as this is only a lower bound, it does not guarantee that larger models will indeed have smaller Lipschitz constant. Interestingly. the result is agnostic to other properties the function, such as its computation cost, or specific architecture.
Given the above result is natural to wonder: can one increase the model size, without increasing the computation cost, and enjoy better robustness? An increasingly popular class of such models are Mixture of Experts (MoE) [23]. MoE models have multiple expert sub-models, and routing function that selects for each input a small subset of the experts and routes the input only to them. Neural networks with MoE layers, such Switch Transformer [10] in NLP. and V-MoE [22] in Computer Vision, have been shown to achieve superior performance in comparison to their dense counterparts,
36th Conference on Neural Information Processing Systems (NeurIPS 2022).

---

by allowing one to scale the model size without increasing the computation cost. In this paper we study the following question: are MoE models more adversarially robust than dense models?
In general, sparse MoE models are not continuous, and hence not even smooth. Small changes to the input can result in the router selecting a different expert for a given example. Unfortunately, in some cases these experts may be very different, resulting in large changes the output. Another factor that affects the robustness of MoE models is the geometry of input data and its routing -how the data is divided between experts. This decides what data an expert is trained on and hence their robustness. Finally, it has been observed that -unless certain auxiliary losses are also applied to encourage balancedness- the number of used experts tends to collapse to very few, and the remaining ones are just ignored [22, 10]. Given these stability issues, it is priori not clear if MoEs, despite having more parameters, are more robust than dense models.
To theoretically study the effect of both of these factors, (1) router stability and (2) data routing, on robustness we consider the smooth MoE models, where each expert output is weighted by their routing probability. In particular, MoE models where all experts are simultaneously applied to every input. We analyze models with fixed routing proving that MoΣs with linear experts can achieve better robustness if the data is well separated and routed accordingly. In the extreme case, MoBs with E experts can have a smaller Lipschitz constant by a factor of 1/VE compared to equivalent dense models. We also characterize the effect of the difference between experts in terms of robustness, showing that MoEs can have a high Lipschitz constant when two experts are very different for inputs that weigh the experts similarly and highly. We show that both these factors, data routing and difference between relevant experts, characterize the Lipschitz constant of MoE. models.
We next evaluate robustness experimentally by using adversarial attacks [20]. We experimentally show that on the ImageNet dataset [6]. MoEs are more robust than dense models that have the same computation cost. Interestingly, we observe that adversarial attacks result in significant routing changes of the inputs, but do not result in lower robustness than dense models. This suggests that in practice, MoEs are robust the choice of the experts an extent. We next perform standard adversarial training of both the dense and the MoE models and again observe that MoEs are more robust against adversarial attacks.
The main contributions of the current work are as follows:
1. We propose a simple theoretical framework to understand the robustness of Mixture of Experts (MoE) models. We provide general and non-trivial sufficient conditions under which MoEs are provably more robust than their dense counterparts for the linear experts setting.
2. We perform extensive experiments to demonstrate that in practice MoEs indeed enjoy better robustness than dense models to norm bounded adversarial attacks. We also uncover intriguing properties of adversarial attacks for MoE models and show that even for robust MoE models, the attacks very often change the routing of data points, thereby pointing to a high degree of redundancy in such models.
2 Preliminaries
2.1 Mixture of Experts
Mixture of Experts combine the outputs of (sub-)modela i.c., the experts, via weighted sum of their outputs [14, 15, 28, 9]. Sparse Mixture of Experts condition the weights of the sum in the inputs, and activate only K (out of E) experts, where K is typically a very small integer compared to E (such as K or K 2) [23, 10]. This form of conditional computation models allows one easily increase the number of parameters in the model (roughly) independently of its compute cost [21]. This approach has been recently applied to significantly increase the model size and quality of models used Natural Language Processing [23, 18, 19, 10, 8] and Computer Vision applications [22, 27].
More concretely, we define the sparse and smooth (or dense) versions of mixtures of experts below. While previous empirical works have sometimes selected more than one expert per input (K > 1), for simplicity we constrain ourselves to the case where only one expert is selected for each input (K = 1). Recently, practical models also tend to this setup to match the FL.PPs of dense models [10].
Sparse MoEs. In these models only one expert is selected for given example ær based on its routing probabilities (z)). These models are not continuous functions of the input. A small change in the input can lead to large changes in the output, due to changes in selected experts. A sparse MoE layer

---

is defined as:
E f(z) = Σ (1)0))) fi(x)).
(I)
Here f()) are the individual expert functions. is the indicator function that takes the value of 1 if the condition is satisfied and 0 else. Note that to do tie breaking in the above definition, in case multiple experts have the same maximum probability, we simply sample uniformly one of the experts with the maximum probability. In practice, MoE models can have multiple sparse layers in combination with dense layers. Further, random noise sometimes added to the p; [23, 22].
We refer to MoEs with only expert as dense models. In dense models, which are the standard in neural networks literature, same function is applied to all input examples. In sparse MoEs introduced above, different part of network / expert is applied to each input example depending on which expert selected. This flexibility allows us to train larger sparse MoEs, that have the same computational cost as the dense models, as only part of MoE activated/selected for each input, with better accuracy than dense models [10, 22].
Smooth MoEs. To theoretically analyze robustness, we consider smooth mixture of expert models, as they are continuous functions. In this case, the output of each expert fi, is weighted according to its routing probabilities:
E f(x) Σ (.(
The routing probabilities pi (xr) are usually computed by routing layer, c.g., pi(x) = (Sx), where is the softmax and S E Ex is trainable variable. Such linear routing layers are the common choice for MoE models in practice [10].
Though these models have recently been shown to achieve state of art performance for tasks in NLP [10] and vision [22]. there have not been many works analyzing their robustness. Recently, Allingham et al. [2] empirically studied robustness of MoEs to natural perturbations in data (e.g.. evaluation across different corrupted versions of CIFARI0 10 [17] and ImageNet [6]), and showed MoEs are more robust for these natural perturbations than the corresponding dense models. To the best of our knowledge ours is the first work theoretically analyzing robustness of MoEs, and empirically exploring the context of adversarial perturbations.
Load balancing loss To prevent MoEs from collapsing and always selecting the same expert for all inputs, it is customary during training add load balancing loss, that encourages equal fraction of inputs being routed different experts. We define and present these losses in detail in appendix E.
2.2 Adversarial robustness
Adversarial robustness models the susceptibility of a function to adversarial perturbations to its input [25, 20J. More concretely, given a function f, loss function E, and an input x E RR,, we can write the adversarial loss incurred x with label y as follows.
max {(f( + 2). #: =
Here is the attack radius per input. For the norm constraint ([|=|), popular choices in practice, are loo and la norms [20, 4]. Adversarial accuracy is the accuracy of the model on data with perturbations that satisfy the above equation, i.e., when the loss function is the 0/1 classification loss.
Since optimizing equation for neural networks is computationally hard, popular methods to find adversarial examples use local search approaches such gradient ascent. The Fast Gradient Signed Method (FGSM) is perhaps the simplest one, with only one gradient update step [11] 1]. For for bounded perturbations the FGSM update is the following:
(4) sgn() is the sign function. The Projected Gradient Descent (PGD) method is a stronger attack that does multiple (7) gradient ascent steps find the perturbation [20]. The update rule for foc bounded perturbations is as follows, starting with x x.
x + z = x + sgn (v+((((z),))
z ++ Hote [x' + a sgn (Va((f(z), v))] Vt E [0, 70 1].
(5)
3

---

is chosen to be f. Here C is the constraint set { | S e) and II is the projection operator.
Existing works have established that neural networks sensitive adversarial attacks resulting a significant drop their accuracy [25]. Interestingly, increasing the size of neural networks, thereby increasing their capacity, leads to improvement in adversarial accuracy, showing that larger neural networks are more robust [20, 26, 12]. recent work, Bubeck and Sellke [3] studied this phenomenon theoretically. They proved lower bound on the Lipschitz constant of any function that fits the training data that scales inversely vpp with the number of function parameters P. However it not clear functions that have more parameters, but not necessarily more computation cost (e.g. MoEs), can achieve better robustness. In this paper we analyze adversarial robustness of MoEs, both theoretically and experimentally.
Notation. We use small bold letters, e.g., æ, to denote vectors and capital bold letters to denote matrices, e.g., W. Scalars are denoted with plain letters. [AV] denotes the integer set from 1 to N. |I-I| denotes the &g norm unless specified otherwise.
3 Robustness analysis
In this section, we present our main results regarding the robustness of the mixture of expert models. We first present a bound on the Lipschitz constant of MoEs for general expert functions Ss. While this is an upper bound for general functions it still highlights the two key components that affect the robustness of MoE models.
3.1 Router stability
In this section, we compute the Lipschitz constant of smooth MoEs with learnable routing. Let,
E
exp( (s;((( x))
f(x) Σ (((x)f((z) where p.(x) 1=1
(6)
Σド= exp (s),
and (8: ket) with each 8; in RDD are learnable variables that decide the routing of an example to different experts. We then prove the following upper bound on the Lipschitz constant of the MoE models with learnable routing.
Lemma 1. Let (frheee be smooth functions with Lipschitz constants HEtE and let L be the Lipschitz constant of f. Let s(x: E3 P.()))s. Then f(x) in equation 6 satisfies,
|V>f(x)]| S Σ L +
and hence
E
I, S max (T.. sup (E(
Σ
ps (a))((())( - -(Z))
x
The above Lemma bounds the Lipschitz constant of MoE models with two terms that depend on (1) data routing, and (2) router stability. The first term above is from the individual Lipschitz constant of the experts. This depends on how the data is partitioned or routed to different experts, which we will discuss in more detail in the next section. The second term arises from the router used to compute probabilities for different experts. One may wonder if we can make this term arbitrarily large by increasing the norm of x. This is not the case as increasing the norm of x usually results in the collapse of the routing probabilities to single expert. This makes (x(x) s(x)) zero for all experts i E [E].
To study the above more concretely, we consider the case of F. = 2 experts. In this setting the second term above reduces to ||p
8(x)) + (1 Pi (=)))2(z)(
-(x)))] Now, in the
extreme case, where pı (x: is either 0 or 1, the above term collapses to 0 as p: f.) (x; 8(x)) )) is 0 in that case. This is also expected for examples that are routed to expert with high probability, the main dominating factor is the individual expert Lipschitz constant, as small perturbations do not cause much change to the probabilities.

---

A more interesting setting is when P1 (z) is 2. In this setting, small perturbations to the input can cause the model to drastically change the weight between the experts, and the above term reduces to ll (f) /2())) (8) - 82)ll- Hence MoE models can suffer from large Lipschitz constants the two experts are very different for points on the boundary, i.c., if [ (z) - -2(x)) >> O. Alternately
fx) = f2) for points on the boundary (i.e. P1 (xx) ~ 2) then this term is small.
We will later see in the experiments that MoEs trained in practice often have good router stability and changes to routing do not result in much degradation of the final accuracy.
3.2 Data routing
In this section we will study the effect of data routing on the Lipschitz constant of the individual experts - the first term in RHS of equation 7 (lemma 1). particular we will try to address how large can max]e(E] [T,,,] be in comparison to the Lipschitz constant of a dense model trained on the same data. Towards this end, we will theoretically analyze the robustness of MoE models with a pre-determined fixed routing and linear experts, in the light of the structure of the data.
3.2.1 Setup
We consider the setting of fixed routing, where assume the routing of individual examples to experts pre-determined and fixed. Moreover, we consider linear models as experts.
More specifically, we consider N input points of dimension D stacked in the matrix X E NxD, together with the corresponding vector of targets would like to find a linear model w" E IRD that minimizes
R In the dense case (i.e., no experts),
min WERD
Xw y1
(8)
The least-squares solution [13] for this problem that minimizes the MSE loss is w = (XTx)
Here denotes the pseudo inverse. On the other hand, for mixture of experts, let the dataset be split into E subsets S ,gg, with X,, y, denoting the data in set S, Now let the data from the set S be routed to the expert i. Below we write the objective for each expert:
min W.ER?
[xw,, yill2 for i [b].
Similarly, let w (X, Xx) y, be the optimal solution for expert that minimizes the MSE.
3.2.2 Analysis
Before introducing our analysis relating the Lipschitz constant of the dense and MoE models, we first present two examples to better understand how routing affects Lipschitz constants. In particular we consider a simple setting with E = 2 experts. Let X =XXT X. be the data routed to the two experts. Recall that the Lipschitz constant of the function f (X) X is [/]l-
Case X,
X2. In this setting we get
|| || [| XT yll = ||[x| X
Xi 91
(X2
22)
X2 3/
w|w1 + w
2 max((1 11, || l)
The second equality follows from the X X,)² = (X X,)² + (X,') x²) and 3) (X O, where steps 2 and 3 follow from the orthogonality of X, and X₂ (see [16]). Hence, experts have smaller Lipschitz constant when data routed to different experts lies in orthogonal subspaces.
steps 1) x X' 2X
Case X1
X2 and y/1 = y2- In this case, we see that:
|| = [I T x) y = x +
x yll
X 2 X y x vll = 0 min[| || I])-
Hence experts have worse Lipschitz constant when the data routed to different experts is aligned. These two simple examples illustrate under which conditions experts have an advantage over a single dense model, and when they do not.
5

---

We now present our main result. To capture this relation between the data geometry and the routing, we introduce the following quamtities. Let [U be the projection matrices onto orthogonal subspaces in RDD One way to construct them is by first taking the singular vectors of X and assigning them to the set with the largest projection. This guarantees that U; is orthogonal to Uj with and they have greatest alignment with subspace spanned by X;-
We first define quantity to capture how well U, captures the span of the data subset X,.
Definition (In-subspace distance: E1). 3e1 2 0, ||UU (x X) - (X; x,) & e1, Vi [R].
Here |l-1|2 for matrix denotes the spectral norm. is small when, Vi, U; captures X; perfectly.
Next define the projection distance between data from two different subsets.
Definition 2 (Cross-subspace distance: 62). 3cg 2 such that for any in the span of X we have
[Ixx" x,)* z|| S #2 =2||=||, Vi j.
Here 62 is small if the data in different subsets X; lies in orthogonal subspaces.
Theorem 1. Let w" be the minimizer of equation 8 and {wi hee be the minimizers of equation Then,
- x yll |001| 1|? Σ -e
Σ Ix] 1 j
E2
where I']+ denotes the projection onto non-negative numbers.
The above result lower bounds the Lipschitz constant of the dense model ||w* in terms Lipschitz constants of the experts || in the MoE model. We present the proof of this theorem in appendix A.
1.=:1/w:3 Assuming a balanced setting where all experts have the same norm parameters, we get the scaling ||w"| Po OVVE)- max,e[e] []) Hence in this setting, experts have a Lipschitz constant that is smaller by factor of VE compared to dense models. This happens when the data lies in E orthogonal subspaces and the data from each subspace is routed to the same expert. This shows that MoE models can have significantly smaller Lipschitz constant than their dense counterparts, while having the same computation cost. For MoEs in practice, data routed to different experts does display some clustering of the features (see Figure Riquelme et [22]).
In the where 61 62 0, we obtain the following bound ||w0||
As data in different partitions X; gets more aligned, é and e2 increase, and reduce the gap between the Lipschitz constant of the dense and the MoE models. This reduces the gap in the Lipschitz constant of the dense model and the experts. In the extreme case, if all the datapoints are the same, then e and 22 are large, eliminating this difference. In appendix D, we give a complementary result relating the Lipschitz constants of the experts that of the dense model when both ei and e2 can possibly large.
Connections to Bubeck and Sellke [3]. Bubeck and Sellke [3] proved universal lower bound on the Lipschitz constant of function required d-memorize N training samples in D dimension.
ND P
1 á 8
For the linear model considered above, this reduces to δi (ōvN) as the number of parameters is D. As MoEs have E times more parameters, they have a lower bound of ÿ (5v) N E
i.e., the lower
bound on the Lipschitz constant of MoEs is smaller by factor of / F. Our result shows that this lower bound is in fact achievable, and hence tight, when the data lies in orthogonal subspaces and data from each subspace is routed to the same expert.

---

4 Experiments
4.1 Setup
We compare the robustness of Vision Transformer (ViT) [7] and Vision MoE (V-MoE) [22] models against adversarial attacks. In particular, we use the ViT-B/32 and V-MoE-B/32 models through all the experiments. These models have the same backbone architecture but the latter replaces one in every two feedforward layers with sparse Mixture of Experts, selecting K = 2 out of E = 32 feedforward experts that are applied on each token'. One could argue that the router in the V-MoE model introduces an overhead that should be accounted for. Thus, we have trained bigger version of the dense ViT-B/32 (which we coin ViT-B++/32) that reaches roughly the same predictive performance as the V-MoE model, but has higher cost. The cost of evaluating an image on the dense ViT models is 8.9 and 17.9 GFLOPs and runtime cost, respectively; and 12.4 GFLOPs on the V-MoE model. Appendix B contains additional experimental details.
We pre-train our models on the private dataset JFT-300M [24] for 7 epochs (517 859 steps with a batch size of 4 096 images), using an image resolution of 224 x 224 pixels, and standard data augmentation (inception crop and horizontal flips). Since JFT-300M is a multi-label dataset, we minimize the sigmoid cross-entropy loss. V-MoE also adds auxiliary losses to encourage a balanced load for all experts; we used the same recipe as in [22]. In both cases we use Adam (33 = 0.9, (32 = 0.999), with peak learning rate of 8 10-4 reached after a linear warm-up of 102 steps and then linearly decayed to a final value of 10 Weight decay of 0.1 was used on all parameters. This is the same pre-training protocol as that used in [7, 22].
After pre-training, the models are fine-tuned on ImageNet [6], at a resolution of 384 x 384 pixels and the same data augmentations as before, for a total of 10t steps, using a batch size of 4 096 images. SGD with Momentum (μ = 0.9) is used for fine-tuning, with a peak learning rate of 0.03, reached after a linear warm-up of 500 steps, and followed with cosine decay to a final value of 10-5, The norm of the flattened vector of gradients is clipped to a maximum value of 10. Since ImageNet images have single label, we minimize the softmax cross-entropy loss during fine-tuning. The same fine-tuning protocol was adopted in [7, 22].
We evaluate the adversarial robustness of both the pre-trained and fine-tuned models, by means of PGD adversarial attacks [20]. We maximize the corresponding loss (sigmoid or softmax cross-entropy), varying the lo norm constraint on the input image, for a total of t = 40 steps.
4.2 Adversarial robustness of V-MoEs
During pre-training the ViT-B/32 model achieves a precision at I of 39.3%, and the V-MoE-B/32 achieves precision-at-1 of 43.5% (conversely, the false discovery rate is 60.7% and 56.5%, respectively). After fine-tuning, the classification error achieved by each model on ImageNet is 19.3% and 17.8%, respectively.
Figure shows in solid lines the false discovery rate (left) and the classification error rate (right) as function of the for constraint. Despite the fact that the V-MoE model contains a router that makes discrete choices among the experts conditioned on the input, which could potentially lead to a severe weakness against the adversarial attacker, we can observe that follows the same trend as the base dense ViT model. It is able to preserve lower error over a wide range of &xo values. A larger version of the dense model matching the quality of the V-MoE has much higher cost.
In addition, we also fine-tuned the base ViT and V-MoE models on ImageNet using PGD adversarial training. We use the same recipe as above but we perform a PGD attack of 10 steps on the input images, with fixed fox- 255 before computing the gradients of the model parameters and updating them. The classification error on the original ImageNet dataset achieved by each model after adversarial fine-tuning is 51.7% and 49.8%, for both models respectively. Figure 1 (right) shows in dashed lines the classification error when these models are evaluated against an adversarial attacker using different lo constraints. As the loo increases, the benefit of adversarial fine-tuning to preserve accuracy is shown in both cases. Once again, both models report similar trends, and the V-MoE model shows better robustness for a wide range of foo values.
Tooen refers to element in the sequence input, which is obtained by projecting input image patches [7]
7

---

JFT-300M
ImageNet
1.0
Classification error rate
1.0
False discovery rate
Model
VIT (8.9 CFLOP)
(17.9 GFLOP)
V-MoE (12.4 GFLOP)
0.6
Adversarial ft.
False
0.7
0.4
True
10 10-3
L 10
10-2
Figure 1: False discovery rate on JFT-300M (left) and classification error rate on ImageNet (right) as function of the fx used in the adversarial attacks. Dashed lines depict models fine-tuned with PGD adversarial training. Although the V-MoE model contains several sparse MoE layers making discrete choices their respective inputs, it shows lower error under adversarial attacks than the base VIT model. much bigger and slower ViT model needed to roughly match the V-MoE.
4.3 Effect of the adversarial attacks on the selected experts
As described in section 3.1, 1, in the region close to the decision boundary of the router, if two experts have very different outputs, the Lipschitz constant of the MoE model could be much higher than that of similar dense model. If the index of the selected experts changes significantly, but the model still shows a reasonably high accuracy under adversarial attacks (compared to a dense model), this would suggest that the outputs of the two selected sets of experts do not differ much.
Figure shows the of changes in the router function of the loo used in the adversarial attack, on the different layers of the V-MoE-B/31 model that have a MoE layer. For each token processed by the model, compute the intersection-over after the adversarial attack. We average the IoU across all processed tokens and define the rate of routing changes as the complement of the average IoU.
(IoU) of the selected set of experts before and
In dashed lines report the rate of changes of V-MoE model fine-tuned using PGD adversarial training. Not only the model has a better accuracy against adversarial attacks when the foo increases, as reported in figure I, but the rate of routing changes is also generally lower across all layers.
fine-tuning
True
changes
Layer 1
Layer 3
Layer 5
Layer 7
Layer 9
Layer 11
0.6
0.2
0.0
10-2
10-2
10-2
Figure 2: Rate of changes in the selection of experts function of the Eoo used in the adversarial attacks on ImageNet against a V-MoE model. The dashed line depicts a V-MoE model fine-tuned with PGD adversarial training. A significant fraction of experts change in each MoE layer. Given that figure shows the V-MoE model keeping advantage over the dense VIT, we hypothesize that the output of different experts for inputs close to the decision boundary of the router is similar.
Despite the fact that significant fraction of choices change on each layer as the fox. increases, figure 1 shows that the V-MoE model still keeps its advantage against adversarial attacks over the VíT model. This suggests that in the regions close to a decision boundary the router, the two corresponding experts have a similar output, hence preventing the full V-MoE model from being less robust in practice. Conversely, the fact that the V-MoF model has better base quality than the ViT counterpart, suggests that the experts are not equivalent for regions far away from the decision boundary, otherwise it would be reduce to a dense model.

---

4.4 Attacking the router's auxiliary losses
Sparse MoE models usually employ auxiliary losses to balance the load among all experts. In particular, in the implementation of V-MoEs, if the load of the experts is highly unbalanced, the experts receiving significantly more tokens than the average could ignore all tokens that exceed the expert's 8 capacity, potentially leading to a significantly worse performance. The question is whether an adversarial attacker can exploit this property in practice. Figure 3 (left) shows the false discovery rate on JFT-300M, for V-MoE model when the router's auxiliary losses are maximized in the adversarial attack, together with the corresponding cross-entropy loss. We use the same weight for the auxiliary losses as the one used train the models. The figure shows that attacking the auxiliary loss does not offer any significant advantage for the attacker. The results are analogous in terms of the classification ervor on ImageNet (not shown here in interest of space).
JFT-300M
JFT-300M
ImageNet
1.0
False discovery rate
1.0
Classilcation erroz rate
1.0
VIT
Model
False discovery
0.9
0.9
0.0
V-MoF (E-4)
V-MoE (E=2)
0.75
V-MoE (E-8)
0.8
0.0
0.6
V-MoE (E=16)
0.70
0.4
V-Mol (D=32)
Attack auxilirry
0.7
0.7
0.05
0.4
0.3
10
True
10-3
10-5
10-1
10-3
10ah
10
10-1
10-2
10-1
Figure 3: (Left) False discovery rate on JFT-300M for VIT-B/32 and V-MoBB-B/32 models (with 32 experts). Dashed lines represent the results when the attacker targets both the cross-entropy and the auxiliary losses used by the V-MoE routers to balance the load among experts in each layer. The auxiliary losses of V-MoE do not present a disadvantage against an adversarial attacker. (Center and Right) False discovery rate JFT-300M and classification error rate on ImageNet for VIT-B/32 and several V-MoE-B/32 models (with increasing number of total experts). On JFT-300M the quality of the model against adversarial attacks improves as more experts are used. After fine-tuning on ImageNet, the number of experts becomes increasingly redundant.
4.5 Increasing the model size by increasing the total number of experts
In section 1 we asked if, given that the lower bound the Lipschitz constant given by Bubeck and Sellke [3] is agnostic to the computational cost of the function, could we make a model more robust by increasing its model size without increasing the total cost? Here we measure how much increasing the total number of experts in a V-MoE model helps against adversarial attacks. Notice that increasing the total number of experts E does not make the V-MoE model more expensive. Figure 3 (center and right) shows the false discovery rate JFT-300M and the classification error ImageNet for an increasing number experts. All V-MoF. models select = 2 experts for each token.
On the one hand, increasing the total number of experts improves the robustness on JFT-300M up to E = 16 experts. The curves for = 16 and E 32 are highly overlapping, thus any difference is most likely due to noise in the training and fine-tuning process. On the other hand, when the V-MoEs fine-tuned on ImageNet, all models with more than two experts achieve roughly the same accuracy under adversarial attacks.
This shows that, although the results presented in section 3 showing better robustmess of MoEs require some assumptions, the conclusions hold to some extent in real scenarios. Increasing the number of parameters by growing the number experts is an effective way of improving the model's robustness.
4.6 Robustness against AutoPGD attacks
We conducted additional experiments using a more sophisticated adversarial attack, AutoPGD [5], which selects the step size to use in each update of the attack. We also increased the number of steps performed in the attack, using 7 = 100 steps. Figure 4a shows the results achieved by two dense (VTT-B/32 and ViT-B++/32) models and a sparse model (V-MoF-B/32), Although AutoPGD is slightly more effective as adversarial attack against all methods (for example, with foo = 30-3 the false discovery rate on JFT-300M of ViT-B/32 is 0.744 using PGD and 0.757 using AutoPGD), the trend of all models is identical to that represented in figure 1.

---

JFT-300M
Rate routing changes (Layers 1, 7, 11)
1.00
0.75
False discovery rate
0.25
0.85
0.00
0.80
10
10-2
10a1 1
10
10-2
10
0.75
0.75
0.70
VIT (8.9 GFLOP)
0.50
ViT++ (17.9
V-MoE (12.4 GFLOP)
0.25
10-3
10-2
10-1
0.00
1enn
10
10
10
10
10
10
(a) False discovery rate on JFT-300M (left) and clas- sification error rate on ImageNet (right) as a function the foo, using AutoPGD and yielding the same con-
(b) Rate of changes function of the &so used in the adversarial attacks on ImageNet against V-MoE model.
selection experts as
clusions as the simpler PGD method.
Figure 4b shows the rate routing changes in the different MoE layers in the V-MoE-B/32 model. This figure is analogous to figure which shows the results for the standard PGD attack. Compared it, AutoPGD able change higher fraction of the selected experts. For example, the maximum fraction changes Layer using PGD was near 0.2, while AutoPGD increases it up to around 0.5.
AutoPGD offers the same conclusion as the PGD attacks: models using sparse MoE layers offer better robustness against adversarial attacks (per GFLOP) than dense models, despite the fact that the router itself can be quite sensible to these attacks.
5 Conclusion
In this work we analyzed the adversarial robustness of MoEs showing their advantage over dense models, with more experts leading better robustness, both theoretically and empirically, We showed how the properties of the data and its routing plays and important role in learning robust MoEs. clustering of Riquelme et al. [22]). currently not
While there is some evidence that routing learned by MoEs in practice display the features in some layers of the model (see Figure explicitly encouraged during training. Hence developing smarter routing strategies that take data geometry into account can be an interesting direction of future work. Currently our analysis limited to linear models, extending this to general models and deriving the dependency of optimal routing on them is another promising research direction.
We have also shown that, for inputs that weigh two experts similarly, the two expert values are very different, then the MoEs can suffer from higher Lipschitz constant. However, for models trained in practice, we saw their predictions be relatively stable, despite significant changes in choice of experts, highlighting potential redundancy of learned experts. However too much redundancy, with all experts learning similar functions is waste of capacity and can affect model performance. Hence is an interesting research problem to balance robustness and accuracy of MoËs by controlling the redundancy of experts.
Acknowledgments and Disclosure F Funding
All authors would like to thank Sven Gowal for sharing their AutoPGD attack codebase. We also thank Neil Houlsby, Basil Mustafa, and André Susano Pinto for providing insightful feedback while working this project.
10

---

References
[1] J.-B. Alayrac, J. Uesato, P.S. Huang, A. Fawzi, R. Stanforth, and P. Kohli. Are labels required for improving adversarial robustness? Advances in Neural Information Processing Systems, 32, 2019.
[2] J. U. Allingham, Wenzel, Z. E. Mariet, B. Mustafa, J. Puigcerver, N. Houlsby, G. Jerfel, V. Fortuin,
B. Lakshminarayanan, J. Snock, D. Tran, C. R. Ruiz, and R. Jenatton. Sparse moes meet efficient ensembles. prggaba///1/1 03360.
CoRR, abs/21 10.03360, 2021. URL. https:t/aaaaaee
[3] S. Bubeck and M. Sellke, A universal law of robustmess via isoperimetry. Advances in Neural Information Processing Systems, 34, 2021.
[4] N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber. D. Tsipras. 1. Goodfellow, A. Madry, and A. Kurakin, On evaluating adversarial rohustness. arXiv preprint arXiv: /00Xi::000200005.. 2019.
[5] F. Croce and M. Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter- free attacks. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 2206 2216. PMLR, 13-18 Jul 2020. URL https:tp/p//.t : /ttttps//prooceeingga mmr.. www.ppeeesssiiie crwce20b.c... .html.
[6] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE Conference on Computer Vision and Puttern Recognition, pages 248-255. leee. 2009.
[7] A. Dosovitskiy, I., Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11922 2020.
[8] N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat, et al. Glam: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning, pages 5547-5500 5569. PMLR, 2022.
[9] D. Eigen, M. Ranzato, and I. Sutskever. Learning factored representations in a deep mixture of experts. arXiv preprint arXiv: 131,,4314, 2013.
[10] W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120) 2022. URL http: // jmmr.. j/llrorrg/////..lell/l//...lllll wwtgg/apers/aaee v23//v- 0992211--9990011 .html.
[11] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In Interna- tional Conference on Learning Representutions, 2015.
[12] S. Gowal, C. Qin, J. Uesato, T. Mann, and P. Kohli. Uncovering the limits of adversarial training against norm- boundeounded adversarial examples. arXiv preprint arXiv:2010.0359) 2020.
[13] T. Hastic, R. Tibshirami, J. H. Friedman, and J. H. Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
[14] R. A. Jacobs, M. I. Jordan, J. Nowlan, and G. F. Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-87, 1991.
[15] M. I. Jordan and R. A. Jacobs. Hicrarchical mixtures of experts and the em algorithm. Neural Computation, 6(2):181 1994. doi: 10.1162/ncco. www.e/cc...999... 1994.6.2,181.
[16] P. Kovanic. On the pseudoinverse of sum of symmetric matrices with applications estimation. Kybernetika, 15(5):341-348, 1979.
[17] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[18] D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang. M. Krikun, N. Shazeer, and 7. Chen. GShard: Scal- ing giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006. 16668, 2020.
[19] M. Lewis. S. Bhosale, T. Dettmers, N. Goyal, and L. Zettlemoyer. Base layers: Simplifying training of large, sparse models. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 6654. PMI.R, 18-24 Jul 2021. URI https: wttp//ddeceeee.eeceoo
www.iinnnamm...mmmm.meeemeeeeeee mlr. ptt..prees/vi1///// /1ewis/ewissae html.
[20] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
[21] D. Patterson, Gonzalez, Q Le. C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104. 104.10300 2021.
[22] C. Riquelme, Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. Susano Pinto, D. Keysers, and N. Houlsby. Scaling Vision with Sparse Mixture of Experts. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 8583 8595. Curran Associates, Inc., 2021. URL https: / //pp/c/eeding.a..///////// wew.peeeiig....uuip......... wttpiipp.ccpppppeeee cc/paper/ Paper. pdf.
2021 f

---

[23] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis. Q. Le, G. Hinton, and J. Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXtv:/701.065538, 2017.
[24] C. Sun, A. Shrivastava, Singh, and Gupta. Revisiting unreasonable effectiveness of data deep learning era. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[25] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I, Goodfellow, and R. Fergus, Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
[26] C. Xie and Yuille. Intriguing properties of adversarial training scale. In International Conference on Learning Representations, 2020.
[27] F. Xue, Z. Shi, F. Wei, Y. Lou, Y. Liu, and Y. You. Go wider instead of deeper. arXiv preprint arXiv:2107.15 108.11177.11,1... 2021.
[28] S. E. Yuksel, J. N. Wilson, and P. D. Gader. Twenty years of mixture of experts. IEEE Transactions on Neural Networks and Learning Systems, 2778)111771199,, 2012. doi: 10.1109/NNISS 2.22001222200 2.2200299.
12


Reviews:
[{"review_id": "wysj8k6T5R_", "reviewer": "Reviewer_uHiB", "summary": "The paper focuses on the adversarial robustness of mixture of expert (MoE) models and proposes theoretical analysis focusing on the Lipschitz constant of an MoE model under some relaxed assumptions such as a smooth MoE and a linear model. \nThe goal is to answer the interesting question of whether MoE models, with significantly higher number of parameters and roughly constant additional computation cost, have better adversarial robustness than their dense (single-model) counterparts.\nThe analysis shows that under certain conditions on the routing function and the structure of the data, MoE models can have a significantly lower Lipschitz constant. Under some ideal conditions, the lower bound on the Lipschitz constant established by Bubeck and Sellke [3] for single over-parameterized models can be reduced by a factor of $\\sqrt{E}$ for a mixture of $E$ experts.\nThey experimentally evaluate the adversarial robustness of the recent Vision Transformers and Vision MoE models on a proprietary dataset (for pre-training) and the ImageNet dataset (for fine-tuning), and report interesting observations about the robustness of MoE models, the choice of experts, and redundancy among the experts.\n", "questions": "Please find a list of my questions and comments below. Major ones are listed first.\n\n**1:** For the case $\\mathbf{X}_1 \\\\,\\bot\\\\, \\mathbf{X}_2$, it is not clear to me how to go from \n$\\\\| (\\mathbf{X}^T \\mathbf{X})^\\dagger \\mathbf{X}^T \\mathbf{y}  \\\\| \\\\,$ to $\\\\, \\\\| (\\mathbf{X}_1^T \\mathbf{X}_1)^\\dagger \\mathbf{X}_1^T \\mathbf{y}_1 \\\\,+\\\\, (\\mathbf{X}_2^T \\mathbf{X}_2)^\\dagger \\mathbf{X}_2^T \\mathbf{y}_2  \\\\| $. Maybe I missed some trick or detail.\n\n**2:** Could the authors clarify the statement on lines 196 - 199 about the construction of the projection matrices and also specify their dimension?\n\n**3:** Lines 242 - 243: it is mentioned that JFT-300M is a multi-label dataset and uses the sigmoid cross-entropy loss. By multi-label, does it mean each sample has 1 or more labels, or does it simply mean that there are multiple classes, with each sample assigned to a single class? Also, does sigmoid cross-entropy loss mean the same as binary cross-entropy loss (which would only be suitable for binary classification)? Please clarify how many classes are present in JFT-300M and provide some additional details on the dataset (that is reasonable to share).\n\n**4:** Please provide a formal  definition of the following metrics (in the appendix if space is limited): Precision at 1, False discovery rate, and Intersection-over-union. \n\n**5:** In the results in Figure 1, the metrics reported are referred to as “classification error rate” and “false discovery rate”. It would be more accurate to call them “robust error rate” and “robust FDR” respectively because these are considering the worst-case adversarial input $x’$ for each clean input $(x, y)$. In other words, the metrics would be defined with a `max` over a $\\ell_\\infty$ norm ball. It is misleading to refer to them as a standard error rate or FDR both in the figures and the text. Same comment for Figure 3.\n\n**6:** In figure 3, it is mentioned that dashed lines are used to indicate that the auxiliary losses are also being attacked. However, I do not see any curves with dashed lines.\n\n**7:** Referring to figure 3 (rightmost figure) and lines 320 - 322, it is mentioned that for finetuning on ImageNet, all models with more than 2 experts achieve roughly the same accuracy (this should be robust accuracy). Can the authors provide some insight on why there is no notable improvement in the robustness (robust accuracy) as the number of experts increases?\n\n**8:** In the interest of completeness and reproducibility, please define all the attack objectives including the ones against the auxiliary losses in an appendix. Please define the auxiliary losses used to balance the load among the experts. \n\n**9:** In section 4.4, it is mentioned that the result for the classification error on ImageNet has a similar trend. This result can be included in an appendix. \n\n**10:** In figures 1 and 3, it would be easier to see the contrast in metrics if the x-axis (perturbation size) is restricted to smaller value (say 0.01). The y-axis could be log-scaled if that helps.\n\n**11:** There are a few places where it stated (for example) that the classification error is x and y respectively, but it is not clear which two models the values refer to. For example, on lines 260 - 262, the classification error of which two models are 19.3% and 17.8%? Same comment for the false discovery rate. \n \n**12:** In lines 85, 86 it is mentioned that only one expert is selected for each input, i.e. $K = 1$. Where is this assumption used?\n\n**13:** In equations 3, 4, and 5, the loss function in the attack objectives should have a class label or target $y$ (currently missing). Also, a distinction should be made whether it is a untargeted or targeted attack. \n\n**14:** In Lemma 1 and other places, please clarify what type of norm is being used. Or state upfront that the norms are by default (e.g.) $\\ell_2$. \n\n**15:** On line 186, I think it should be $\\mathbf{X}^T = [\\mathbf{X}_1^T ~\\mathbf{X}_2^T]$ in order to have the right dimensions.\n\n**16:** Nit: On line 142, it should be $\\\\{ \\mathbf{s}_i \\in \\mathbb{R}^D \\\\}_\\{i \\in [E]\\}$. It seems like the set of $\\mathbf{s}_i$ lives in $\\mathbb{R}^D$.\n\n**17:** On line 168, I think it should be equation 7, not 6.\n\n**18:** Nit: Hyphen missing in definitions 1 and 2. Should be “In-subspace distance” and “Cross-subspace distance”.\n\n**19:** Lines 237 - 238: Please clarify that the 8.9 GFlops corresponds to the ViT-B/32, while the 17.9 GFlops corresponds to the ViT-B++/32 model. \n\n**20:** A table with the compute in GFlops and the (order of) number of parameters for the different architectures, as well as for different number of experts would be useful. Please have a forward reference to Table 1 in the Appendix.\n\n**21:** Please clarify what is meant by a Token. \n\n", "limitations": "The authors discuss some limitations of their work in section 5. One of them is the focus on linear models for simplicity of analysis. Additionally, the paper focuses on the analysis of smooth mixture of experts, whereas in practice sparse MoEs are not smooth or continuous. This should be discussed under limitations.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n**S1:** The paper focuses on the theoretical and empirical analysis of the adversarial robustness of mixture of expert (MoE) models. This is an important problem since the robustness of neural networks to bounded adversarial perturbations is a desirable property. \n\n**S2:** The work seems to be relatively novel to my knowledge since existing works have not focussed much on the analysis of robustness of over-parameterized MoEs through the lens of Lipschitz constant. It is inspired by the recent work of Bubeck and Sellke [3], who establish a result on the lower bound of the Lipschitz constant of over-parameterized neural networks in terms of the number of parameters, samples, and the dimension.\n \n**S3:** The presentation of technical details is fairly clear to follow, except in a few places where some clarifications would help (please see my questions). \n\n### Weaknesses\n\n**W1:** Given that practical MoEs follow the sparse combination of experts as specified by equation (1), they are not continuous functions. Hence, small changes in their input can lead to large changes in their output. The analysis presented in the paper assumes a smooth weighted mixture of experts. It is not clear how this analysis can be extended to the sparse MoEs, or how one can draw insights from the analysis for the case of sparse MoEs.\n\n**W2:** Some of the details about the experiments are not clear or missing (e.g., attack objectives, definition of metrics). This would be important to make the work more reproducible (please see my questions). \n\n**W3:** (Minor) The analysis could be supported with some intuitions and maybe a figure (a toy example) for illustration. That would convey the key ideas in a better way. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "hmisZmRxriI", "reviewer": "Reviewer_P3QL", "summary": "This paper gives a theoretical analysis to the adversarial robustness of MoEs, it shows MoEs achieve better robustness than the dense models through lower bounding the Lipschitz constant of MoEs by that of the dense models theoretically, and then shows superior robustness of MoEs compared to the dense models experimentally.", "questions": "As stated in the previous part, firstly, I think it is better to run the experiments on more attacks, just running it on the PGD attacks is not convincing; secondly, the theory in this paper lower bounds the Lipschitz constant of MoEs by that of the dense models, and then the authors claim that MoEs achieve better robustness than the dense counterparts theoretically, what I concern is that as a global property, can the Lipschitz constant fully describe the robustness of a model? Perhaps a smaller Lipschitz constant means the model will have better robustness in the worst case, i.e. the mass of the data is concentrated on the most steep part of the model.", "limitations": "The authors addressed the limitations of this work in the conclusion part.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- This paper lower bounds the Lipschitz constant of MoEs by that of the dense models, showing that MoEs achieve better robustness than the dense models.\n- This paper does some experiments to verify the superior robustness of MoEs compared to the dense models and uncovers some intriguing properties of adversarial attacks for MoEs.\n- Although the linear regression model and the kernel regression is quite simple, this paper shows some interesting results about the robustness of MoEs.\n\nWeaknesses:\n- In the experiments, the paper just uses the PGD adversarial attacks, it is not convincing, as is said in [1], just use PGD attacks is not enough, I suggest to do further experiments on AutoAttack[1] and C&W[2].\n- Lipschitz constant is a global property, but adversarial robustness is not a global property, it depends on the data distribution and other factors. So I think the Lipschitz constant is not sufficient to describe the robustness of a classifier.\n\n[1] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.\n\n[2] Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. IEEE Symposium on Security and Privacy (SP), pages 39–57, 2017.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Ny7pWduz1Bz", "reviewer": "Reviewer_wLgw", "summary": "This paper studies the robustness of mixture-of-experts (MOE) models.  The authors study the Lipschitz properties of MOEs, and they study MOEs in a simple linear setting.  Ultimately, they seek to make connections to recent work that has studied how over-parameterization (OP) plays a role in robustness.  They also provide a set of experiments to empirical study this setting.", "questions": "* The authors say that the $\\ell_\\infty$ and $\\ell_2$ norms are \"popular choices in practice.\"  What does \"in practice\" mean here?  I am not aware of any settings \"in practice\" that actually use $\\ell_p$ robustness.  I think that in the broader context of the literature, $\\ell_p$ robustness is seen as a stepping stone toward understanding more practically relevant security concerns, such as distribution shifts in autonomous vehicles tasks or medical imaging.\n\n* In the sentence:\n\n> \"The compute needed to evaluate an image on the dense ViT models are 8.9 and 17.9 respectively.\"\n\nWhat are the units on 8.9 and 17.9?", "limitations": "I did not see a discussion of this.  At the very least, the authors could comment on the limitations of their analysis.  Furthermore, they could comment on the potential for nefarious actors to use adversarial technology.  Although in principle I agree that this paper is theoretical in nature and that there aren't any concrete concerns from a societal point of view.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n---\n\n**Similar linear setting.**  I liked the analysis that the authors performed for the case of linear MOEs.  Although the proofs seemed to follow standard techniques, I think that this part of the paper was a nice first step toward measuring the Lipschitzness of MOE models.\n\n**New problem setting.** I have not seen many works that consider the problem setting of the robustness of MOEs.  If the robustness of MOEs is fundamentally different from that of other model classes, then this would represent a useful direction.\n\n\n### Weaknesses\n---\n\n**Preliminaries** The authors repeatedly refer to the idea that MOEs have smaller computation cost.  I do not understand what this means.  Does it mean that the models are more computationally efficient at inference time?  Does it mean that they are less costly to train?  Without a description of how MOE models work and what their practical advantages are, I think that many readers will be left wondering the same thing.  For instance, they authors say that\n\n> \"Practical models also tend to this setup to match the FLOPs of dense models.\" (sic)\n\nWhat does \"FLOPS of dense models\" mean?  Which dense models are the authors referring to?  What is even meant by a dense model -- e.g. convolutional neural networks, MLPs, etc?  I think this points to the need for a stronger preliminary section, where MOE and dense models are both introduced and defined.  Readers coming from the robustness literature may not be familiar with MOEs, so it's important to set those readers up so that they can understand the rest of the paper.\n\n**Is this a problem?**  I'm having trouble discerning whether MOEs suffer from robustness issues.  The authors give some intuitive evidence for the lack of robustness of MOEs (e.g., that routing protocols can change given small perturbations), but there isn't any concrete  empirical evidence that MOEs are not robustness.  Therefore, I think it's not unreasonable to question why these results are meaningful.  \n\nLet me explain a bit more.  In the standard robustness literature, it has been repeatedly demonstrated that CNNs and MLPs are highly vulnerable to attacks.  For example, a PGD attack can degrade performance on CIFAR10 for ResNet18 from ~94% to essentially 0% (i.e., worse than random guessing).  This empirical result spawned all of the research concerning the robustness of these models.\n\nIn this paper, we are considering a new model -- MOEs -- which the authors describe as being more computationally efficient and using a fundamentally different design.  Therefore, I think that before studying the theoretical aspects of the robustness of MOEs, it would be worthwhile to demonstrate whether or not MOEs actually lack robustness to small perturbations.  \n\n**Related work.**  I think that the discussion of related work could be significantly improved.  At the very least, there should be a section called \"Related Work\" where the authors discuss past approaches for robustness and lines of work important to the development of MOE models.  Indeed, there seems to be previous work that studies the robustness of MOE models:\n\n> Xu, Kaidi, et al. \"Mixture of robust experts (more): A robust denoising method towards multiple perturbations.\" arXiv preprint arXiv:2104.10586 (2021).\n\nThe authors should cite works like these to distinguish their contribution from that of the literature.\n\n**Claims in the introduction.**  Some of the claims made in the introduction seem to be inaccurate.  For instance, the claim that adversarial robustness only refers to studying *bounded* changes.  In fact, the field has evolved to study more general (non-norm-bounded) shifts in data; see e.g.,\n\n> Wong, Eric, and J. Zico Kolter. \"Learning perturbation sets for robust machine learning.\" arXiv preprint arXiv:2007.08450 (2020).\n\n> Robey, Alexander, Hamed Hassani, and George J. Pappas. \"Model-based robust deep learning: Generalizing to natural, out-of-distribution data.\" arXiv preprint arXiv:2005.10247 (2020).\n\n> Hendrycks, Dan, et al. \"The many faces of robustness: A critical analysis of out-of-distribution generalization.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\nFurthermore, the authors claim that poor adversarial robustness is due to over-parameterization.  This also seems inaccurate, as poor robustness has also been attributed to adversarial subspaces\n\n> Gilmer, Justin, et al. \"Adversarial spheres.\" arXiv preprint arXiv:1801.02774 (2018).\n\nand to the local linearity of DNNs with piece-wise linear activations\n\n> Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. \"Explaining and harnessing adversarial examples.\" arXiv preprint arXiv:1412.6572 (2014).\n\namong other causes.  These claims highlight the need for a more comprehensive literature review in my opinion.\n\n**Connections to Bubeck and Selke [3].**  I don't understand the impact of the connections to [3].  Essentially, the argument seems to be that as MOEs have $E$ times more parameters, one can replace $P$ by $P\\cdot E$ and then you get a constant factor in the bound.  The thing is, while we may consider taking the number of data points to infinity, we will never have very large number of experts in the same way from my understanding.  Sure, if you take $E\\to\\infty$, the bound gets small, but is this practically possible?\n\n**Lack of a practical takeaway.**  One question I have about this work: What is are practical takeaways?  The goal of this paper seems to be to study the robustness of MOEs.  However, it's hard to draw any concrete conclusions based on my reading.  For example, does the theory inform the reader of any algorithm concerning how to improve MOE robustness?  Are there settings where MOEs are *significantly* more robust than ViT models?  The experiments seem rather non-conclusive to me in this regard, as the ViT models perform very similarly to the MOE models.  Indeed, in Figure 1, the curve are all almost overlapping.  The same is true in Figure 3.  So setting the theory aside, I'm not sure what the practical takeaways are from this paper.\n\n**Proof of Lemma 1.**  I believe that there is an error in the proof of Lemma 1.  The proof begins by calculating the gradient of the mixture of experts:\n\n$$ \\nabla f(x) = \\sum_{i=1}^E p_i(x) \\cdot \\nabla f_i(x) + \\sum_{i=1}^E \\nabla p_i(x) \\cdot f_i(x)$$\n\nWe then take the norm of both sides, and apply the triangle inequality:\n\n$$ || \\nabla f(x) || \\leq \\sum_{i=1}^E p_i(x) \\cdot ||\\nabla f_i(x)|| + \\left|\\left| \\sum_{i=1}^E \\nabla p_i(x) \\cdot f_i(x) \\right|\\right|. $$\n\nNext we take the supremum over $x$ on both sides, yielding \n\n$$ \\text{Lip}(f) \\leq \\sup_{x} \\sum_{i=1}^E p_i(x) \\cdot ||\\nabla f_i(x)|| + \\sup_{x}\\left|\\left| \\sum_{i=1}^E p_i(x) [s_i - \\bar{s}(x)] \\cdot f_i(x) \\right|\\right|. $$\n\nWe can further upper bound the RHS by moving the supremum inside the sum, meaning that\n\n$$ \\text{Lip}(f) \\leq  \\sum_{i=1}^E \\sup_{x} p_i(x) \\cdot ||\\nabla f_i(x)|| + \\sup_{x}\\left|\\left| \\sum_{i=1}^E p_i(x) [s_i - \\bar{s}(x)] \\cdot f_i(x) \\right|\\right|. $$\n\nSo far, this aligns with the proof given in the paper (although I've written out the steps in more detail here).  This is the point at which I feel problems start to arise.  For any index $i\\in[E]$, consider the expression\n\n$$ \\sup_{x} p_i(x) \\cdot ||\\nabla f_i(x)||  $$\n\nThe authors assert that this expression is less than or equal to (or simply equal to -- the proof is ambiguous here) $p_i(x) \\text{Lip}(f_i)$, which I believe is incorrect (although if I'm wrong, I'd welcome a discussion with the authors).  \n\nIn general, my intuition for this being wrong is that in general, for real-valued functions $q(x)$ and $g(x)$, it does **not** hold that $\\sup_x(q(x) \\cdot g(x)) = q(x) \\cdot \\sup_x(g(x))$ $\\forall x$.  (In this particular case, we have $q(x) = p_i(x)$ and $g(x) = ||\\nabla f_i(x)||$.)  To make this more concrete, let's consider a counterexample.  Let $f(x)$ be defined piecewise (here I've dropped the $i$ subscript): \n\n$$ f(x) = (-1/2)x + 1/2 \\quad\\text{for}\\quad x<-1 $$\n$$f(x) = -x \\quad\\text{for}\\quad -1\\leq x\\leq 1$$\n$$f(x) = (-1/2)x - 1/2 \\quad\\text{for}\\quad x>1$$\n\nThe Lipschitz constant of this function is clearly $\\text{Lip}(f)=1$.  Now let \n\n$$ p(x) = \\epsilon \\quad\\forall x : |x| \\leq 1$$\n$$ p(x) = 1-\\epsilon \\quad\\forall x : |x| > 1$$\n\nwhere again, I've dropped the $i$ index for simplicity.  Then we have that \n\n$$ p(x) ||\\nabla f(x)|| = \\epsilon \\quad \\text{for}\\quad |x| \\leq 1$$\n$$ p(x) ||\\nabla f(x) || = (1-\\epsilon)/2 \\quad\\text{for}\\quad |x| >1 $$\n\nand therefore $\\sup_x p(x) ||\\nabla f(x) || = (1-\\epsilon) / 2$ provided that $\\epsilon < 1/3$.  On the other hand, we have that\n\n$$ p(x) \\text{Lip}(f) = \\epsilon \\quad\\text{for}\\quad |x|\\leq 1 $$\n$$ p(x) \\text{Lip}(f) = 1-\\epsilon \\quad\\text{for}\\quad |x| > 1 $$\n\nTherefore, we find that \n\n$$ \\epsilon = p(x) \\text{Lip}(f) < \\sup_x p(x) ||\\nabla f(x)|| = (1-\\epsilon)/2 $$\n\nif $\\epsilon < 1/3$ and $p(x) = \\epsilon$ (since $p(x)$ is a function of $x$, which is a free variable, we can pick $p(x)$ to be whatever we like from the above definition).  Therefore, it does not hold that $\\sup_x p(x) ||\\nabla f(x)|| \\leq p(x) \\text{Lip}(f)$, breaking the proof of Lemma 1.\n\nAs several results and much of the discussion in the paper follow from this Lemma, I think it's important that this concern be addressed.\n\n**Grammar.**  The grammar and structure of the paper could also be revised.  There are several typos as well, which should be addressed.\n\n### Final thoughts\n---\n\nOverall, while I think that this paper goes in an interesting direction, I also believe that it would need to be fundamentally rewritten and expanded to be considered for acceptance.  The motivation for this problem is shaky, given that it is not clear whether MOE models lack robustness in the first place.  The literature review is sparse, and some of the proofs appear to be incorrect.  Furthermore, the experiments are not convincing, as the difference in robustness between dense and non-dense models doesn't seem to be large.  For these reasons, I cannot recommend accept at this time, especially given the possibility that the proof of Lemma 1 is incorrect.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8ANvDdiC90", "reviewer": "Reviewer_XB3a", "summary": "The paper investigates the adversarial robustness of sparse Mixture of Experts models for image classification, in order to answer the question: do models with more parameters (but not necessarily more computational power) have better robustness? Bounds on the Lipschitz constant are derived to compare the sparse MoE models with their dense counterpart, to show that sparse MoE models are more robust. The claim is then verified with empirical experiments on large data sets.\n", "questions": "* In Section 3.2, the setup of the analysis is a regression problem instead of a classification problem. How do the results presented are impacted by the actual setup of ViT-MoE, where we have a non-linear model with multiple MoE layers in a classification problem? The layers between the MoE layers could amplify the impact of the perturbations and reduce the \"smoothness\" gained by using MoE.\n* In Definition 1, the statement is trivial. There's no restriction in $\\epsilon_1$, so it could be any arbitrarily large value. The definition is just saying that there is a number larger than another number.\n* In Section 4, the term \"token\" is used, but it is never described in the paper.\n* From Figure 1, ViT++ and ViT-MoE are indistinguishable. Is the improvement in robustness coming from the MoE or just from the increased model size?\n* Related to Figure 2: the figure has the rate of changes per layer. It would also be interesting to see if the attack always modifies the routing in all layers or if there is some pattern (e.g. for some inputs the change only happens in later layers; or, can the change in one layer have no impact on the next?)\n* In Figure 3, again, how significant is the difference? How does MoE helps with robustness?\n* Adaptive attacks: in Section 4.4, the auxiliary losses are included in the attack. What are those losses and how are they optimized? Does the attacker specifically tries to change the output of the routing? Can an attacker force all images to be routed to a single expert?\n* Please expand the discussion of related works, for example: [1, 2].\n* Improve the mathematical description in Section 2:\n    - Add the dependency on the label $y$ in the formulation of the adversarial examples in eqs 3, 4, 5.\n    - line 115: the clarification after \"i.e.\" is actually confusing (is the 0/1 loss used for evaluation only or for generation of the examples in the preceding equation?)\n    - Eq 5: in the current text the variable $t$ is defined as a real number. Replace with $\\forall t \\in \\{0, \\dots, \\tau -1\\}$\n\n### References\n[1]: Wu, B., Chen, J., Cai, D., He, X., & Gu, Q. (2021). Do Wider Neural Networks Really Help Adversarial Robustness? https://arxiv.org/abs/2010.01279  \n[2]: Xu, K., Wang, C., Cheng, H., Kailkhura, B., Lin, X., & Goldhahn, R. (2021). Mixture of robust experts (more): A robust denoising method towards multiple perturbations. https://arxiv.org/abs/2104.10586", "limitations": "The main limitation of the work is how applicable their theoretical analysis is to the actual large models of interest. It is not clear how the limited setup of their analysis actually applies to more complex models as used in the empirical experiments.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n+ The topic of the paper is relevant, both from the perspective of robustness analysis (what makes a model robust?) and the MoE perspective (with trend towards large models, MoE is a viable alternative, so what other benefits we get besides computational cost?).\n+ The paper has a very clear goal: are sparse MoE models more robust than their dense counterpart?\n+ The paper is well organize and, generally, it was easy to read.\n+ The empirical experimentation is performed on very large data sets and non-trivial models.\n+ Originality: while there are papers investigating robustness of large models, they do not analyze MoE models.\n\n## Weaknesses\n- The mathematical expressions/descriptions are sometimes imprecise/incomplete (e.g. from the eqs in Sec 2.2, it would seem that the true label of the image is not used to generate the adversarial example). See more in the questions.\n- The theoretical analysis (Sec 3) focus on linear (single layer) experts for a regression task and fixed routing. There's little discussion on how these results transfer to the much more complex setting of ViT-MoE (non-linear, multiple layers of MoE, non-fixed routing) for classification. \n- In the empirical evaluation, it is difficult to distinguish improvement (in particular for JFT-300M) and the significance of the results\n- Part of the analysis is conducted in a private data set (JFT-300M), which makes impossible to reproduce those results.\n- Limited discussion of related works\n\n## Summary\nOverall, the paper is clear, easy to follow and pursuits an original analysis of an interesting and relevant problem. However, it needs to be polished more as there are several issues that put in question the quality and significance of the results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept