S YMMETRIC -APL A CTIVATIONS : T RAINING I NSIGHTS AND R OBUSTNESS TO A DVERSARIAL A TTACKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Deep neural networks with learnable activation functions have shown superior performance over deep neural networks with ﬁxed activation functions for many different problems.
The adaptability of learnable activation functions adds expressive power to the model which results in better performance.
Here, we propose a new learnable activation function based on Adaptive Piecewise Linear units (APL), which 1) gives equal expressive power to both the positive and negative halves on the input space and 2) is able to approximate any zero-centered continuous non-linearity in a closed interval.
We investigate how the shape of the SymmetricAPL function changes during training and perform ablation studies to gain insight into the reason behind these changes.
We hypothesize that these activation functions go through two distinct stages: 1) adding gradient information and 2) adding expressive power.
Finally, we show that the use of Symmetric-APL activations can signiﬁcantly increase the robustness of deep neural networks to adversarial attacks.
Our experiments on both black-box and open-box adversarial attacks show that commonly-used architectures, namely Lenet, Network-in-Network, and
ResNet-18 can be up to 51% more resistant to adversarial fooling by only using the proposed activation functions instead of ReLUs.
I NTRODUCTION
Deep neural networks (DNNs) are a series of linear transformations followed by point-wise nonlinear functions.
These non-linear functions are also referred to as activation functions.
Activation functions directly inﬂuence the training dynamics and, therefore, affect the ﬁnal performance of the
DNN.
For instance, activations such as Sigmoid and Tanh suffer from what is known as the “vanishing gradient” problem, when the inputs are in the saturation regions of these functions.
This can adversely affect network performance and cause slow convergence.
Rectiﬁed linear units (ReLU) (Nair & Hinton, 2010) have been shown to perform better than Sigmoid and Tanh in many scenarios.
Instead of using a ﬁxed activation function, one can learn a parameterized activation function to add expressive power to the model.
Towards this goal, signiﬁcant efforts have been made to introduce different learnable activation functions (Agostinelli et al., 2014;
He et al., 2015; Ramachandran et al., 2017; Jin et al., 2016;
Li et al., 2016).
Although most of these adaptive activation functions outperform ﬁxed activation functions in multiple tasks, there is very little insight into why.
In this paper, we propose the Symmetric Adaptive Piecewise Linear unit (Symmetric-APL; pronounced as Symmetric apple: "æp.@l ) as a new adaptive activation function which builds upon the adaptive piecewise linear unit (APL) (Agostinelli et al., 2014).
Similar to APL units, S-APL has multiple hinges which are connected by linear pieces.
The S-APL activation is designed to give the same expressive power to both the positive and negative sides of the x-axis.
We show that SAPL can approximate any continuous and zero-centered function, which encompasses many types of learned and ﬁxed activations functions.
Using this ﬂexible learned activation function, we can then analyze how the shape changes during training and use ablation studies to investigate the role that these changes play in optimization.
This helps us better understand learned activation functions and explain why they often provide increases in performance.
Finally, we evaluate the robustness of S-APL activated networks to adversarial attacks.
Experiments are done using both black-box and open-box adversarial attacks where we observe S-APL equipped networks are more robust to attacks than their ReLU counterparts.
Experiments show that the robustness improves against differential evolution attacks (Su et al., 2019b) by (up to) 51% on Lenet , 47% on ResNet-18 (He et al., 2016), 43% on pure CNN, and 38% on Net in Net (Lin et al., 2013).
For open-box adversarial attacks, the robustness also improves against FGSM and CW-L2 (Goodfellow et al., 2014) attack by (up to) 36%, 28%, and 25% for Lenet, pure CNN, and ResNetarchitectures, respectively using CIFAR-10 dataset.
R ELATED W ORKS
ReLU and its variants such as Leaky-ReLU (Maas et al., 2013), P-ReLU (He et al., 2015), exponential linear units (ELU) (Clevert et al., 2015), and scaled exponential linear units (SELU) (Klambauer et al., 2017) are believed to solve the vanishing gradient problem and some are shown to better normalize the network.
By changing the negative side of x-axis, these variants push the mean activations of each unit closer to zero and accelerate learning.
Some of the pioneering attempts to learn activations in a neural network can be found in Poli (1996),
Weingaertner et al.
(2002), and Khan et al.
(2013) where the authors proposed novel approaches to learning the best activation function per neuron among a pool of candidate activations by employing genetic and evolutionary algorithms.
Maxout (Goodfellow et al., 2013) has been introduced as an activation function aimed at enhancing the model averaging properties of dropout (Srivastava et al., 2014).
However, not only is it limited to approximating convex functions, but it also requires a signiﬁcant increase in parameters.
Neural architecture search (Ramachandran et al., 2017) uses a combination of exhaustive and reinforcement learning based search.
Although the authors discovered multiple novel activation functions, of which f ( x ) = x ∗ sigmoid ( βx ) performed slightly better than ReLU, the search space of activation functions is limited.
S-ReLU (Jin et al., 2016) was proposed as an adaptive activation function which can mimic both convex and non-convex functions.
It is worth mentioning that, in Lin et al.
, the authors proposed the network in network approach where they replace activation functions in convolutional layers with small multi-layer perceptrons.
Theoretically, due to Universal Approximation Theorem, this is the most expressive activation; however, it requires many more parameters.
A DAPTIVE P IECEWISE A CTIVATION F UNCTION 3. A DAPTIVE P IECEWISE L INEAR U NITS
Agostinelli et al.
(2014) proposed an adaptive activation function which consists of multiple linear pieces.
The formulation of this adaptive piecewise linear (APL) activation function is shown in
Equation 1.
[Equation]
Where h i is the non-linear transformation for the hidden unit i and S is a hyperparameter that determines the number of hinges.
a i is the slope of the linear piece and b i is the location of the hinge.
However, APL units are not zero-centered as they can have a non-zero output for a input of zero.
This behavior provides no apparent beneﬁcial purpose.
Furthermore, APL units can only represent piecewise linear functions whose output h i ( x ) =
x for x > u , for some scalar u .
This signiﬁcantly restricts the class of piecewise linear functions that APLs can express.
3. S YMMETRIC A DAPTIVE P IECEWISE L INEAR U NITS (APL: "æp.@l )
To overcome the shortcomings of APL activation function, we deﬁne a new adaptive piecewise linear activation function that is zero-centered and gives the same expressive power to the positive and negative half of the input space.
Since S-APL shares the variables a s + , b s + , a s − , and b s − among all the neurons of a layer (e.i.,
h i ( x, S ) does not depend on i ), it requires fewer parameters in compare to the plain APL.
We formulate the activation h i of the hidden unit i as the summation of symmetric hinge-shaped pieces.
[Equation]
Similar to APL units, S is a hyperparameter which determines the number of hinges (on each side of the x-axis).
a s + and a s − control the slope of linear pieces in both the positive and negative sides of the x-axis, while b s + , b s − ∈
[0 , + ∞ ) determine the location of hinge s .
The total number of additional parameters is 4 S for a single layer.
Because our networks are trained with batch normalization (Ioffe & Szegedy, 2015) before each S-APL layer, the distribution of input to the S-APL function will follow that of a normal distribution.
Therefore, we ﬁx the hinges to be a pre-determined number of standard deviations away from the mean, decreasing the number of parameters to 2 S .
Based on Equation 2, S-APL activation is zero-centered which leads to more stable statistical behavior during training.
In case of using ﬁxed hinges’ locations for APL, due to increasing the number of parameters, S-APL offers more ﬂexibility than the normal APL.
However, this increase is of O ( constant ) which is negligible.
With the assumption of a s + = a s − and b s + = b s − , since h si ( x ) = h si ( − x ) , S-APL allows neurons to activate symmetrically.
As it is mentioned in Zhao &
Grifﬁn (2016), networks with symmetric activation functions are more robust to adversarial fooling.
We will delve into this property in later sections.
S-APL can better handle the limits.
It could be inferred from the deﬁnition that, in contrast to normal
APL, increasing the absolute value of the input x will also increase the number of terms affecting the h i ( x ) .
However, normal APL requires the rightmost linear section in all the component functions to have a unit slope and bias 0, which is not an appropriate constraint and undermines its representation ability.
The following theorem shows that S-APL can approximate any non-linear, zero-centered, and Lipschitz continuous function in a closed interval of real numbers.
It is worth mentioning that APL was not ﬂexible enough to do so.
Theorem 3.1 For any function f : [ A, B ] → R and (cid:15) ∈ R , ∃ S ∈ N , where | f
( x ) − S-APL(x) | ≤ (cid:15) , assuming:
• A and B are ﬁnite real numbers.
• f is M-Lipschitz continuous.
Proof: We start by dividing the interval [ A, B ] into S equal sub-intervals [ A i , B i ] .
In other words, we put S equally distanced hinges in the interval [ A, B ] .
So the length of each sub-interval will be B − AS .
We start approximating f from the left most sub-interval which is [ A, A + B − AS ] .
The approximation of f in that sub-interval will be a line which connects f ( A ) to f ( A + B − AS ) .
Due to
M-Lipschitz continuity of f , for any x ∈ [ A, A + B − AS ] :
[Equation]
Again, due to M-Lipschitz continuity of f , we know that:
[Equation]
Using Equation 4, we can re-write the inequality 3 as follows:
[Equation]
So, by choosing S ≥ 2 M ( B − A ) (cid:15) , we satisfy that | f ( x ) − S-APL(x) | ≤ (cid:15) .
in order to complete the proof, we need to show that this approximation can be applied to other sub-intervals of [ A, B ] as well.
We do so by induction as follows:
Suppose that the approximation is done for sub-intervals [ A 1 , B 1 ] , ..., [ A i , B i ] , now we prove that for sub-interval [ A i +1 ,
B i +1 ] we can do the approximation which is connecting f
( A i +1 ) to f ( B i +1 ) .
The slope of S-APL in the subinterval [ A i , B i ] is (cid:80)
i 1 a i + or (cid:80) i 1 a
i − (depending on the sign of the sub-interval).
However, the slope of S-APL in the sub-interval [ A i +1 , B i +1 ] will be either (cid:80) i + a i + or
(cid:80) i
+ a i − .
In both cases, the extra term a
i +
+ or a i +1 − is not ﬁxed and can change the slop to any arbitrary value.
This fact plus the assumption of continuity of S-APL guarantee that f ( A i +1 ) can be connected to f ( B i +1 ) which was our proposed approximation.
As we showed the approximation for the left most sub-interval, the induction is complete.
(cid:3)
C OMPARISON TO C OMMONLY U SED A CTIVATION F UNCTIONS
In order to show that S-APL unit is beneﬁcial for deep neural networks, we compare it with wellknown activation functions in different architectures.
We employed ﬁve different neural networks trained on three different datasets, MNIST, CIFAR-10, and CIFAR-100.
We used S = 4 and set the hinges at positions x = − 2 .
5 , − 2 , − 1 , 0 , +1 , +2 , +2 .
5 (0 considered as two hinges).
Slope of a s +is initialized to 1 and the remaining slopes are initiazlied to 0 .
With this initialization, the initialized
S-APL mimics the shape of ReLU.
The results of the experiments are shown in Table 1.
The table shows that S-APL leads to a better validation accuracy in almost all architectures.
[Table]
H OW S-APL C HANGES D URING T RAINING AND P OSSIBLE R EASONS W HY
Figures 1 and 2 show how the shape of S-APL changes for each layer during training.
From these ﬁgures, we can see that during the early stages of training, the shape of S-APL looks similar to that of leaky ReLU: having a negative output for x < 0 and a positive output for x ≥ 0 .
During the later stages of training, the activation function has a positive output for both x < 0 and x ≥ 0 .
In addition, the slope of the output decreases as the magnitude of x increases.
We hypothesize that there are two distinct stages during the optimization of these learned activation functions.
In the ﬁrst stage, the function returns negative values for x < 0 and in the second stage, the function returns positive values for x < 0 .
To investigate this hypothesis, we train S-APL under two different conditions where we: 1) force the output to only be negative for x < 0 (S-APLnegative) and 2) force the output to only be positive for x < 0 (S-APL-positive).
We accomplish this by taking the absolute value of all a s − and multiplying by +1 or − 1 .
Figure 2 shows that, although S-APL-positive has the ability to mimic the ﬁnal learned shape of the
S-APL function
, it barely deviates from the ReLU initialization.
This shows that the ability to give a negative output for negative inputs is crucial for S-APLs.
Figure 2 shows that S-APL-negative does initially go negative in the 4th layer and then returns to the ReLU shape.
To gain further insight, we compare the loss of these activation functions in Figure 3.
We see that S-APL has a lower loss compared to S-APL-positive and S-APL-negative.
Furthermore, S-APL-negative has a lower loss compared to S-APL-positive which has a lower loss than ReLU.
These results lead us to believe that it is possible that the ﬁrst stage, S-APL-negative, adds gradient information by giving negative output for negative inputs.
It is possible that S-APL-positive is not capable of doing so because giving positive outputs for negative inputs creates an identiﬁability issue.
In other words, outputs for x < 0 and x ≥ 0 look the same.
However, we can see that this property is useful for the S-APL function in the later stages of training.
It is possible that this is because the DNN has adapted its parameters such that outputs on both the negative and positive sides capture a similar concept, adding expressive power to the model.
Furthermore, one cannot achieve the ﬁnal S-APL shape without the ﬁrst stage of having negative outputs for negative inputs.
To conclude, it appears that the S-APL units ﬁrst go through two stages: 1) adding gradient information and 2) adding expressive power and that the second stage depends on the ﬁrst stage.
This information can be used to inform the design of learnable activation functions and even DNN optimization techniques.
R OBUSTNESS TO A DVERSARIAL A TTACKS
DNNs have been shown to be vulnerable to many types of adversarial attacks (Szegedy et al., 2013;
Goodfellow et al., 2014).
Research suggests that activation functions are a major cause of this vulnerability (Zantedeschi et al., 2017).
Zhang et al.
(2018) bounds a given activation function with a few linear and quadratic functions and allows it to tackle general activation functions.
This adds up with applying a different activation for each neuron so the resulting network shows efﬁciency and robustness to adversarial foolings.
Wang et al.
(2018) proposed a data-dependent activation function and empirically shows its robustness to both black-box and gradient-based adversarial attacks.
Other studies such as Rakin et al.
(2018), Dhillon et al.
(2018), and Song et al.
(2018) focused on other additional properties of activation such as quantization and pruning and showed they can improve the robustness of DNNs against adversarial examples.
Recently, Zhao & Grifﬁn (2016) theoretically showed that DNNs with symmetric activations are less likely to get fooled.
The authors proved that “symmetric units suppress unusual signals of exceptional magnitude which result in robustness to adversarial fooling and higher expressibility.”
Due to the symmetric convergence of S-APL units and also their hinges’ locations ( b + i = b − i ) which result in h ( x ) =
h ( − x ) , S-APL units are capable of increasing the robustness of DNNs against adversarial attacks.
In this section, relying on the properties of S-APL such as data dependency, piecewise linearity, noninjective behavior, better handling of extreme values, and most importantly, symmetric shape, we show that adding S-APL to a DNNs greatly improves the robustness against adversarial attacks.
This claim is veriﬁed through a wide range of experiments with the CIFAR-10 dataset under both blackbox and open-box methods, including one-pixel-attack and Fast Gradient Sign Method.
Figure 4 provides an intuition for the robustness of S-APL activated network in comparison to ReLU activated ones.
For each of the two networks, we take 100 random samples of frog and ship images and visualize the pre-softmax representations using tSNE visualization (Maaten & Hinton, 2008) in
Figure 4.
As it is depicted, for S-APL activated networks, the two classes are better separated.
We suspect that red samples such as those that are lying on the outer circle in the left plot of Figuremight be somewhere on the manifold of the data where the network is more susceptible.
As one can see, there is no red point on the outer circle of the right plot which might be showing the robustness of S-APL networks.
For other samples on the inner circle and the cloud next to circles, we can see better separation by the S-APL network.
6. B LACK -B OX A DVERSARIAL A TTACK
For black-box attacks against a DNN, we assume the adversary has no information about the structure or parameters of the DNN and does not have access to any large training dataset.
The adversary’s only capability is to observe labels assigned by the DNN for chosen inputs in a manner analog to a cryptographic oracle.
A successful attacking technique of this type is Su et al.
(2019a) which is based on differential evolution.
We can iteratively generate adversarial images to try to minimize the conﬁdence (probability) of the network’s classiﬁcation.
The process starts with random modiﬁcations of a few pixels to generate adversarial examples.
At each step, one feeds several adversarial images to the DNN and only observes the output as the probability of different classes.
Examples that lowered the conﬁdence of the true class will be kept to generate the next generation of adversaries.
The next generation modiﬁcations are then evolved through a certain mutation scheme to generate new adversarial images.
By repeating these steps for a few iterations, the adversarial modiﬁcations generate more and more misleading images.
The last step returns an adversarial modiﬁcation that reduces the conﬁdence of the true class the most.
This means that the conﬁdence of the true class would be reduced so much that a new incorrect category now has the highest classiﬁcation conﬁdence.
This technique is called untargeted attack since we don’t specify the labels of the modiﬁed images.
In the following experiment, we modify one, three, and ﬁve pixels of images to generate adversarial examples.
The mutation scheme we used for this experiment is as follows:
[Equation]
Where r 1 , r 2 , and r 3 are three non-equal random indices of the modiﬁcations at level l .
x l +1 i will be an element of a new candidate modiﬁcation.
To evaluate the effect of S-APL activation on the robustness of DNNs, we employ commonlyused architectures, namely, Lenet5, Network-in-Network (Lin et al., 2013), pure CNN (ours), and
ResNet-18 (He et al., 2016).
Each architecture is trained with ReLU, APL, and Swish activation functions and once with S-APL activation functions.
The results in Table 2 show that Lenet
and
ResNet-18 architectures are 51% and 47.8% more resistant to back-box adversarial attacks.
As authors in Zhao & Grifﬁn (2016) theoretically proved, that one can be robust to adversarial examples with symmetric behavior.
After observing adversarial samples which are deceiving to both networks, we found that S-APL activated network still assigns high conﬁdence to the true labels of the perturbed images.
In other words, the difference between the conﬁdence of the true label and the conﬁdence of adversarially
[Table]
wrong label is much smaller in S-APL network than networks that are activated with ReLU, APL, or Swish.
More precisely, we measure the average of | Z ( x
[Table]
) true
[Table]
− Z ( x
[Table]
) adversarial label | over all adversarial samples where each network is fooled.
Z ( . ) is the post-softmax output and x (cid:48) is the adversarial sample.
The avg ( | Z ( x (cid:48) ) true label − Z ( x (cid:48) ) adversarial label | ) for each model is also included in
Table 2.
Due to the reported averages, we can conclude that S-APL is much more robust against being adversarially deceived.
[Equation]
A TTACK
To further explore the robustness of S-APL activated networks, in this section, we consider two of the popular benchmarks of open-box adversarial attacks: Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014) and CW attack (Carlini & Wagner, 2017).
For both attacking strategy, we consider three different architectures and compare the rate of successful attacks for each of the networks with ReLU, APL, or Swich activations versus S-APL activation functions.
The dataset and architectures are same as those were used for black-box adversarial attacks.
6.2. FGSM
Fast Gradient Sign Method generates an adversarial image x (cid:48) from the original image x by maximizing the loss L ( x (cid:48) , y ) , where y is the true label of the image x .
This maximization problem is subjected to || x − x (cid:48)
|| ∞ ≤ (cid:15) where (cid:15) is considered as the attack strength .
Approximating with the ﬁrst term of Taylor expansion, we have:
[Equation]
So the adversarial image x (cid:48) would be:
[Equation]
Where the (cid:15) is considered as the power of the attack.
This form of FGSM is considered to be an untargeted attack where there is no pre-speciﬁed label t that x (cid:48) should be classiﬁed as.
This method only yields an adversary x (cid:48) which will not be classiﬁed as y .
To see how S-APL increases the robustness against FGSM attack, we employ three architectures all trained to classify CIFAR-10 dataset.
For different range of (cid:15) the results are summarized in Table 3.
[Table] [Table]
In the best case, for Lenet architecture and (cid:15) = 0 .
02 , S-APL activated network is 36% more robust than the ReLU activated one.
The robustness also improves by 28% and 25% for pure CNN and
ResNet-18 respectively.
We can see the overall increase in the robustness over all networks and (cid:15) values.
However, one can conclude that for smaller values of (cid:15) the robustness of S-APL activated network is higher in comparison to ReLU activated networks.
As we increase the attack strength ( (cid:15) ), the robustness of S-APL networks gets closer to the robustness of ReLU activated networks.
6.2. CW-L
We showed that replacing ReLU with S-APL activation units can greatly improve the robustness of DNNs against FGSM attack.
To further investigate the robustness that S-APL is adding to the network, we are using a more powerful open-box adversarial attack.
Another open-box adversarial attack which is generally more powerful than FGSM was introduced in Carlini & Wagner (2017).
For a given pair of image x and label y , this technique is trying to ﬁnd the minimum perturbation δ , so that the perturbed image x (cid:48) is classiﬁed as t (cid:54) =
y .
Considering
L 2 norm as the goal of minimization, this perturbation minimization problem can be formulated as follows:
[Equation]
Or equally, to ease the satisfaction of equality, Equation 9 can be rephrased as min ||
δ ||
22 + c.g ( x + δ ) where g ( x ) = max ( max t (cid:54) =
y ( logit ( x ) t − logit ( x ) y )) , c is Lagrange multipliyer, and logit ( x ) is the pre-softmax vector for the input x .
Here, we examine the robustness of S-APL activated networks against CW-L2 attack.
As it is shown in the table 4, CW-L2 attack is more powerful than FGSM.
Although the S-APL activated network is less robust to CW-L2 attack than to FGSM attack, it is still showing more robustness in comparison to ReLU, APL, and Swish activated networks.
Similar to black-box attack, the metric avg | Z true −
Z adv | is calculated for successfully attacked images.
The smaller values of avg | Z true − Z adv | for
S-APL activated networks in both open-box attacks shows that S-APL activated networks are more resistant to being fooled.
C ONCLUSION AND F UTURE W ORKS
We extend the idea of piecewise linear activation functions by introducing S-APL as a learnable and potentially symmetric activation function.
We investigate the properties of S-APL and their evolution during training to understand how these functions accelerate learning and increase expressive power in DNNs.
Finally, we show that S-APL networks show more robustness to adversarial attacks than ReLU networks.
One can potentially design a hybrid defense method that combines other defense techniques and S-APL units.
A PPENDIX 8. N UMBER OF H INGES AND THE S YMMETRY OF S-APL
In this section, we perform a variety of experiments to ﬁnd the best setting for S-APL activation in terms of complexity and expressibility.
Before going to experiments, it is necessary to mention that we used batch normalization layer before every S-APL layer in our experiments.
This means that the input of the S-APL activation is unit Gaussian.
Relying on this fact and the experiments done in Agostinelli et al.
we chose a ﬁxed and symmetric set of hinges on ± σ, ± 2 σ, ± 2 .
5 σ, ± 3 σ, and ... .
This will reduce the number of
S-APL’s parameters by a factor of 2 which makes it much less computationally expensive.
First, we assess the effect of S on the performance of S-APL.
Due to Theorem 3.1, greater S values increase the expressive power of the S-APL which results in better ﬁtting the training data.
We tried
S ∈ [2 , 3 , 4 , 5 , 6] , (which means 4, 6, 8, 10, and 12 hinges) with ﬁxed hinges’ locations on MNIST (LeCun & Cortes, 2010), CIFAR-10 (Krizhevsky et al.) and binary classiﬁcation of IMDB review dataset (Maas et al., 2011) to cover MLP, convolution, and recurrent architectures respectively.
Each network is trained with two types of S-APL activation; 1) A shared S-APL : a shared unit among all neurons of a layer and 2)
An independent S-APL unit for each neuron of a layer.
As it is summarized in Table 5, in all cases for S ≥ 4 the performance is not decreasing signiﬁcantly.
On the other hand, due to the increase in the number of parameters of S-APL, the unit becomes more computationally expensive.
In Table 6 we compare the per-epoch training run-time for different number of hinges of a shared S-APL .
[Table] [Table]
For small values of S , we can see that S-APL is comparable to an exponential activation unit such as Tanh, and much faster than heavier activation such as Maxout.
As one can conclude from both Table 5 and 6, there is a trade-off between the complexity and expressibility.
We believe that S = 4 , or equally 8 hinges, is the best choice for the number of hinges where the hinges’ locations are symmetrically ﬁxed.
8. E XPERIMENTS ’ D ETAILS AND S TATISTICAL S IGNIFICANCE
In this section, we explain the experimental conditions and all the parameters used for each experiment.
Then, in order to make the results of Table 1 more interpretable, we perform a t-test (Kim, 2015) on all the error rates achieved in that experiment.
In section 4 , the experiment corresponding to Table 1 was performed using six different architectures.
Lenet 5 was used as it was introduced in LeCun et al.
It has two convolution layers followed by two MLPs that are connected to a softmax layer.
The MLP architecture which is only used for classifying MNIST dataset has three fully connected feed-forward layers with 258, 128, and 10 hidden units.
A dropout of rate 0 .
3 was added in between each layer.
{+The batch size was 
+}
and we trained the MLP network for 50 epochs.
Pure CNN architecture which is only taking advantage of convolutional layers, was introduced in Springenberg et al.
All the speciﬁcations, including kernel size, stride, learning rate, batch size, and the number of epochs are taken from the main article.
ResNet18 architecture as a variant of residual networks (He et al., 2016) was taken from Napoletano et al.
(2018) which has ﬁve residual convolution blocks, one MLP, and a softmax layer.
The size of the convolution kernels, strides, padding, pooling size, pooling operation, learning rate, batch size, and epochs are taken as the defaults mentioned in Napoletano et al.
EffectiveNets was introduced in Tan & Le (2019) and is used in our experiments exactly the way in was originally introduced.
It is necessary to mention that we use B 0 variant of EffectiveNets.
Lastly, Net in Net architecture which is using an MLP instead of a ﬁxed nonlinear transformation is taken from Agostinelli et al.
It is worth mentioning that in section [Table] all the experiments were performed ﬁve times.
To reduced the effect of noise and randomness, the average of the ﬁve times is reported.
In Tabel [Table] we show the statistical signiﬁcance of the experiments performed in section 4. Since each number is the average of ﬁve experiments
[Table]
we are able to perform a t-test and provide p-values and statistical signiﬁcance for each individual experiment.
As one can see in Table 7, most of the numbers of Table 1 are statistically signiﬁcant
[Table] [Table] In section 5 , we used two architectures to visualize S-APL shapes at different stages of the training process.
For Figure 1 and Figure 2 we used the same MLP and Lenet 5 architectures as described above.
In section 6 , we started by a tSNE visualization of 100 random samples of frogs and ships from the cifar10 test set.
The tSNE mapping was done with a learning rate of 30 and a perplexity of 40.
For the black-box adversarial attack experiments, each network was attacked ﬁve times and the reported number is the average of successful modiﬁcations in ﬁve different attacks.
Attacks were done using the maximum number of iteration to be 40 and the pop size to be 400.
As for the open-box attacks, for both FGSM and CW-L2 attack, we used the implementation and default hyper-parameters in Rauber et al.
Lastly, for activation function of networks in section 6 , ReLU ( y = x for x > 0 , 0 otherwise ), APL ( S = 5 , with ﬁxed hinges on 0 , ± 1 , and ± 2) , Swish ( y = x.sigmoid ( βx ) with β = 0 . 2) , and
S-APL (with the conﬁgurations mentioned in the previous section) were used.