Bandlimiting Neural Networks Against Adversarial Attacks
Authors: Yuping Lin
Kasra Ahmadi K. A.
Hui Jiang
Venue:  (2020)
Keywords: adversarial examples
adversarial attack defense
neural network
Fourier analysis

Abstract: In this paper, we study the adversarial attack and defence problem in deep learning from the perspective of Fourier analysis. We first explicitly compute the Fourier transform of deep ReLU neural networks and show that there exist decaying but non-zero high frequency components in the Fourier spectrum of neural networks. We then demonstrate that the vulnerability of neural networks towards adversarial samples can be attributed to these insignificant but non-zero high frequency components. Based on this analysis, we propose to use a simple post-averaging technique to smooth out these high frequency components to improve the robustness of neural networks against adversarial attacks. Experimental results on the ImageNet and the CIFAR-10 datasets have shown that our proposed method is universally effective to defend many existing adversarial attacking methods proposed in the literature, including FGSM, PGD, DeepFool and C&W attacks. Our post-averaging method is simple since it does not require any re-training, and meanwhile it can successfully defend over 80-96% of the adversarial samples generated by these methods without introducing significant performance degradation (less than 2%) on the original clean images.

Under review as a conference paper at ICLR 2020
BANDLIMITING NEURAL NETWORKS AGAINST ADVERSARIAL ATTACKS
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we study the adversarial attack and defence problem in deep learning from the perspective of Fourier analysis. We first explicitly compute the Fourier transform of deep ReL.U neural networks and show that there exist decaying but non-zero high frequency components in the Fourier spectrum of neural networks. We then demonstrate that the vulnerability of neural networks towards adversarial samples can be attributed to these insignificant but non-zero high frequency compo- simple post-averaging technique
nents. Based this analysis, we propose to smooth out these high frequency components to improve the robustness of neural networks against adversarial attacks. Experimental results the ImageNet and the CIFAR-10 datasets have shown that our proposed method is universally effective defend many existing adversarial attacking methods proposed in the literature, including FGSM, PGD, DeepFool and C&W attacks. Our post-averaging method is simple since it does not require any re-training, and meanwhile can successfully defend over 80-96% of the adversarial samples generated by these methods without introducing significant performance degradation (less than 2%) on the original clean images.
INTRODUCTION
Although deep neural networks (DNN) have shown to be powerful in many machine learning tasks, Szegedy et al. (2013) found that they are vulnerable adversarial samples. Adversarial samples are subtly altered inputs that can fool the trained model to produce erroneous outputs. They are more commonly seen image classification task and typically the perturbations to the original images are so small that they are imperceptible to human eye.
Research in adversarial attacks and defences is highly active in recent years. the attack side, many attacking methods have been proposed (Szegedy et al. 2013 Goodfellow et al.] 2014 Papernot
et al. 2016a Papernot et al. 2017 Mooosvvi-Dezooo et 2016 Kurakin et al. [2016] Madry et al.
2017 Carlini and Wagner, 2017a Chen et al.) [017]] Alzantot et al.) 2018: Brendel et al. (017))
with various ways to generate effective adversarial samples to circumvent new proposed defence methods. However, since different attacks usually are effective to different defences datasets, there is no consensus on which attack the strongest. Hence for the sake of simplicity, in this work, we will evaluate our proposed defence approach against four popular attacks for empirical analysis. In the defence side, various defence mechanisms have also been proposed, including adversarial training (Rozsa al. 2016 Kurakin al. 2016 Tramer et al., 2017 Madry et al. 2017 network distillation (Papermot et al. (0066b), gradient masking (Nguyen and Sinha|[007] adversarial detection (Feinman al., 2017) and adding modifications neural networks (Xie et al.. (017). Nonetheless, many of them were quickly defeated by new types of attacks (Carlini and Wagner 2016 2017b Athalye et al. 20188 Athalyy and Carlini 2018 Alzantot al.) 2018). Maddy et al.) (2017) tried to provide a theoretical security guarantee for adversarial training by a min-max loss formulation, but the difficultics in non-convex optimization and finding the ultimate adversarial samples for training may loosen this robustness guarantee. As result, far there is defence that universally robust to all adversarial attacks.
Along the line of researches, there were also investigations into the properties and existence of adversarial samples. [Szegedy et al. (2013) first observed the transferability of adversarial samples across models trained with different hyper-paramesn and across different training sets. They also

---

Under review as a conference paper at ICLR 2020
attributed the adversarial samples to the low-probability blind spots in the manifold. In (Goodfellow et al., 2014). the authors explained adversarial samples as "a result models being too linear, rather than too nonlinear." In (Papernot et al. 2016), the authors showed the transferability occurs across models with different structures and even different machine learning techniques in addition to neural networks. In summary, the general existence and transferability of adversarial samples well known but the reason of adversarial vulnerability still needs further investigation.
Generally speaking, when we view neural network as multivariate function f(x) of input x, small imperceptible perturbation Ax leads to a huge fluctuation A [x),, the large quantity Ax(x) Ax essentially corresponds high frequency components the Fourier spectrum of f(x). In this paper, we will start with the Fourier analysis of neural networks and elucidate why there always exist some decaying but nonzero high frequency response components in neural networks. Based on this analysis, we show that neural networks are inherently vulnerable to adversarial samples due to the underlying model structure. Next, we propose simple post-averaging method tackle this problem. Our proposed method is fairly simple since it works post-processing stage of any given neural network models and does not require re-training the networks at all. Furthermore, we have evaluated the averaaiggg method against four popular adversarial attacking methods and method is shown to be universally effective in defending all examined attacks. Experimental results on the ImageNet and the CIFAR-10 datasets have shown that simple post-averaging method can successfully defend over 80-96% of the adversarial samples generated by these attacks with little performance degradation (less than 2%) on the original clean images.
FOURIER ANALYSIS OF NEURAL NETWORKS
In order understand the behaviour of adversarial samples, it is essential to find the Fourier transform of neural networks. Fortunately, for some widely used neural networks, namely fully-connected neural networks using ReLU activation functions, we may explicitly derive their Fourier transform under some minor conditions. As we will show, these theoretical results will shed light on how adversarial samples happen in neural networks.
2.1 FOURIER TRANSFORM OF FULLY-CONNECTED RELU NEURAL NETWORKS
As we know, any fully-connected ReLU neural networks (prior to the softmax layer) essentially form piece-wise linear functions input space. Due to space limit, we will only present the main results in this section and the proofs and more details may be found in Appendix.
Definition 2.1. A piece-wise linear function is contimuous function f RR" The such that there are some hyperplanes passing through origin and dividing R² into M pairwise disjoint regions Rms (m 1,2, ... M). on each of which linear:
Ri W W2 X
x E Rz
f(x)
WM X RM
Lemma 2.2. Composition of piece-wise linear function with ReLU activation function is also piece-wise linear function.
Theorem 2.3. The output of any hidden unit in an a piece-wise linear function.
ReLU neural network is
This is straightforward because the input to any hidden node is a linear combination of piece-wise linear functions and this input is composed with the ReLU activation function to yield the output, which is also piece-wise linear. However, each region R., is the intersection of different number of half-spaces, enclosed by various hyperplanes in R". In general, these regions Ry, (m = 1,... .M) do not have simple shapes. For the purpose of mathematical analysis, need to decompose each region into a union some well-defined shapes having a uniform form, which is called infinite simplex.
2

---

Under review as a conference paper at ICLR 2020
Definition 2.4. Let V
V2,... -V.. be a set of linearly independent vectors in R". An infinite simplex, Riy, is defined as the region linearly spanned by V using only positive weights:
" VgVE
an 200 K 1.2,
Theorem 2.5. Each piece-wise linear function f(x) can be formulated as summation of some simpler functions: f(x) Et= Ji(x), each of which is linear and non-zero only in infinite
simplex as follows:
f((x) =
W X R
otherwise
where Vi is set n linearly independent vectors, and Wi is weight vector.
In practice, computational convenience, we may normalize all inputs x into the unit hyper-cube, U, = [0. 1]". Obviously, this assumption
always assume that the input to neural networks, is bounded. As result, for
be easily incorporated into the above analysis by multiplying each fi(x) in eq 2 by I[; h(x,), (h(1 Σ,) where h(x) is the Heaviside step function. Alternatively, we may simplify this term by adding n² additional hyperplanes to further split the input space to ensure all the elements of x do not change signs within each region Rv.. In this case, within each region At the largest absolute value among all elements of x is always achieved by specific element, as rg. In other words, the dimension Tr, achieves the largest absolute value inside A Similarly, the normalized piece-wist wise linear function may be represented
which is denoted
summation of some functions: f(x) Eg (x),, where each (x) (g 1,2, Q) has the following form:
9 = Wg x h(1 Irr.)
RV
otherwise
For every V, there exists an n x n invertible matrix Ag to linearly transform vectors of V, into standard basis vectors e, in R" As result, each function 9x(x) may be represented in terms of standard bases V. {el, .. ....... e,) as follows:
9q (x) {
Wy Xg h(1 -1--1 11)8,) x, Ri otherwise
0
where &, XA q. and w, = 1, Lemma 2.6. Fourier transform of the following function:
s(x) { h(1 - 11-1xx
x E Rt otherwise
may be presented as:
iw
S(w) = 2n
(3)
(wip wi)
where is the r-th component of frequency vector w (r 1,... n), and wp = 0.
Finally we derive the Fourier transform of fully-connected ReLU neural networks as follows.
Theorem 2.7. The Fourier transform of the output of any hidden node in a fullycconnectedd unbi- a ReLU neural network may be represented as Ee A,A," VS(WA -1), where v denote the differential operator.
For mathematical convenience, we assume neural networks have no biases here. However, regular neural networks with biases may be reformulated as unbiased ones by adding another dimension of constants. Thus. the main results here equally applicable both cases. Note that regular neural networks with biases are used experiments in this paper.

---

Under review as a conference paper at ICLR 2020
Figure 1: Illustration of input space divided into sub-regions by biased neural network. The black lines are the hyperplanes for the first network layer, while the blue lines are for the second layer and the red lines are for the third layer. A small perturbation from point A to point B may possibly cross many hyperplanes.
Obviously, neural networks are the so-called approximated bandlimited models as defined in (Jiang 2019), which have decaying high frequency components in Fourier spectrum. Theorem 2.7 further suggests that the matrices A, may contribute to the high frequency components when the corre- sponding region Rt are too small. This is clear because the determinant of A, is proportional to the volume of Rt, in R". In summary, the high frequency components of neural networks are mostly attributed to these tiny regions in the input space. As we will show later, these small regions may be explicitly exploited generate adversarial samples for neural networks.
2.2 UNDERSTANDING ADVERSARIAL SAMPLES
As shown in Theorem 2.3 neural network may be viewed as a sequential division of the input space into many small regions, as illustrated in Figure [ Each layer further division of the existing regions from the previous layers, with each region being divided differently. Hence a neural network with multiple layers would result in a tremendous amount of sub-regions in the input space. For example, when cutting n-dimensional space using N hyperplanes, the maximum number of regions may be computed as (*) + (+) +... + (.). For a hidden layer of N = 1000 nodes and input dimension is 200, the maximum number of regions is roughly equal to 20000, In other words, even a middle-sized neural network can partition input space into a huge number of sub-regions, which neural network, we can not expect there is at least one training sample inside each region. For those regions that do have any training sample, the resultant linear functions in them may be arbitrary since they do not contribute to the training objective function at all. Of course, most of these regions are extremely small in size. When we measure the expected loss function over the entire space, their contributions are negligible since the chance for a randomly sampled point to fall into these tiny regions extremely small. However, adversarial attack is imposing a new challenge since adversarial samples are not naturally sampled. Given that the total number of regions is huge, those tiny regions are almost everywhere in the input space. For any data point in the input space, we almost surely can find such tiny region in proximity where the linear function is arbitrary. If point inside this tiny region is selected, the output of the neural network may be unexpected. We believe that these tiny unlearned regions may be major reason why neural networks are vulnerable to adversarial samples.
easily exceed the total number of atoms in the universe. When we learn
In layered deep neural networks, the linear functions in all regions are not totally independent. If we use (!) to denote the weight matrix in layer the resultant linear weight wa in eq. is actually the sum of all concatenated v(!) along all active paths. When we make a small perturbation Ax to any input x, the fluctuation in the output of any hidden node can be approximated represented as:
Af x N. I [EM
[MP]]]
where N denotes the total number of hyperplanes to be crossed when moving x to x + Ax. In any practical neural network, we normally have at least tens of thousands of hyperplanes crossing the hypercube Un = [0, 1]". In other words, for any input x in a nigh-dimensional space, a small perturbation can always easily cross large number of hyperplanes to enter a tiny unlearned region. When is fairly large, the above equation indicates that the output of neural network can still fluctuate dramatically even after all weight vectors are regularized by Ia or La norm. As reference,
4

---

Under review as a conference paper at ICLR 2020
we have verified this on some ImageNet data using a VGG16 model. When PGD is used generate adversarial samples with average perturbation ||Ax|| I 0.35, which is extremely small perturbation since x has over hundred thousand dimensions ImageNet, have observed that in average about N 5278 hyperplanes are crossed per layer even after such small perturbation is added.
At last, since the ubiquitous existence of unlearned tiny regions is intrinsic property of neural networks given its current model structure, we believe that adversarial training strategies will not be sufficient to completely get rid of adversarial samples. In principle, neural networks must be strictly bandlimited to filter out those decaying high frequency components in order to completely eliminate all adversarial samples. We definitely need more research efforts to figure how to do this effectively and efficiently for neural networks.
3 THE PROPOSED DEFENCE APPROACH: POST-AVERAGING
3.1 POST-AVERAGING
In this paper, we propose simple post-processing method to smooth out those high frequency components as much as possible, which relies on simple idea similar to moving- dimensional sequential data. Instead of generating prediction merely from one data point, we use the averaged value within small neighborhood around the data point, which is called post-averaging here. Mathematically, the post-averaging computed as an integral over a small neighborhood centered at the input:
in one-
fc(x) J// / S f(x 16-x'' x') dx'
(5)
where x is the input and f(x) represents the output of the neural network, and C' denotes a small neighborhood centered at the origin and Ve denotes its volume. When choose C' to be an n-sphere in IR" of radius r, we may simply derive the Fourier transform of fc(x) as follows:
[
r + 1) Jz (rlw))
(6) where (.) is the first kind Bessel function of order n/2. Since the Bessel functions, [w),, decay
Fc(w)=F(w)
-ix'-w dx' = F(w)
Vc
(r(w))
with rate 1/ 1/ as |w] + xo (Wtson 1995), we have Fe (r/w)) "TT as |wا - co. Therefore,
F(w)
if r is chosen properly. the post-averaging operation can significantly bandlimit neural networks by smoothing out high frequency components. Note that the similar ideas have been used in (Jiang et al. 1999. Tiang and Lee 2003) improve robustness in speech recognition.
3.2
SAMPLING METHODS
However, it is intractable to compute the above integral for any meaningful neural network used in practical applications. In this work, we propose use simple numerical method approximate For any input x, we select K points in the neighborhood C centered at x, i.e. (x1,X2 ,xx} xx) to approximately compute the integral as
K K
1
fx(x)
f(xx).
Obviously, in order to defend against adversarial samples, it important to have samples outside the current unlearned tiny region. In the following, we use a simple sampling method based on directional vectors. To generate a relatively even set of samples for eq.(7), we first determine some directional vectors v. and then move the input along these directions using several step sizes within the sphere of radius r:
x
where = [ tr]. and V is a selected unit-length directional vector. For each selected direction, generate six samples within C along both the positive and the negative directions to ensure efficiency and even sampling. We use this implementation for the convenience to extend with different types of sampling strategies.
5

---

Under review as a conference paper at ICLR 2020
We tried several direction sampling strategies, including using the directions towards the closest region boundaries, and found that the simple random direction sampling gives the best performance. In this sampling method, we fill the directional vectors with random numbers generated from standard normal distribution, and then normalize them to have unit length.
EXPERIMENTS
In this section, we evaluate the above post-averaging method on defending against several popular adversarial attacking methods.
4.1 EXPERIMENTAL SETUP
Dataset: We evaluated our method on both the ImageNet (Russakovsky al. 2015) and CIFAR-I0 (Krizhevsky et al., 2009) datasets. Since our proposed post-averaging method does not need re-train neural networks, we do not need to use any training data in our experiments.
For evaluation purpose, we use the validation set of the ImageNet dataset. The validation set consists of 50000 images labelled into 1000 categories. For computational efficiency, we randomly choose 5000 images from the ImageNet validation set and evaluate model on these 5000 images.
For the CIFAR-10 dataset, we use the full test set, which consists of 10000 images labelled into 10 categories.
Target model: For model on ImageNet, we use a pre-trained ResNet- 152 (He et al. 2016 network that is available from PyTorch, while for CIFAR-10, 10, we use pre-traines trainer ResNet- In our experiments, we directly use these pre-trained
110 network from Yerlan Idelbayev g models without any modification.
Source of adversarial attacking methods: We Foolbox (Rauber et al. 2017), an open source tool box to generate adversarial samples using different adversarial attacking methods. In this work, tested our method against four popular attacking methods in the literature: Fast Gradient Sign method (FGSM) (Goodfellow et al. 2014). Projected Gradient Descent (PGD) method (Kurakin al. 2016 Madry al. 2017), DeepFool (DF) attack method Mezzovi-Dezlooli et al.) 2016) and Carlini & Wagner (C&W) L2 attack method (Carlini and Wagner 2017a). We used these attack methods in their default settings.
Threat model: In our experiments, we use 400 norm to constrain the allowed perturba- tion distance.
4.2 EVALUATION CRITERTA
For each experiment, define:
Clean set: The dataset that consists of the original images from ImageNet or CIFAR-10.
Attacked set: For every correctly classified image in the Clean set, if adversarial sample is successfully generated under the attacking criteria, the original sample is replaced with the adversarial sample; if no adversarial sample is found, the original sample is kept in the dataset. Meanwhile, all the misclassified images are kept in the dataset without any change. Therefore the Attacked has the same number of images the clean set.
In our experiments, we evaluate the original network and the network defended by post-averaging on both the Clean and the Attacked sets. The performance is measured in terms of:
Accuracy: number of correctly classified images over the whole dataset.
Defence rate: number of successfully defended adversarial samples over the total number of adversarial samples in the Attacked set. By "successfully defended",, it refers to the case where an adversarial sample is correctly classified after the original model defended by the post-averaging approach.
/tttps://githb.. http://www....oo.aaaeaeeeerrreee pretraiind__mcce
far wtt.t///aeteeeer

---

Under review as a conference paper at ICLR 2020
0.98
0.96
Defence
0.94
PGD
DF
0.0
10
15
20
25
Sampling radius
Figure 2: Defence rate of post-averaging when using different sampling radius ImageNet.
Table 1: Performance of post-averaging defending against different attacking methods (c 8/255,
and K 15).
Original Model
Defended by Post- Averaging
Top-I
Accuracy
Top-1
Accuracy Attacked
Defence
attack, defence
Dataset
Clean
Attacked
Clean
Rate
#Adv
FGSM, random(r=30)
0.0750
0.7734
0.7488
0.9426
3500
PGD, random(r=30)
0.0004
0.7732
0.7606
0.9639
3873
ImageNet
0.7750
DE, random(r=30)
0.0350
0.7730
0.7614
0.9639
3710
C&W, random(r=30)
0.0090
0.7734
0.7548
0.9554
3830
FGSM, random(r=6)
0.1816
0.9247
0.8022
0.8080
7552
PGD, random(r=6)
0.0000
0.9255
0.8841
0.9330
9368
CIFAR-10
0.9368
DE, random(==6)
0.1968
0.9254
0.8626
0.8872
7413
C&W, random(1=6)
0.0322
0.9257
0.8902
0.9367
9046
4.3 EXPERIMENTAL RESULTS
Table shows the performance of our defence approach against different attacking methods. this table, the samples for post-averaging are selected within n-sphere of radius in eq.(8), with K different directions. Thus results in a total input) for each input image to be used in eq. (7). Moreover, all the adversarial samples generated are restricted to be within the perturbation range e 8/255. We show the top-1 accuracy of the original model and the defended model both the Clean and the Attacked set respectively, well as the defence rate of the defended model. Besides, we also show the number of adversarial samples successfully generated by each attacking method in the last column.
x 3 + 91 samples (including the
From Table [ we can see that our proposed defence approach is universally robust to all of the attacking methods have examined. It has achieved above 80-96% defence rates in all the experiments with only a minor performance degradation in the Clean set (less than 2%). Especially on the ImageNet dataset, our method is able to defend about 95% of the adversarial samples. However, an interesting observation from the experimental results is that the defence rate in the CIFAR-I0 dataset is lower than the usually more challenging ImageNet dataset, We think this may be because data points are sparser in the ImageNet space than in the CIFAR-10 10 space, as ImageNet has much larger dimensionality.
Generally, using larger sampling radius r can increase the chance of moving out of the unlearned regions as we desired, but it will also introduce more noise that can harm the prediction accuracy; On the other hand, using smaller sampling radius r can reduce the performance degradation but may not be sufficient to defend against adversarial samples. The optimal value for r varies with different datasets due to their dimensionality and data sparsity. In experiments, we found that r = 30 for ImageNet and r = for CIFAR-10 10 achieved relatively better performance. Figure shows how the model defence rate ImageNet varies with different r. As shown in the figure, the optimal value for r also varies in different attacking methods, but the performance variations are small. general, model retains high defence throughout the range (15, 30).
7

---

Under review as a conference paper at ICLR 2020
0.8
0.7
PGD
DF
0.5
28s
255
255
200
255
Epsilon (Linf)
Figure 3: Defence rate post-averaging on different allowed perturbation distances on CIFAR-10.
Table 2: Performance of post-averaging on ImageNet with different number of sampling directions
8/255 and
30).
Original Model
Defended by Post- lostravergg
Averaged
Top-1 Accuracy
Top-1
Accuracy Attacked
Defence
Inference
attack, defence
Clean
Attacked
Clean
Rate
#Adv
Time
PGD, random(K=6)
0,0004
0.7736
0.7606
0.9636
3873
0.18s
PGD, random(K=15)
0.0004
0.7730
0.7604
0.9636
3873
0.40s
0.7750
PGD, random(K=30)
0,0006
0.7732
0.7614
0.9644
3874
0.86s
PGD, random(K=60)
0.0004
0.7734
0.7618
0.9649
3874
1.74s
We also tested the effect of K, the number of sampling directions used, on the model performance. From Table
that model performance is not very sensitive K. It able to achieve good defence rate with only K = 6, that is, 37 samples used for each input image. In implementation. these samples can be easily packed into mini-batch for fast computation in GPUs. When running on the same machine, we measured the averaged inference time for a single input image on the original network 0.04 seconds, while the inference time for our models with different K are shown in Table 2 By comparison, we can know that the inference time after adding post-averaging is roughly K the original inference time.
At last, we evaluated our post-averaging defence approach against attacks with different allowed perturbation ranges c. The results are shown in Figure 3 we can see, our model retains very good attack defence rate up to 32/255. Note that the defence rate against PGD and C&W doesn't 't change much along the variation e, this is because PGD and C&W W have already successfully generated adversarial samples for most of the correctly classified inputs when is small. Hence their generated adversarial samples will not change much when using larger e. For FGSM, our method yields lower defending performance. The possible reason is that FGSM tends generate much larger perturbations than other three stronger attacking methods under the same setting. A large perturbation more likely move samples across class-specific decision boundaries to generate much more confusing samples. In our opinion, this is a general phenomenon in pattern classification, particular adversarial attacks.
5
FINAL REMARKS
In this paper, we have presented some theoretical results by Fourier analysis of ReLU neural networks. These results are useful for us to understand why neural networks are vulnerable to adversarial samples. Based on the results, we hypothesize that the inevitable and ubiquitous existence of tiny unlearned regions in the model function mapping may be major reason for adversarial vulnerability. As possible defence strategy, we have proposed simple post-averaging method. Experimental results on the ImageNet and the CIFAR-10 datasets have demonstrated that our simple defence technique turss out to be very effective against many popular attack methods in the literature. Finally, will interesting to see whether our post-averaging method will be still robust against any new attack methods in the future.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, and Mani B. Srivastava. Genattack: Practical black-box attacks with gradient-free optimization. CoRR, abs/1805. 11090, 2018. URL http:
//arxiv. org. abs/ 11090
Anish Athalye and Nicholas Carlini. On the robustness of the evpr 2018 white-box adversarial example defenses. arXiv preprint arXiv:1804.03288 2018.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. CoRR, abs/ (000/8802,0040,, 2018. URL
http://arxiv/ org/ abs 181.20022000000
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint 1rXiv:1112.04248, 2017.
Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples. arXiv preprint arXiv: 1607:1600.000 00431, 2016.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39-57. IEEE, 2017a.
Nicholas Carlini and David Wagner. Adversarial examples not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pages 3- 14. ACM, 2017b.
Nicholas Carlini and David Wagner. Magnet and "efficient defenses against adversarial attacks" are not robust to adversarial examples. arXiv preprint arXiv: 11Xiv:/71..087777 2017c
Pin- Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsich. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop Artificial Intelligence and Security, AlSec 17, pages 15- 26, New York, NY, USA, 2017. ACM. ISBN 777-1145003-522-- doi: 10. 10.1145/3128572.31444
URL http:
acm. org 2tg/10.000 10. www.1111111122.. 3140448
Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. Detecting adversarial samples from artifacts. CoRR, abs/1703.00410, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 770-778, 2016.
Hui Jiang. A new perspective machine learning: How to do perfect supervised learning. arXiv preprint arXiv:1901.0000 2019.
Hui Jiang and Chin-Hui Lee. A new approach to utterance verification based on neighborhood information in model space. IEEE Transactions on Speech and Audio Processing, 11(5), 2003.
Hui Jiang, Keikichi Hirose, and Qiang Huo. new approach to utterance verification based on neighborhood information in model space. IEEE Transactions Speech and Audio Processing, 7 (4), 1999.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:161 1.0/366, 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.
Towards deep learning models resistant to adversarial attacks. arXiv preprint 1rXiv:1706.066 2017.

---

Under review as a conference paper at ICLR 2020
Seyed- Seyed-Mohsen Moosavi-Dezfooli,, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2574-2582, 2016.
Linh Nguyen and Arunesh Sinha. A learning approach and masking approach to secure learning. CoRR, abs/ abs/1700 709. 2017. URL htttp:////aaxxi http://aaavvav.a.aa/aaoo
1799. 04447
N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami. The limitations of deep learning in adversarial settings. 2016 IEEE European Symposium on Security and Privacy (EuroS P), pages 372-387, March 2016a. doi: N0.1109/EmrSPP2006...0
N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pages 992-597, May 2016b. doi: 10.1 10//S/
Nicolas Papernot, Patrick McDanicl, and Ian Goodfellow. Transferability in machine learning: from phenomena to bla attacks using adversarial samples. CoRR, abs/ 0605007277, 2016. URL http:://txiiiiiie org/ /100..//1666..00 07277
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, 7. Berkay Celik, and Ananthram
Swami. Practical black-box attacks against machine learning. In Proceedings the 2017 ACM on Asia Conference on Computer and Communications Security, ASIA CCS 17, pages 506-519, New York, NY, USA, 2017. ACM. ISBN 978- 1 4503-4494444 doi: 10. 10.114513552973383300009 URL http:httt//ooo.. acm. wwtp:/oog...../.....111 (tt..11115/352933... 3053009
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:700.04131/ 2017. URL
http://arxiv: org/ abs 11tpp//7770003333
Andras Rozsa, Ethan M. Rudd, and Terrance E. Boult. Adversarial diversity and hard positive generation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Workshops. June 2016.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 13)2211-252, 2015. doi: 10.1007/s1 12630/11-112--001111811818
F.M. Stein and G. Weiss. Introduction to Fourier Analysis on Euclidean Spaces. Mathematical Series. Princeton University Press, 1971. ISBN
URL
googlo
ca/books ?1 2id didd----u YUCV67 YUCV678MNAIC 8MNAIC
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties neural networks. arXiv preprint 131.v:1312.6199 2013.
Florian Tramèr, Alexey Kurakin, Nicolas Paperno,, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07200 2017.
George Neville Watson. A treatise the theory of functions,, Cambridge university press, 1995.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan L. Yuille. Mitigating adversarial effects through randomization. CoRR, abs/1711.01991, 2017. URL abs/1711.0199)
10

---

Under review as a conference paper at ICLR 2020
APPENDIX: MATHEMATICAL PROOFS
Definition B.1. A piece-wise linear function is continuous function f R - R such that there are some hyperplanes passing through origin and dividing R" into M pairwise disjoint regions Rms. (m 1, .... M), each of which is linear:
W x Ri W2 X
X E R2
f(x)
WM Ras
Lemma B.2. Composition of a piece-wise linear function with a ReLU activation uunitatiionunction is also piece-wise linear function.
Proof. Let r() denote the ReLU activation function. If fx) on region Rm takes both positive and negative values, rx)()) will break it into two regions RD and Rd. On the former r (f(x)) f(x) and on the latter (x)) O, which both are linear functions. As f(x) on R, is linear, common boundary of 2 and R2 lies inside hyperplane passing through origin which is the kernel of the linear function. Therefore, if f(x) is picce-wise linear function defined by & hyperplanes resulting in M regions, r (x)) will be a piece-wise linear function defined by at most k+ m hyperplanes. L
Theorem B.3. The output of any hidden unit in an unbiased fully- ReLU neural network is a piece-wise linear function.
Proof. This proposition immediately follows lemma B.2
Definition B.4. Let V (V1, V2.-..; Vn} be of independent vectors in R An infinite simplex, Ry, defined as the region linearly spanned by V using only positive weights:
VAVE VK aq > 0}
Theorem B.5. Each pisee-wise linear function f(x) can be formulated as a summation of some
functions: f(x) Σk fK(x), each of which is linear and non-zero only in infinite simplex
follows:
fa(x) =
WE
X KERY
otherwise
where V& is a set of independent vectors, and Wj is a weight vector.
Proof. Each region Rd of piece- wise linear function, (x), which describes the behavior of ReLU node intersects with an affine hyper-plane results in a convex polytope. This convex polytope can be triangulated into some simplices. Define Vx, (k: 1, 2.... K). sets of vertexes of these simplices. The infinite simplexes created by these vector sets will have the desired property and f(x) can be written as: f(x) Ek= (x).
As explained earlier in the original article by adding n² hyper-planes those defining the piece-wise linear function, the output of a ReLU node may represented as f(x) Σ (x). These hyper-planes those perpendicular to standard basis vectors and subtraction of one of these vectors from another one. That is, (i = ..... n. and e; e, (1 < i < s n). Given this representation, the final step to achieve the Fourier transform is the following lemma:
Lemma B.6. Fourier transform of the following function:
s(x) = h(1 1.x)
x E Rt otherwise
11

---

Under review as a conference paper at ICLR 2020
may be presented as:
S(w)
(10)
2m
(40p) Wi)
r=0
r'zr
where wp is the rth component of frequency vector (r (r=11 1,,-. nt), and wo O.
Proof. Alternatively, s(x) may be represented as:
s sxx = h(1 x)h(1 - 1 x) II
hhzz))h(1- xz,)
(11)
j=1
Therefore, we need to compute Fourier transform of h(x)h(1 z):
1
e lzw h(x)(1 - x)dx 1
e
-iw dæ
(12)
2x
/2T 0
-00
-i
1
e e-
(13)
2TT w
By taking the inverse Fourier transform of the function:
- 1- ( - (1) dc
(V2m)
(14)
2 C
where 8n n-dimensional Dirac Delta function, can be shown that is the Fourier transform of 1(1 x) 1. x):
1
-
1 E-c
12
j.xx x (v
2πT
n
5n(w - (1) dc dw
(15)
22m
2
5
Rm
00
1
N00
li
giw.x
((1 6n(w-(1)
6n(w
dc dw (16)
2T
B
--x0
V2TT
C
1
- E
iis
... /
(iwxx mw (1) dw de
(17)
2T
2T
C
R"
1
-i 1 iic
e ds
(18)
2 -0 22T S
h(1 xx)((11 - 1 1xx x)
(19)
Now we can find the Fourier transform of s(x)
i
000 -i l-ck
S(w)
[II
(V2T) n-1
6p(w -(1 d
(20)
2TT
Wr
J-x 22 5
r=1
1 i(w,-()
i(
n+2
ic
dc
(21)
V2T
Wp
1=0
where is convolution operator. The final integrand may be represented as:
i(w--C) ()
ic
i(w, i(w,-()
(22)
Wp
C
Wp
T=0
T=0
r=0
A
e i0
i(u, (3)
(23)
Wp
r=0
r=0
11
A,
i0
(-1) BB| (B(1)
(24)
Wp - C
BCQ
Σ
(-1)
Bl 100 ((B)-1))
(25)
W,
1=0
BCQ
12

---

Under review as a conference paper at ICLR 2020
where 92 fwb. ..., ) TB is the summation over elements of B and A, II
1
Therefore:
4, #r
i(w,-()
dc
(26)
Wr C
0
10
A
(-1)1 11 ilan (181-1)4) dc
(27)
Wy
pm 0
BCS
(-1)1 ioo (0aon-(18111)))) dç
(28)
Wp 5
BCS
dc
(29)
BCS
=
Σ A Σ (-1) r=0
Blin it sign( [B|
1)e i(r (1
(30)
BCS
If B does not contain w,, and have at least 2 elements then the terms for B and B U {wp} will cancel
each other out. Also, sign( B 1) will vanish if B has only one element. Therefore, there only remains empty set and sets with two elements one of them being wr.. Given the fact that Σ A, = 0. the result of the integral will be:
5e 16
00
1/ww- -)
dg = in Σ
Ar
iwr + Σ -www))
(31)
Wr
r=0
r'
-ivi
A
(32)
r=0
Finally, substituting 32 into 21 yields the desired result.
Theorem B.7. The Fourier transform of the output of any hidden node in a conleccee ReLU neural network may be represented as IQ WgA (VS(AA)))) where denote the differential operator.
Proof. As discussed in the original paper, f(x) E 9q (x) where:
(x(x) = {
Wg X, h(1 - 1 xy)
Xg E Ri.
(33)
otherwise
or equivalently:
05(x) Wg xxx)
(34)
Therefore:
Q F(w)= ()(w)
(35)
Q Wg. (S(Wg))
(36)
where W. = wA-1 -1
DERIVATION OF FQ. 6
As for the Fourier transform computed in section 3.1 it should be mentioned that the integral in equation 6 is the Fourier transform of:
(x) h((r (x])
(37)
13

---

Under review as a conference paper at ICLR 2020
which can be derived utilizing the property of the Fourier transforms for radially symmetric func- tions (Stein and Weiss 1971):
|wl - - / J Juy (wWpp) - h(r p)p
(+((w
(38)
J. الحي p)p dp
|w
(39)
r
J& (r|w])
(40)
|w
Given this transform:
Vc S "Jee --x-'w dx
Fo(w)= = F(w)
(41)
= =F(w) 1 + 1) Jy (r(w))
(42)
π (rlw)) 2
14


Reviews:
[{"review_id": "rJxkmnwOKS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper makes three contributions: 1) analyzing the number of linear regions and the Fourier spectrum of fully-connected neural networks with ReLU activations; 2) proposing an ensemble method as a defense against adversarial evasion attacks; 3) analyzing the performance of that defense against state-of-the-art attacks on CIFAR-10 and ImageNet data.\n\nMy main concern is regarding the originality and significance of this work. Most of the results regarding the piece-wise linear geometry of fully-connected neural networks with ReLU activations are known. The authors should reference and discuss the relation of their work in particular to [1]. The Fourier transformation results look straightforward to me. I am not convinced either that they are indeed \"essential\", as the authors write, for analyzing the nature of adversarial samples. In fact, I find the discussion in Section 2.2 rather vague; for instance, what is the formal notion of \"tiny unlearned regions\" of neural networks? The authors mention an experiment in which they observed that an adversarial perturbation on ImageNet led to the crossing of N=5278 hyperplanes \"per layer\", which sounds like an interesting finding, however, more detail is needed to understand how it was obtained.\n\nThe proposed defense is not very original either; the authors should discuss it in comparison e.g. to [2] and [3] who also considered ensemble-based methods to defend against adversarial perturbations. The experimental results in Table 1 & 2 look promising at a first glance, however, [4] has demonstrated that ensemble-based, randomized defenses can be fairly easily defeated (e.g. using expectations over transformations when computing the gradients for adversarial sample generation). The authors need to report the performance of their defense under such white-box settings.\n\nThe strong performance of the FGSM attack in Figure 3 is surprising. PGD is strictly stronger than FGSM, so its defense rate shouldn't be higher than the one for FGSM. I disagree with the authors' explanation that FGSM \"generates much larger perturbations\"; if the epsilon parameter is set properly in the PGD attack, it should exhaust that perturbation budget (assuming that the step size and number of iterations are large enough). On the other hand, I don't understand how the authors were able to constrain the L-inf norm of the C&W and DeepFool adversarial samples (since those attacks aim to minimize L2 norms).\n\n\n[1] Montufar et al., On the Number of Linear Regions of Deep Neural Networks. NeurIPS, 2014.\n[2] Liu et al., Towards robust neural networks via random self-ensemble. ECCV, 2018.\n[3] Cao & Gong, Mitigating evasion attacks to deep neural networks via region-based classification. ACSAC, 2017.\n[4] Athalye et al., Obfuscated gradients give a false sense of security. ICML, 2018.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SyxSBsG15r", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overview:\nThe paper is dedicated to studying adversarial attack and defense problems from the perspective of Fourier analysis. They demonstrate that the adversarial vulnerability of neural networks can be attributed to non-zero high-frequency components. Then, the author proposes a simple post average approach to smooth out the insignificant high-frequency components, which can improve the adversarial robustness of neural networks. They conduct extensive experiments on ImageNet and CIFAR-10 to defend existing attacks, including FGSM, PGD, DeepFool, and C&W attacks.\n\nStrength Bullets:\n1. The logic chain of this paper is complete. The author first gives an interesting observation that the non-zero high-frequency components may cause the vulnerability of neural networks. Then they propose a post average technique to reduce the high-frequency components and improve robustness.\n2. The paper is well organized. Some figures, like fig1, are intuitive and interesting.\n\nWeakness Bullets:\n1. How can we get formula (4)? The author needs to provide more explanation.\n2. It is confused about how to pick directional vectors v for input x. Does the author treat the input RGB image (i.e. 3 times 32 times 32) as a one-dimensional vector? Then the author chooses k other vectors in the neighborhood. There need more details and rational explanations. The author can use the visualization method (i.e. tSNE) the illustrate the relationship among samples.\n3. For different sampling strategies, the author needs to provide an ablation study.\n4. The most arguable point is the experiment setting. I think it is a weak white-box experiment setting. The author uses the attacked set which is generated from the undefended model. It means the author doesn't attack his post-processing method. Thus, the improved robustness can be the result of gradient masking introduced by the author's techniques. BTW, it is totally traceable to attack the paper's approach. We can formula equation (7) and equation (8) as two layers in the front and at the end of networks. Then you can calculate the gradient for a complete model setting. The author needs to provides more convincing results.\n5. The author needs to compare with other post- or pre- processing methods, even the normal adversarial training. Also, the paper should contain detailed settings about attacks. For example, how many iterations do you run for PGD attacks? This is an important factor for trustworthy results.\n6. [Minor] Quotation mask mistake in the second row of page 2.\n\nRecommendation:\nDue to the limits of experiments design and setting, this is a weak reject.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "Syg-EVrJ5r", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an approach for improving robustness of already trained artificial neural networks with relu activation functions. The main motivation comes from signal processing where robustness is typically obtained via averaging moduli of Fourier coefficients over some frequency band (e.g., mel-frequency coefficients and deep scattering spectrum are based on this principle). The strategy amounts to sampling several random direction vectors in a ball of constant radius centered at a training example and averaging their predictions. The empirical estimate of the expected predictor value over the ball centered at a training example is used as its hypothesis value.\n\nThe approach is introduced by first expressing an artificial neural network with relu activations as a piecewise linear function (Definition 2.1, Lemma 2.2, Theorem 2.3). The paper then makes an observation that the instance space can be further sub-divided such that the network can be written as a sum of piecewise functions defined over regions given by positive linear combinations of linearly independent vectors (Definition 2.4, Theorem 2.5). A linear transformation of the instance space then allows for writing that function in canonical basis and computing its Fourier transform (Lemma 2.6, Theorem 2.7). The paper then makes an observation that the inverse of the linear transform used for making a change of basis (mapping to the canonical basis) can introduce instabilities to piecewise linear components defined over small regions (the inverse matrix appears in the Fourier transform). To address this instability, the paper relies on prior work by Jiang et al. (1999, 2003) and assigns the expected value over some ball centered at a training example as its prediction (which should be equivalent to performing averaging over bands in the Fourier domain). The integral is analytically intractable and, thus, the authors do an empirical estimate by averaging values in different random directions around the example. The experiments show that the method does not exhibit any serious instability with respect to the number points selected in that way.\n\nThe strategy does not require any additional training of the network and can be easily applied to already trained models with relu activations.\n\nIn the experiments, the approach is evaluated on standard adversarial attacking mechanisms: fast gradient sign method (Goodfellow et al., 2014), projected gradient descent method (Kurakin et al., 2016; Madry et al., 2017), deep fool attack method (Moosavi-Dezfooli et al., 2016) and L_2 method (Carlini & Wagner, 2017). I am not very familiar with the related work but this seems to be sufficient number of baselines to assess the effectiveness of the approach. The experiments are performed on ImageNet and Cifar10 datasets and show that the approach can make standard ResNet models more robust to the listed attacking strategies. It might be useful to test the approach with several different architectures (e.g., multi-layer perceptrons with different number of hidden layers, mix of convolutional and fully connected blocks etc).\n\nIn summary, the paper is well written and easy follow. The idea itself is simple but the intuition behind it is rather interesting and (to the best of my knowledge) provides novel insights into the workings of artificial neural networks with relu activations.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers recommend rejection due to various concerns about novelty and experimental validation. The authors have not provided a response.
Decision: Reject