Original:  P EER N ETS : E XPLOITING P EER W ISDOM A GAINST A DVERSARIAL A TTACKS
Revised: c P EER N ETS : E XPLOITING P EER W ISDOM A GAINST A DVERSARIAL A TTACKS

Original: A BSTRACT
Revised: A BSTRACT

Original: Deep learning systems have become ubiquitous in many aspects of our lives.
Revised: Deep learning systems have become ubiquitous in many aspects of our lives.

Original: Unfortunately, it has been shown that such systems are vulnerable to adversarial attacks, making them prone to potential unlawful uses.
Revised: Unfortunately, it has been shown that such systems are vulnerable to adversarial attacks, making them prone to potential unlawful uses.

Original: Designing deep neural networks that are robust to adversarial attacks is a fundamental step in making such systems safer and deployable in a broader variety of applications (e.g. autonomous driving), but more importantly is a necessary step to design novel and more advanced architectures built on new computational paradigms rather than marginally building on the existing ones.
Revised: Designing deep neural networks that are robust to adversarial attacks is a fundamental step in making such systems safer and deployable in a broader variety of applications (e.g. autonomous driving), but more importantly is a necessary step to design novel and more advanced architectures built on new computational paradigms rather than marginally building on the existing ones.

Original: In this paper we introduce PeerNets , a novel family of convolutional networks alternating classical Euclidean convolutions with graph convolutions to harness information from a graph of peer samples.
Revised: In this paper we introduce PeerNets , a novel family of convolutional networks alternating classical Euclidean convolutions with graph convolutions to harness information from a graph of peer samples.

Original: This results in a form of non-local forward propagation in the model, where latent features are conditioned on the global structure induced by the graph, that is up to 3 × more robust to a variety of white- and black-box adversarial attacks compared to conventional architectures with almost no drop in accuracy.
Revised: This results in a form of non-local forward propagation in the model, where latent features are conditioned on the global structure induced by the graph, that is up to 3 × more robust to a variety of white- and black-box adversarial attacks compared to conventional architectures with almost no drop in accuracy.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep convolutional networks (CNN) are the de-facto standard in almost any computer vision application, ranging from image recognition (Krizhevsky et al., 2012;
Revised: Deep convolutional networks (CNN) are the de-facto standard in almost any computer vision application, ranging from image recognition (Krizhevsky et al., 2012;

Original: He et al., 2015), object detection (Yuan et al., 2017; Redmon et al., 2016; Ren et al., 2015), semantic segmentation (He et al., 2017; Badrinarayanan et al., 2017) and motion estimation (Liu et al., 2017; Vijayanarasimhan et al., 2017; Niklaus et al., 2017).
Revised: He et al., 2015), object detection (Yuan et al., 2017; Redmon et al., 2016; Ren et al., 2015), semantic segmentation (He et al., 2017; Badrinarayanan et al., 2017) and motion estimation (Liu et al., 2017; Vijayanarasimhan et al., 2017; Niklaus et al., 2017).

Original: The ground breaking results of deep learning for industry-relevant problems have made them also an important ingredient in many real world systems for applications such as autonomous driving and user authentication.
Revised: The ground breaking results of deep learning for industry-relevant problems have made them also an important ingredient in many real world systems for applications such as autonomous driving and user authentication.

Original: Unfortunately, it has been shown (Szegedy et al., 2013) that such systems are vulnerable to easy to fabricate adversarial examples opening potential ways for their untraceable and unlawful exploitation.
Revised: Unfortunately, it has been shown (Szegedy et al., 2013) that such systems are vulnerable to easy to fabricate adversarial examples opening potential ways for their untraceable and unlawful exploitation.

Original: Adversarial attacks.
Revised: Adversarial attacks.

Original: Szegedy et al.
Revised: Szegedy et al.

Original: (2013) found that deep neural networks employed in computer vision tasks tend to learn very discontinuous input-output mappings, and can be caused to misclassify an image by applying an almost imperceptible ‘adversarial’ perturbation, which is found by maximizing the network’s prediction error.
Revised: (2013) found that deep neural networks employed in computer vision tasks tend to learn very discontinuous input-output mappings, and can be caused to misclassify an image by applying an almost imperceptible ‘adversarial’ perturbation, which is found by maximizing the network’s prediction error.

Original: While it was expected that addition of noise to the input can ruin the classiﬁcation accuracy of a neural network, only tiny adversarial perturbations are needed as a great surprise to the machine learning community.
Revised: While it was expected that addition of noise to the input can ruin the classiﬁcation accuracy of a neural network, only tiny adversarial perturbations are needed as a great surprise to the machine learning community.

Original: Multiple methods for designing adversarial perturbations have been proposed, perhaps the most dramatic being single pixel perturbation by
Revised: Multiple methods for designing adversarial perturbations have been proposed, perhaps the most dramatic being single pixel perturbation by

Original: Su et al.
Revised: Su et al.

Original: Because of its potentially dramatic implications on the security of deep learning technology (which are employed for example, in face recognition systems Sharif et al.
Revised: Because of its potentially dramatic implications on the security of deep learning technology (which are employed for example, in face recognition systems Sharif et al.

Original: (2016) or autonomously driving cars), adversarial attacks and defenses against them have become a very active topic of research (Rauber et al., 2017; Kurakin et al., 2018).
Revised: (2016) or autonomously driving cars), adversarial attacks and defenses against them have become a very active topic of research (Rauber et al., 2017; Kurakin et al., 2018).

Original: Adversarial attacks can be categorized as targeted and non-targeted .
Revised: Adversarial attacks can be categorized as targeted and non-targeted .

Original: The former aim at changing a source data sample in a way to make the network classify it as some pre-speciﬁed target class (imagine as an illustration a villain that would like a computer vision system in a car to classify all trafﬁc signs as STOP signs).
Revised: The former aim at changing a source data sample in a way to make the network classify it as some pre-speciﬁed target class (imagine as an illustration a villain that would like a computer vision system in a car to classify all trafﬁc signs as STOP signs).

Original: The latter type of attacks, on the other hand, aims simply at making the classiﬁer predict some different class.
Revised: The latter type of attacks, on the other hand, aims simply at making the classiﬁer predict some different class.

Original: Furthermore, we can also distinguish between the ways in which the attacks are generated: white box attacks assume the full knowledge of the model and of all its parameters and gradient, while in black box attacks, the adversary can observe only the output of a given generated sample and has no access to the model.
Revised: Furthermore, we can also distinguish between the ways in which the attacks are generated: white box attacks assume the full knowledge of the model and of all its parameters and gradient, while in [Table] attacks, the adversary can observe only the output of a given generated sample and has no access to the model.

Original: It has been recently found thatfor a given model and a given dataset there exists a universal adversarial perturbation by Moosavi-Dezfooli et al.
Revised: It has been recently found that

Original: (2016) that is network-speciﬁc but input-agnostic, i e., a single noise image that can be applied to any input of the network. Such perturbations were furthermore shown to be insensitive to geometric transformations. Defense against adversarial attacks.
Revised: [Table]

Original: There has been signiﬁcant recent effort trying to understand the theoretical nature (Goodfellow et al., 2015; Fawzi et al., 2018) and strategies against adversarial perturbations. The search for solution has focused on the network architecture, training procedure, and data pre-processing (Gu & Rigazio, 2014).
Revised: and a given

Original: Intuitively, one could argue that robustness to adversarial noise could be achieved by adding adversarial examples during training. However, recent work showed that this brute-force approach does not work in this case and that it forces the network to converge to a bad local minimum (Tram`er et al., 2018). As an alternative, Tram`er et al. (2017) introduced ensemble adversarial training augmenting training data with perturbations transferred from other models.
Revised: [Table]

Original: Papernot & McDaniel (2016) proposed defensive distillation training. Mahdizadehaghdam et al.
Revised: there exists a universal adversarial perturbation by Moosavi-Dezfooli et al [Table]

Original: (2018) showed that deep dictionary learning architecture provides greater robustness against adversarial noise. Lately, Athalye et al. (2018) have pointed to the problem of so called gradient obfuscation , which has to be carefully treated during design of a new defense method. They have proven it to be the main reason why many state-of-the-art methods work, which further emphasizes the fact that defense against adversarial attacks are still an open question.
Revised: is

Original: Main contributions.
Revised: [Table]

Original: In this paper, we introduce Peer-Regularized Networks (PeerNet), a new family of deep models that uses a graph of samples to perform non-local forward propagation . The output of the network depends not only by the given test sample at hand, but also by its interaction with several training samples.
Revised: [Table]

Original: We experimentally show that such a novel paradigm leads to substantially higher robustness (up to 3 × ) to adversarial attacks at the cost of a minor classiﬁcation accuracy drop when using the same architecture.
Revised: (up to 3 × ) to adversarial attacks at the cost of a minor classiﬁcation accuracy drop when using the same architecture.

Original: We also show that the proposed non-local propagation acts as a strong regularizer that makes possible to increase the model capacity to match the current state-of-the-art without incurring in overﬁtting.
Revised: We also show that the proposed non-local propagation acts as a strong regularizer that makes possible to increase the model capacity to match the current state-of-the-art without incurring in overﬁtting.

Original: We provide experimental validation on established benchmarks such as MNIST, CIFAR10 and CIFAR100 using various types of adversarial attacks.
Revised: We provide experimental validation on established benchmarks such as MNIST, CIFAR10 and CIFAR100 using various types of adversarial attacks.

Original:  R ELATED WORKS
Revised:  R ELATED WORKS

Original: Our approach is related to three classes of approaches: deep learning on graphs, deep learning models combined with manifold regularization, and non-local ﬁltering.
Revised: Our approach is related to three classes of approaches: deep learning on graphs, deep learning models combined with manifold regularization, and non-local ﬁltering.

Original: In this section, we review the related literature and provide the necessary background.
Revised: In this section, we review the related literature and provide the necessary background.

Original: 2. D EEP L EARNING ON G RAPHS
Revised: 2. D EEP L EARNING ON G RAPHS

Original: In the recent years, there has been a surge of interest in generalizing successful deep learning models to non-Euclidean structured data such as graphs and manifolds, a ﬁeld referred to as geometric deep learning in Bronstein et al.
Revised: In the recent years, there has been a surge of interest in generalizing successful deep learning models to non-Euclidean structured data such as graphs and manifolds, a ﬁeld referred to as geometric deep learning in Bronstein et al.

Original: First attempts of learning on graphs date back to the works of
Revised: First attempts of learning on graphs date back to the works of

Original: Gori et al.
Revised: Gori et al.

Original: ; Scarselli et al.
Revised: ; Scarselli et al.

Original: (2009), where the authors considered steady state of learnable diffusion process (more recent works Li et al.
Revised: (2009), where the authors considered steady state of learnable diffusion process (more recent works Li et al.

Original: ; Gilmer et al.
Revised: ; Gilmer et al.

Original: (2017) improved this approach using modern deep learning schemes).
Revised: (2017) improved this approach using modern deep learning schemes).

Original: Spectral domain graph CNNs.
Revised: Spectral domain graph CNNs.

Original: Bruna et al.
Revised: Bruna et al.

Original: ; Henaff et al.
Revised: ; Henaff et al.

Original: (2015) proposed formulating convolution-like operations in the spectral domain, deﬁned by the eigenvectors of the graph Laplacian.
Revised: (2015) proposed formulating convolution-like operations in the spectral domain, deﬁned by the eigenvectors of the graph Laplacian.

Original: Among the drawbacks of this architecture is O ( n 2 ) computational complexity due to the cost of computing the forward and inverse graph Fourier transform, O ( n ) parameters per layer, and no guarantee of spatial localization of the ﬁlters.
Revised: Among the drawbacks of this architecture is O ( n 2 ) computational complexity due to the cost of computing the forward and inverse graph Fourier transform, O ( n ) parameters per layer, and no guarantee of spatial localization of the ﬁlters.

Original: A more efﬁcient class of spectral graph CNNs was introduced in Defferrard et al.
Revised: A more efﬁcient class of spectral graph CNNs was introduced in Defferrard et al.

Original: ; Kipf &
Revised: ; Kipf &

Original: Welling (2016) and follow up works, who proposed spectral ﬁlters that can be expressed in terms of simple operations (such as additions, scalar- and matrix multiplications) w.r.t.
Revised: Welling (2016) and follow up works, who proposed spectral ﬁlters that can be expressed in terms of simple operations (such as additions, scalar- and matrix multiplications) w.r.t.

Original: the Laplacian.
Revised: the Laplacian.

Original: In particular, Defferrard et al.
Revised: In particular, Defferrard et al.

Original: (2016) considered polynomial ﬁlters of degree p , which incur only p times multiplication by the Laplacian matrix (which costs O ( |E| ) in general or O ( n ) if the graph is sparsely connected), also guaranteeing ﬁlters that are supported in p -hop neighborhoods.
Revised: (2016) considered polynomial ﬁlters of degree p , which incur only p times multiplication by the Laplacian matrix (which costs O ( |E| ) in general or O ( n ) if the graph is sparsely connected), also guaranteeing ﬁlters that are supported in p -hop neighborhoods.

Original: Levie  which were carried 
Revised: Levie et al. (2017) proposed rational ﬁlter functions including additional inversions of the Laplacian, which were carried our approximately using an iterative method. Monti et al. (2018) used multivariate polynomials w.r.t.

Revised: multiple Laplacians deﬁned by graph motifs; Monti et al.

Revised: (2017b) used Laplacians deﬁned on products of graphs in the context of matrix completion problems.

Original: Spatial domain graph CNNs.
Revised: Spatial domain graph CNNs.

Original: On the other hand, spatial formulations of graph CNNs operate on local neighborhoods on the graph (Duvenaud et al., 2015; Monti et al., 2017a; Atwood & Towsley, 2016; Hamilton et al., 2017; Veliˇckovi´c et al., 2017; Wang et al., 2018).
Revised: On the other hand, spatial formulations of graph CNNs operate on local neighborhoods on the graph (Duvenaud et al., 2015; Monti et al., 2017a; Atwood & Towsley, 2016; Hamilton et al., 2017; Veli ˇ ckovi ´ c et al., 2017; Wang et al., 2018).

Original: Monti et al.
Revised: Monti et al.

Original: (2017a) proposed the Mixture Model networks (MoNet), generalizing the notion of image ‘patches’ to graphs.
Revised: (2017a) proposed the Mixture Model networks (MoNet), generalizing the notion of image ‘patches’ to graphs.

Original: The centerpiece of this construction is a system of local pseudo-coordinates u ij ∈ R d assigned to a neighbor j of each vertex i .
Revised: The centerpiece of this construction is a system of local pseudo-coordinates u ij ∈ R d assigned to a neighbor j of each vertex i .

Original: The spatial analogue of a convolution is then deﬁned as a Gaussian mixture in these coordinates.
Revised: The spatial analogue of a convolution is then deﬁned as a Gaussian mixture in these coordinates.

Original: Veliˇckovi´c et al.
Revised: Veliˇckovi´c et al.

Original: (2017) reinterpreted this scheme as graph attention (GAT), learning the relevance of neighbor vertices for the ﬁlter result,
Revised: (2017) reinterpreted this scheme as graph attention (GAT), learning the relevance of neighbor vertices for the ﬁlter result,

Original: [Equation]
Revised: [Equation]

Original: where α ij are attention scores representing the importance of vertex j w.r.t.
Revised: where α ij are attention scores representing the importance of vertex j w.r.t.

Original: i , and the p × q (cid:48) matrix
Revised: i , and the p × q (cid:48) matrix

Original: A and 2 p -dimensional vector b are the learnable parameters.
Revised: A and 2 p -dimensional vector b are the learnable parameters.

Original: 2. L OW -D IMENSIONAL R EGULARIZATION
Revised: 2. L OW -D IMENSIONAL R EGULARIZATION

Original: There have recently been several attempts to marry deep learning with classical manifold learning methods (Belkin & Niyogi, 2003; Coifman & Lafon, 2006) based on the assumption of local regularity of the data, that can be modeled as a low-dimensional manifold in the data space.
Revised: There have recently been several attempts to marry deep learning with classical manifold learning methods (Belkin & Niyogi, 2003; Coifman & Lafon, 2006) based on the assumption of local regularity of the data, that can be modeled as a low-dimensional manifold in the data space.

Original: Zhu et al.
Revised: Zhu et al.

Original: (2017) proposed a feature regularization method that makes input and output features to lie on a low dimensional manifold.
Revised: (2017) proposed a feature regularization method that makes input and output features to lie on a low dimensional manifold.

Original: Garcia & Bruna (2017) cast few shot learning as supervised message passing task which is trained end-to-end using graph neural networks.
Revised: Garcia & Bruna (2017) cast few shot learning as supervised message passing task which is trained end-to-end using graph neural networks.

Original: One of the key disadvantages of such approaches is the difﬁculty to factor out global transformations (such as translations) to which the network output should be invariant, but which ruin the local similarity.
Revised: One of the key disadvantages of such approaches is the difﬁculty to factor out global transformations (such as translations) to which the network output should be invariant, but which ruin the local similarity.

Original: 2. N ON -L OCAL I MAGE F ILTERING
Revised: 2. N ON -L OCAL I MAGE F ILTERING

Original: The last class of methods related to our approach are non-local ﬁlters
Revised: The last class of methods related to our approach are non-local ﬁlters

Original: Sochen et al.
Revised: Sochen et al.

Original: ; Tomasi &
Revised: ; Tomasi &

Original: Manduchi (1998); Buades et al.
Revised: Manduchi (1998); Buades et al.

Original: (2005) that gained popularity in the image processing community about two decades ago.
Revised: (2005) that gained popularity in the image processing community about two decades ago.

Original: Such non-shift-invariant ﬁlters produce, for every location of the image, a result that depends not only on the pixel intensities around the point, but also on a set of neighboring pixels and their relative locations.
Revised: Such non-shift-invariant ﬁlters produce, for every location of the image, a result that depends not only on the pixel intensities around the point, but also on a set of neighboring pixels and their relative locations.

Original: For example, the bilateral ﬁlter uses radiometric differences together with Euclidean distance of pixels to determine the averaging weights.
Revised: For example, the bilateral ﬁlter uses radiometric differences together with Euclidean distance of pixels to determine the averaging weights.

Original: The same paradigm has brought into Non-local networks by Wang et al.
Revised: The same paradigm has brought into Non-local networks by Wang et al.

Original: (2017), where the response at a given position is deﬁned as a weighted sum of all the features across different spatio-temporal locations, whose interpolation coefﬁcients are inferred by a parametric model, usually another neural network.
Revised: (2017), where the response at a given position is deﬁned as a weighted sum of all the features across different spatio-temporal locations, whose interpolation coefﬁcients are inferred by a parametric model, usually another neural network.

Original:  P EER R EGULARIZATION
Revised:  P EER R EGULARIZATION

Original: In this paper, we propose a new deep neural network architecture that takes advantage of the data space structure.
Revised: In this paper, we propose a new deep neural network architecture that takes advantage of the data space structure.

Original: The centerpiece of our model is the Peer Regularization (PR) layer, designed as follows.
Revised: The centerpiece of our model is the Peer Regularization (PR) layer, designed as follows.

Original: Let X 1 , . . .
Revised: Let X 1 , . . .

Original: X N be n × d matrices representing the feature maps of N images, to which we refer as peers (here n denotes the number of pixels and d is the dimension of the feature in each pixel).
Revised: X N be n × d matrices representing the feature maps of N images, to which we refer as peers (here n denotes the number of pixels and d is the dimension of the feature in each pixel).

Original: Given a pixel of image i , we consider its K nearest neighbor graph in the space of d -dimensional feature maps of all pixels of all the peer images, where the neighbors are computed using e.g. the cosine distance.
Revised: Given a pixel of image i , we consider its K nearest neighbor graph in the space of d -dimensional feature maps of all pixels of all the peer images, where the neighbors are computed using e.g. the cosine distance.

Original: The k th nearest neighbor of the p th pixel 
Revised: The k th nearest neighbor of the p th pixel x ip taken from image i is the q k th pixel x j k q k taken from peer image j k , with k = 1 , . . . , K and j k ∈ { 1 , . . . , N } , q k ∈ { 1 , . . . , n } .

Original: We apply a variant of graph attention network (GAT) by Veliˇckovi´c et al.
Revised: We apply a variant of graph attention network (GAT) by Veli ˇ ckovi ´ c et al.

Original: (2017) to the nearestneighbor graph constructed this way,
Revised: (2017) to the nearestneighbor graph constructed this way,

Original: [Equation]
Revised: [Equation]

Original: where a () denotes a fully connected layer mapping from 2 d -dimensional input to scalar output, and α ij k pq k are attention scores determining the importance of contribution of the q k th pixel of image j to the output p th pixel ˜ x ip of image i .
Revised: where a () denotes a fully connected layer mapping from 2 d -dimensional input to scalar output, and α ij k pq k are attention scores determining the importance of contribution of the q k th pixel of image j to the output p th pixel ˜ x ip of image i .

Original: This way, the output feature map ˜ X
Revised: This way, the output feature map ˜ X

Original: i is pixel-wise weighted aggregate of the peers.
Revised: i is pixel-wise weighted aggregate of the peers.

Original: Peer Regularization is reminiscent of non-local means denoising of Buades et al.
Revised: Peer Regularization is reminiscent of non-local means denoising of Buades et al.

Original: (2005), with the important difference that the neighbors are taken from multiple images rather than from the same image,
Revised: (2005), with the important difference that the neighbors are taken from multiple images rather than from the same image,

Original: and the combination weights are learnable.
Revised: and the combination weights are learnable.

Original: Full graph approximation.
Revised: Full graph approximation.

Original: In principle, one would use a graph built out of a nearest neighbor search across all the available training samples.
Revised: In principle, one would use a graph built out of a nearest neighbor search across all the available training samples.

Original: Unfortunately, this is not feasible due to memory and computation limitations.
Revised: Unfortunately, this is not feasible due to memory and computation limitations.

Original: We therefore use a Monte Carlo approximation, as follows.
Revised: We therefore use a Monte Carlo approximation, as follows.

Original: Let
Revised: Let

Original: X 1 , . . .
Revised: X 1 , . . .

Original: , X N (cid:48) denote the images of the training set.
Revised: , X N (cid:48) denote the images of the training set.

Original: We select randomly with uniform distribution M smaller batches of N (cid:28) N (cid:48) peers, batch m containing images { l m 1 , . . .
Revised: We select randomly with uniform distribution M smaller batches of N (cid:28) N (cid:48) peers, batch m containing images { l m 1 , . . .

Original: , l mN } ⊂ { 1 , . .
Revised: , l mN } ⊂ { 1 , . .

Original: N (cid:48) } .
Revised: N (cid:48) } .

Original: The nearest-neighbor graph is constructed separately for each batch m , such that the k th nearest neighbor of pixel p in image i is pixel q mk in image j mk , where m = 1 , . . .
Revised: The nearest-neighbor graph is constructed separately for each batch m , such that the k th nearest neighbor of pixel p in image i is pixel q mk in image j mk , where m = 1 , . . .

Original: , M , j mk ∈ { l m 1 , . . .
Revised: , M , j mk ∈ { l m 1 , . . .

Original: , l mN } , and q mk ∈ { 1 , . . . , n } .
Revised: , l mN } , and q mk ∈ { 1 , . . . , n } .

Original: The output of the ﬁlter is approximated by empirical expectation on the M batches, estimation as follows
Revised: The output of the ﬁlter is approximated by empirical expectation on the M batches, estimation as follows

Original: [Equation]
Revised: [Equation]

Original: In order to limit the computational overhead, M = 1 is used during training, whereas larger values of
Revised: In order to limit the computational overhead, M = 1 is used during training, whereas larger values of

Original: M are used during inference (see Section 4 for details).
Revised: M are used during inference (see Section 4 for details).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the robustness of Peer Regularization to adversarial perturbations on standard benchmarks (MNIST LeCun (1998), CIFAR-10, and CIFAR-100 Krizhevsky (2009)) using a selection of common architectures (LeNet by LeCun et al.
Revised: We evaluate the robustness of Peer Regularization to adversarial perturbations on standard benchmarks (MNIST LeCun (1998), CIFAR-10, and CIFAR-100 Krizhevsky (2009)) using a selection of common architectures (LeNet by LeCun et al.

Original: (1998) and ResNet by He et al.
Revised: (1998) and ResNet by He et al.

Original: The modiﬁcations of the aforementioned architectures with the additional PR layers are referred to as PR-LeNet and
Revised: The modiﬁcations of the aforementioned architectures with the additional PR layers are referred to as PR-LeNet and

Original: PR-ResNet and depicted in Figure 2.
Revised: PR-ResNet and depicted in Figure 2.

Original: Additional details and results appear in the appendices.
Revised: Additional details and results appear in the appendices.

Original: 4. G RAPH CONSTRUCTION
Revised: 4. G RAPH CONSTRUCTION

Original: Training.
Revised: Training.

Original: During training, the graph is constructed using all the images of the current batch as peers.
Revised: During training, the graph is constructed using all the images of the current batch as peers.

Original: We compute all-pairs distances and select K -nearest neighbors.
Revised: We compute all-pairs distances and select K -nearest neighbors.

Original: To mitigate the inﬂuence of the small batch sizes during training of PR-Nets, we introduce high level of stochasticity inside the
Revised: To mitigate the inﬂuence of the small batch sizes during training of PR-Nets, we introduce high level of stochasticity inside the

Original: PR layer, by using dropout of 0 .
Revised: PR layer, by using dropout of 0 .

Original: 2 on the all-pairs distances while performing K -nearest neighbors and 0 .
Revised: 2 on the all-pairs distances while performing K -nearest neighbors and 0 .

Original: 5 on the attention weights right before the softmax nonlinearity.
Revised: 5 on the attention weights right before the softmax nonlinearity.

Original: We used K = 10 for all the experiments.
Revised: We used K = 10 for all the experiments.

Original: Testing.
Revised: Testing.

Original: For testing, we select N ﬁxed peer images and then compute distances between each testing sample and all the N peers.
Revised: For testing, we select N ﬁxed peer images and then compute distances between each testing sample and all the N peers.

Original: Feeding a batch of test samples, each test sample can be adjacent only to the N samples in the ﬁxed graph, and not to other test samples.
Revised: Feeding a batch of test samples, each test sample can be adjacent only to the N samples in the ﬁxed graph, and not to other test samples.

Original: Because of the approximation in the graph construction (see Section 3), we have to ensure that our random pre-selection of the graph nodes does not inﬂuence the performance.
Revised: Because of the approximation in the graph construction (see Section 3), we have to ensure that our random pre-selection of the graph nodes does not inﬂuence the performance.

Original: We therefore perform a Monte Carlo sampling over
Revised: We therefore perform a Monte Carlo sampling over

Original: M forward passes with different uniformly sampled graphs with N nodes and we average over the M runs.
Revised: M forward passes with different uniformly sampled graphs with N nodes and we average over the M runs.

Original: We used N = 50 for MNIST and CIFAR-10 and N = 500 for CIFAR-100, over M = 10 runs.
Revised: We used N = 50 for MNIST and CIFAR-10 and N = 500 for CIFAR-100, over M = 10 runs.

Original: 4. A RCHITECTURES
Revised: 4. A RCHITECTURES

Original: MNIST.
Revised: MNIST.

Original: We modify the LeNet-5 architecture by adding two PR layers, after each convolutional layer and before max-pooling (Figure 2, top).
Revised: We modify the LeNet-5 architecture by adding two PR layers, after each convolutional layer and before max-pooling (Figure 2, top).

Original: CIFAR-10.
Revised: CIFAR-10.

Original: We modify the ResNet-32 model, with A = 16 , B = 32 and C = 64 , as depicted in
Revised: We modify the ResNet-32 model, with A = 16 , B = 32 and C = 64 , as depicted in

Original: Fig. 2 by adding two PR layers at the last change of dimensionality and before the classiﬁer.
Revised: Fig. 2 by adding two PR layers at the last change of dimensionality and before the classiﬁer.

Original: Each
Revised: Each

Original: ResNet block is a sequence of Conv + BatchNorm + ReLU + Conv + BatchNorm + ReLU layers.
Revised: ResNet block is a sequence of Conv + BatchNorm + ReLU + Conv + BatchNorm + ReLU layers.

Original: CIFAR-100.
Revised: CIFAR-100.

Original: For this dataset we take ResNet-110 ( A = 16 , B = 32 and C = 64 ) and modify it in the same way as for CIFAR-10.
Revised: For this dataset we take ResNet-110 ( A = 16 , B = 32 and C = 64 ) and modify it in the same way as for CIFAR-10.

Original: Each block is of size 18.
Revised: Each block is of size 18.

Original: 4. R ESULTS
Revised: 4. R ESULTS

Original: We tested robustness against several different types of white-box adversarial attacks: gradient descent, fast-gradient sign method of Goodfellow et al.
Revised: We tested robustness against several different types of white-box adversarial attacks: gradient descent, fast-gradient sign method of Goodfellow et al.

Original: (2015), projected gradient descent and universal adversarial perturbation of Moosavi-Dezfooli et al.
Revised: (2015), projected gradient descent and universal adversarial perturbation of Moosavi-Dezfooli et al.

Original: Similarly to previous works on adversarial attacks, we evaluate robustness in terms of the fooling rate , deﬁned as the ratio of images for which the network predicts a different label as the result of the perturbation.
Revised: Similarly to previous works on adversarial attacks, we evaluate robustness in terms of the fooling rate , deﬁned as the ratio of images for which the network predicts a different label as the result of the perturbation.

Original: As suggested in Moosavi-Dezfooli et al.
Revised: As suggested in Moosavi-Dezfooli et al.

Original: (2016), random perturbations aim to shift the data points to the decision boundaries of the classiﬁers.
Revised: (2016), random perturbations aim to shift the data points to the decision boundaries of the classiﬁers.

Original: For the method of Moosavi-Dezfooli et al.
Revised: For the method of Moosavi-Dezfooli et al.

Original: even if the perturbations should be universal across different models, we generated it for each model separately to achieve the fairest comparison.
Revised: even if the perturbations should be universal across different models, we generated it for each model separately to achieve the fairest comparison.

Original: Timing.
Revised: Timing.

Original: On CIFAR-10 our unoptimized PR-ResNet-32 implementation, during training, processes a batch of 64 samples in 0 .
Revised: On CIFAR-10 our unoptimized PR-ResNet-32 implementation, during training, processes a batch of 64 samples in 0 .

Original: 15 s compared to the ResNet-32 baseline that takes 0 .
Revised: 15 s compared to the ResNet-32 baseline that takes 0 .

Original: 07 s .
Revised: 07 s .

Original: At inference time a batch of 100 samples with a graph of size 100 is processed in 1 .
Revised: At inference time a batch of 100 samples with a graph of size 100 is processed in 1 .

Original: 5 s whereas the baseline takes 0 .
Revised: 5 s whereas the baseline takes 0 .

Original: 4 s .
Revised: 4 s .

Original: However, it should be noted that much can be done to speed-up the PR layer, we leave it for future work.
Revised: However, it should be noted that much can be done to speed-up the PR layer, we leave it for future work.

Original: Gradient obfuscation.
Revised: Gradient obfuscation.

Original: With regard to recent work of Athalye et al.
Revised: With regard to recent work of Athalye et al.

Original: (2018), we make sure that
Revised: (2018), we make sure that

Original: PeerNet models yield reasonable gradients during perturbation generation, and therefore do not suffer from this phenomenon.
Revised: PeerNet models yield reasonable gradients during perturbation generation, and therefore do not suffer from this phenomenon.

Original: Following experiments further support our claim, as they show that PeerNets do not exhibit the behaviour that typically accompanies this problem (e.g. our model can be always fooled with an unbounded attack, iterative attacks are more efﬁcient than the single-step ones, etc. ).
Revised: Following experiments further support our claim, as they show that PeerNets do not exhibit the behaviour that typically accompanies this problem (e.g. our model can be always fooled with an unbounded attack, iterative attacks are more efﬁcient than the single-step ones, etc. ).

Original: 4.3. S AMPLE - SPECIFIC ATTACKS
Revised: 4.3. S AMPLE - SPECIFIC ATTACKS

Original: Let us ﬁrst evaluate our model on sample-speciﬁc non-targeted attacks.
Revised: Let us ﬁrst evaluate our model on sample-speciﬁc non-targeted attacks.

Original: This means, having a testing sample with label y , we aim to generate an adversarial sample particularly for this testing sample, which will have label ˆ y (cid:54) =
Revised: This means, having a testing sample with label y , we aim to generate an adversarial sample particularly for this testing sample, which will have label ˆ y (cid:54) =

Original: y , while satisfying || x − ˆ x || ∞ ≤ (cid:15) , where (cid:15) is some small value to produce a perturbation v that is nearly perceivable.
Revised: y , while satisfying || x − ˆ x || ∞ ≤ (cid:15) , where (cid:15) is some small value to produce a perturbation v that is nearly perceivable.

Original: For best reproducibility we used implementations of the following methods provided in Foolbox by Rauber et al.
Revised: For best reproducibility we used implementations of the following methods provided in Foolbox by Rauber et al.

Original: Gradient Descent attack.
Revised: Gradient Descent attack.

Original: Generation of the adversarial example is posed as a constrained optimization problem and solved using gradient descent.
Revised: Generation of the adversarial example is posed as a constrained optimization problem and solved using gradient descent.

Original: It can be summarized by the following formula:
Revised: It can be summarized by the following formula:

Original: [Equation]
Revised: [Equation]

Original: where ten (cid:15) values are scanned in ascending order, until the algorithm either succeeds or fails.
Revised: where ten (cid:15) values are scanned in ascending order, until the algorithm either succeeds or fails.

Revised: [Equation]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Fast Gradient Sign method.
Revised: Fast Gradient Sign method.

Original: For the next series of experiments, we test against the Fast Gradient
Revised: For the next series of experiments, we test against the Fast Gradient

Original: Sign Method (FGSM) of Goodfellow et al. (2015).
Revised: Sign Method (FGSM) of Goodfellow et al. (2015).

Original: FGSM generates an adversarial example ˆ x for an input x with label y as follows:
Revised: FGSM generates an adversarial example ˆ x for an input x with label y as follows:

Original: [Equation]
Revised: [Equation]

Original: where J ( θ, x , y ) is the cost used during training (cross-entropy in our case), and (cid:15) controls the magnitude of the perturbation.
Revised: where J ( θ, x , y ) is the cost used during training (cross-entropy in our case), and (cid:15) controls the magnitude of the perturbation.

Original: Projected Gradient Descent.
Revised: Projected Gradient Descent.

Original: Compared to the previous, this is a much stronger adversary, often referred to as a multi-step method
Revised: Compared to the previous, this is a much stronger adversary, often referred to as a multi-step method

Original: FGSM k
Revised: FGSM k

Original: (Madry et al.
Revised: (Madry et al.

Original: Formally, given an input x with label y , single iteration generating adversarial sample x t +1 is deﬁned:
Revised: Formally, given an input x with label y , single iteration generating adversarial sample x t +1 is deﬁned:

Original: [Equation]
Revised: [Equation]

Original: where J ( θ, x , y ) is the cost used during training (cross-entropy in our case), and (cid:15) controls the magnitude of the perturbation and operator (cid:81) deﬁnes the projection on a S -ball around x .
Revised: where J ( θ, x , y ) is the cost used during training (cross-entropy in our case), and (cid:15) controls the magnitude of the perturbation and operator (cid:81) deﬁnes the projection on a S -ball around x .

Original: Discussion of results.
Revised: Discussion of results.

Original: From Figure 5, we can clearly see that sample-speciﬁc gradient-based attacks have to generate much stronger perturbation in order to fool the PeerNet architectures.
Revised: From Figure 5, we can clearly see that sample-speciﬁc gradient-based attacks have to generate much stronger perturbation in order to fool the PeerNet architectures.

Original: Some qualitative examples are shown in Figure 6.
Revised: Some qualitative examples are shown in Figure 6.

Original: Careful reader will notice that PeerNet often cannot be fooled without generating attack observable by a human eye.
Revised: Careful reader will notice that PeerNet often cannot be fooled without generating attack observable by a human eye.

Original: As expected, the strongest PGD attack is able to generate a perturbation for all the samples in the test set even for PeerNet.
Revised: As expected, the strongest PGD attack is able to generate a perturbation for all the samples in the test set even for PeerNet.

Original: However, much stronger perturbations need to be generated.
Revised: However, much stronger perturbations need to be generated.

Original: For ResNet-32, it is sufﬁcient to set (cid:15) ≥ 0 .
Revised: For ResNet-32, it is sufﬁcient to set (cid:15) ≥ 0 .

Original: 013 , while for PR-Resnet-32 we need (cid:15) ≥ 0 .
Revised: 013 , while for PR-Resnet-32 we need (cid:15) ≥ 0 .

Original: 651 , which obviously results in much stronger perturbation.
Revised: 651 , which obviously results in much stronger perturbation.

Original: This is caused by the iterative nature of PGD and also supports the fact that our method does not suffer from gradient obfuscation, as a viable perturbation can be found based on the iterative processing of network gradients for any test sample.
Revised: This is caused by the iterative nature of PGD and also supports the fact that our method does not suffer from gradient obfuscation, as a viable perturbation can be found based on the iterative processing of network gradients for any test sample.

Original: 4.3. U NIVERSAL ADVERSARIAL PERTURBATIONS
Revised: 4.3. U NIVERSAL ADVERSARIAL PERTURBATIONS

Original: Finally, we used the method of Moosavi-Dezfooli et al.
Revised: Finally, we used the method of Moosavi-Dezfooli et al.

Original: (2016), implemented in their DeepFool toolbox to generate a non-targeted universal adversarial perturbation v .
Revised: (2016), implemented in their DeepFool toolbox to generate a non-targeted universal adversarial perturbation v .

Original: The strength of the perturbation was bound by || v || 2 ≤ ρ E
Revised: The strength of the perturbation was bound by || v || 2 ≤ ρ E

Original: ||
Revised: ||

Original: x
Revised: x

Original: || 2 , where the expectation is taken over the set of training images and the parameter ρ controls the strength.
Revised: || 2 , where the expectation is taken over the set of training images and the parameter ρ controls the strength.

Original: We ﬁrst evaluted on MNIST dataset by LeCun (1998) and compared to LeNet-5 as baseline.
Revised: We ﬁrst evaluted on MNIST dataset by LeCun (1998) and compared to LeNet-5 as baseline.

Original: The same evaluations were performed on CIFAR-10 and CIFAR-100 datasets, using ResNet-32 and ResNetas baselines, respectively.
Revised: The same evaluations were performed on CIFAR-10 and CIFAR-100 datasets, using ResNet-32 and ResNetas baselines, respectively.

Original: We additionally compare our method on CIFAR-10 to other baselines, namely BRELU with Gaussian Additive Noise by Zantedeschi et al.
Revised: We additionally compare our method on CIFAR-10 to other baselines, namely BRELU with Gaussian Additive Noise by Zantedeschi et al.

Original: (2017), and MagNet of Meng &
Revised: (2017), and MagNet of Meng &

Original: Chen (2017).
Revised: Chen (2017).

Original: PeerNet performs signiﬁcantly better than MagNet and better than BRELU for strong noise.
Revised: PeerNet performs signiﬁcantly better than MagNet and better than BRELU for strong noise.

Original: Compared to MagNet in particular, the key advantage of our architecture is that it is fully trained end-to-end together with regularization layers instead of as a separate independent module.
Revised: Compared to MagNet in particular, the key advantage of our architecture is that it is fully trained end-to-end together with regularization layers instead of as a separate independent module.

Original: We argue that learned features are more discriminative and robust if trained end-to-end.
Revised: We argue that learned features are more discriminative and robust if trained end-to-end.

Original: Moreover, it is worth mentioning that training a separate module is very prone to gradient obfuscation problem mentioned before.
Revised: Moreover, it is worth mentioning that training a separate module is very prone to gradient obfuscation problem mentioned before.

Original: Results in Figure 7 clearly show that our PeerNet variants are much more robust than the respective baselines.
Revised: Results in Figure 7 clearly show that our PeerNet variants are much more robust than the respective baselines.

Original: We observe minor loss in accuracy at ρ = 0 , which is a consequence of the regularization due to the averaging in feature space, and we argue that it could be mitigated by increasing the model capacity.
Revised: We observe minor loss in accuracy at ρ = 0 , which is a consequence of the regularization due to the averaging in feature space, and we argue that it could be mitigated by increasing the model capacity.

Original: For this purpose, the same conﬁguration with double number of maps in PR layers (marked as v2) was trained, and results show indeed a sensible improvement that does not come at the cost of a much higher fooling rate as for the ResNet baselines.[Figure]
Revised: For this purpose, the same conﬁguration with double number of maps in PR layers (marked as v2) was trained, and results show indeed a sensible improvement that does not come at the cost of a much higher fooling rate as for the ResNet baselines.

Original: A visual comparison of perturbations generated for classical CNN and our PR-Net is depicted in
Revised: A visual comparison of perturbations generated for classical CNN and our PR-Net is depicted in

Original: Figure 4.
Revised: Figure 4.

Original: It can be immediately appreciated that the perturbations for PR-Net have more localized structures.
Revised: It can be immediately appreciated that the perturbations for PR-Net have more localized structures.

Original: We argue that this is due to the attempt of fooling the KNN mechanism of the PR layers, resulting in strong noise in ‘background’ areas rather than in the central parts usually containing the object. 
Revised: We argue that this is due to the attempt of fooling the KNN mechanism of the PR layers, resulting in strong noise in ‘background’ areas rather than in the central parts usually containing the object. [Figure]

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We introduced PeerNets, a novel family of deep networks alternating Euclidean and Graph convolutions to harness information from peer images and showed their robustness to adversarial attacks in a variety of scenarios through extensive experiments for white-box attacks in targeted and non-targeted settings.
Revised: We introduced PeerNets, a novel family of deep networks alternating Euclidean and Graph convolutions to harness information from peer images and showed their robustness to adversarial attacks in a variety of scenarios through extensive experiments for white-box attacks in targeted and non-targeted settings.

Original: PeerNets are simple to use and can be added to any baseline model with minimal changes and are able to deliver remarkably lower fooling rates with negligible loss in performance.
Revised: PeerNets are simple to use and can be added to any baseline model with minimal changes and are able to deliver remarkably lower fooling rates with negligible loss in performance.

Original: Interestingly, the amount of noise required to fool PeerNets is much higher and results in the generation of new images where the noise has clear structure and is signiﬁcantly more perceivable to the human eye.
Revised: Interestingly, the amount of noise required to fool PeerNets is much higher and results in the generation of new images where the noise has clear structure and is signiﬁcantly more perceivable to the human eye.

Original: In future work, we plan to provide a theoretical analysis of the method and scale it to ImageNet-like benchmarks.
Revised: In future work, we plan to provide a theoretical analysis of the method and scale it to ImageNet-like benchmarks.

Revised: A CKNOWLEDGMENTS

Revised: We gratefully acknowledge the generous support by ERC Consolidator Grant No. 724228 (LEMAN), multiple Google Research Faculty awards and Nvidia equipment grants.

Revised: MB is also partially supported by the Royal Society Wolfson Research Merit award and Rudolf Diesel industrial fellowship at TU

Revised: Munich. LG is supported by NSF grant DMS-1546206, a Vannevar Bush Faculty Fellowship, and a gift from Amazon Web Services

Original: A T RAINING HYPER - PARAMETERS
Revised: A T RAINING HYPER - PARAMETERS

Original: The training hyper-parameters used in our experiments are summarized in Table 1.
Revised: The training hyper-parameters used in our experiments are summarized in Table 1.

Original: [Table]
Revised: [Table]

Original: It worths noticing that due to the limited GPU memory
Revised: It worths noticing that due to the limited GPU memory

Original: [Table]
Revised: [Table]

Original: we typically had to decrease the batch size for the PR network architectures.
Revised: we typically had to decrease the batch size for the PR network architectures.

Original: B S AMPLE - SPECIFIC GRADIENT - BASED PERTURBATIONS
Revised: B S AMPLE - SPECIFIC GRADIENT - BASED PERTURBATIONS

Original: [Table]
Revised: [Table]

Original: C U NIVERSAL ADVERSARIAL PERTURBATIONS
Revised: C U NIVERSAL ADVERSARIAL PERTURBATIONS

Original: Results for universal adversarial perturbations attacks using dataset MNIST are in Table 3, for
Revised: Results for universal adversarial perturbations attacks using dataset MNIST are in Table 3, for

Original: CIFAR-10 in Table 4 and CIFAR-100 in Table 5.
Revised: CIFAR-10 in Table 4 and CIFAR-100 in Table 5.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D A DVERSARIAL TRAINING
Revised: D A DVERSARIAL TRAINING

Original: We have compared our approach to adversarial training method using the code 1 provided by Madry et al.
Revised: We have compared our approach to adversarial training method using the code 1 provided by Madry et al.

Original: The ResNet-32 baseline model provided by TensorFlow (the same we use as CNN baseline in our paper) was trained using the script provided in the repository mentioned above.
Revised: The ResNet-32 baseline model provided by TensorFlow (the same we use as CNN baseline in our paper) was trained using the script provided in the repository mentioned above.

Original: We have used two training conﬁgurations producing two baseline models:
Revised: We have used two training conﬁgurations producing two baseline models:

Original: ResNet-32 A - the default hyperparameters provided by the repository 2.
Revised: ResNet-32 A - the default hyperparameters provided by the repository 2.

Original: ResNet-32 B - the same hyperparameters as in our paper
Revised: ResNet-32 B - the same hyperparameters as in our paper

Original: PeerNet model was trained traditionally using SGD with momentum without adversarial training.
Revised: PeerNet model was trained traditionally using SGD with momentum without adversarial training.

Original: Selection and conﬁguration of the attack was left as provided by the repository mentioned above.
Revised: Selection and conﬁguration of the attack was left as provided by the repository mentioned above.

Original: Results shown in the Table 6 show superiority of PeerNet on this benchmark.
Revised: Results shown in the Table 6 show superiority of PeerNet on this benchmark.

Original: Moreover, PR-ResNet was trained without considering any speciﬁc attacks and still outperforms ResNet-32, which was adversarially trained using this speciﬁc attack, by margin of 20%.
Revised: Moreover, PR-ResNet was trained without considering any speciﬁc attacks and still outperforms ResNet-32, which was adversarially trained using this speciﬁc attack, by margin of 20%.

Original: [Table]
Revised: [Table]

Original: E B LACK - BOX ATTACKS
Revised: E B LACK - BOX ATTACKS

Original: For completeness, additional comparison of our method to some of the strong black box attacks has been carried out using repository 2 by Bhagoji et al.
Revised: For completeness, additional comparison of our method to some of the strong black box attacks has been carried out using repository 2 by Bhagoji et al.

Original: The evaluation has been done on small subset of 100 test samples.
Revised: The evaluation has been done on small subset of 100 test samples.

Original: We have used untargeted both single-step and iterative query black-box attack computed using ﬁnite difference method, compared to CW-loss based white-box attack.
Revised: We have used untargeted both single-step and iterative query black-box attack computed using ﬁnite difference method, compared to CW-loss based white-box attack.

Original: The results are listed in the Table 7.
Revised: The results are listed in the Table 7.

Original: [Table]
Revised: [Table]

Original: The results above show that our method is efﬁcient of black-box attacks as well.
Revised: The results above show that our method is efﬁcient of black-box attacks as well.

Original: Moreover, it veriﬁes that our method does not cause gradient masking.
Revised: Moreover, it veriﬁes that our method does not cause gradient masking.

Original: Athalye et al.
Revised: Athalye et al.

Original: (2018) mentions that iterative attacks should be stronger than single step, which in the results above holds.
Revised: (2018) mentions that iterative attacks should be stronger than single step, which in the results above holds.

Original: Also, it states that black-box attacks should be strict subset of white-box attacks, which is conﬁrmed as well.
Revised: Also, it states that black-box attacks should be strict subset of white-box attacks, which is conﬁrmed as well.

Original: F U NIVERSAL ADVERSARIAL PERTURBATIONS FOR CIFAR-10 VISUALLY
Revised: F U NIVERSAL ADVERSARIAL PERTURBATIONS FOR CIFAR-10 VISUALLY

Original: G N ON - TARGETED SAMPLE - SPECIFIC PERTURBATIONS FOR CIFARVISUALLY
Revised: G N ON - TARGETED SAMPLE - SPECIFIC PERTURBATIONS FOR CIFARVISUALLY

Original: [Figure]
Revised: [Figure]