user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Defensive Quantization Layer For Convolutional Network Against Adversarial Attack

Paper content:
Recent research has intensively revealed the vulnerability of deep neural networks, especially for convolutional neural networks (CNNs) on the task of image recognition, through creating adversarial samples which `"slightly" differ from legitimate samples.
This vulnerability indicates that these powerful models are sensitive to specific perturbations and cannot filter out these adversarial perturbations.
In this work, we propose a quantization-based method which enables a CNN to filter out adversarial perturbations effectively.
Notably, different from prior work on input quantization, we apply the quantization in the intermediate layers of a CNN.
Our approach is naturally aligned with the clustering of the coarse-grained semantic information learned by a CNN.
Furthermore, to compensate for the loss of information which is inevitably caused by the quantization, we propose the multi-head quantization, where we project data points to different sub-spaces and perform quantization within each sub-space.
We enclose our design in a quantization layer named as the Q-Layer.
The results obtained on MNIST and Fashion-MNSIT datasets demonstrate that only adding one Q-Layer into a CNN could significantly improve its robustness against both white-box and black-box attacks.
In recent years, along with the massive success of deep neural networks (DNNs) witnessed in many research fields, we have also observed their impressive failures when confronted with adversarial examples, especially for image recognition tasks.
Prior work (Szegedy et al. (2014) ; Goodfellow et al. (2015) ) has demonstrated that an adversarial image can be easily synthesized by adding to a legitimate image a specifically crafted perturbation, which is typically imperceptible for human visual inspection.
The generated adversarial image, however, is strikingly effective for causing convolutional neural network (CNN) classifiers to make extreme confident misclassification results.
This vulnerability of DNNs has stimulated the unceasing arms race between research on both attacking (Goodfellow et al. (2015) ; Kurakin et al. (2017) ; Carlini & Wagner (2017) ; Moosavi-Dezfooli et al. (2016) ; Chen et al. (2017) ; Brendel et al. (2018) ) and defending (Madry et al. (2018) ; Samangouei et al. (2018b) ; Buckman et al. (2018) ; Zhang & Liang (2019) ) these powerful models.
Among much existing work and a large variety of defense methods, several prior studies (Xu et al. (2018) ; Buckman et al. (2018) ; Zhang & Liang (2019) ) have spent concerted efforts on defending adversarial attacks through input quantization.
The principle idea of these methods is to use quantization to filter out small-scale adversarial perturbations.
Recall that in prior work (Bau et al. (2017) ; Zeiler & Fergus (2014) ; Zhou et al. (2015) ), it has been shown that the shallow layers of a CNN mostly capture fine-grained features including lines and curves.
In the meantime, deeper layers learn coarse-grained yet semantically more critical features, which essentially discriminate different samples.
Especially for classification tasks, it is natural to expect samples with the same classification label to share similar semantic information.
As such, the semantic similarity between samples may be better revealed if we attend to their latent features learned by the intermediate layers of a CNN.
Here we hypothesize that data points with similar semantic information should be distributed densely in the latent feature space.
Thus, in order to more effectively filter out adversarial perturbations, we propose an alternative approach which quantizes the data representations embedded in the feature space produced by the intermediate layers of CNN classifiers.
Interestingly, there have been other studies that develop similar approaches but for different purposes.
For example, Wang et al. (2017; have applied k-means clustering on the intermediate feature maps of CNN models to discover explainable visual concepts.
Recent methods, including VQ-VAE (van den Oord et al. (2017) ) and SOM-VAE (Fortuin et al. (2019) ), were proposed to construct generative models for images and time-series data with discrete latent representations, which offer better explainability.
However, to the best of our knowledge, the approach of applying intermediate layer quantization for CNN models has not been explored in the context of defending adversarial examples.
In this work, we propose a quantization method that is realized by an extra intermediate layer, i.e., the quantization layer (Q-Layer).
Our Q-Layer can be easily integrated into any existing architecture of CNN models.
Specifically, the Q-Layer splits the mainstream of information that flows forward in a regular CNN model into two separate flows.
Both flows share the same information passed by layers before the Q-Layer, but differ in the subsequent networks after the Q-Layer.
These two flows produce two outputs, one is the quantized output, and the other is the Non-quantized output.
Specifically, the non-quantized path is introduced to facilitate the gradient-based training, and to regularize the quantization operation.
In the quantized path, we introduce non-differentiability to defend gradient-based attacks.
It is important to note that, while gradient-based attacks cannot be directly applied to the quantized network, they can still be conducted by following the nonquantized path.
Also, similar to most input transformation methods (Xu et al. (2018) ; Buckman et al. (2018) ) proposed for defending adversarial examples, our quantization will inevitably lose some feature information, which might be useful for classification.
In order to compensate for this loss of information, we further propose multi-head quantization, where we project data points to different sub-spaces and perform quantization within each sub-space.
In particular, we perform the projection by re-weighting the input-channels of CNN with trainable parameters.
This projection process can be interpreted as performing feature extraction from different points of view, hence help retain the overall effectiveness of our method without causing much performance degradation for the model to be protected.
Last but not least, our proposed method can be readily combined with other existing defenses, e.g., adversarial training (Goodfellow et al. (2015) ), to jointly improve the adversarial robustness of a protected CNN classifier.
In summary, we make the following contribution:
• We propose a quantization-based defense method for the adversarial example problem by designing a quantization Layer (Q-Layer) which can be integrated into existing architectures of CNN models.
Our implementation is online available 1 .
• We propose multi-head quantization to compensate for the possible information loss caused by the quantization process, and bring significant improvement to the adversarial robustness of an armed model under large perturbation.
• We evaluate our method under several representative attacks on MNIST and Fashion-MNIST datasets.
Our experiment results demonstrate that the adoption of the Q-Layer can significantly enhance the robustness of a CNN against both black-box and white-box attack, and the robustness can be further improved by combining our method with adversarial training.
2 RELATED WORK 2.1 ADVERSARIAL ATTACK Given a neural network classifier N with parameters denoted by w, N can be regarded as a function that takes an input x ∈ R dx and produces an classification label y, i.e., N (x;
w) = y or N
(x) = y for notation simplicity.
In principle, the goal of the adversarial attack is to create a perturbation δ ∈ R dx to be added to a legitimate sample x for creating an adversarial example, i.e., x + δ, which causes the target model N to produce a wrong classification result.
Depending on different threat models, adversarial attacks are categorized as black-box attacks or white-box attacks (Papernot et al. (2018) ).
Specifically, it is commonly assumed in the white-box attack scenario, that an attacker knows every detail of the target model.
This dramatically eases the generation of impactful adversarial examples, and has stimulated researchers to propose various white-box attack methods, including the fast gradient sign method (FGSM) (Goodfellow et al. (2015) ), the basic iterative method (BIM) (Kurakin et al. (2017) ), the Carlini-Wagner (CW) attack (Carlini & Wagner (2017) ), and DeepFool (Moosavi-Dezfooli et al. (2016) ).
On the contrary, in the black-box attack scenario, an attacker is typically assumed to be restricted for accessing detailed information, e.g., the architecture, values of parameters, training datasets, of the target model.
There have been many black-box attack methods proposed in prior work (Chen et al. (2017) ; Brendel et al. (2018) ; Papernot et al. (2016) ).
Representative black-box attacks typically exploit the transferability (Papernot et al. (2016) ) of the adversarial examples, hence is also referred to as transfer black-box attacks.
Explicitly, in transfer black-box attacks, an attacker can train and maintain a substitute model, then conduct white-box attacks on the substitute model to generate adversarial samples which retain a certain level of attack power to the target model.
Since both black-box and white-box attacks rely on the white-box assumption, in the following, we mainly introduce several representative white-box attacks, namely the FGSM, BIM and CW attacks, which are also employed in our experiments due to their wide adoption as the benchmark attack methods (Samangouei et al. (2018a; b) ).
Fast gradient sign method (FGSM) Goodfellow et al. (2015) proposed FGSM, in which δ is calculated by scaling the l ∞ norm of the gradient of the loss function L with respect to a legitimate input x as follows:
where represents the maximally allowed scale of perturbation.
This method represents a one-step approximation for the direction in the input space that affects the loss function most significantly.
Basic iterative method (BIM) Kurakin et al. (2017) proposed the BIM attack, which iteratively performs the FGSM hence generates more impactful adversarial examples at the expense of computational efficiency.
Carlini-Wagner (CW) attack Carlini & Wagner (2017) aimed to find the smallest perturbation to fool the target model, by solving the following optimization problem:
where c > 0 is a tunable positive constant and p represents different norms.
In our experiment, we consider l ∞ norm.
L is designed to satisfy that L(x, δ) < 0 if and only if N (x + δ) = N (x).
In this paper, we have designed and implemented a quantization layer (Q-Layer) to protection CNN classifiers from the adversarial attacks, and presented the experiment results which show that, by simply inserting one Q-Layer into a regular CNN, its adversarial robustness under both white-box and black-box attacks obtains significant improvement.
Moreover, we have combined our method in tandem with adversarial training.
The empirical results show that the Q-layer can make a CNN benefit more from adversarial training and even perform well under attacks with larger perturbations.
One limitation of this work is due to the uncertainty introduced by the random initialization of concept matrix.
This issue also exists in many other clustering algorithms.
In this work, we alleviate the impact of this issue by reactivating inactivate concepts.
Future work would pursue other approaches on constructing the concept matrix, e.g., regularizing the concept matrix with specific semantic constrains, and using the E-path as a learned index to retrieve information stored in the concept matrix, which acts as an external memory.

assistant: text: We propose a quantization-based method which regularizes a CNN's learned representations to be automatically aligned with trainable concept matrix hence effectively filtering out adversarial perturbations.