# Improving Adversarial Robustness of CNNs via Maximum Margin
## 1 Introduction  
Convolutional neural networks (CNNs) have been widely used in various domains, especially in computer vision. CNNs have satisfying learning capabilities as they can extract image features of different scales by multiple convolutional layers. However, various CNN architectures are vulnerable. The models will predict wrong results with small perturbations adding to the inputs that humans cannot recognize. In other words, adversarial examples [1; 2; 3], which are similar to clean examples, will mislead the CNN to output incorrect results. Moreover, adversarial examples can transfer across different CNN models [4; 5]. These phenomena indicate that the CNN learns some simple patterns to accomplish tasks exactly on specified data but does not recognize more robust advanced features like the human brain. So the CNN is not safe for medical care [6], autonomous driving [7], and other fields.  
The potential serious harm of adversarial examples has drawn the attention of the scientific research community, forming a new field called adversarial machine learning. There are two main types of defense methods against adversarial examples, i.e., certified methods and empirical methods. Certified methods [8; 9] can obtain provable robustness, but the provable robustness is low and most of them only aim at the attack with disturbance of \(l_{2}\) norm. Empirical methods account for the majority of defense methods, including stochastic activation pruning (SAP) [10], distillation defense [11], asymmetrical adversarial training [12], ME-net [13], etc. However, the robustness of empirical methods is not reliable, and it may be broken by stronger attacks [14; 15].  
Adversarial training (AT) [16] is one of the most successful empirical defense methods at present which is a data augmentation technique for training models on both natural and adversarial examples. Although AT is an empirical method, it has been proved to be highly robust even under adaptive attack [14]. AT effectively robustifies CNN buthas two challenges: lower clean accuracy [17] and the overfitting problem [18; 19]. To address these challenges, many variants of AT have been proposed, such as TRADES [20], customized adversarial training (CAT) [21], dynamic adversarial training (DAT) [22], and geometry-aware instance-reweighted adversarial training (GAIRAT) [23]. However, these two problems are still unsolved very well. So this paper is devoted to them by considering the maximum margin.  
The key insight is that the purpose of adversarial machine learning, i.e., the maximizing margin is very similar to that of SVM. Theoretically, the margin of the classifier in the input space is positively correlated with its robustness. However, for multi-layer neural networks, the margin on the input space is difficult to be calculated and constrained. AT can indirectly increase the margin of the classifier in the input space because the adversarial examples are obtained by moving clean examples toward the decision boundary. We find that ordinary training methods such as using cross-entropy loss cannot effectively increase the margin. So we propose adversarial training with supported vector machine (AT-SVM) that increases the margin by SVM. Specifically, we insert an SVM auxiliary classifier before the last layer in the network, which can maximize the margin on this feature space. The SVM is a better classifier for high-level feature space, so we use it to identify more confusing points which will be trained on the original network. With continuous learning, the boundary of the original network will gradually approach the boundary of the SVM. At the same time, in order to ensure that the margin on the feature space is consistent with the margin on the input space, we constrain the distance between the clean example and the corresponding adversarial example in the margin direction of this feature space. Experiments indicate that our proposed method can effectively improve the robustness against adversarial examples.