MMA Training: Direct Input Space Margin Maximization through Adversarial Training
We study adversarial robustness of neural networks from a margin maximization perspective, where margins are defined as the distances from inputs to a classifiers decision boundary.Our study shows that maximizing margins can be achieved by minimizing the adversarial loss on the decision boundary at the "shortest successful perturbation", demonstrating a close connection between adversarial losses and the margins.We propose Max-Margin Adversarial training to directly maximize the margins to achieve adversarial robustness.Instead of adversarial training with a fixed, MMA offers an improvement by enabling adaptive selection of the "correct" as the margin individually for each datapoint.In addition, we rigorously analyze adversarial training with the perspective of margin maximization, and provide an alternative interpretation for adversarial training, maximizing either a lower or an upper bound of the margins.Our experiments empirically confirm our theory and demonstrate MMA trainings efficacy on the MNIST and CIFAR10 datasets w.r.t. and robustness.
We propose MMA training to directly maximize input space margin in order to improve adversarial robustness primarily by removing the requirement of specifying a fixed distortion bound.An adaptive margin-based adversarial training approach to train robust DNNs, by maximizing the shortest margin of inputs to the decision boundary, that makes adversarial training with large perturbation possible.A method for robust learning against adversarial attacks where the input space margin is directly maximized and a softmax variant of the max-margin is introduced.