Rademacher Complexity for Adversarially Robust Generalization ∗ Sourya Basu Course Project : ECE 543-Statistical Learning Theory Course instructor : Prof .

Machine learning algorithms have shown incredible performance in several inference tasks such as image classi cation, speech recognition, and game playing [2 4]. Although these algorithms work excellently in natural settings (i.e., test data generated in the same way as the train data), their performance drop signi cantly when the test data is corrupted adversarially, at times in a manner that is imperceptible by human beings [5]. Such examples where the input to a trained machine learning model is perturbed so as to increase the probability of wrong inference is called adversarial examples. In image classi cation, adversarial examples might include something as simple as adding small perturbations imperceptible to humans, changing surrounding areas of the main object in an image or even simple rotations or translations [6,7]. The existence of such adversarial examples is alarming in several situations, because of the use machine learning algorithms in several critical examples, such as medical diagnosis or selfdriving cars, e.g. self-driving cars that rely on real-time image recognition might end up with making wrong predictions in the presence of, say, di erent lighting or weather than the one it is trained in, or medical diagnosis might end up predicting some hence making it important for us to analyze the performance of machine learning algorithms in the presence of adversaries. The existence of such examples raise serious questions about the robustness of the existing state-of-the-art machine learning algorithm and risk of using them in critical applications. The existence of such adversarial examples have motivated researchers to come up with algorithms that can defend against particular examples e ectively, however, this also resulted in creation of more and more of such adversarial examples which the existing algorithms does not perform well on, to the extent that there is a virtual race between designing adversarial examples and designing algorithms that can defend them. As reported in the paper [1], in the current scenario, the attackers generating adversarial examples are winning, e.g. it has been shown in [8], that carefully designed gradient-based algorithms may fool most of the existing defense algorithms. Hence, instead of designing defense algorithms against particular adversarial examples, this paper [1] takes a di erent approach and establishes theoretical guarantees on the performance of learning algorithms in the presence of adversaries during the testing phase. The paper [1] uses adversarial training as it appears to be quite e ective against adversarial examples as described in literature [10,11] which optimizes over adversarial loss during the training phase as described next. Let (x, y) ∈ X × Y be the data points drawn according to some unknown distribution D. Let F be a hypothesis class and l(f(x), y) be the loss associated with f ∈ F . This paper [1] considers l∞ adversarial attacks wherein, an -attack can be described as follows: the adversary is allowed to observe the trained model, and given a test data, x, the adversary nds a data point x′ such that ‖x−x‖∞ ≤ and l(f(x′), y) is maximized. Thus, to have better test performance, the learning algorithm should solve the following optimization problem, called as the adversarial risk minimization problem: