Practical Inference-Time Attacks Against Machine-Learning Systems and a Defense Against Them
Creators: name: Sharif, Mahmood
Publisher: name: Carnegie Mellon University
Year: 2020
Subjects: subject: 90699 Electrical and Electronic Engineering not elsewhere classified
subject: FOS: Electrical engineering, electronic engineering, information engineering
subject: FOS: Electrical engineering, electronic engineering, information engineering

description: Prior work has shown that machine-learning algorithms are vulnerable to evasion by socalled adversarial examples. Nonetheless, the majority of the work on evasion attacks<br>has mainly explored L<sub>p</sub>-bounded perturbations that lead to misclassification. From a computer-security perspective, such attacks have limited practical implications. To fill<br>the gap, we propose evasion attacks that satisfy multiple objectives, and show that these attacks pose a practical threat to computer systems. In particular, we demonstrate how to produce adversarial examples against state-of-the-art face-recognition and malwaredetection systems that simultaneously satisfy multiple objectives (e.g., smoothness and robustness against changes in imaging conditions) to mislead the systems in practical settings. Against face recognition, we develop a systematic method to automatically generate attacks, which are realized through printing a pair of eyeglass frames. When worn by attackers, the eyeglasses allow them mislead face-recognition algorithms to evade recognition or impersonate other individuals. Against malware detection, we develop an attack that guides binary-diversification tools via optimization to transform binaries in a functionality preserving manner and mislead detection. The attacks that we initially demonstrate achieve the desired objectives via ad hoc optimizations. We extend these attacks via a general framework to train a generator neural network to emit adversarial examples satisfying desired objectives. We demonstrate<br>the ability of the proposed framework to accommodate a wide range of objectives, including imprecise ones difficult to model, in two application domains. Specifically,<br>we demonstrate how to produce adversarial eyeglass frames to mislead face recognition with better robustness, inconspicuousness, and scalability than previous approaches, as well as a new attack to fool a handwritten-digit classifier. Finally, to protect computer-systems from adversarial examples, we propose n-ML—a novel defense that is inspired by n-version programming. n-ML trains an ensemble of n classifiers, and classifies inputs by a vote. Unlike prior approaches, however, the classifiers are trained to classify adversarial examples differently than each other,<br>rendering it very difficult for an adversarial example to obtain enough votes to be misclassified. In several application domains (including face and street-sign recognition),<br>we show that n-ML roughly retains the benign classification accuracies of state-of-theart models, while simultaneously defending against adversarial examples (produced<br>by our framework, or L<sub>p</sub>-based attacks) with better resilience than the best defenses known to date and, in most cases, with lower inference-time overhead.<br>