Adversarial Examples for k-Nearest Neighbor Classifiers Based on Higher-Order Voronoi Diagrams

Adversarial examples are a widely studied phenomenon in machine learning models . While most of the attention has been focused on neural networks , other practical models also suffer from this issue . In this work , we propose an algorithm for evaluating the adversarial robustness of k-nearest neighbor classification , i.e. , finding a minimum-norm adversarial example . Diverging from previous proposals , we propose the first geometric approach by performing a search that expands outwards from a given input point . On a high level , the search radius expands to the nearby higher-order Voronoi cells until we find a cell that classifies differently from the input point . To scale the algorithm to a large k , we introduce approximation steps that find perturbation with smaller norm , compared to the baselines , in a variety of datasets . Furthermore , we analyze the structural properties of a dataset where our approach outperforms the competition . 1 Introduction . It is well-known that machine learning models can easily be misled by maliciously crafted inputs , called adversarial examples , which are generated by adding a tiny perturbation to test samples [ Biggio et al. , 2013 , Szegedy et al. , 2013 , Goodfellow et al. , 2015 ] . Adversarial examples are often studied in the context of neural networks , leaving the problem largely unexplored for other classifiers . The knearest neighbor , or simply k-NN , classifier is a simple yet widely used model in various applications such as data mining , recommendation , and anomaly detection systems where interpretability and simplicity are preferred [ Wu et al. , 2008 ] . This non-parametric classifier does not require a training phase and has a well-understood and elegant geometric foundation [ Okabe et al. , 1992 ] . k-NN is also an active area of research with lots of developments in popular libraries like Google ’ s ScaNN [ Guo et al. , 2020 ] and Facebook ’ s FAISS [ Johnson et al. , 2017 ] . In recent works [ Papernot and McDaniel , 2018 , Dubey et al. , 2019 , Sitawarin and Wagner , 2019b ] , k-NN is combined with neural networks to enhance the robustness and the interpretability . Despite the importance of k-NN , there are only a handful of works that study its robustness [ Wang et al. , 2018 , Yang et al. , 2020 , Wang et al. , 2019 , Sitawarin and Wagner , 2019a , 2020 ] . The first step towards evaluating the robustness of a classifier is to generate an adversarial example that is close to a given test point , under some definition of closeness . In the case of k-NN where k > 1 , this problem is challenging as the computation involves high-dimensional polytopes that satisfy a set of geometric properties , the so-called Voronoi cells . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we propose the GeoAdEx algorithm for finding adversarial examples for k-NN classifiers . GeoAdEx is the first algorithm that exploits the underpinning geometry of the k-NN classifier . Specifically , our approach performs a principled search on the underlying high-order Voronoi diagram by expanding the search radius around the test point until it finds an adversarial , or simply incorrect , classification . The geometric foundation of GeoAdEx allows us to locate nearby adversarial cells that the other attacks typically miss . GeoAdEx follows the footsteps of the work by Jordan et al . [ 2019 ] who exploited the geometry of neural networks for verifying the outputs . Our algorithmic approach stands in sharp contrast to previous attacks for k-NN [ Yang et al. , 2020 , Wang et al. , 2019 ] which refine the exhaustive approach by heuristic-based filtering . Furthermore , we introduce optimizations and approximations to the main algorithm , and as a result , the experiments show that GeoAdEx discovers the smallest adversarial distance compared to all of the baselines in the vast majority of our experiments for k ∈ { 3 , 5 , 7 } . GeoAdEx finds up to 25 % smaller mean adversarial distance compared to the second best attack . We note that one inherent shortcoming of our geometric approach is the increased computation time , an aspect that can be further improved . Finally , we present experiments that demonstrate that GeoAdEx performs significantly better when points from different classes are “ mixed '' together , i.e. , there is no clear separation between classes . On a high level , such a setup generates a more intricate spatial tessellation with nearby cells that alternate classes . In this challenging case , GeoAdEx performs up to 5× better than the baselines . 2 Background and Related Work . 2.1 Background . Let X = ( x1 , . . . , xn ) be a set of points from Rd , also called generators , and let Y = ( y1 , . . . , yn ) be the class of each point from a set of possible classes { 1 , . . . , c } . Let d ( x , x′ ) denote a metric between x , x′ ∈ Rd . Let Z ( k ) ( X ) be the set of all possible subsets consisting of k points from X , i.e. , Z ( k ) ( X ) = { L ( k ) 1 , . . . , L ( k ) i , . . . , L ( k ) l } , where L ( k ) i = { xi1 , . . . , xik } , xij ∈ X and l = ( n k ) . We define as order-k Voronoi cell associated with L ( k ) i the set V ( L ( k ) i ) that includes the points of Rd that are closer to L ( k ) i than any other k points of X . More formally : V ( L ( k ) i ) = { p | max x { d ( p , x ) |x ∈ L ( k ) i } ≤ min x′ { d ( p , x′ ) |x′ ∈ X \ L ( k ) i } } With the term Voronoi facet , or simply facet , we refer to a boundary of an order-k Voronoi cell . We define as bisector B { xa , xb } the set of points from Rd that are equidistant from xa ∈ X and xb ∈ X . There is a tight connection between bisectors and facets . Every Voronoi facet is part of a bisector , but not every bisector , ( n 2 ) in total , includes a facet . We call the bisectors that do include a facet active bisectors and those that do not inactive bisectors . We also note that an order-k Voronoi cell has at most k ( n− k ) facets . The collection of order-k Voronoi cells for all L ( k ) i is called order-k Voronoi diagram . In this work , we assume the standard non-cocircularity assumption1 to avoid degenerate cases . We only consider the Euclidean distance for the k-NN , i.e. , d ( xa , xb ) = ‖xa − xb‖2 . Up to this point , we have not used the classes of the X . In this work , we focus on k-NN classifiers that classify based on the majority ( class ) vote of the k-nearest points . We define as adversarial cell with respect to ( x , y ) any order-k Voronoi cell that has different majority than label y . We denote with A ( x ) the set of all adversarial cells with respect to ( x , y ) . With the term adversarial facet with respect to ( x , y ) we refer to a facet that belongs to a cell from A ( x ) . 2.2 Related Work . Adversarial Robustness of k-NN Classifiers . Wang et al . [ 2018 ] study the robustness property of k-NN classifiers from a theoretical perspective . The authors show that a k-NN classifier can be as robust as the optimal Bayes classifier given a sufficiently large number of generators and k. Wang et al . [ 2018 ] and Yang et al . [ 2020 ] also propose potential methods for improving the robustness of k-NN by pruning away some of the “ ambiguous ” generators . Other works consider different alternatives for 1In practice this can be achieved by adding a small random noise on each generator to break potential cocircularity between them . enhancing the robustness of neural networks by combining them with k-NN . [ Papernot and McDaniel , 2018 , Sitawarin and Wagner , 2019b , Dubey et al. , 2019 ] Attacks on k-NN Classifiers . Sitawarin and Wagner [ 2019a , 2020 ] propose a method for finding minimum-norm adversarial examples on k-NN and deep k-NN classifiers [ Papernot and McDaniel , 2018 ] . Their method approximates a k-NN classifier with a soft differentiable function so adversarial examples can be found via a gradient-based optimization algorithm . While the approach is efficient and scalable to large k , it provides no guarantee for finding the nearest adversarial example and overlooks the geometry of k-NN . Yang et al . [ 2020 ] and Wang et al . [ 2019 ] take a similar approach for finding the adversarial example closest to a given test x . Their method follows an exhaustive search by computing the distance between x and all the Voronoi cells that have a different class from x . Both of them show that this can be done exactly when k = 1 but does not scale well for k > 1 . Consequently , they introduce heuristics to improve the efficiency by choosing only a subset of the Voronoi cells . However , in the process , they lose the optimality guarantee on the resulted adversarial example . More importantly , the heuristics will miss an exponential number of cells as k increases . Attacks Based on Geometric Insights . The problem of verifying or finding the nearest adversarial examples in neural networks is an active research direction where solutions still do not scale to a satisfactory degree [ Huang et al. , 2017 , Katz et al. , 2017 , Tjeng et al. , 2017 , Wong and Kolter , 2018 , Weng et al. , 2018 , Cohen et al. , 2019 , Jordan et al. , 2019 ] . Among these works , Jordan et al . [ 2019 ] proposed an alternative for a ReLU network , using a geometric approach . Specifically , the authors notice that piecewise linear networks partition the input space into numerous polytopes where points of the same polytope are classified with the same label . Armed with this observation , they propose an algorithm that iteratively searches through these polytopes that are further away from the test sample until they find a polytope with a different label . 3 The GeoAdEx Algorithm . Objective . The goal of the algorithm is to find the smallest perturbation δ∗ that moves a test point ( x , y ) to an adversarial cell . This objective can be expressed as the following optimization problem : δ∗ = arg min δ ‖δ‖22 ( 1 ) s.t . x+ δ ∈ A ( x ) We define as ∗ optimal adversarial distance where ∗ : = ‖δ∗‖2 . The norm of any other ( non-optimal ) perturbation δ that misclassifies ( x , y ) , i.e. , x+ δ ∈ A ( x ) , is simply called adversarial distance . A First Approach . The constraint of the above formulation implies that x+ δ must be a member of an adversarial cell from A ( x ) . A simple approach to solve this minimization is to build a series of optimization problems , one for each of the cells in A ( x ) , and pick the solution with the minimum adversarial distance . Unfortunately , this would require solving O ( ( n k ) ) quadratic programs each of which has k ( n − k ) constraints . While this complexity may be manageable when k = 1 and n is small , it does not scale with k. On a high level , Yang et al . [ 2020 ] and Wang et al . [ 2019 ] take this approach and additionally develop heuristics to scale to cases with k > 1 . The GeoAdEx Approach . The core idea of GeoAdEx is to perform a principled geometric exploration around the test point x. GeoAdEx processes order-k Voronoi cells à la breadth-first search until it discovers an adversarial cell . Algorithm 1 provides a pseudocode of the main steps of GeoAdEx , and Figure 1 illustrates some steps of GeoAdEx on a 3-NN classifier with two classes . In the following subsections , we explain these steps in detail and describe performance speedups as well as approximation steps to scale the algorithm to k > 1 .

Summary: The paper describes a "white box" adversarial attack on k-NN classifiers that is based on the geometry of the Voronoi diagram. The paper first defines the relevant geometric elements - Voronoi cells and their facets (which reside on bisectors - hyperplanes that are equidistant to two data points) and when they are considered *adversarial*. The paper then proposes an algorithm (GeoAdEx) for searching for the nearest adversarial facet for a given test point, using a priority queue in a BFS manner. GeoAdEx is shown to provide an optimal (min L2) attack. To allow scaling GeoAdEx to k>1 and large data, several approximations are suggested (which may result in a suboptimal attack). The experimental section compares the mean attack size for seven datasets against three other kNN attacks (for k=3, 5, 7) and shows that GeoAdEx results in a smaller-norm attack.