Adversarial Examples for k-Nearest Neighbor Classifiers Based on Higher-Order Voronoi Diagrams

Adversarial examples are a widely studied phenomenon in machine learning models . While most of the attention has been focused on neural networks , other practical models also suffer from this issue . In this work , we propose an algorithm for evaluating the adversarial robustness of k-nearest neighbor classification , i.e. , finding a minimum-norm adversarial example . Diverging from previous proposals , we propose the first geometric approach by performing a search that expands outwards from a given input point . On a high level , the search radius expands to the nearby higher-order Voronoi cells until we find a cell that classifies differently from the input point . To scale the algorithm to a large k , we introduce approximation steps that find perturbation with smaller norm , compared to the baselines , in a variety of datasets . Furthermore , we analyze the structural properties of a dataset where our approach outperforms the competition . 1 Introduction . It is well-known that machine learning models can easily be misled by maliciously crafted inputs , called adversarial examples , which are generated by adding a tiny perturbation to test samples [ Biggio et al. , 2013 , Szegedy et al. , 2013 , Goodfellow et al. , 2015 ] . Adversarial examples are often studied in the context of neural networks , leaving the problem largely unexplored for other classifiers . The knearest neighbor , or simply k-NN , classifier is a simple yet widely used model in various applications such as data mining , recommendation , and anomaly detection systems where interpretability and simplicity are preferred [ Wu et al. , 2008 ] . This non-parametric classifier does not require a training phase and has a well-understood and elegant geometric foundation [ Okabe et al. , 1992 ] . k-NN is also an active area of research with lots of developments in popular libraries like Google ’ s ScaNN [ Guo et al. , 2020 ] and Facebook ’ s FAISS [ Johnson et al. , 2017 ] . In recent works [ Papernot and McDaniel , 2018 , Dubey et al. , 2019 , Sitawarin and Wagner , 2019b ] , k-NN is combined with neural networks to enhance the robustness and the interpretability . Despite the importance of k-NN , there are only a handful of works that study its robustness [ Wang et al. , 2018 , Yang et al. , 2020 , Wang et al. , 2019 , Sitawarin and Wagner , 2019a , 2020 ] . The first step towards evaluating the robustness of a classifier is to generate an adversarial example that is close to a given test point , under some definition of closeness . In the case of k-NN where k > 1 , this problem is challenging as the computation involves high-dimensional polytopes that satisfy a set of geometric properties , the so-called Voronoi cells . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we propose the GeoAdEx algorithm for finding adversarial examples for k-NN classifiers . GeoAdEx is the first algorithm that exploits the underpinning geometry of the k-NN classifier . Specifically , our approach performs a principled search on the underlying high-order Voronoi diagram by expanding the search radius around the test point until it finds an adversarial , or simply incorrect , classification . The geometric foundation of GeoAdEx allows us to locate nearby adversarial cells that the other attacks typically miss . GeoAdEx follows the footsteps of the work by Jordan et al . [ 2019 ] who exploited the geometry of neural networks for verifying the outputs . Our algorithmic approach stands in sharp contrast to previous attacks for k-NN [ Yang et al. , 2020 , Wang et al. , 2019 ] which refine the exhaustive approach by heuristic-based filtering . Furthermore , we introduce optimizations and approximations to the main algorithm , and as a result , the experiments show that GeoAdEx discovers the smallest adversarial distance compared to all of the baselines in the vast majority of our experiments for k ∈ { 3 , 5 , 7 } . GeoAdEx finds up to 25 % smaller mean adversarial distance compared to the second best attack . We note that one inherent shortcoming of our geometric approach is the increased computation time , an aspect that can be further improved . Finally , we present experiments that demonstrate that GeoAdEx performs significantly better when points from different classes are “ mixed '' together , i.e. , there is no clear separation between classes . On a high level , such a setup generates a more intricate spatial tessellation with nearby cells that alternate classes . In this challenging case , GeoAdEx performs up to 5× better than the baselines . 2 Background and Related Work . 2.1 Background . Let X = ( x1 , . . . , xn ) be a set of points from Rd , also called generators , and let Y = ( y1 , . . . , yn ) be the class of each point from a set of possible classes { 1 , . . . , c } . Let d ( x , x′ ) denote a metric between x , x′ ∈ Rd . Let Z ( k ) ( X ) be the set of all possible subsets consisting of k points from X , i.e. , Z ( k ) ( X ) = { L ( k ) 1 , . . . , L ( k ) i , . . . , L ( k ) l } , where L ( k ) i = { xi1 , . . . , xik } , xij ∈ X and l = ( n k ) . We define as order-k Voronoi cell associated with L ( k ) i the set V ( L ( k ) i ) that includes the points of Rd that are closer to L ( k ) i than any other k points of X . More formally : V ( L ( k ) i ) = { p | max x { d ( p , x ) |x ∈ L ( k ) i } ≤ min x′ { d ( p , x′ ) |x′ ∈ X \ L ( k ) i } } With the term Voronoi facet , or simply facet , we refer to a boundary of an order-k Voronoi cell . We define as bisector B { xa , xb } the set of points from Rd that are equidistant from xa ∈ X and xb ∈ X . There is a tight connection between bisectors and facets . Every Voronoi facet is part of a bisector , but not every bisector , ( n 2 ) in total , includes a facet . We call the bisectors that do include a facet active bisectors and those that do not inactive bisectors . We also note that an order-k Voronoi cell has at most k ( n− k ) facets . The collection of order-k Voronoi cells for all L ( k ) i is called order-k Voronoi diagram . In this work , we assume the standard non-cocircularity assumption1 to avoid degenerate cases . We only consider the Euclidean distance for the k-NN , i.e. , d ( xa , xb ) = ‖xa − xb‖2 . Up to this point , we have not used the classes of the X . In this work , we focus on k-NN classifiers that classify based on the majority ( class ) vote of the k-nearest points . We define as adversarial cell with respect to ( x , y ) any order-k Voronoi cell that has different majority than label y . We denote with A ( x ) the set of all adversarial cells with respect to ( x , y ) . With the term adversarial facet with respect to ( x , y ) we refer to a facet that belongs to a cell from A ( x ) . 2.2 Related Work . Adversarial Robustness of k-NN Classifiers . Wang et al . [ 2018 ] study the robustness property of k-NN classifiers from a theoretical perspective . The authors show that a k-NN classifier can be as robust as the optimal Bayes classifier given a sufficiently large number of generators and k. Wang et al . [ 2018 ] and Yang et al . [ 2020 ] also propose potential methods for improving the robustness of k-NN by pruning away some of the “ ambiguous ” generators . Other works consider different alternatives for 1In practice this can be achieved by adding a small random noise on each generator to break potential cocircularity between them . enhancing the robustness of neural networks by combining them with k-NN . [ Papernot and McDaniel , 2018 , Sitawarin and Wagner , 2019b , Dubey et al. , 2019 ] Attacks on k-NN Classifiers . Sitawarin and Wagner [ 2019a , 2020 ] propose a method for finding minimum-norm adversarial examples on k-NN and deep k-NN classifiers [ Papernot and McDaniel , 2018 ] . Their method approximates a k-NN classifier with a soft differentiable function so adversarial examples can be found via a gradient-based optimization algorithm . While the approach is efficient and scalable to large k , it provides no guarantee for finding the nearest adversarial example and overlooks the geometry of k-NN . Yang et al . [ 2020 ] and Wang et al . [ 2019 ] take a similar approach for finding the adversarial example closest to a given test x . Their method follows an exhaustive search by computing the distance between x and all the Voronoi cells that have a different class from x . Both of them show that this can be done exactly when k = 1 but does not scale well for k > 1 . Consequently , they introduce heuristics to improve the efficiency by choosing only a subset of the Voronoi cells . However , in the process , they lose the optimality guarantee on the resulted adversarial example . More importantly , the heuristics will miss an exponential number of cells as k increases . Attacks Based on Geometric Insights . The problem of verifying or finding the nearest adversarial examples in neural networks is an active research direction where solutions still do not scale to a satisfactory degree [ Huang et al. , 2017 , Katz et al. , 2017 , Tjeng et al. , 2017 , Wong and Kolter , 2018 , Weng et al. , 2018 , Cohen et al. , 2019 , Jordan et al. , 2019 ] . Among these works , Jordan et al . [ 2019 ] proposed an alternative for a ReLU network , using a geometric approach . Specifically , the authors notice that piecewise linear networks partition the input space into numerous polytopes where points of the same polytope are classified with the same label . Armed with this observation , they propose an algorithm that iteratively searches through these polytopes that are further away from the test sample until they find a polytope with a different label . 3 The GeoAdEx Algorithm . Objective . The goal of the algorithm is to find the smallest perturbation δ∗ that moves a test point ( x , y ) to an adversarial cell . This objective can be expressed as the following optimization problem : δ∗ = arg min δ ‖δ‖22 ( 1 ) s.t . x+ δ ∈ A ( x ) We define as ∗ optimal adversarial distance where ∗ : = ‖δ∗‖2 . The norm of any other ( non-optimal ) perturbation δ that misclassifies ( x , y ) , i.e. , x+ δ ∈ A ( x ) , is simply called adversarial distance . A First Approach . The constraint of the above formulation implies that x+ δ must be a member of an adversarial cell from A ( x ) . A simple approach to solve this minimization is to build a series of optimization problems , one for each of the cells in A ( x ) , and pick the solution with the minimum adversarial distance . Unfortunately , this would require solving O ( ( n k ) ) quadratic programs each of which has k ( n − k ) constraints . While this complexity may be manageable when k = 1 and n is small , it does not scale with k. On a high level , Yang et al . [ 2020 ] and Wang et al . [ 2019 ] take this approach and additionally develop heuristics to scale to cases with k > 1 . The GeoAdEx Approach . The core idea of GeoAdEx is to perform a principled geometric exploration around the test point x. GeoAdEx processes order-k Voronoi cells à la breadth-first search until it discovers an adversarial cell . Algorithm 1 provides a pseudocode of the main steps of GeoAdEx , and Figure 1 illustrates some steps of GeoAdEx on a 3-NN classifier with two classes . In the following subsections , we explain these steps in detail and describe performance speedups as well as approximation steps to scale the algorithm to k > 1 .

Summary: This paper focuses on the problem of generating adversarial perturbations for $k$-nearest-neighbor classifiers. Focusing on the geometric nature of $k$-nearest-neighbor classification, the proposed scheme GeoAdEx works with order-$k$ Voronoi cells and facets to search around the test point being attacked by gradually increasing the search radius around it and updating the upper and lower bounds to the optimal adversarial perturbation norm. Beyond this novel algorithm to generate the optimal perturbation, the paper focuses on making this algorithm meaningfully scalable in high dimensional data with $k>1$ nearest-neighbors. The first optimization is the observation that the task of finding the neighboring order-$k$ Voronoi cells to the current cell in question can be solved by finding the neighboring order-$1$ Voronoi cells -- this is an improvement though finding order-$1$ Voronoi cells in high dimensions is still a very significant bottleneck, and hence needs to be approximated by just using the $m$ nearest neighbors of all the $k$ points in the current cell. This introduces a significant approximation that appears to be necessary to make the proposed algorithm tractable. Without this approximation, the proposed algorithm is guaranteed to find optimal perturbation; the above approximation does not have any such property. Empirically, the proposed algorithm is evaluated against existing baselines, with significant improvement, however, with an order of magnitude more computational cost. The empirical evaluation also study the effect of the class separation on the relative performance of the proposed GeoAdEx against baselines, demonstrating the intuitive result that GeoAdEx performs better with small class separations since the baselines tend to miss the closeby small order-$k$ Voronoi cells which GeoAdEx almost exhaustively searches over.