Nearest neighbor classification expects the class conditional probabilities to be locally constant, and suffers from bias in high dimensions. We propose a locally adaptive form of nearest neighbor classification to try to ameliorate this curse of dimensionality. We use a local linear discriminant analysis to estimate an effective metric for computing neighborhoods. We determine the local decision boundaries from centroid information, and then shrink neighborhoods in directions orthogonal to these local decision boundaries, and elongate them parallel to the boundaries. Thereafter, any neighborhood-based classifier can be employed, using the modified neighborhoods. The posterior probabilities tend to be more homogeneous in the modified neighborhoods. We also propose a method for global dimension reduction, that combines local dimension information. In a number of examples, the methods demonstrate the potential for substantial improvements over nearest neighbor classification. Index Terms-Classification, nearest neighbors, linear discriminant an,alysis, curse of dimensionality. + - T. Hastie is with the
Local linear discriminative analysis REF estimates a local distance metric using the local linear discriminant analysis.
Discriminant adaptive nearest neighbor classification
IEEE Trans. Pattern Anal. Mach. Intell.
IEEE Trans. Pattern Anal. Mach. Intell.
Computer Science