Given a set F of classifiers and a probability distribution over their domain, one can define a metric by taking the distance between a pair of classifiers to be the probability that they classify a random item differently. We prove bounds on the sample complexity of PAC learning in terms of the doubling dimension of this metric. These bounds imply known bounds on the sample complexity of learning halfspaces with respect to the uniform distribution that are optimal up to a constant factor. We then prove a bound that holds for any algorithm that outputs a classifier with zero error whenever this is possible; this bound is in terms of the maximum of the doubling dimension and the VC-dimension of F and * Part of this research was done while the first author was visiting Google in Mountain View, California. Email addresses: bshouty@cs.technion.ac.il (Nader H. Bshouty), liy3@gis.a-star.edu.sg strengthens the best known bound in terms of the VC-dimension alone. Finally, we show that there is no bound on the doubling dimension of halfspaces in R n in terms of n that holds independently of the domain distribution. This implies that there is no such a bound in terms of the VCdimension of F (in contrast with the metric dimension).
Bshouty, Li and Long REF provided a generalization bound in terms of the maximum of the doubling dimension and the VCdimension of the hypothesis class F .
Using the Doubling Dimension to Analyze the Generalization of Learning Algorithms
J. Comput. Syst. Sci.
J. Comput. Syst. Sci.
Mathematics Computer Science