Improved lower bounds for learning from noisy examples: An information-theoretic approach
This paper presents a general information-theoretic approach for obtaining lower bounds on the number of examples required for Probably Approximately Correct (PAC) learning in the presence of noise. This approach deals directly with the fundamental information quantities, avoiding a Bayesian analysis. The technique is applied to several different models, illustrating its generality and power. The resulting bounds add logarithmic factors to (or improve the constants in) previously known lower bounds. (C) 2001 Academic Press.
lower bounds noisy examples pac learning entropy mutual information