We study the rates of convergence in generalization error achievable by active learning under various types of label noise. Additionally, we study the general problem of model selection for active learning with a nested hierarchy of hypothesis classes and propose an algorithm whose error rate provably converges to the best achievable error among classifiers in the hierarchy at a rate adaptive to both the complexity of the optimal classifier and the noise conditions. In particular, we state sufficient conditions for these rates to be dramatically faster than those achievable by passive learning. 1. Introduction. Active learning refers to a family of powerful supervised learning protocols capable of producing more accurate classifiers while using a smaller number of labeled data points than traditional (passive) learning methods. Here we study a variant known as pool-based active learning, in which a learning algorithm is given access to a large pool of unlabeled data (i.e., only the covariates are visible), and is allowed to sequentially request the label (response variable) of any particular data points from that pool. The objective is to learn a function that accurately predicts the labels of new points, while minimizing the number of label requests. Thus, this is a type of sequential design scenario for a function estimation problem. This contrasts with passive learning, where the labeled data are sampled at random. In comparison, by more carefully selecting which points should be labeled, active learning can often significantly decrease the total amount of effort required for data annotation. This can be particularly interesting for tasks where unlabeled data are available in abundance, but label information comes only through significant effort or cost.
In pool-based active learning REF , the learning algorithm selects samples to be labeled from a pool of unlabeled data.
Rates of convergence in active learning
Annals of Statistics 2011, Vol. 39, No. 1, 333-361
Mathematics