For many t ypes of machine learning algorithms, one can compute the statistically optimal" way to select training data. In this paper, we review how optimal data selection techniques have been used with feedforward neural networks. We then show h o w the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression. While the techniques for neural networks are computationally expensive and approximate, the techniques for mixtures of Gaussians and locally weighted regression are both e cient and accurate. Empirically, w e observe that the optimality criterion sharply decreases the number of training examples the learner needs in order to achieve good performance.
Cohn et al. REF describe how optimal data selection techniques can be applied to statistically-based learning algorithms like a mixture of Gaussians and locally weighted regression.
Active learning with statistical models
J. Artificial Intelligence Research
Computer Science