document: Quantifying trends accurately despite classifier error and class imbalance This paper promotes a new task for supervised machine learning research : quantification - the pursuit of learning methods for accurately estimating the class distribution of a test set , with no concern for predictions on individual cases . A variant for cost quantification addresses the need to total up costs according to categories predicted by imperfect classifiers . These tasks cover a large and important family of applications that measure trends over time . The paper establishes a research methodology , and uses it to evaluate several proposed methods that involve selecting the classification threshold in a way that would spoil the accuracy of individual classifications . In empirical tests , Median Sweep methods show outstanding ability to estimate the class distribution , despite wide disparity in testing and training conditions . The paper addresses shifting class priors and costs , but not concept drift in general .
doc_bio_tags: O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O