document: Learning and making decisions when costs and probabilities are both unknown In many data mining domains , misclassification costs are different for different examples , in the same way that class membership probabilities are example-dependent . In these domains , both costs and probabilities are unknown for test examples , so both cost estimators and probability estimators must be learned . After discussing how to make optimal decisions given cost and probability estimates , we present decision tree and naive Bayesian learning methods for obtaining well-calibrated probability estimates . We then explain how to obtain unbiased estimators for example-dependent costs , taking into account the difficulty that in general , probabilities and costs are not independent random variables , and the training examples for which costs are known are not representative of all examples . The latter problem is called sample selection bias in econometrics . Our solution to it is based on Nobel prize-winning work due to the economist James Heckman . We show that the methods we propose perform better than MetaCost and all other known methods , in a comprehensive experimental comparison that uses the well-known , large , and challenging dataset from the KDD '98 data mining contest .
doc_bio_tags: O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O