Cost-sensitive learning with data acquisition
Authors: Shengli Sheng
Venue:  (2007)

Cost-sensitive learning is one of the most important topics in machine learning and data mining, and attracts high attentions in recent years. It takes costs into consideration in the learning process. Many real-world applications involve different types of costs. Two most important types of costs are identified as misclassification costs and data acquisition costs. Misclassification costs are an extension of error rate (or one minus accuracy) as different types of errors (such as false positive and false negative) can have different costs. Data acquisition costs reflect how expensive it is to acquire extra information for building more accurate learning models. This thesis addresses how to minimize the sum of the two different types of costs under three different types of data acquisition paradigms. In the first paradigm, we study attribute value acquisition of testing examples. For example, medical doctors always have to obtain a series of medical test results (i.e., attribute values in machine learning and data mining) before making diagnosis (classification) for their patients (i.e., testing examples in machine learning and data mining). We use cost-sensitive learning to model the diagnosis process, and propose three categories of test strategies to determine which medical tests and in what order to be performed, in order to minimize the sum of the misclassification costs and test costs, under the corresponding policies. To further investigate the effect of the delay cost of each test in diagnosis, we develop a novel multiple batch strategy. It can determine the size of each batch and the number of batches needed to minimize the sum of not only misclassification costs and test costs, but also delay costs. In the second paradigm, we study label acquisition in cost-sensitive learning. Label acquisition is well known as a specific form of active learning, which acquires the labels of unlabeled examples from domain experts (this involves the label acquisition cost). Most previous works of active learning do not unify the misclassification cost and label acquisition cost. We unify the two costs and develop a cost-sensitive active learning algorithm, which achieves a smaller total cost with fewer numbers of examples than other active learning algorithms. Finally, in many real-world applications, there are often not enough training examples available to build a learning model. Acquiring additional training examples is the first step. To avoid acquire more examples than necessary is a very important issue. We investigate and develop two training example acquisition strategies: complete attribute strategy and partial attribute strategy in the cost-sensitive learning. Partial attribute strategy further reduces the example acquisition cost by acquiring a partial example which contains values of a subset of attributes. The learning algorithm decides what attribute values are needed in the partial example acquisition. In summary, this dissertation proposes several algorithms to solve the problems of real-world applications, in terms of minimizing the total cost of misclassification and data acquisition. Keywords: Cost-Sensitive Learning, Data Acquisition, Active learning, Cost-Sensitive Active Learning, Classification, Testing Strategy, Attribute Value Acquisition, Label Acquisition, Example Acquisition