Robust Learning with Highly Skewed Category Distributions Selen

Highly skewed category distributions are abundant in many real-world tasks in data mining, such as medical diagnosis (rare diseases), text categorization (rare topics), and fraud detection (when most transactions are legitimate). Under extreme class skew, most supervised learning algorithms tend to minimize loss by labeling every instance with the majority class(es), leading to poor recall on the minority class(es). However, true misclassification costs may be much greater when minority class instances are missed, e.g. a massive but rare fraud missed, or an uncommon life-threatening condition misclassified as benign. Hence, a means of detecting rare but consequential classes is required, and that is the topic of this dissertation. Whereas learning under extreme class skew has been previously investigated, many challenges remain: e.g. disjunctive majority classes and minority-majority class overlap. Prior research did not consider incorporating the structure of minority class into the learning process. In this dissertation, we address class imbalance under the compactness hypothesis, i.e. minority class forms one or more compact clusters in the feature space. Furthermore, we introduce several learning algorithms to address class imbalance under two other assumptions: disjunctive majority class and overlapping classes. We also propose new active learning strategies in cases when there are insufficient labeled minority class instances to learn accurate concept descriptions under highly-skewed settings. Our algorithms are based on a variety of methods/paradigms, including multiple kernel learning, maximum mean discrepancy, and cost-sensitive learning. We evaluate the new and baseline methods on several real-world datasets with a particular focus on the Womens’ Ischemic Syndrome Evaluation (WISE) dataset, to demonstrate a practical application in medical diagnosis. We show that when the assumptions are satisfied, leveraging the structure of classes, such as compact minority class, disjunctive majority class, leads to better prediction performance, quantified by the improvement in F-1 and AUC measures. Our empirical results reveal an improvement in F-1 as much as 28%. Acknowledgments First and foremost, I would like to thank my advisor, Jaime Carbonell, for his mentorship during this PhD. His unconditional support allowed me to freely pursuit my research interests, and his close guidance made sure that I was always on the right track. He taught me how to analyze a problem from different perspectives, and helped me become the independent researcher that I am. I always admired his endless knowledge and wisdom; he will always remain a role model of mine. I was extremely fortunate to have Anatole Gershman, Charles Elkan, and Jeff Schneider on my thesis committee. Their invaluable feedback and suggestions helped me greatly improve this thesis. I would like to thank Cédric Archambeau for his mentorship during my internship at Xerox Research Centre Europe. The insightful discussions we had helped me broaden my vision as a researcher. I would like to thank my former co-advisor Eric Xing for sparking my interest in bioinformatics, and for his guidance through the early bumps in PhD. Thanks to Judith Klein-Seetharaman for introducing me to challenging problems in data mining, and for her deep confidence that I will overcome them. During my time at Carnegie Mellon, I have had the opportunity to learn from various faculty, collaborators, post-docs, and fellow graduate students. I am grateful to everyone of them, they greatly influenced my development as a researcher. I have had a splendid time in Pittsburgh, I have been fortunate to have amazing friends, amazing housemates. This thesis would not be possible without their support. Last but not least, I would like to thank my family, especially my mother. She has always stressed the importance of education, “the gold bracelet” that I must have. This dissertation is dedicated to her.