Semi-Supervised AUC Optimization Without Guessing Labels of Unlabeled Data

Semi-supervised learning, which aims to construct learners that automatically exploit the large amount of unlabeled data in addition to the limited labeled data, has been widely applied in many real-world applications. AUC is a well-known performance measure for a learner, and directly optimizing AUC may result in a better prediction performance. Thus, semi-supervised AUC optimization has drawn much attention. Existing semi-supervised AUC optimization methods exploit unlabeled data by explicitly or implicitly estimating the possible labels of the unlabeled data based on various distributional assumptions. However, these assumptions may be violated in many real-world applications, and estimating labels based on the violated assumption may lead to poor performance. In this paper, we argue that, in semi-supervised AUC optimization, it is unnecessary to guess the possible labels of the unlabeled data or prior probability based on any distributional assumptions. We analytically show that the AUC risk can be estimated unbiasedly by simply treating the unlabeled data as both positive and negative. Based on this finding, two semi-supervised AUC optimization methods named SAMULT and SAMPURA are proposed. Experimental results indicate that the proposed methods outperform the existing methods.  In many real-world applications, collecting a large amount of unlabeled data is relatively easy, while obtaining the labels for the collected data is rather expensive since much human effort and expertise is required for labeling. Semisupervised learning (Chapelle, Schlkopf, and Zien 2006; Zhu et al. 2009), aiming to construct learners that automatically exploit the large amount of unlabeled data in addition to the limited labeled data in the purpose of improving the learning performance, has drawn significant attention. Many semi-supervised learning methods have been proposed. To effectively exploit the unlabeled data, almost all of these methods elaborate to link the labeled data and the unlabeled data based on certain distributional assumption (Chapelle, Schlkopf, and Zien 2006), such as the cluster assumption, the manifold assumption, etc., and construct the learner by explicitly or implicitly estimating the labels of unlabeled instances.  AUC (area under ROC curve) (Hanley and McNeil 1982), which measures the probability of a randomly drawn positive instance being ranked before a randomly drawn negative instance, is a widely-used performance measure for a learner, especially when the data distribution exhibits certain imbalance. Directly optimizing the AUC during the learning procedure usually lead to a better prediction performance. Many studies are elaborated to show that AUC can be effectively and efficiently optimized (Herschtal and Raskutti 2004; Gao and Zhou 2015; Gao et al. 2013; Ying, Wen, and Lyu 2016). Some efforts have been devoted to AUC optimization in semi-supervised settings (Amini, Truong, and Goutte 2008; Fujino and Ueda 2016). These methods rely on the aforementioned distributional assumptions. However, those assumptions may be violated in real-world applications and the learner may be biased when explicitly or implicitly estimating the labels based on the violated assumptions, resulting in poor performance or even performance degradation (Cozman and Cohen 2002). Sakai, Niu, and Sugiyama (2017) proposed an unbiased semisupervised AUC optimization method based on positiveunlabeled learning. However, such a method requires an accurate estimation of the prior probability to reweight the unlabeled data, which is usually difficult especially when the number of labeled data is extremely small. In this paper, we argue that in semi-supervised AUC optimization problem, it is unnecessary to guess the possible labels of the unlabeled data or prior probability based on any distributional assumptions. We theoretically show that the AUC risk can be estimated unbiasedly by treating the unlabeled data as both positive and negative without any distributional assumptions. Such a theoretical finding enables us to address semisupervised AUC optimization problem by simply treating the unlabeled data as both positive and negative data, without designing specific strategies to identify the possible labels of the unlabeled data. Based on this theoretical finding, we propose two novel semi-supervised AUC optimization methods: SAMULT, a straightforward unbiased method by regarding the unlabeled data as both positive and negative data, and SAMPURA, an ensemble method by random partitioning the unlabeled data into pseudo-positive and pseudonegative sets to train base classifiers. The experimental reThe Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)