SlowGuess/NeurIPS_2023_OCR huggingface dataset:
PUE: Biased Positive-Unlabeled Learning Enhancement by Causal Inference
Xutao Wang, Hanting Chen, Tianyu Guo, Yunhe Wang*
Huawei Noah's Ark Lab.
{xutao.wang, chenhanting, tianyu.guo, yunhe.wang}@huawei.com,
Positive-Unlabeled (PU) learning aims to achieve high-accuracy binary classification with limited labeled positive examples and numerous unlabeled ones. Existing cost-sensitive-based methods often rely on strong assumptions that examples with an observed positive label were selected entirely at random. In fact, the uneven distribution of labels is prevalent in real-world PU problems, indicating that most actual positive and unlabeled data are subject to selection bias. In this paper, we propose a PU learning enhancement (PUe) algorithm based on causal inference theory, which employs normalized propensity scores and normalized inverse probability weighting (NIPW) techniques to reconstruct the loss function, thus obtaining a consistent, unbiased estimate of the classifier and enhancing the model's performance. Moreover, we investigate and propose a method for estimating propensity scores in deep learning using regularization techniques when the labeling mechanism is unknown. Our experiments on three benchmark datasets demonstrate the proposed PUe algorithm significantly improves the accuracy of classifiers on non-uniform label distribution datasets compared to advanced cost-sensitive PU methods. Codes are available at https://github.com/huawei-noah/Noah-research/tree/master/PUe and https://citee.com/mindspore/models/tree/master/research/cv/PUe.
In the era of big data, deep neural networks have achieved outstanding performance across various tasks, even surpassing human performance in many instances, particularly in traditional binary classification problems. The success of these deep neural networks often hinges on supervised learning using large quantities of labeled data. However, in reality, acquiring even binary labels can be challenging. For instance, in recommendation systems, users' multiple clicks on films may be considered as positive samples. Nonetheless, all other films cannot be assumed uninteresting and thus should not be treated as negative examples; instead, they should be regarded as unlabeled ones.
The same issue emerges in text classification, where it is typically more straightforward to define a partial set of positive samples. However, due to the vast diversity of negative samples, it becomes difficult or even impossible to describe a comprehensive set of negative samples that represent all content not included in the positive samples. Similar situations occur in medical diagnostics, malicious URL detection, and spam detection, where only a few labeled positives are available amidst a plethora of unlabeled data. This scenario is a variant of the classical binary classification setup, known as PU. In recent years, there has been a growing interest in this setting. Positive-Unlabeled (PU) learning primarily addresses the challenge of learning binary classifiers solely from positive samples and unlabeled data.
In previous research, numerous PU algorithms have been developed, with cost-sensitive PU learning emerging as a popular research direction. Methods such as [19, 18, 6] reweight positive and negative risks by hyper-parameters and minimize it. In addition, Self-PU [4] introduces self-supervision to nnPU through auxiliary tasks, including model calibration and distillation; ImbPU [23] oversamples and modifies sample weights to address unbalanced data. Dist-PU[24] corrects the negative preference of the classification model through prior information. PUSB [15] maintains the order-preserving assumption. However, these methods necessitate the assumption that their checked set is uniformly sampled from the population, or else the PU learning risk estimator ceases to be an unbiased or consistent estimator, otherwise resulting in reduced model accuracy.
In reality, the labeled set is often biased and does not conform to the Selected Completely At Random (SCAR) assumption [10, 3], which posits that the observed labeled examples are a random subset of the complete set of positive examples. Consequently, it is essential to relax the assumption of the labeled set and replace it with a more general assumption about the labeling mechanism: the probability of selecting positive examples to be labeled depends on their attribute values, known as the Selected At Random (SAR) assumption [10].
The work most closely related to us is the paper [7] by W Gerych et al. The focus of this article is to develop a method for generating identifiable propensity scores. Moreover, this paper proves that the general propensity score is unidentifiable in any standard PU data assumption except for the probability gap scenario. In this paper, two sets of assumptions are proposed to estimate propensity scores: one is Local Certainty and the other is Probabilistic Gap. Under the local certainty hypothesis, there is no overlap between the positive and negative hypotheses,  $p(x^{P}|y = 0) = 0$ . Under the Probabilistic Gap assumption, it is assumed that  $e = k * p(y = 1|x)$  is linear and there is an anchor point.
However, the above assumptions are too strict. We show in the appendix that the estimated propensity scores cannot be completely unbiased. To address the aforementioned problem, we propose a causal-inference-inspired
PU learning framework, termed PUs. Our approach aims to weight the original PU learning risk estimator under biased conditions, thereby obtaining a new unbiased or consistent PUs learning risk estimator, which is achieved through the inverse probability weighting technique of known normalized propensity scores. Given that propensity scores for samples are typically unknown, we apply regularization techniques to the deep learning classifier to estimate the propensity score of each sample in the marker set. An illustration of the proposed method is shown in Figure 1. We also provide an estimate of the generalization error bound for empirical risk minimization. Experimental results demonstrate that the proposed PUs framework can be applied to advanced PU algorithms, such as PUbN and Dist-PU, to enhance their performance in the SAR scenario. Our main contributions are summarized as follows:
Labeled Positive Samples Unlabeled Negative Samples
Tradition PU
Figure 1: Our method and traditional PU method classification diagram, our solution uses reweighting to make the classification plane more accurate.
Unlabeled Positive Samples Classification Boundary
Proposed PUE
- We relax our assumption of local determinism.
- We use a deep learning model to estimate propensity scores instead of a linear model.
- Our propensity score estimation method can be extended to the case of negative classes (e.g. PUbNe).
- Our new algorithm can simply be coupled with most cost-sensitive algorithms to improve performance. The PUe algorithm achieved superior results over existing solutions on standard benchmarks (MNIST, CIFAR10) and the Alzheimer's Disease Neuroimaging Initiative (ADNI) database.
In this section, we conduct a thorough review of existing PU algorithms, examining their limitations and challenges, particularly in the context of biased labeling scenarios that result in diminished accuracy. To address these issues, we introduce a novel PUs algorithm, specifically designed to
overcome the aforementioned limitations, thus enhancing the performance of PU algorithms under biased labeling conditions.
2.1 Review of PU classification
In standard PN classification, let  $\pmb{x} \in \mathbb{R}^d$  and  $y \in \{+1, -1\}$  be the input samples and its corresponding labels. We are given positive data and negative data that are sampled independently from  $p_P(\pmb{x}) = p(\pmb{x} | y = +1)$  and  $p_N(\pmb{x}) = p(\pmb{x} | y = -1)$  as  $\chi_P = \{\pmb{x}_i^P\}_{i=1}^{n_P}$  and  $\chi_N = \{\pmb{x}_i^N\}_{i=1}^{n_N}$ . We denote the class prior probability as  $\pi = p(y = 1)$ , where we follow the convention to assume  $\pi$  as known throughout the paper [16]. In fact, in the paper [14] by S Jain et al., it is pointed out that class prior can be estimated in the case of the proportion of classes that are biased in the labelling data. Let  $g: \mathbb{R}^d \to \mathbb{R}$  be the binary classifier and  $\theta$  be its parameter and the  $L: \mathbb{R} \times \{+1, -1\} \to \mathbb{R}_+$  be a loss function.
Let us denote by  $R_P(g, +1) = \mathbb{E}_{\boldsymbol{x} \sim p_P(\boldsymbol{x})}[L(g(\boldsymbol{x}, +1))]$ ,  $R_N(g, -1) = \mathbb{E}_{\boldsymbol{x} \sim p_N(\boldsymbol{x})}[L(g(\boldsymbol{x}, -1))]$ .  $R_P(g, +1)$  is the expected risk loss of a positive sample and  $R_N(g, -1)$  is the expected risk loss of a negative sample.
The risk of classifier  $g$ ,  $\hat{R}_{PN}(g)$  can be approximated by:
$$
\hat {R} _ {P N} (g) = \pi \hat {R} _ {P} (g, + 1) + (1 - \pi) \hat {R} _ {N} (g, - 1), \tag {1}
$$
where  $\hat{R}_P(g, + 1) = \frac{1}{n_P}\sum_{i = 1}^{n_P}L(g(\pmb{x}_i^P), + 1)$  and  $\hat{R}_N(g, - 1) = \frac{1}{n_N}\sum_{i = 1}^{n_N}L(g(\pmb{x}_i^N), - 1)$ . By minimizing  $\hat{R}_{PN}(g)$ . We obtain the ordinary empirical risk minimizer  $\hat{g}_{PN}$ .
In standard PU classification, instead of N data  $\chi_N$ , we only have a set of U samples drawn from the marginal density  $p(\pmb{x})$ ,  $\chi_U = \{\pmb{x}_i^U\}_{i}^{n_U}\sim p(\pmb{x})$ . Because of the fact  $p(\pmb {x}) = \pi p_P(\pmb {x}) + (1 - \pi)p_N(\pmb {x})$  we have the unbiased risk estimator, which key idea is to use the following equality:
$$
(1 - \pi) R _ {N} (g, - 1) = R _ {U} (g, - 1) - \pi R _ {P} (g, - 1). \tag {2}
$$
where  $R_U(g, -1) = \mathbb{E}_{x \sim p(\pmb{x})}[L(g(\pmb{x}), -1)]$  and  $R_P(g, -1) = \mathbb{E}_{x \sim p_P(\pmb{x})}[L(g(\pmb{x}), -1)]$ . As a result, we can approximate the unbiased classification risk (1) by:
$$
\hat {R} _ {u P U} (g) = \pi \hat {R} _ {P} (g, + 1) + \hat {R} _ {U} (g, - 1) - \pi \hat {R} _ {P} (g, - 1), \tag {3}
$$
where  $\hat{R}_P(g, -1) = \frac{1}{n_P} \sum_{i=1}^{n_P} L(g(\boldsymbol{x}_i^P), -1)$  and  $\hat{R}_U(g, -1) = \frac{1}{n_U} \sum_{i=1}^{n_U} L(g(\boldsymbol{x}_i^U), -1)$ . By minimizing  $\hat{R}_{PU}(g)$  we obtain the ordinary empirical risk minimizer  $\hat{g}_{PU}$ .
In theory, the risk  $(1 - \pi)R_{N}(g, - 1) = R_{U}(g, - 1) - \pi R_{P}(g, - 1)$  is also positive. However, if the model of  $g$  is too flexible,  $\hat{R}_{PU}(\hat{g}_{PU})$  indeed goes negative and the model severely overfits the training data. We can use the non-negative risk estimator for PU learning to alleviate overfitting[16]:
2.2 PU Classification in Biased Scenarios
PU learning mostly assumes that it runs under an ideal situation, that is, all labeled samples are selected completely at random from all positive samples, which is also called The Selected Completely At Random (SCAR).
Definition 1. SCAR(Selected Completely At Random [6]): Labeled examples are selected completely at random, independent from their attributes, from the positive distribution. The propensity score  $e(x)$ , which is the probability for selecting a positive example, is constant and equal to the label frequency:
$$
e (\boldsymbol {x}) = p (s = 1 | \boldsymbol {x}, y = 1) = p (s = 1 | y = 1) = c. \tag {4}
$$
However, many PU learning applications suffer from labeling bias. The Selected Completely At Random (SCAR) assumption does not conform to reality. For example, whether someone clicks
on a sponsored search ad is influenced by the position in which it is placed. Similarly, whether a patient with a disease will see a doctor depends on his socio-economic status and the severity of his symptoms. The Selected At Random (SAR) assumption, is the most general assumption about the labeling mechanism: the probability for selecting positive examples to be labeled depends on its attribute values [1].
Definition 2. SAR(Selected At Random [3]): The labeling mechanism depends on the values of the attributes of the example. However given the attribute values, it does not depend on the probability of the example being positive. Instead of assuming a constant probability for all positive examples to be labeled, it assumes that the probability is a function of a subset of an example's attributes:  $e(\boldsymbol{x}) = p(s = 1|\boldsymbol{x}, y = 1)$ .
In biased labeling scenarios, the accuracy of common PU algorithms decreases significantly. So we need to develop the PUe algorithm. The mathematical representation of equation is exactly the same as that of the propensity score in causal inference. Causal inference is the process of determining the actual and independent effects (effects) of a given phenomenon (cause) within a larger system. Causality inference can give evidence of causality model established by causal reasoning. Causal inference is therefore fundamentally a missingdata problem [13], which is very similar to the PU problem. The PU problem is essentially a classification problem of missing negative labels.
What we're actually using here is reverse causal reasoning. In general, we study the average causal effect of a treatment. However, we first complete the causal analysis of the processing (i.e. selection) and the label itself, and then use the known causal analysis, combined with the selected samples and the unselected samples to determine the distribution of positive samples. In this paper, accept processing means that the sample is selected for labeled. And the result is refers to whether the sample itself is a positive sample. Without noise in the detection, whether the sample itself is a positive sample is determined at the time when the sample is generated, and the behavior of selecting is performed after the sample is generated. In causal inference, what happens after the result is produced has no effect on the result, i.e. whether the sample is selected has a causal effect on the label value is 0. In essence, individual causality is also 0. Based on such a simple causality, we use the method of reweighting the sample to estimate the positive probability of the sample.
Since the causal effect of the labeling mechanism on the sample category is zero, known causality can be utilized to improve the performance in the case of biased PU problems. Rubin Causal Model of (RCM) [12] causal inference are extensively used within Statistics, Political Science, Economics, and Epidemiology for reasoning about causation. Due to the presence of confounders, estimating the average treatment effect directly using the average of observed treatment/control outcomes would include a spurious effect, which is called selection bias, brought by the confounders. A key concept in RCM is balancing score [11], reweighting samples by which is one of the most effective methods to overcome selection bias. Besides, propensity score is a special case of balancing score. The propensity score is defined as the conditional probability of treatment given background variables [22]. Propensity score is one of the core concepts of assignment mechanism in RCM. Propensity scores play a central role in observational studies of causality. Propensity scores can be used to reduce selection bias by equating groups based on these covariates, which is called inverse propensity weighting (IPW) [21].
In causal inference, inverse probability weighting (IPW) [21] is a standard statistical technique about propensity score for calculating statistics standardized to a pseudo-population different from that in which the data was collected. Study designs with a disparate sampling population and population of target inference are common in application. Weighting, when correctly applied, can potentially improve the efficiency and reduce the bias of unweighted estimators. In the context of causal inference and survey methodology, propensity scores are estimated (via methods such as logistic regression, random forests, or others), using some set of covariates. These propensity scores are then used as estimators for weights to be used with Inverse probability weighting methods. We incorporate the propensity score when learning in a PU setting by using the propensity scores to reweight the data. The examples are weighted with the inverse of their propensity score.
A crucial difference with the propensity score from causal inference is that our score is conditioned on the class being positive. IPW cannot be applied when working with positive and unlabeled data, because we have zero probability for labeling negative examples. But we can only do a weighting on positive. For each labeled example  $(x_{i},s = 1)$ , which has a propensity score  $e_i$ , there are expected
to be  $\frac{1}{e_i}$  positive examples. So we can use normalized inverse probability weighting on positive to estimate the unbiased expectations on positive.
$$
R _ {P} (g, + 1) = \mathbb {E} _ {\boldsymbol {x} \sim p _ {P} (\boldsymbol {x})} [ L (g (\boldsymbol {x}, + 1)) ] = \mathbb {E} _ {c \in (0, 1 ]} \left\{\mathbb {E} _ {\boldsymbol {x} \sim p _ {P} (\boldsymbol {x})} [ L (g (\boldsymbol {x}, + 1)) | e (x) = c ] \right\}. \tag {5}
$$
$$
\hat {R} _ {P} ^ {e} (g, + 1) = \sum_ {i = 1} ^ {n _ {P}} \frac {1}{\tilde {e} \left(\boldsymbol {x} _ {i} ^ {P}\right)} L \left(g \left(\boldsymbol {x} _ {i} ^ {P}\right), + 1\right). \tag {6}
$$
Similarly,  $R_{P}(g, - 1) = \mathbb{E}_{\boldsymbol {x}\sim p_{P}(\boldsymbol {x})}[L(g(\boldsymbol {x}, - 1))] = \mathbb{E}_{c\in (0,1)}\{\mathbb{E}_{\boldsymbol {x}\sim p_{P}(\boldsymbol {x})}[L(g(\boldsymbol {x}, - 1))|e(x) = c]\} .$
$$
\hat {R} _ {P} ^ {e} (g, - 1) = \sum_ {i = 1} ^ {n _ {P}} \frac {1}{\tilde {e} \left(\boldsymbol {x} _ {i} ^ {P}\right)} L \left(g \left(\boldsymbol {x} _ {i} ^ {P}\right), - 1\right), \tag {7}
$$
where  $e(\pmb{x}_i^P)$  is the true propensity scores of positive samples and  $\hat{e}(\pmb{x}_i^P)$  is the estimate of propensity scores, while  $\tilde{e}(\pmb{x}_i^P)$  and  $\tilde{\hat{e}}(\pmb{x}_i^P)$  is the normalized inverse probability weighting on positive samples. We can use the real normalized propensity score of positive samples  $\tilde{e}(\pmb{x}_i^P)$  to construct the PUE's loss function, or use the estimated normalized propensity score of positive samples  $\tilde{\hat{e}}(\pmb{x}_i^P)$  to construct the loss function. Since then, we use  $\tilde{e}(\pmb{x}_i^P)$  to represent them.
From the normalization, we can know the following formula:  $\sum_{i=1}^{n_P} \frac{1}{\hat{e}(\boldsymbol{x}_i^P)} = 1$ ,  $\sum_{i=1}^{n_P} \frac{1}{\hat{\tilde{e}}(\boldsymbol{x}_i^P)} = 1$ ,  $\frac{1}{\hat{e}(\boldsymbol{x}_i^P)} = \frac{\frac{1}{e(\boldsymbol{x}_i^P)}}{\sum_{j=1}^{n_P} \frac{1}{e(\boldsymbol{x}_j^P)}}$ , and  $\frac{1}{\hat{\tilde{e}}(\boldsymbol{x}_i^P)} = \frac{\frac{1}{\hat{e}(\boldsymbol{x}_i^P)}}{\sum_{j=1}^{n_P} \frac{1}{\hat{e}(\boldsymbol{x}_j^P)}}$ . Note,  $\sum_{j=1}^{n_P} \frac{1}{e(\boldsymbol{x}_j^P)} = N_P$ , where  $N_P$  means the number of all positive data.
$n_{P} = n_{L}$  is the number of labeled samples,  $N_{P}$  is the number of positive samples,  $n$  is the number of samples. It is clear that to be unbiased, we have to make  $E_{p(x|y=1)} = \frac{n_{P}}{N_{P}}$ , which indicates that  $\sum_{j=1}^{n} e(x) = n_{P}$ . And  $E_{p(x|s=1)} \frac{1}{e(x_j^P)} = N_{P}$ . Since  $e(x_i^N) > 0$ , existing propensity score estimation methods such as SAR-EM in [3] often underestimate the propensity score of positive samples, which will cause co-directional bias or produce an unidentifiable result (overfitting) that all labeled samples have a propensity score close to 1 and all unlabeled samples have a propensity score close to 0.
The normalization technique is used to make the deviations no longer in the same direction (co-directional error will produce worse results). Moreover, the maximum estimation error ratio of sample weights is reduced. The regularization technique is used to alleviate the overfitting problem, and the work does not assume a known propensity score or a positive sample distribution. After normalization is used,  $\theta A < \frac{\frac{1}{\hat{e}(x_i^L)}}{\sum_j \hat{e}(x_j^L)} < A / \theta$  does not have such co-direction error, where  $\frac{\frac{1}{e(x_i^L)}}{\sum_j e(x_j^L)} = A$  and  $\theta < P(y = 1|x_i^L) \leq 1$ .
To calculate the unbiased PU loss in the biased scenario, we must know the propensity score of each labeled sample. But this propensity score we usually don't know. So we consider two cases: (1) the true propensity scores we know and (2) we must estimate the propensity scores from data.
Case 1: Know the True Propensity Scores. In standard PN Classification, the true PN risk with real class  $y$  of size  $n$  can be formulated as follows:
$$
R _ {P N} (g | y) = \pi \hat {R} _ {P} (g, + 1) + (1 - \pi) \hat {R} _ {N} (g, - 1) = \frac {1}{n} \sum_ {i = 1} ^ {n} y _ {i} L (g (\boldsymbol {x} _ {i}), + 1) + (1 - y _ {i}) L (g (\boldsymbol {x} _ {i}), - 1). \tag {8}
$$
Definition 3 (IPW). Given the propensity scores  $e$  and PU labels, the inverse probability weighting estimator of  $\hat{R}_{PUe}(g)$  is:
$$
\hat {R} _ {P U e} (g) = \pi \hat {R} _ {P} ^ {e} (g, + 1) + \hat {R} _ {U} (g, - 1) - \pi \hat {R} _ {P} ^ {e} (g, - 1). \tag {9}
$$
Figure 2: Overview of the PUs framework. Our goal is to estimate the propensity score of the labeled samples and modify the sample weights using the normalized inverse probability weighting technique to obtain the loss function uniformly unbiased estimator.
This estimator is unbiased.
$$
\mathbb {E} \left[ \hat {R} _ {P U e} (g) \right] = R _ {P N} (g | y) = \frac {1}{n} \sum_ {i = 1} ^ {n} y _ {i} L \left(g \left(\boldsymbol {x} _ {i}\right), + 1\right) + \left(1 - y _ {i}\right) L \left(g \left(\boldsymbol {x} _ {i}\right), - 1\right). \tag {10}
$$
When the propensity score is known, not only the loss function reconstructed by the PUs algorithm is unbiased, the expectation of the PUs loss function is equal to the real PN loss function, but also the difference between the PUs loss function value and the real PN loss function value is probability bounded. However, in other PU algorithms where the label distribution is biased, the expectation of the loss function is biased [3].
Theorem 1: Inverse Probability Weighting Error Bound. For any predicted classes  $\hat{y}$  and the real class  $y$ , with probability  $1 - \eta$ , the inverse probability weighting estimator  $\hat{R}_{PUE}(g)$  does not differ from the true  $PN$  loss function  $R_{PN}(g|y)$  more than:  $|R_{PN}(g|y) - \hat{R}_{PUE}(g)| \leq \sqrt{\frac{L_{\max}^2 \ln \frac{2}{\eta}}{2n}}$ . with  $L_{\max}$  is the maximum absolute value of cost function  $L(g, y)$ .
Theorem 2: Error Bounds of Common PU Algorithm in Biased Scenarios. For any predicted classes  $\hat{y}$  and the real class  $y$ , with probability  $1 - \eta$ , the inverse probability weighting estimator  $\hat{R}_{PU}(g)$  does not differ from the true PN loss function  $R_{PN}(g|y)$  more than:
$$
\left| R _ {P N} (g | y) - \hat {R} _ {P U} (g) \right| \leq 2 \pi L _ {\max } * \frac {n _ {u}}{n _ {u} + n _ {p}} + \sqrt {\frac {L _ {\max } ^ {2} \ln \frac {2}{\eta}}{2 n}}.
$$
In the biased scenario, the error bound of PU algorithm is larger than that of PUe algorithm.
Case 2: Estimate the Propensity Scores from Data. But in practice, the probability of the sample being labeled is usually unknown. For example, in ad recommendation, we do not actually know the probability of everyone clicking on the ad, that is, the propensity score is unknown. So in order to get the loss function of PUs algorithm for the unknown label distribution, we must have an estimate  $\hat{e}$  of the propensity score.
$$
\hat {R} _ {P U \hat {e}} (g) = \pi \hat {R} _ {P} ^ {\hat {e}} (g, + 1) + \hat {R} _ {U} (g, - 1) - \pi \hat {R} _ {P} ^ {\hat {e}} (g, - 1). \tag {11}
$$
In this case, the bias of the inverse probability weighting estimator is:
$$
\operatorname {b i a s} \left(\hat {R} _ {P U \hat {e}} (g)\right) = \pi \sum_ {i = 1} ^ {n} y _ {i} \left\{\left(\frac {1}{N _ {P}} - \frac {\tilde {e} \left(\boldsymbol {x} _ {i}\right)}{N _ {P} \tilde {\hat {e}} \left(\boldsymbol {x} _ {i}\right)}\right) * [ L (g (\boldsymbol {x} _ {i}), + 1) - L (g (\boldsymbol {x} _ {i}), - 1) ] \right\}. \tag {12}
$$