Deep Positive Unlabeled Learning with a Sequential Bias
Authors: Walter Gerych
Thomas Hartvigsen
Luke Buquicchio
Kavin Chandrasekaran
Hamid Mansoor
Abdulaziz alajaji
Venue:  (2021)

Abstract: For many domains, from video stream analytics to human activity recognition, only weakly-labeled datasets are available.
Worse yet, the given labels are often assigned sequentially, resulting in sequential bias. Current Positive Unlabeled (PU) classifiers, a state-of-the-art family of robust semi-supervised methods, are ineffective under sequential bias. In this work, we  propose DeepSPU, the first method to address this sequential bias problem. DeepSPU tackles the two interdependent subproblems of learning both the latent labeling process and the true class likelihoods within one architecture. We achieve this by developing a novel iterative learning strategy aided by theoretically-justified cost terms to avoid collapsing into a naive classifier. Our experimental studies demonstrate that DeepSPU outperforms state-of-the-art methods by over 10% on diverse real-world datasets.

Under review as a conference paper at ICLR 2021
DEEP POSITIVE UNLABELED LEARNING WITH A SEQUENTIAL BIAS
Anonymous authors Paper under double-blind review
ABSTRACT
For many domains, from video stream analytics to human activity recognition, only weakly-labeled datasets are available. Worse yet, the given labels are often assigned sequentially, resulting in sequential bias. Current Positive Unlabeled (PU) classifiers, state-of-the-are family of robust semi-supervised methods, are ineffective under sequential bias. In this work, we propose DeepSPU, the first method to address this sequential bias problem. DeepSPU tackles the two interde- pendent subproblems of learning both the latent labeling process and the true class likelihoods within one architecture. We achieve this by developing novel itera- tive learning strategy aided by theoreticalll-justine into a naive classifier. Our experimental studies demonstrate that DeepSPU out- performs state-of -f--th-----rrrr-- methods by over 10% on diverse real-world datasets.
cost terms to avoid collapsing
INTRODUCTION
Motivation. State-of-the-art approaches for learning from data with only only incomplete positive labels require an accurate estimation of the likelihood that any given positive instance receives label, known as the propensity score. However, all existing approaches overlook the fact that the annotations given for sequential data are often clustered together, and thus the likelihood that given instance is labeled is dependent on the labels of the surrounding instances. We refer to this as sequential bias. Overlooking this sequential bias results in an incorrect propensity score and significantly reduced classification performance. Ours is the first work to make this observation and we propose the first solution to this open problem.
Human Activity Recognition (HAR) is prime example of sequential bias in data. To collect HAR data, subjects are asked to report their activities while wearing mobile sensors. As study-length in- creases (collection may take many days), participants leave many activities unlabeled. Additionally, wearable sensors record data rapidly so large blocks of time get labeled consecutively, also creating sequential bias. Many more applications, such intrusion detection from video illness prediction from medical records, have similar sequentiallly-abblee data are are susceptible to sequential bias (Rodriguez-Moreno et al. [2019] [Schekkermann et al. (0188). This is crucial issue as existing meth- ods show drastically reduced accuracy when sequential bias is not accounted for (as demonstrated in our Experimental Results).
State-of-the-Art. Positive Unlabeled (PU) classifiers are a family of semi-supervised methods that learn from incompletely-labeie data without requiring any labeled negative examples (Bekker &
Davis 2020
Elkan & Noto 2008 Li & Liu 2005 Hsich et al. 2015 Du Plessis et al. 2015 Kiryo al., 2017; Bekker Davis 2018a| Kato et al. 2019). This is a key strength of PU methods
because representative negative examples, typically required by semi-supervised methods, are often not feasible to acquire. For instance, in the HAR example, there are infinitely many activities that
an individual is not performing at any given time. Consequentially, participants are only expected to provide some positive labels for their activities (Vaizman et al., (017).
Unfortunately, existing PU methods make unrealistically restrictive simplifying assumptions on how the labels were applied. Specifically, they either assume that there is no bias in the labeling process (the probability of sample being unlabeled positive instance is uniform) (Elkan & Noto 2008 Du Plessis et al., 2013) Kiryo et al. 2017) or otherwise only depends on the local attributes of each instance (Bekker & Davis 2018a Kato et al. 2019). This means that existing methods do not

---

Under review as a conference paper at ICLR 2021
model sequential bias. And, demonstrate in experiments, these methods are significantly negatively impacted when a sequential bias is present.
Problem Description and Technical Challenges. Given dataset of sequences, predict the true class likelihoods of each instance within the given sequence given a subset of labeled positive instances during training. In particular, focus on the difficult case where labels have been assigned with a sequential bias, with sequential bias defined as the case where the likelihood that a positive instance is labeled varies depending on whether its neighboring instances were labeled.
goal is to
This problem is challenging due to two difficult interdependent subproblems. First, we have the dependency problem: if had model of the latent labeling process (which we call the propen- sity model) that allowed us to identify the true unlabeled positive instances, then we could use this propensity model to train a classifier to produce the true class likelihoods. However, we need these same true class likelihoods in order to train the propensity model causing a cyclic dependency. Sec- ond, standard maximum likelihood estimation inherently assumes all instances are labeled, leading to naive classifier in the presence of labeling bias. To capture unlabeled positive instances, PU classifier must instead predict an appropriate number of positive instances without simply assuming all positives instances are labeled.
Our Approach: DeepSPU. We propose Deep Sequential PU (DeepSPU), which is the first Posi- tive Unlabeled method to use propensity score model that predicts the likelihood that any given positive instance is labeled while taking sequential bias into account. The propensity score allows train classifier network given only partially labeled data. We achieve this by developing novel learning method that overcomes the cyclic dependency problem by iteratively learning the propensity score model and the classifier using weakly-labeled data. Further, we introduce the two novel PU cost terms: the Prior-Matching Costs (PMC) and the Obserration-Matchingg Costs (OMC), which prohibit the propensity model and classifier from collapsing into incorrect naive solutions.
Contributions. The main contributions of our work are:
We identify sequential bias, a labeling pattern characteristic of many real-world labeling pro- cesses, and demonstrate how ignoring this bias significantly impacts the performance state-of- the-art PU classifiers.
We propose the first learning strategy to minimize the bias incurred from sequentially biased PU data. Namely, we propose an iterative learning strategy and design two novel PU cost terms, Prior- Matching and Observation-Matthing which prohibit collapse into certain incorrect adversarial solutions, as justified through theoretical analysis.
We develop DeepSPU, the first model to mitigate sequential bias. DeepSPU uses the aforemen- tioned learning strategy to jointly estimate the two interdependent latent variables: the propensity score and the true class probabilities without any direct supervision for either learning task.
RELATED WORK
PU classifiers, a special type of semi-supervised models, learn from weakly-labeled training data without requiring any labeled negative examples (Bekker & Davis, 2020). In contrast, semi- supervised methods lean heavily on having access both labeled positive and labeled negatives to learn from weakly supervised data, making them less robust than PU alternatives (Van Engelen & Hoos, 2020). PU methods are also more robust than similar family of classifiers known as One Class (OC) classifiers, as general OC classifiers do not make use of unlabeled data during training Khan Madden 2009).
There are many approaches to PU learning, such as re-weighting predictions (Zhang & Lee 2005 Elkan & Noto 2008), iteratively identifying reliable examples (Ienco & Pensal 2016), and most notably risk minimization (Northcutt et al. 2017 Du Plessis al. 2015 Kiryo et al. 2017). How- ever, all these ofate-of-thh-ar approaches share the dften-unrealistee assumption that no bias exists in the labeling process, sequential or otherwise. When bias is present, these methods are sus- ceptible to learning skewed decision boundaries and thus are prone to making biased and incorrect classifications (Bekker & Davis 2020
2

---

Under review as a conference paper at ICLR 2021
A few recent PU methods have begun to model bias in the labeling process, though none have captured sequential bias. One approach assumes the likelihood that a given true positive instance is labeled depends solely on its distance from the negative distribution (Kato et al. 2019), while others explore the case when this labeling likelihood depends on the general position of the positive instance in the feature space (Bekker & Davis] 2018a). Our Dee method extends beyond these methods by coping with both this feature-level bias and the previously-overlooke sequential bias.
3 PROBLEM SETTING
3.1 POSITIVE UNLABELED LEARNING
Positive Unlabeled (PU) learning is the task of training a classifier to predict the true class of each instance given set of mostly unlabeled data and only some labeled positive instances (Bekker & Davis 2020). Solving this problem requires modeling the likelihood that any given true positive instance is labeled. This labeling likelihood is referred to the propensity score (Bekker & Davis, 2018a). Estimating the propensity score is nof the same as predicting whether an instance is labeled, which is often relatively easy. This is because estimating the propensity score corresponds to deter- mining the likelihood that an instance is labeled conditioned on the instance being a true positive. We cannot directly estimate this value from the data as we do not know whether or not any given unlabeled instance is a true positive.
be dataset D of N sequence pairs, where X a sequence of real values and ( is a sequence of binary label indicators, |x |c(,) For readability, we drop the superscript and describe our approach in terms of one sequence. Let X = (I1, be a sequence of 7 real values (which we refer to as sequence of instances), and L= 62, er) be an associated sequence of label indicators such that & otherwise. Additionally, for each feature-label sequence pair there is an unobserved binary true class sequence, Y (y/1 1/2.... WT), y, E (0. representing the underlying classes of the instance. In addition to y/ being unavailable during training, Pr(s, 1 1) 1 as assume there are no labels for negative instances, while Pr(e, = 1 = 1) * O as we assume not all positive instances are labeled.
Formally, let D
FT)
if x; is labeled positive, and
We consider both feature-lever and sequential biases in the labeling process. Feature-level bias assumes the propensity score for a positive instance depends local features of the instance and thus Pr(& = 1y: = 1) * Pr(e: = 1/y: = 1, 1,4:). Sequential bias assumes the propensity score of a positive instance
also depend on the label status of preceding instances, f1.1-1, and thus Pr(e, = 1|8 = 1) * (r(,, = 1|m = 1, T1s. (1.-1). We call propensity score that captures sequential and feature-level bias sequential propensity score q, Pr(e, 1ly = 11 114
Our goal is to train a classifier ge(-) with parameters 0 to solve the binary classification problem, such that go (X) = Pry (Y During training, only the features X and label status indicator C are observed while the true class Y is unobserved. Notation used summarized in Appendix A.6
3.2 BACKGROUND ON EMPIRICAL PU RISK MINIMIZATION
In standard positive-negative binary classification, the risk of a classifier is given as:
R(ge) = TEE + (1 (C=
where Ep and Ep are the distribution over the positive and negative instances respectively, and C++ is the loss incurred from predicting go(x)) given that the true instance is positive while C' is the loss incurred from predicting go (x) when the true instance is negative. Several recent works have focused on reformulating the above risk into uunaaiie-unllabeeed risk that takes expectations over the labelled and unlabeled distributions rather than the positive and negative distributions, as the latter two distributions can not be estimated directly from PU data (Kiryo al., 2017] Du Plessis et al. 2015). Directly minimizing the empirical PU risk has been successful in the unbiased SCAR setting (Du Plessis et al. 2015 Kiryo et al. 2017)
Additionally, Bekker et al. proposed a PU risk for the SAR setting, where there is bias in the labeling that is a function of the feature values of each instance (Bekker & Davis 2018a). For classifier go
3

---

Under review as a conference paper at ICLR 2021
this risk is given as:
R(go) =xCBE c+ (+(xa)) + (1
1 [e(x) + -1-na [e) (g0(())))
(1 (1-c()) c(x)
c-((()))
(o(x))
(1)
where e(x) is the propensity score and represents the probability that positive instance is labeled. In the formulation proposed by Bekker et al. the propensity score is only function of the local feature values x.
A classifier that directly minimizes Bekker's PU risk has not been proposed for the case where both the propensity core e(x) and the posterior Py (Y are unknown. This is due to the difficulty of estimating both the propensity score and the posterior jointly, which arises from the fact that in the above risk the perceived performance of the estimated posterior ge (and thus gradients incurred) is based on the estimate of the propensity score and vice versa. Ergo, poor estimate of the propensity score can lead the classifier ge to a poor solution while a poor classifier go leads to an inaccurate estimator of the propensity score.
As described in the following section our proposed DeepSPU method does succeed in learning the propensity score and go jointly by minimizing the PU risk directly. DeepSPU overcomes the aforementioned difficulty of training the two latent variables through risk minimization by using a novel iterative training algorithm coupled with additional regularization terms.
METHODOLOGGY
4.1
OVERVIEW
We now describe a general estimation procedure for sequentially biased PU data, along with spe- cific model for learning in this setting.
4.2 SEQUENTIAL BIAS LEARNING STRATEGY
There are 2 major components to our proposed learning strategy. First, we employ an innovative Iterative Learning Strategy, iteratively training classifier model and a propensity score model by minimizing the positive unlabeled risk (Bekker & Davis, 2020). This achieved without explicit feedback of the two estimated latent variables. Second, we design two novel PU cost (regularization) terms that are employed during the iterative training. These cost terms prevent the networks from converging naive solutions which minimize the PU risk but do not result in correct probability distributions for the latent target variables.
4.2.1 ITERATIVE LEARNING STRATEGY
As stated in section 3.2
typical classifiers for fulle-labeled data trained to minimize the expected
value of a loss function C, known as the risk. If the propensity score is known then the standard risk R can be expressed in terms of expectations over only positive and unlabeled distributions, instead of the positive and negative distributions (Du Plessis et al. 2015 Bekker & Davis (018a)). Thus, train our model we express the empirical positive unlabeled risk, RpU, in terms of our novel sequential propensity score as:
Rpuls.glX, T 1 (E < (-ot(((a))) +
+ (0
2116=1
2/6:=0
where C+ is the loss incurred from predicting g(x) assuming that the true class is positive, C- the loss incurred from predicting g(x) assuming the true class is negative, and qi = Pr(é, = 1 I1i: (1--1) is the propensity minimizing the above equation will correspond to minimizing the true risk. This means that in effect can learn the same classifier that would have found if we had been given fully labeled data.
of the i-th instance. If our propensity scores accurate, then
4

---

Under review as a conference paper at ICLR 2021
Theorem 1. Ifq, Pr(é = 1/ 1, T1.. (1-1)) = 1: then Rp q L) is unbiased estimation of the true risk R(g(Y).
A proof of this theorem is given in Section A.4 of the Appendix. As stated, Theorem indicates that we can train a classifier on positive and unlabeled data if we have accurate propensity scores. However, the propensity score is in general an unobserved observed variable to use as target during training, and thus there no straightforward way to minimize the PU risk. The naive approach would be to simply learn the parameters of the classifier and propensity model & simultaneously by minimizing Equation However, there is no guarantee that Equation is is an unbiased estimate of the true risk if the estimated propensity scores are incorrect (and thus no indication that this would yield a good classification model). We propose to overcome this challenge with a novel Iterative Learning Strategy, outlined below.
We begin training the a classification model and propensity score model by initializing the propensity scores with "good" estimates, acquired by assuming the labeling likelihood e = Pr(e, = 11y; 1) is constant for all positive instances. c is easily computed given a prior on the class π = Pr(y 1) (Bekker& & Davis 2020), and has been shown to be successfully estimated from the initial data set Jain et al. (016)) We show how to derive c from π in the appendix.
Next, we train a propensity model and a classification model iteratively in separate interleaved stages. That is, at each stage of training, either the parameters of the propensity model or clas- sification model are updated independently from each other.
4.2.2 REGULARIZING COST TERMS
It is possible to minimize Equation 2 by naively setting gi 1 A i and g(zz)) = L, In this case, the classifier does not predict the instance's 's true class, and instead erroneously predicts whether or not an instance is labeled (Bekker & Davis 2018a). To avoid this adversarial solution, we propose two cost terms. First, we add the Prior-Matchun Matching Costs (PMC). which drives the percentage of predicted positives to match the percentage of true positive
expect according our prior. The PMC is given by the KL divergence between the unconditioned distribution of predicted positives and Bern(r), the Bernoulli distribution parameterized by the true class prior π: PMC(X) = (L = [[o)((Ber((TT)
(3)
Thus, the number of predicted positives are driven to match the expected number of true positives. By definition, the PMC will be larger than the number of labeled instances, and thus Equation T1 will incur high penalty cost for the adversarial solution.
Additionally, if the estimated propensity scores match the true propensity scores, then our propensity score multiplied by the class probability of the i-th instance will equal the probability that the i-th instance is labeled, as stated in Theorem 2 This informs our next cost term.
1) is the sequential propensity score of the i-th instance, then Pr(f. = 1 1/2, 1-1 = 9s Pr(s)(z.).
Theorem 2. If qi = Pr(e, = 1|21:3 f15-11,
The proof is given in Appendix A.5
In short, we propose explicitly encourage the product of the estimated propensity scores and predicted class probabilities to match the observed labels. We accomplish this by adding the Binary Cross Entropy (BCE) between them next cost term. We refer to this the Abservation-Matchiie Costs (OMC), as it requires our propensity score and class predictions to match the observed data:
OMC(X) = = - EΣ ( log Vi
(1) 9:
(4)
j=1
We thus propose final combined cost function sequentially biased PU data as follows:
L( θ) = I JQ, (ge) + (1 -) Joo (Q+), where / is an indicator variable that equals if we' we're updating the parameters of the classification model ge and is 0 otherwise, and J is the cost term defined as:
(b)
Rp X PMC A₂OMC,
(5)
with A and A2 being weights on the corresponding cost terms and (b) corresponding to the above equation function of b while is held constant.
5

---

Under review as a conference paper at ICLR 2021
4.3 THE DEEPSPU MODEL FOR SEQUENTIALLY BIASED DATA
We now propose specific model to minimize Equation [ Our model, DeepSPU, consists of three sub-networks: 1) the Representation Network, which learns a robust representation of the input data, 2) the Sequential Propensity Network, which models the likelihood that an instance is labeled given that it positive, and 3) the Classification Network, which models the likelihood that a given instance belongs to the positive class. The Sequential Propensity Network, used only during the training stage, is crucial in the training of the Classification Network as it allows us to train the Classification Network given only weakly-labeled training data. After training, the Classification Network can be deployed without the Sequential Propensity Network to predict the true class of new instances. Detailed pseudo-code is available in the appendix.
4.3.1 REPRESENTATION NETWORK
The Sequential Propensity Network and the Classification Network both share a common base representation of the input data. This shared representation is modeled by a recurrent neu- ral network (RNN) Ba with parameters S2 such that Bg takes the sequence of input data X (2,, ITT and maps each element of the sequence to a corresponding latent repre- sentation = (h1, hu..... ,.)) Each latent representation h2 is given by h; = FI; h,-1), with the specific form of Fh determined by the choice of RNN network. Our implementation of Deep- SPU uses a Gated Recurrent Unit (GRU) for the RNN (Cho et al. 2014), though the same principles apply other RNN architectures.
4.4 SEQUENTIAL PROPENSITY NETWORK
The aptly-named Sequential Propensity Network models the propensity score conditioned on the local feature values of each instance and the feature values of all preceding instances, along with the label indicators of all preceding instances. The first component of this sub-network another GRU that for each label & produces latent representation s; conditioned on all previous labels. In effect, 8; summarizes the label subsequence & to E, into one latent variable. This GRU is coupled with feed-forward network (FFN) that takes in the latent label representation 8:-1, the previous label indicator 4-1, and the representation the input features learned by the Representation Network h, in order produce the propensity score q; of the ith instance. With & representing the Iearnable parameters of the Sequential Propensity Network Qo, Qo models the sequential propensity score Q (he, 88-1, 9: Pr(f, 1|r1; 11--1).
4.5 CLASSIFICATION NETWORK
The shared hidden representation h; also passed as input into the DeepSPU classifier subnetwork, which models the positive class probability given the features of the input data, Our classifier model ge (x;) P( Q) with parameters is given by: (x) (r,)): O), where F, is a fully connected network with parameters O. The predicted class value for an instance x; is given by round(g((x))) where round(z)
= > 0.5 and otherwise. We chose 0.5 as the cutoff as ge (z:) represents the probability that y/ 1.
EXPERIMENTS
5.1 DATA AND EXPERIMENTAL DETAILS
Datasets. We evaluate our models and a relevant set of baselines on several real-world sequential human activity recognition datasets: UCI HAR (Anguita et al. (013)). Older Healthy (OH) Torres al. 2013), and Ext ExtesSooy (ES) Vaizman al. 2017). ExtraSensory was subsampled at one reading for every 10 minutes of collected data due to the size of the dataset.
HAR
Due to extreme class imbalance, we combine the multiple classes into one for some of the datasets.
https:/archive.ics..uu
wwwpwiiiiiii...iii........ http://extrrsensoryaued.d.o

---

Under review as a conference paper at ICLR 2021
Royains Gracker
Predised Cless Bequaince
Instantisted Classification Network
(Frzeen]
Incoetaaee Model
Festure Sequence
Trainable Classification Network
Yea
Network
Classifier Training Stage?
Loss
Trainsble
Label Sequence
Propensity Network
Propensity Model
Yea
Fropensity Sequence
Figure 1: An overview of the DeepSPU learning process. The Representation Network learns hidden representation of the input sequence that fed to both the Classification Network and the Sequential Propensity Network. Networks are alternately trained iteratively, with one being frozen while the other is updated.
We combine UCT HAR into 3 classes: one representing Walking-related activities, one that com- bines Going Up and Down Stairs, and one that combines stationary activities. We likewise combine stationary activities in OH HAR. We also evaluate on RealityComnon
(RC Flu) (Madan al. 201T), an in-the-wild health dataset, where the task is classify whether the individual is experiencing flu symptoms given day using mobile sensor data.
Compared methods. We compare DeepSPU to the following state-of- saute-of-the-ar the--he-rtttt methods:
Posirive-Negative (PN) Classifier (hoo et al. 2014): As baseline, we adopt standard binary positive-negative classifier that treats all unlabeled instances as negatives. The model otherwise has the same structure the DeepSPU classification network.
uPU Duu Plessis et al. 2015): uPU is a recent highly-influenticn approach to training deep net- works PU data. uPU's convex approach minimizes the empirical PU risk. However, this approach assumes the propensity score is constant.
nnPU Kiryo al. 2017): Similarly uPU, anPU also assumes that the propensity is constant to minmiiee the empirical PU risk. Additionally, nnPU clips the risk of the unlabeled instances O to avoid infinitely negative risks and is thus liable overfit during training.
SAR-EM [Bekker & Davis 2018a1: SAR-EM is the leading PU method for learning under feature-level biased labeling. SAR-EM jointly trains a classifier and propensity network using an Expectation Expectation-Maximizaa Maximization algorithm, such that the propensity network can handle feature-level but not sequential bias.
We do not compare against any general semi-supervised methods. This is not oversight. As per our problem definition, we assume we have no labeled negative examples. This means general semi-supervised methods are inapplicable. (Van Engelen & Hoos 2020).
Evaluation metric. To measure the performance of our compared methods, we use balanced accu-
TN+FF BA is similar to the standard accuracy metric, but accounts for class imbalance: BA of 0.5 is achieved by a naive classifier, regardless of class imbalance. An optimal classifier will achieve BA 1.0.
racy (BA) (Brodersen al. 2010), which is defined
TP4FN TP
TN
Implementation details. We use an 70%/10%//20% train/validation/test split for each dataset. The base classifier for all methods a GRU with a 10-dimensional hidden layer. A 1-layer feed-forward neural network uses the GRU's latent representation as input for each instance and serves as the classifier. DeepSPU has an additional I-layer GRU for the propensity network. Each method trained until convergence (200 epochs). All methods are implemented in PyTorch and are publicly available
s hwwp:///...omoo....o.cc......./e

---

Under review as a conference paper at ICLR 2021
Datasets:
ES Lyying
ES Sitting
ES Walking
ES Sleeping
RealiyCommons
PN:
0.60
0.62
0.66
uPU:
0.70
0.62
0.65
0.70
0.60
nnPU:
0.70
0.62
0.66
0.72
0.62
SAR EM:
0.66
0.63
0.58
0.69
0.54
DeepSPU:
0.77
0.66
0.68
0.78
0.66
Table 1: Performance of DeepSPU vs compared methods on naturally weakly labeled real-world
datasets experiencing sequential bias. DeepSPU outperforms the others. Results reported as Bal- anced Accuracy. Higher is better.
Method:
PN
uPU
noPU
SAR-EM
DeepSPU
% Labeled:
|
15
|
15
5
10
OH Stationary
0.50
0.50
0.51
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
053
0,53
0.55
OH Ambulating
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.51
0.52
0.50
0.50
0.50
0.50
0.52
0.54
UCI Walking
0.50
0.50
0.51
0.53
0.50
0.51
0.52
0.54
0.58
UCT Stairs
0,50
0.50
0.50
0.59
0.62
0.72
0.64
0.65
0.67
0.52
0.53
0,58
0.79
0.80
0.86
UCI Stationary
0.50
0.51
0.53
0.91
0.91
0.93
0.91
0.91
0.94
0.79
0.79
0.85
0.96
0.96
0.97
ES Walking
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.51
0.52
0.54
ES Sitting
0.50
0.50
0.50
0.50
0.50
0.52
0.50
0.50
0,52
0.53
0.53
0.54
0.55
0.58
0.60
ES Sleeping
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.51
0.50
0.50
0.50
0.58
0.65
0.66
ES Lying
0.50
0.50
0.51
0.50
0.50
0.51
0.50
0.51
0.53
0.50
0.50
0.50
0.62
0.69
0.72
Table 2: Classification performance for various levels of labeling on the Older Healthy, UCI HAR, and ExtraSensory HAR datasets. Results reported as Balanced Accuracy (higher is better).
5.2 EXPERIMENTAL. STUDY ON CLASSIFYING NATURALLY-UNIAFFIIE DATA
First, we demonstrate SPepSPU's ability to classify data that naturally exhibits sequential bias. This can be clearly shown for the Ext raSnsorryy and Real tyCommons datasets, because both were collected "in the wild" with study participants labeling their own data sequentially. Thus they are thus naturally weakly labeled. Further, due to the study design it unclear if an instance that is not labeled positive is in fact negative or an unlabeled positive (Chang et al.) 2017] Madan et al.) 2011), thus fitting the positive unlabeled problem description. For this experiment, we estimate the class prior for nnPU and DeepSPU using TIEE (Bekker & Davis (2018b). As shown in Table [ Deep- SPU significantly outperforms the state-of-the-ard methods for both datasets. This demonstrates that tackling sequential bias inherent in these real-world datasets is impactful. Additionally, the base- line underperforms the PU methods in almost all cases, highlighting the importance of leveraging PU classifiers for these weakly-labeled real-world problems. Overall, these results confirm that the pervasive problem of sequential bias in labeling can be mitigated by DeepSPU.
5.3 EXPERIMENTAL STUDY ON LEARNING FROM VARIOUS LEVELS OF LIMITED LABELS
In practice, the percentage of labeled positives available during training will vary from dataset to dataset. Thus, we perform this next experiment to study the impact of the proportion of labeling on each method's performance. In line with much of the recent PU work (Bekker & Davis 2020 2018a Kiryo et al.) 2017). we achieve this by removing subsets of labels from each dataset prior training. We range label availability from 5% 15% of all positive instances, and remove the labels from all negative instances. To encourage sequential bias, unlabeling is done sequentially: A binary Markov Chain decides whether or not to remove the label for each instance in turn. The likelihood the Markov Chain switches from "labeling" to "unlabeling" states is varied according to the desired level of unlabeling. Details of this unlabeling process are in Appendix A.7 As shown in T DeepSPU significantly outperforms all other methods for every level of unlabeling. This demonstrates that even with very few labels, modeling sequential bias leads to significant improve- ments in performance. As expected, the Positive Negative performs the worst, as it does not account for unlabeled positives. Surprisingly, SAR-EM is outperformed by nnPU, despite nnPU assuming that no bias arises in the labeling process. This may be explained by mmPU's natural aversion to overfitting (Kiryo et al.) 2017). while SAR-EM may learn spurious relationships in the labels while mistakenly modeling sequential bias feature-level bias.
8

---

Under review as a conference paper at ICLR 2021
Balanced Accuracy
Accuracy
0.65
0.65
0.60
0.60
0.55
0.50
DeepSPU
No OMC
No PMC No lterative Training
0.50
Estimated Prior
(a) Ablation study
(b) Parameter study
Figure 2: a) The ablation study of DeepSPU indicates that each of its major components significantly improves its performance. b) Performance of DeepSPU varying values for the class prior т estimate, with the true prior being about 0.36.
5.4 ABLATION STUDY
DeepSPU minimizes PU risk using three components: The iterative training algorithm, the inservation-matccing cost (OMC), and the prior-matching cost (PMC). In this experiment, we demonstrate the necessity of each component in ablation study the ExtraSensory Sitting dataset. Removing any of the three components results significantly lower classifi- cation accuracy. Specifically, removing the iterative training strategy impacts the performance most significantly. This expected, as without the iterative training strategy the cost function is likely be biased. We also see that the PMC divergence, which encourages the percentage of predicted positives to match the class prior, is the more important of the two novel cost terms.
5.5 EXPERIMENTAL STUDY EVALUATING PARAMETERS FOR ESTIMATED PRIOR
PU methods rely on prior estimates of class label likelihoods т Pr(y 1). In particular, DeepSPU uses this prior for the PMC term. With this value being estimated (Jain et al. 2016). the selected prior could be inaccurate in a practical setting. Therefore, we now study the impact of poorly-estimated priors on DeepSPU by varying this estimation. We train DeepSPU on the ES sitting dataset, where the true class prior is about 0.36. As shown in Figure 2b DeepSPU achieves the strongest performance when the estimated prior is closest the true prior. However, even when the prior is overestimated, the performance is not significantly impacted until the esti- mated prior becomes severely over underestimated. This indicates that in practice DeepSPU is generally robust even to incorrectly estimated priors.
CONCLUSION
We propose DeepSPU, the first PU solution learning from weakly labeled data with sequential bias in the labeling. We formulate a novel iterative learning strategy to jointly train a classification model and labeling likelihood (propensity) model, along with designing two theoreticalll-justiffe terms to account for this bias. Through a series extensive experimental results we demonstrate sequential labeling bias naturally arises in real-world datasets. Also,
PU cost
that the previously-overlookee the state-of-the-art PU methods have poor performance when this type bias present, while DeepSPU achieves robust classification performance under sequential labeling bias for rich variety of real-world data sets.
REFERENCES
Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. A public domain dataset for human activity recognition using smartphones. In Esann, 2013.
Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data under the selected at random assumption. Journal of Machine Learning Research, 2018a.
Jessa Bekker and Jesse Davis. Estimating the class prior in positive and unlabeled data through decision tree induction. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018b.
Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: a survey. Mach. Learn,, 109(4):719-760 2020.

---

Under review as a conference paper at ICLR 2021
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th International Conference on Pattern Recognition, pp. 3121-3124 IEEE, 2010.
Yung-Ju Chang, Gaurav Paruthi, Hsin- Hsin-Ying Wu, Hsin-Yu Lin, and Mark W Newman. An inves- tigation of using mobile and situated crowdsourcing to collect annotated travel activity data in real-word settings. International Journal of Human-Computer Studies, 102:81-102, 2017.
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using mn macoder-decode for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing, pp. 1724-1734, 2014.
Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Convex formulation for learning from positive and unlabeled data. In International Conference on Machine Learning, pp. 1386-1394, 2015.
Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 213-220, 2008
Cho-Jui Hsieh, Nagarajan Natarajan, and Inderjit S Dhillon. Pu learning for matrix completion. In ICML, pp.
2015.
Dino Ienco and Ruggero G Pensa. Positive and unlabeled learning in categorical data. Neurocom- puting, 196: 13-1----111 124, 2016.
Shantanu Jain, Martha White, and Predrag Radivojac. Estimating the class prior and posterior from noisy positives and unlabeled data. In Advances In Neural Information Processing Systems (NeurIPS), pp. 2693-2701, 2016.
Masahiro Kato, Takeshi Teshima, and Junya Honda. Learning from positive and unlabeled data with selection bias. International Conference on Learning Representations, 2019.
Shehroz S Khan and Michael G Madden. A survey of recent trends in one class classification. In Irish conference on artificial intelligence and cognitive science, pp. 188-197. Springer, 2009.
Ryuichi Kiryo, Gang Niu, Marthinus C du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In Advances In Neural Information Processing Systems (NeurIPS), pp. 1675-16855 2017.
Xiao-Li Li and Bing Liu. Learning from positive and unlabeled examples with different data distri- butions. In European conference on machine learning, pp. 218-229. Springer, 2005.
Anmol Madan, Manuel Cebrian, Sai Moturu, Katayoun Farrahi, et al. Sensing the" health state" of community. IEEE Pervasive Computing, 11(4):36-45, 2011.
Curtis Northeutt, Tailin Wu, and Isaac Chuang. Learning with confident examples: Rank pruning for robust classification with noisy labels. In Proceedings of the Thirty-Third Conference Uncertainty in Artificial Intelligence, 2017.
Itsaso Rodríguez-Moreno, José María Martinez-Orzeta, Basilio Sierra, Igor Rodriguez, and Ekaitz Jauregi. Video activity recognition: State-of-the-art. Sensors, 19(14)3110,, 2019.
Mike Schaekermann, Edith Law, Kate Larson, and Andrew Lim. Expert disagreement in sequential labeling: A case study on adjudication in medical time series analysis. In SAD/CrovdBias@ HCOMP, pp. 55 65-66,22 2018
Roberto I. Shinmoto Torres, Damith C Ranasinghe, Qinfeng Shi, and Alanson Sample. Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. In 2013 IEEE Interna- tional Conference on RFID (RFID), pp. 191-198. IEEE, 2013.
Yonatan Vaizman, Katherine Ellis, and Gert Lanckriet. Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Computing, 16(4):62-74, 2017. 2017.
10

---

Under review as a conference paper at ICLR 2021
Jesper E Van Engelen and Holger H Hoos. A survey semi-supervised learning. Machine Learn- ing, 109(2):333-44)) 4400, 2020.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nel- son, Eric Jones, Robert Kern, Eric Larson, Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, lan Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mul- bregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020. doi: (5000////159--99-006660
Dell Zhang and Wee Sun Lee. A simple probabilistic approach to learning from positive and unla- beled examples. In Proceedings of the 5th annual UK workshop computational intelligence (UKCI), pp. 83-87, 2005.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 STRATEGIES FOR CALCULATING THE CLASS PRIOR
Our DeepSPU method and the state-of-the-ar nnPU method both take in the class prior π = p(y 1) parameter. While this can be approximated from PU data (Jain et al. 2016), we calculated this from the true class labels for all experiments. This was done follows:
21
NT
where N is the number of time series in the training set and T is the length of each series.
A.2 DEEPSPU TRAINING
Algorithm 1: Dee Training Process
input Initialized propensity network Q Initialized classification network Initialized shared representation B
Dataset of sequences with features and labels C of split into batches
Prior estimate ft
output: Trained classifier
Trained propensity function
nuo Inbeted instances
define /rozzengpropnnsitt(saa train-classifier True
for 1 mar epochs do
for botch in num. batches do if brain_classi ver then
Yooth = (((a((aat)t)))
Qbatch = frozee.prroennsityfffooo (aanaeeennBBBatthiiie Xbatch))
else
Yoatch frozen classi ier (arasisier((rrzeer (Joozen B(Xaatch)
Qbatch = ((((baatch)
end
loss Joss(Quateh, Voatch) Rpu + BCE + KTadio
end
Update weights of B with Adam optimizer using loss if train, classi fier then
Update weights of G with Adam optimizer using loss else
Update weights of Q with Adam optimizer using loss end
train classifies = not train classifie ier copy Q) ier = copy (G)
Brozen_B (B
end
For the sake of reproducibility provide the pseudocode for training DeepSPU,
12

---

Under review as a conference paper at ICLR 2021
A.3 FINDING LABELING LIKELIHOOD GIVEN THE CLASS PRIOR
As described in the Methodology section, DeepSPU initially considers the propensity score to be constant for the first few epochs of training. This constant propensity score is found given knowledge of the class prior as follows:
c= Pr(l 1Iy 1)
1r(e=1,y=1
Pr(y= 1)
Pr(L= = 1)
Pr(y =
Pr([=1] 1)
Thus, we need only to estimate Pr(e- 1) by finding the fraction of labeled instances in the dataset, and dividing this value by the class prior I.
A.4 PROOF OF THEOREM 1 Proof.
E[RPu (9,9) g|xx))x)] = Σ y:qa
1
L
(g)xa()))) + - (a(a:))) (L--))))(())))) + (1 -. (o(x;))
m
4
qi
1 m
Σ
VeL (g(x:)) + -1-18 L (g(x:))
m
R(ggY)
A.5
PROOF OF THEOREM 2
Proof.
Pr(l. = 1|11.1 (11-1)
Pr(e, = 1/2,, 11.-1:1 P)Pr(y 1/2,, 11:1-1)
+ Pr((f 1 | 4-1, Pr = 0 0| (111-1
Pr(( 1/2,, 4-118 1)Pr(y; 1|2,, 111
= Prr(, 112,, 1)Pr(M =1/2,)
13

---

Under review as a conference paper at ICLR 2021
A.6
TABLE OF NOTATION
Notation
Description
N
Number of sequences in the dataset
T
Length of sequence
3z
Features of ith instance
Binary class indicator variable for ith instance
yi
y/ if 3/s is of the positive class, 0 otherwise.
Binary label indicator variable for it/ instance.
4
f, 1 if the timestep was labeled,
L, 0 otherwise.
Sequential propensity score of the ith instance.
Q: = Pr(c, [1 f14-1 Ui = 1)
π
Class prior. π Pr(y = 1)
Label frequency. c= Pr(e = 11y = 1)
Sequence of corresponding variable. i.e., X (Z),'.. .T)
Capital letter
where i 1, .N and 1, 1...... T
Table 3: Reference for symbols used in this work
14

---

Under review as a conference paper at ICLR 2021
True Class (y)
0
0
1
1
1
1
1
Markov Chain
1
1
0
0
0
0
0
0
Labels (t)
0 1
0
0
Figure example of unlabeling with Markov chain.
A.7 INTRODUCING SEQUENTIAL. BIAS
For each sub-sequence of consecutive positive instances within each sequence, a binary Markov chain was run to decide which instances were to be unlabeled. We initiated the Markov chain at state '1', where state '1' indicates that the corresponding positive instance remains labeled. The Markov chain could transition to state "0" with probability X. We varied the value of À to evaluate DeepSPU's performance for various levels mislabeling. For instances where the Markov chain is in state '0" the corresponding positive instance was set to be unlabeled. The chain has 1% probability of transitioning back process introduces sequential bias, the likelihood that given instance is mislabeled depends whether the previous instance mislabeled (a given instance has a 100 (1 A) % chance of being labeled if the preceding instance is labeled).
from state An example of this is shown in figure 3 This
15

---

Under review as a conference paper at ICLR 2021
50
40
Frequency
30
20
10
11
|+
mm
100
200
300
400
500
600
Consequtive Minutes of Labeling
Figure 4: Histogram of lengths of consecutive labels (in minutes).
A.8 QUANTITATIVE ANALYSIS EXTRASENSORY LABELING PATTERNS
Figure 4 shows the frequency of lengths of labeled subsequences in the ExtraSensory dataset. Clearly, the participants do enter phases of labeling there are many long subsequences of con- secutive labels.
16

---

Under review as a conference paper at ICLR 2021
Likelihond of Labeling
0.8
0.4
0.2
0.0
10
True Propensity Score
Estimated Propensity Score
Number of timesteps since start positive subsequence
Figure 5: DeepSPU's estimated propensity the propensity scores. The estimated scores match the true scores almost perfectly.
A.9 EVALUATING PERFORMANCE OF PROPENSITY MODEL
We perform an additional experiment in order to demonstrate DeepSPU's propensity network's s abil- ity to learn the propensity score. In this experiment, created subsequences of positive in- stances and subsequences of negative instances. The feature values for the positive instances where drawn from normal distribution with mean 0 and unit variance, while the features for the nega- tive instanced were drawn from a normal distribution with mean 10 and unit variance. The positive subsequences were labeled using a gamma distribution decide which positive instances received labels. We used Scipy's 's (Virtanen et al., (020) gamma distribution with shape, location, and scale parameters all set to We then train DeepSPU on this data and extracted learned propensity scores. As Figure 5 shows, the estimated propensity scores match the true propensity scores in- curred by the gamma distribution almost perfectly.
17


Reviews:
[{"review_id": "yPQ-QECOBiT", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overall comments: Novel formulation and good experimental results. The method is claimed to be a general solution to PU settings with biased positives. However, there might be potential issues related to identifiability of the posterior from the labels. Mathematical writing lacks rigor.\n \nStrengths:\n1) The formulation is Novel.\n2) Experiments show superior performance then the state of the art.\n\nWeaknesses:\n1) The algorithms for class prior estimation in PU learning are only derived for the no bias case. Thus class prior estimated using biased positives can be wildly incorrect. And consequently, it can introduce significant bias in the learnt posterior.\n2) The new loss function is a function of the posterior and the propensity function which are both learnt from the data. As implied by the authors it can’t alone be used to learn the true posterior as there is a trivial solution to the loss which assigns q_i=1 for all i and g(x_1) = l_i. The authors add terms corresponding to the prior matching cost and observation matching cost to solve the issue. However, it seems that the observation matching cost is not helpful in that regard since it is minimum when q_i*g(x_i)=l_i and consequently q_i = 1 and g(x_i) = l_i. \n3) In general, it seems that q_i and posterior cannot be uniquely identified from the labeled data. The true posterior is the minimizer of the new PU loss only if the estimated q_i indeed corresponds to the true labeling mechanism. Is it not possible that the both estimated q_i and the estimated posterior are far away from the true labeling mechanism and the true posterior, respectively? In my opinion, a consistent estimate of the true posterior is not feasible, unless assumptions are made on the nature of the bias. \n4) Lack of rigor in mathematical writing: g_\\theta(X)=Py(Y|X) implies that  g_\\theta is a function of the sequence, but used as the posterior at a given x in the sequence. It would be better to have separate notations for the theoretical quantities and their estimates. In equation 2 shouldn’t it be Bern(Pr(\\hat{y}=1|x)). G_\\theta is defined twice, as the posterior probability and also as the output of the model. In reality it is the output of the model and the probability is what it is intended to estimate. OMC is not just a function of X it is also a function of the label. Please define OMC completely as a summation over the dataset. J is used as a function of g_\\theta and also as a function of Q_\\phi.  q_i is used in two different ways: one where y_i is given and the other where y_i is not given. \n5) It will be useful to mention that the loss functions belong to the proper loss family so that g(x) indeed corresponds to the posterior [1].\n6) Please give details on how are hyperparameters \\lambda_1 and \\lambda_2 are tuned.\n\n[1] Reid MD, Williamson RC. Composite binary losses. The Journal of Machine Learning Research. 2010 Dec 1;11:2387-422.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "uU03mSQT0Mp", "reviewer": "AnonReviewer6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**: This paper makes the observation that sequential labelling processes introduce a bias in the obtained labels. This is relevant for weakly supervised sequential settings (eg human activity recognition), where examples are either assigned a positive label or are unlabelled. For this setting, a novel method is introduced that models this (latent) labelling process and uses the latent labels in the objective function,  this results in better performance on two datasets. While the observation of sequential bias is interesting and the performance is promising, I rate the current paper as a 'OK but not good enough': it seems to miss the insights behind the observation of sequential labelling bias and it seems to miss some experiments (ablation study, multi-class, etc).\n\n**Sequential Labelling Bias**:\nThe observation about sequential bias is interesting and probably valid.\nBut some claims are doubtful without further backup by experiments or references, for example: \"Hence, an instance is more likely to be labeled if its neighbours are also labeled, and vice versa\" [P1, first paragraph]. This depends completely on the labelling procedure, eg for Human Activity Recognition: when at random times people are asked what they do now, the labels dont follow this assumption. Also in the next paragraph: it is described that HAR data might have sequential bias, but it is not shown, what the properties of this bias are. Given that this is so important in the current manuscript, I'd like to see more insights:\n1. What kind of bias is observed? I'd expect that people like to label in the beginning, and then forget about it later on. So that early in the sequence more labels are provided than later. \n2. How can this biased be modelled? Please show some qualitative and quantitative results from different type of annotation/labelling procedures. Does the bias differ from domain to domain (eg medical vs human activity recognition)?\n3. Does labelling bias depend on the annotator/person how labels the sequence?\n4. Can the (obtained) propensity score be used for predicting labelling bias. How good is it at this? Or would a temporal smoothing over labels do equally well?\n\n**Problem Setting**\n- The problem setting assumes there are sets of sequences. Why is that? What defines such a single sequence? \n- Can the setting be generalised to multi-class  classification? In that case a positive label for one class is a (strong indicator for) a negative label for all other classes.\n- x_t seems to be a scalar-value, why is that? Can the model be used with more complex input (eg images)?\n\n**Methodology**\n- Figure 1 seems to contain an error: both the SPM and CN always take the learned representation of the feature sequence, not the feature sequence itself. It is also a rather complicated figure, if the only thing to show is that the networks are trained alternately.\n- For my understanding it would be nice if the relation with related work is made here more clearly. Where does this method differ from the PN network? Only in the loss? If yes, which loss is used then? \n\n**Experiments**\n- I miss an ablation study for the effect of Rpu, PMC and OMC in the final objective (below Eq 3). What is the influence of adding PMC to the training pipeline? Or is Rpu alone sufficient?\n- I miss experiments on multi-class datasets\n- I miss the details on the datasets (type of features, number of examples, influence of subsampling etc).\n\n**Conclusion**\nTo restate again, I think the observation of sequential bias is valid and it has to be present in many (sequential) labelling tasks. While in this paper some form of a solution is provided, I do miss the insights into this bias. I hope these can be provided during the rebuttal phase.\n\n**Post-Rebuttal**\nThe authors have addressed some of my concerns during the rebuttal phase. Thanks! For me the major drawback of the current manuscript remains that 'sequential bias' is not really defined, and not really showed to exist in the datasets at hand. Only indirectly, by our new method performs better than existing methods. I think this should be improved, to increase the understanding of this topic.", "extra_scores": {}, "extra_text": {}}, {"review_id": "EztJ4gBV1QN", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "--- Update after discussion ---\n\nI thank the authors for considering my recommendations. I think the clarity of the updated paper is much improved, particularly the introduction. I further think that the authors have adequately addressed the concerns raised by the other reviewers and recommend accepting the paper.\n\n--- Review ---\n\nOverall, I thought this paper was good, but not great. I found the problem poorly motivated and the contributions limited in scope. The methods were reasonable and appear technically correct, if somewhat ad hoc. I recommend acceptance, though I think the paper could be substantially improved.\n\n--- Comments ---\n\n1. After reading the intro, it was very unclear to me what problem the paper was trying to solve and what the contributions of the paper were. In part, this is because the main problem addressed by the paper, \"sequential bias\", is not defined until the second page despite being reference earlier and in the abstract. Even the fact that we are dealing with sequential data wasn't made very clear until later. I recommend rewriting the intro to start by defining the problem that the paper solves, something like: Existing methods for learning instance-level classification models from sequential data with positive only labels rely on accurately estimating the conditional probability that an instance receives a label, known as the propensity score. However, these methods generally ignore the fact that positive labels are often temporally clustered together --- referred to as sequential bias --- resulting in inaccurate propensity scores and lower classification performance. We propose a method that ...\n\n2. In section 3, I recommend giving more of a tutorial on how propensity scores are used in non-sequential settings. As it is, I wasn't sure how the propensity scores could be used to recover the true risk (and thus, why they were important) until page 5.\n\n3. The main contribution of this paper doesn't really have anything to do with deep learning and I think tying the method so closely to NNs artificially limits the contributions. Specifically, most of section 4.5 and 4.6 could be presented without any reference to NNs. I recommend moving 4.5 and 4.6 before 4.1 - 4.3 and writing them as a general estimation procedure. 4.1 - 4.3 can then be framed as discussing the specific model specification details used in the paper. Even the name DeepSPU references deep learning when it doesn't really need to.\n\n--- Minor comments ---\n\n1. Page 2, par 2, line 2: \"Instance\" not yet defined.\n\n2. Page 2, par 4, line 7: Prior-Matching --> Prior-Weighted\n\n3. Page 3, par 3, line 2: I recommend $|\\cdot|$ or $length(\\cdot)$ rather than $||\\cdot||$ to refer to the length of a sequence so as to avoid confusion with a norm.\n\n4. Page 3, par 4, line 5: $y_i^{(j)}$ --> $y_i$\n\n5. Page 3, par 4, line 8: I don't follow the logic that a low propensity score implies a high probability of being a true positive. Since we are conditioning on $x_i$, couldn't it also just be an instance that is obviously negative and thus unlikely to be labeled?\n\n6. Page 3, par 5, line 3: Is there notation defined in the appendix that is not defined somewhere in the main text? If not, I'm not sure I would include this table.\n\n7. Page 4, par 3, line 2: local feature values --> local feature values and the feature values of all preceding instances\n\n8. The notation in the appendix differs from that used in the main paper and has several typos. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Dear Authors,

Thanks for your detailed feedback to and even communications with the reviewers. Your additional input certainly clarified some of the concerns raised by the reviewers and also improved their understanding of your work.

However, we still think that the notion of sequential bias is unclear, and the authors overclaim what they have done.
For these reasons, this paper cannot be recommended for acceptance.
I hope that the detailed feedback from the reviewers will help improve this work for future publication.

Decision: Reject