Paper: Learning with Temporal Label Noise (ICLR-2024)

Reviewer comment (Novelty):
What motivates the introduction of this new label noise setting? Does the temporal label noise exist in real-world applications?

Author response:
**"What motivated you to introduce this new label noise setting? Does the temporal label noise exist in the real-world?"**

Thank you for asking this question! We were actually motivated to work on this because of real-world applications that we work on in the healthcare domain. In particular, user-annotated human activity time series data, where participants notoriously mislabel their activities depending on when and what they are doing. 

Stepping back, we identified that this problem arises in a substantial class of real-world applications where labels are collected over time, especially when people label their own data over time. Well-known examples include:

1) **Human activity recognition**: Human activity recognition models detect people’s activities over time, usually from wearable devices or smartphones. For example, smartphone apps may tailor notifications to someone’s behaviour. But training labels are provided by people annotating their own activities over time [1,2]. Such self-annotation is notoriously noisy. For example, people often inaccurately report when they fall asleep or wake up [3]. Such annotations suffer from temporal label noise because some times have more noise than others (e.g., around sleep).
2) **Modelling mental health over time**: Consider suicide-risk prediction tools that use wearable sensors to alert doctors of patient risks. These models are trained on self-reported indicators of suicide-risk [4]. But studies consistently show that users can fear repercussions and underrepresent their true suicide risk [5]. In this case, periods of time where individuals are suicidal are subject to increased label noise.
3) **Temporal bias in grading**: When grading student assignments, later assignments can receive worse grades, likely as graders tire [6]. This sequential bias implies label noise increased over time in the 30 million records studied in this work. We have included a new citation to this case in the Introduction.

Reviewer comment (Theory):
The assumption that "the sequence of noisy labels is independent of the features given the true labels" can be strong, since label noise in the real world can be instance-dependent and class-dependent.

Author response:
**"The authors assumption that ‘the sequence of noisy labels is independent of the features given the true labels’ can be strong, since the label noise in the real-world can be instance-dependent and class-dependent."**

You are right that this may be a strong assumption. We wanted to be explicit about this assumption in the manuscript. 

You are right that there are other variants of real-world label noise that can arise. Stepping back, we actually wanted to focus on the canonical setting in which we had class-conditional label noise that is feature-independent – this was important to pinpoint how methods and performance would change as a result of temporal effects. 

Our method is built to handle class-dependent label noise. Handling instance-dependent is out of scope (and generally a challenging problem even in the non-temporal, static setting)  [7-9]. 

We also consider the setting where noise is simultaneously class- and time-dependent (that is, a unique time-dependent noise process for each class). The temporal noise type “Mixed” reported in the paper represents class- and time-dependent noise. We showcase the superior performance of TENOR in this case (Table 1)! Additionally, we have also added experiments on synthetic and real-world multiclass sequential classification tasks, showing that TENOR has strong performance in these settings as well (Appendix E.7). We believe this further increases the utility and generalizability of our work.

Reviewer comment (Experiments):
The comparison methods can be outdated. It is better to consider more state-of-the-art methods as baselines. Furthermore, clarification is needed on how the baselines are adapted to fit the specific setting.

Author response:
**"The comparison methods can be outdated. It is better to consider more SOTA methods as baseline. Furthermore, how you adaptive the baseline to fit your specific setting"**

We tried to choose baselines that were used in prior work:  VolMinNet (ICML 2021) [10] and Anchor (CVPR 2017) [11]. We’re happy to run experiments against other methods if you would like (just let us know). In general, we think that there is a marginal benefit in adding more baseline methods since they assume that label noise is static over time and will fail when the label noise is temporal (albeit at different rates). We show that such static methods are insufficient, even if they estimate the noise perfectly (Fig 2), so more-extensive comparisons are out of scope. As it stands, these points apply to other methods and future attempts to make existing methods time-dependent.

One point that we would like to make is that some of the “baseline methods” that we present in the experiments are actually methods that we developed ourselves (e.g.: VolMinTime and AnchorTime)! These are methods where we take the existing static method and modify it to a temporal setting as described in Section 4.4 . The results show that some of the issues resulting from temporal label noise can be mitigated by allowing for temporal variation. Our results show that this improves their performance, though our approach remains superior.

Reviewer comment (Experiments):
Is the proposed method only applicable to binary classification? It is advisable to adapt the real-world noisy label learning dataset Clothing1M to the framework to validate the efficacy of the proposed method.

Author response:
**"It is advisable to adapt the real-world noisy label learning dataset Clothing1M to your framework to validate the efficacy of the proposed method"**

Thanks for mentioning this. We considered applying this dataset since it contains real label noise. Unfortunately, Clothing1M is not naturally applicable to our setting since each instance only has a single label – i.e., there is no “sequence” of labels over time.

Given your suggestion for real-world noisy labels, we did run experiments  with the “ExtraSensory” dataset [12] which also contains “real world noisy labels” for a time series prediction task.  ExtraSensory includes human activity data from smartphones and smartwatches from 60 users over a timespan of 300K minutes.

In this case, we then train binary classifiers to predict when users are asleep or awake, which is a key benchmark task. In contrast to HAR and HAR70 datasets in our experiments, the labels in ExtraSensory are user-provided and subject to label noise (since users often misreport falling asleep and waking up, especially when they are sleepy or in a rush to leave in the morning). 

Our results for this dataset show that TENOR successfully identifies an interpretable temporal noise function (Appendix E.2) and that TENOR predicts there exists higher label noise near sleep/awake transitions. We also see that TENOR is responsive to these temporal effects. In this case, we cannot validate if these are the correct predictions since we do not have any cleaned ground truth labels (like HAR, HAR70, or even Clothing1M), but this is the kind of behaviour we would expect to see from a good model in this setting. We included these results in Appendix E.2 and will work to integrate them into the Experiments if you think this is valuable.

**"Is the proposed method only applicable to binary classification?"**

Our proposed method works for multiclass classification problems! We have included results from new experiments on multiclass data for synthetic and real-world tasks (ranging from 2 to 4 classes). We find that our method still outperforms existing methods under a variety of temporal noise functions. Please refer to the following table, demonstrating performance on 3-class classification for the synthetic dataset. Complete results can be found in Appendix E.7.

|             | Time Independent | Time Independent | Exponential  | Exponential    | Linear       | Linear         | Sigmoidal    | Sigmoidal      | Sinusoidal   | Sinusoidal     | Mixed        | Mixed          |
|-------------|------------------|------------------|--------------|----------------|--------------|----------------|--------------|----------------|--------------|----------------|--------------|----------------|
|             | *Accuracy*         | *MAE*              | *Accuracy*     | *MAE*           | *Accuracy*     | *MAE*            | *Accuracy*     | *MAE*            | *Accuracy*     | *MAE*            | *Accuracy*     | *MAE*            |
| Uncorrected | 81.7$\pm$3.4     | NA            | 77.2$\pm$1.3 | NA         | 77.0$\pm$1.6 | NA         | 79.5$\pm$2.9 | NA          | 79.5$\pm$1.7 | NA          | 76.6$\pm$2.4 | NA          |
| Anchor      | 87.5$\pm$2.0     | 0.05$\pm$0.006   | 84.3$\pm$0.9 | 0.06$\pm$0.005 | 84.1$\pm$1.1 | 0.06$\pm$0.009 | 84.4$\pm$2.0 | 0.09$\pm$0.019 | 85.9$\pm$0.9 | 0.07$\pm$0.004 | 84.3$\pm$1.0 | 0.07$\pm$0.010 |
| VolMinNet   | 92.4$\pm$3.1     | 0.05$\pm$0.014   | 89.1$\pm$0.8 | 0.04$\pm$0.002 | 89.5$\pm$0.6 | 0.05$\pm$0.003 | 92.9$\pm$2.8 | 0.07$\pm$0.013 | 90.2$\pm$0.4 | 0.06$\pm$0.003 | 89.0$\pm$1.5 | 0.05$\pm$0.013 |
| AnchorTime  | 87.1$\pm$2.1     | 0.05$\pm$0.006   | 85.0$\pm$0.5 | 0.06$\pm$0.005 | 85.4$\pm$0.9 | 0.06$\pm$0.007 | 87.4$\pm$2.3 | 0.06$\pm$0.008 | 85.3$\pm$0.8 | 0.07$\pm$0.004 | 86.4$\pm$2.0 | 0.05$\pm$0.006 |
| VolMinTime  | 90.0$\pm$3.3     | 0.03$\pm$0.008   | 86.8$\pm$1.0 | 0.05$\pm$0.001 | 87.0$\pm$1.3 | 0.04$\pm$0.002 | 90.9$\pm$3.3 | 0.04$\pm$0.007 | 88.4$\pm$0.8 | 0.05$\pm$0.001 | 87.8$\pm$2.1 | 0.04$\pm$0.004 |
| TENOR       | **96.8$\pm$0.6**     | **0.01$\pm$0.002**   | **96.0$\pm$0.3** | **0.01$\pm$0.002** | **96.6$\pm$0.4** | **0.01$\pm$0.002** | **97.3$\pm$0.6** | **0.01$\pm$0.002** | **96.8$\pm$0.3** | **0.02$\pm$0.002** | **96.8$\pm$0.5** | **0.01$\pm$0.002** |

Reviewer comment (Novelty):
Named entity recognition (NER) with label noise is a highly related domain (both dealing with sequentially correlated label noise), some existing works have already been proposed in this area [1,2], and authors should discuss the similarities and differences between those domains, and if possible, apply TENOR to NER to better showcase the performance superiority.

Author response:
We want to flag this comment as a potential misunderstanding: these NER papers study neither temporal noise nor time series. Instead, they randomly unlabel named entities in classic NLP datasets. Further, they use short sentences containing 10-20 words. We study time series, which often span hundreds or thousands of timesteps. However, we agree that TENOR is likely applicable broadly and we hope our work inspires such adaptations.

Reviewer comment (Theory):
Authors should consider more mild assumptions that enable loss correction to be applied to temporal classification.

Author response:
We would like to flag another potential misunderstanding: the focus of our work is loss correction for temporal classification. As we mention in Section 3, the conditional independence assumptions we make are common in such sequential classification problems [1-3]. Our findings also apply to our synthetic and real-world datasets, suggesting our assumptions are realistic.  

Also as mentioned in Appendix C, our assumptions provide a strong foundation for future work. Existing work in statistical learning theory on highly-dependent sequences may be a promising direction for future research, particularly in the context of temporal label noise, which we define in this work.

Reviewer comment (Theory):
Learning a matrix function that changes with time seems very challenging even independently, but learning it together with the classifier is even more so. The paper lacks clarity on why this approach works. If even if there are not any theoretical guarantees for TENOR method, it would be good to discuss why the optimization works and is not derailed by too much label noise (in the experiments, I see even for about 30% noise, the learning is pretty robust).

Author response:
**"It would be good to discuss why the optimization works and is not derailed by too much label noise”**

Thanks for your question, we’re glad to have a chance to elaborate!

*"Why the Optimization works”*

A naive approach would be to treat each time-step independently and optimize a separate function or set of weights for each $Q_t$ - this is the VolMinTime baseline in our experiments. The clever part of our implementation is how we link each $Q_t$  using the same function (a neural network with a single set of weights). Now in order to see how we can identify $Q(t)$, we need to consider how the noisy posterior decomposes. Specifically, the noisy-label posterior at time $t$ can be expressed as a product of a noise-transition matrix and a clean-label posterior 

$P(\tilde{y}\_t|x\_{1:t}) = Q\_t ^ {\top} P(y\_t|x\_{1:t})$

In our case, $Q_t$ is identifiable provided the posterior distribution is *sufficiently scattered* – i.e., it is not uniformly distributed in the unit-simplex. This follows from an important theoretical result in Non-negative Matrix Factorization (NMF) [1, 2] and motivates a different method to estimate the noise transition-matrix estimation matrix [3], which we use as a baseline in our experiments.

*"Why isn’t TENOR derailed by too much noise"*

If we assume that the posterior distribution is sufficiently scattered, then $Q_t$ is identifiable, and its columns form the minimum-volume simplex that encloses $P(\tilde{y}\_t|x\_{1:t})$. Regardless of the amount of noise, the noisy posterior is a linear combination of the columns of $Q_t$ and the clean posterior. In our experiments, we go up to 40% label noise and find our methods are still effective (see Fig 3 and Appendix E).

In order to efficiently estimate $Q_t$ at all timesteps, we set up the equality-constrained optimization problem in Eq 3 as an augmented Lagrangian objective [4], which is guaranteed to converge under some regularity conditions [4].

Reviewer comment (Theory):
It's unclear how the TENOR objective imposes regularity across time 't', e.g. if we know the label noise say increases with time in the instances, or varies sinusoidally, etc., it's surprising that the model can learn such Q(t) functions, without any additional regularization or at least a good initial point. The Frobenius norm reg. can't do this. Part of this confusion arises because of my lack of clarity in how Q_w(t) is modeled via a neural network -- are there just one set of weights and an additional 't' parameter (integer) that gives the noise matrix estimates across 't'?

Author response:
**“It's unclear how the TENOR objective imposes regularity across time 't… Part of this confusion arises because of my lack of clarity in how $Q(t)$ is modeled via a neural network”**

We briefly discuss how we model $Q_{\omega}(t)$ in Appendix D.2 and are happy to elaborate more here. We use a fully-connected DNN with weights $\omega$ that takes a real-valued time-step as input ($t \rightarrow Q_t$) and outputs a *valid* noise transition matrix as per (Def 1). To ensure that the output is a valid we take the output and:

1) reshape it into a square matrix $C^2 \times 1$  vector is converted into a (C x C) matrix)
2) apply a softmax operation row-wise to ensure entries are valid transition probabilities
3) add an identity matrix to ensure diagonal dominance
4) and finally we row-normalize to ensure the final matrix is row-stochastic.

Since these operations are differentiable, we can then fit the underlying function $Q(t)$ using standard training algorithms.

In this case, the loss function that we use represents the aggregate volume across all timesteps. The *optimal* weights of the network are therefore the weights that generate the minimum volume transition matrix at each time step. Because the network modelling $Q(t)$ shares one set of weights for all timesteps, we can then model any time-dependent function, due to the universal representation properties of neural networks.

Stepping back, one important point that we want to make is that TENOR can work with any model $Q(t)$ – i.e., beyond the ones that we learn from data using the method described above. For example, we could also use a Neural ODE to parameterize $dQ/dt$ and estimate $Q(t)$ for all t in 1…T by solving an initial value problem. We describe this in greater detail in Appendix C.

Reviewer comment (Theory):
Theorems 1 and 2 are about equivalences, but there are no forms of estimation/finite sample guarantees in the paper.

Author response:
**"Theorems 1 and 2 are about equivalences, but there are no forms of estimation/finite sample guarantees in the paper.”**

Thanks for raising this point! We agree that these are infinite sample guarantees. We prioritized these since it seems to be the standard in previous work [5,6]. With that being said, we agree that finite sample guarantees are most important. We will work to see if anything can be said for the finite sample condition. Meanwhile we have updated the text to state this clearly and explained its limitations (Appendix C - Limitations and Future Work).