Learning under Temporal Label Noise

## Abstract

Abstract Many sequential classification tasks are affected by label noise that varies over time.
Such noise can cause label quality to improve, worsen, or periodically change over time. We first propose and formalize temporal label noise , an unstudied problem for sequential classification of time series. In this setting, multiple labels are recorded in sequence while being corrupted by a time-dependent noise function. We first demonstrate the importance of modelling the temporal nature of the label noise function and how existing methods will consistently underperform. We then propose methods that can train noise-tolerant classifiers by estimating the temporal label noise function directly from data. We show that our methods lead to state-of-the-art performance in the presence of diverse temporal label noise functions using real and synthetic data. † † Our code is available on GitHub .

## 1 Introduction

Many supervised learning datasets contain *noisy* observations of ground truth labels. Such *label noise* can arise due to issues in human annotation or data collection [, ], including lack of expertise among annotators [, ], discrepancies in labelling difficulty [, , ], subjective labeling tasks [, , ], and systematic issues in automatic annotation like measurement error [, ].

Label noise is a key vulnerability of modern supervised learning [, , ]. Intuitively, models trained with noisy labels may learn to predict noise rather than the ground truth. Such models will then underperform at test time when they must instead predict the ground truth. This problem is exacerbated by deep learning methods, which have the representational capacity to overfit the noise [, , , ].

While there are many methods designed to be robust to label noise, they are all designed for static, time-invariant data [, , , ].
Real-world problems (e.g., healthcare) often involve time series data, where multiple labels are collected over time. Labels typically represent the change in underlying states over time - for example an individual transitioning in and out of testing positive for COVID-19. As these labels are generated over time, label *noise* can surely change over time. Existing approaches are ill-equipped to handle this phenomenon. Here, we introduce the concept of *temporal label noise*. We argue that accounting for temporal label noise can lead to time series models robust to noise.
To the best of our knowledge, learning from temporal label noise is unstudied, yet exists in a range of tasks. For example:

- Longitudinal Self-Reporting for Mental Health: Mental health studies often collect self-reported survey data over long periods of time. Such self-reporting is known to be biased [, , , ], where participants are more or less likely to report certain features. For example, the accuracy of self-reported alcohol consumption is often seasonal [].
- Human Activity Recognition. Wearable device studies often ask participants to annotate their activities over time. But participants may mislabel due to recall bias, time of day, or labelling-at-random for monetized studies [, ].
- Clinical Measurement Error: The labels used to train clinical prediction models are often derived from clinician notes in electronic health records. These labels may capture noisier annotations during busy periods, when a patient is deteriorating, or the bedside situation is more chaotic [].

Figure: Figure 1: Label quality can vary over time due to *temporal* label noise. Existing methods assume noise is time-invariant (*static*) leading to loss in performance. By accurately modeling temporal label noise, we can improve performance and outperform existing approaches. Here, we show performance improvements on synth dataset comparing reconstruction error and accuracy between static and temporal methods subject to 30% temporal label noise (see Appendix E for details).
Refer to caption: x1.png

Classifying time series with temporal label noise is challenging. First, existing losses that are robust to static label noise [, ] underperform when facing temporal noise—as we show in our experiments. This means that classifiers can still suffer, even for known temporal noise functions. Worse yet, the temporal label noise function is generally not known. Often times, all that is available is a dataset with temporally noisy labels with no indication of which instances or time-steps are more likely to be correct.

We propose novel time series classification objectives which *learn* the temporal noise function from the data. These methods leverage temporal loss functions, which we prove are *robust to temporal label noise*. These methods can be used out of the box and allow practitioners to learn noise-tolerant time series classifiers, even with unknown, temporal label noise.

Our contributions are as follows:

- 1.
We formalize the problem of learning from noisy labels in temporal settings. We are the first to study this important problem on which we show that prior works under-perform.
- 2.
We propose two loss functions for training models that are robust to temporal label noise. On their own, each can be used to improve prior methods.
- 3.
We propose TENOR, the first method for learning temporal noise functions. By pairing a neural network with our proposed loss functions, TENOR can model any noise function and thereby lead to better classifiers.
- 4.
Our experiments identify that existing methods underperform in the presence of temporal noise, while our proposed methods lead to better classifiers. This highlights the necessity of accounting for temporal noise.

### Related Work

#### Time Series

Machine learning for time series, especially in healthcare, often relies on combining autoregressive approaches to modelling sequences with deep neural network based methods [, , ]. Primarily in the context of sequence-to-sequence modelling, Recurrent Neural Networks (RNNs) and state space models have gained attention for their ability to model observed data as emanations from underlying latent states that evolve over time. Attention-based mechanisms have further enabled models to prioritize relevant segments of data [], thereby enhancing performance in tasks such as patient outcome prediction and treatment optimization [, , , , , ]. These approaches primarily rely on supervised learning to model changes in latent states over time. In healthcare for example, underlying latent states can be understood as clinical labels (i.e: sick vs healthy) that patients can transition in and out of. An understudied area in this domain is label noise - what happens when the labels we use to learn time series models are corrupted?

#### Label Noise

Our work identifies and addresses gaps in the literature on noisy labels [, , , ]. Prior to this, the vast majority of work on noisy labels has focused on static classification tasks, where label noise cannot change over time.
We instead consider a setting in which label noise arises over time, and where the noise rates can therefore change over time. While a few methods have considered noise in time series data [, ], they focus on tools to identify individual noisy samples rather than developing general-purpose methods robust to temporal label noise. These studies also suggest that noisy samples can be identified because neighboring time points are unlikely to be corrupted together. However, we propose that neighboring time points are *more* likely to be corrupted together when they were labelled together.

We present a way to learn via empirical risk minimization with noise-robust loss functions [, , , , ].
Many approaches in the static setting utilize methods for identifying correctly-labeled instances [, ], or regularization techniques to minimize the impact of incorrect label [, ].
Our results highlight the potential to learn from noisy labels when we have knowledge of the underlying noise process [c.f., , ]. We also develop methods to fit the noise model directly from noisy data [see e.g., , , , , , ].

## 2 Framework

#### Preliminaries

We consider a sequential classification task over $C$ classes and $T$ time steps. Each instance is characterized by a triplet of *sequences* over $T$ time steps $({\mathbf{x}}_{1:T},{\mathbf{y}}_{1:T},\tilde{{\mathbf{y}}}_{1:T})\in\mathcal{
X}\times\mathcal{Y}\times\mathcal{Y}$. Where $\mathcal{X}\subseteq{\mathbb{R}}^{d\times T}$ and $\mathcal{Y}=\{1,\dots,C\}^{T}$. Here, ${\mathbf{x}}_{1:T}$, ${\mathbf{y}}_{1:T}$, and $\tilde{{\mathbf{y}}}_{1:T}$ are sequences of instances, *clean labels* and *noisy labels*, respectively.

When learning under temporal label noise, we do not observe the *true* label sequence ${\mathbf{y}}_{1:T}$. Thus, we only have access to a set of $n$ instances $D=\{({\bm{x}}_{1:T},\tilde{{\bm{y}}}_{1:T})_{i}\}_{i=1}^{n}$. We assume that the sequences for each instance are generated i.i.d. from a joint distribution $P_{{\mathbf{x}}_{1:T},{\mathbf{y}}_{1:T},\tilde{{\mathbf{y}}}_{1:T}}$, where the label noise process can vary over time. This distribution obeys two standard assumptions in sequential modeling and label noise [, , ]:

###### Assumption 1 (Markov Assumption) .

A label at time $t$ depends only on the sequence of feature vectors up to $t$:

$$ $\displaystyle p({\mathbf{y}}_{1:T}\mid{\mathbf{x}}_{1:T})=\prod_{t=1}^{T}p({ \textnormal{y}}_{t}|{\mathbf{x}}_{1:t})$ $$

###### Assumption 2 (Feature-Independence) .

The sequence of noisy labels is conditionally independent of the features given the true labels: $\tilde{{\mathbf{y}}}_{1:t}\perp\!\!\!\!\perp{\mathbf{x}}_{1:t}\mid{\mathbf{y}}
_{1:t}$ for $t=1,\ldots,T$

1 allows us to elegantly factorize the joint distribution of a sequence: $p(\tilde{{\mathbf{y}}}_{1:T}|{\mathbf{x}}_{1:T})=\prod_{t=1}^{T}q_{t}(\tilde{{
\textnormal{y}}}_{t}|{\mathbf{x}}_{1:t})$. 2 allows for the noisy label distribution at $t$ to be further decomposed as:

$$ $\displaystyle q_{t}(\tilde{{\textnormal{y}}}_{t}|{\mathbf{x}}_{1:t})=\sum_{y \in\mathcal{Y}}q_{t}(\tilde{{\textnormal{y}}}_{t}\mid{\textnormal{y}}_{t}=y)p( {\textnormal{y}}_{t}=y\mid{\mathbf{x}}_{1:t})$ (1) $$

###### Assumption 1 (Markov Assumption) .

###### Assumption 2 (Feature-Independence) .

### 2.1 Learning from Temporal Label Noise

Our goal is to learn a sequential classification model ${\bm{h}}_{\theta}:\mathcal{X}\rightarrow\mathbb{R}^{C}$ with model parameters $\theta\in\Theta$. Here, ${\bm{h}}_{\theta}({\mathbf{x}}_{1:t})$ returns an estimate of $p({\textnormal{y}}_{t}|{\mathbf{x}}_{1:t})$.
To infer the label at time step $t$, ${\bm{h}}_{\theta}$ takes as input a sequence of feature vectors up to $t$, and outputs a sequence of labels by taking the $\operatorname*{arg\,max}$ of the predicted distribution for each time step (see e.g., [, ]).

We estimate parameters ${\hat{\theta}}$ for a model robust to noise, by maximizing the expected accuracy as measured in terms of the *clean* labels:

$$ $\displaystyle\hat{\theta}=\operatorname*{argmax}_{\theta\in\Theta}\mathbb{E}_{ {\mathbf{y}}_{1:T}|{\mathbf{x}}_{1:T}}\prod_{t=1}^{T}p({\textnormal{y}}_{t}={ \bm{h}}_{\theta}({\mathbf{x}}_{1:t})\mid{\mathbf{x}}_{1:t})$ $$

However, during training time we only have access to sequences of *noisy* labels. To demonstrate how we can sidestep this limitation, we first need to introduce a flexible way to model sequential noisy labels. Existing methods assume that noise is time-invariant (Fig. 1). To relax this assumption, we capture the temporal nature of noisy labels using a *temporal label noise function*, in Def. 1.

###### Definition 1 .

Given a sequential classification task with $C$ classes and noisy labels, the *temporal label noise function* is a matrix-valued function ${\bm{Q}}:\mathbb{R}_{+}\to[0,1]^{C\times C}$ that specifies the label noise distribution at any time $t>0$.

We denote the output of the temporal noise function at time $t$ as ${\bm{Q}}_{t}:={\bm{Q}}(t)$. This is a $C\times C$ matrix whose $i,j^{\textrm{th}}$ entry encodes the flipping probability of observing a noisy label $j$ given clean label $i$ at time $t$: $q_{t}(\tilde{{\textnormal{y}}}_{t}=j\mid{\textnormal{y}}_{t}=i)$. We observe that ${\bm{Q}}_{t}$ is positive, row-stochastic, and diagonally dominant — ensuring that ${\bm{Q}}_{t}$ encodes a valid probability distribution [, ].

${\bm{Q}}_{\omega}$ denotes a temporal noise function parameterized by a function with parameters $\omega$. This parameterization can be constructed to encode essentially any temporal noise function. As shown in Table 1, we can capture a wide variety of temporal noise in time series classification tasks using this representation.

**Table 1: Overview of noise functions for time series classification tasks. We show the noise model ${\bm{Q}}(t)$ and parameters $\omega$ needed to model $q({\tilde{{\textnormal{y}}}_{t}=j\mid{\textnormal{y}}_{t}=i})$. Practitioners can model for label noise by choosing a parameteric class to capture effects and fit parametric representations from data. The choice of noise model can differ across classes and subgroups (see the Mixed model in Section 4). We include other models and details in Appendix D.**
| Pattern | Depiction | Noise Model – ${{\bm{Q}}}_{ij}$ | Parameters | Applications |
| --- | --- | --- | --- | --- |
| Periodic |  | $\frac{1}{2}+\frac{1}{2}\sin(\alpha_{ij}t+\phi_{ij})$ | $\omega_{ij}=(\alpha_{ij},\phi_{ij})$<br>$\alpha$ controls frequency<br>$\phi$ controls shift | Annotation reliability varies over time of day – e.g., due to changes in annotator attentiveness over the day []. |
| Decay |  | $\alpha_{ij}\exp(-\beta_{ij}t)$ | $\omega_{ij}=(\alpha_{ij},\lambda_{ij})$<br>$\alpha$ controls initial noise<br>$\beta$ controls decay rate | Annotation reliability improves rapidly (e.g., diagnostic accuracy of COVID-19 rapidly improved at the onset of the pandemic []) or improves and plateaus (e.g., irreducible uncertainty in labels []) |
| Growth |  | $\frac{\alpha_{ij}}{1+\exp(-\beta_{ij}(t-\gamma_{ij}))}$ | $\omega_{i,j}=(\alpha_{ij},\beta_{ij},\gamma_{ij})$<br>$\alpha$ controls limit<br>$\beta$ controls growth rate<br>$\gamma$ controls inflection point | Annotation reliability decreases abruptly – e.g., due to rapid adoption of improved clinical guidelines []. Or, it decreases gradually – e.g., due to underdiagnosis during the pandemic due to healthcare disruptions [] |

###### Definition 1 .

### 2.2 Loss Correction

Modeling temporal label noise is the first piece of the puzzle in training time series classifiers robust to label noise. However, we still need to consider how to leverage these noise models during empirical risk minimization. It remains unclear if and how existing loss correction techniques work for time series. Here we present theoretical results showing that learning is possible in our setting when we know the true temporal noise function ${\bm{Q}}(t)$. We include proofs in Appendix A.

We begin by treating the noisy posterior as the matrix-vector product of a noise transition matrix and a clean class posterior (Eq. 1). To this effect, we define the *forward sequence loss*:

###### Definition 2 .

Given a sequential classification task over $T$ time steps, a noise function $\bm{Q}(t)$, and a proper composite loss function (^1^11A loss that is well-calibrated for probability estimation (e.g., NLL loss) and incorporates a link function mapping model outputs to probability estimates (see [] for more detail)) $\ell_{t}$, the *forward sequence loss* of a model ${\bm{h}}_{\theta}$ on an instance $(\tilde{{\bm{y}}}_{1:T},{\bm{x}}_{1:T})$:

$$ $\displaystyle\overrightarrow{\ell}_{seq}({\tilde{{\bm{y}}}_{1:T}},{{\bm{x}}_{1 :T}},{{\bm{h}}_{\theta}}):=\sum_{t=1}^{T}\ell_{t}(\tilde{y}_{t},{\bm{Q}}_{t}^{ \top}{\bm{h}}_{\theta}({\bm{x}}_{1:t}))$ $$

An intriguing property of the *forward sequence loss* is that the minimizer of the *forward sequence loss* over the noisy labels maximizes the likelihood of the data over the clean labels. This suggests that the *forward sequence loss* is robust to label noise:

###### Theorem 1 .

A classifier that minimizes the empirical *forward sequence loss* over the noisy labels maximizes the empirical likelihood of the data over the clean labels.

$$ $\displaystyle\operatorname*{argmin}_{\theta\in\Theta}\mathbb{E}_{\tilde{{ \mathbf{y}}}_{1:T},{\mathbf{x}}_{1:T}}\overrightarrow{\ell}_{seq}({{\mathbf{ \tilde{y}}}_{1:T}},{{\mathbf{x}}_{1:T}},{{\bm{h}}_{\theta}})$ $\displaystyle=\operatorname*{argmin}_{\theta\in\Theta}\sum_{t=1}^{T}\mathbb{E} _{{\mathbf{y}}_{1:t},{\mathbf{x}}_{1:t}}\ell_{t}({\textnormal{y}}_{t},{\bm{h}} _{\theta}({\mathbf{x}}_{1:t}))$ $$

An alternative strategy is the *backward sequence loss*, which extends the backward loss of Patrini et al. [] to a temporal setting:

###### Definition 3 .

Given a sequential classification task over $T$ time steps, a noise function $\bm{Q}(t)$, the *backward sequence loss* of ${\bm{h}}_{\theta}$ on an instance $({\mathbf{\tilde{y}}}_{1:T},{\mathbf{x}}_{1:T})$ is:

$$ $\displaystyle\overleftarrow{\ell}_{seq}({{\mathbf{\tilde{y}}}_{1:T}},{{\mathbf {x}}_{1:T}},{{\bm{h}}_{\theta}}):=\sum_{t=1}^{T}\overleftarrow{\ell}_{t}({ \tilde{{\textnormal{y}}}_{t}},{{\bm{h}}_{\theta}({\mathbf{x}}_{1:t})}).$ $$

Here $\overleftarrow{\ell}_{t}({c},{{\bm{h}}_{\theta}({\mathbf{x}}_{1:t})})=[{\bm{Q}
}_{t}^{-1}]_{c,:}\cdot\boldsymbol{\ell}({\bm{h}}_{\theta}({\mathbf{x}}_{1:t}))$ and $\boldsymbol{\ell}({\bm{h}}_{\theta}({\mathbf{x}}_{1:t})):=[\ell(c,{\bm{h}}_{
\theta}({\mathbf{x}}_{1:t}))]_{c=1:C}^{\top}$ is a vector containing the negative log-likelihood loss of $\ell(c,{\bm{h}}_{\theta}({\mathbf{x}}_{1:t}))$ for observed class $c$.

Intuitively, the term “backward” reflects the fact that the function corrects for label noise by using the noise model and the noisy labels to infer the clean labels. In Theorem 2, we show that we can maximize accuracy on the clean labels using only noisy labels with the *backward sequence loss*:

###### Theorem 2 .

Minimizing the expected *backward sequence loss* over noisy label sequences maximizes the joint likelihood over clean label sequences.

$$ $\displaystyle\operatorname*{argmax}_{\theta\in\Theta}\mathbb{E}_{{\mathbf{y}}_ {1:T}|{\mathbf{x}}_{1:T}}\log\big{(}p_{\theta}({\mathbf{y}}_{1:T}|{\mathbf{x}} _{1:T})\big{)}$ $\displaystyle=\operatorname*{argmin}_{\theta\in\Theta}\mathbb{E}_{\tilde{{ \mathbf{y}}}_{1:T}|{\mathbf{x}}_{1:T}}\boldsymbol{\overleftarrow{\ell}}_{seq}( {{\bm{h}}_{\theta}({\mathbf{x}}_{1:T}}))$ $$

Both theorems imply that we can *train on the noisy distribution* and learn a noise-tolerant classifier in expectation. Though both loss functions provide this guarantee, the *backward sequence loss* requires a costly and potentially numerically unstable matrix-inversion at *every* time step. This can be especially problematic when $C$ and $T$ are large. More importantly, we find that the *backward sequence loss* performs *substantially* worse than *forward sequence loss* in practice (e.g., see Section 4.2 for Analysis). For these reasons, we utilize the *forward sequence loss* in subsequent sections.

###### Definition 2 .

###### Theorem 1 .

###### Definition 3 .

###### Theorem 2 .

## 3 Methods

We have shown how to leverage the temporal label noise function to train models robust to noise. However, the label noise function is generally not available *a priori* and should ideally be estimated from data. In this section we propose methods for *learning* the temporal noise function from noisy data alone. We present a number of methods for different use cases, including: joint methods, which learn the model and temporal noise function in a single-step procedure; as well as a plug-in approach, which first estimates the temporal noise function and "plugs in" to a loss correction method. Finally, we discuss practical considerations when choosing between them.

### 3.1 Joint Continuous Estimation

We start with a method that can simultaneously learn a time series classifier and the temporal noise function. Given a noisy dataset, we learn these elements by solving for $t=1,\dots,T$ an optimization problem of the form:

$$ $\displaystyle\begin{split}\min_{\omega,\theta}\quad&\text{Vol}(\bm{Q}_{\omega} (t))\\ \textrm{s.t.}\quad&\bm{Q}_{\omega}(t)^{\top}{\bm{h}}_{\theta}({\bm{x}}_{1:t})= p(\tilde{y}_{t}\mid{\bm{x}}_{1:t})\end{split}$ (2) $$

This objective, which we refer to as the Temporal Noise Robust objective (TENOR), is designed to return a faithful representation of the noise function $\hat{\omega}$ by imposing the *minimum-volume simplex* assumption [], and a noise-tolerant sequential classifier $\hat{\theta}$ by minimizing the *forward sequence loss* in Def. 2.

Here, the objective minimizes the volume of the noise matrix, denoted as $\text{Vol}({\bm{Q}}_{t})$. This returns a matrix ${\bm{Q}}_{t}$, at each time step, that obeys the *minimum-volume simplex* assumption, which is a standard condition used to ensure identifiability in static classification tasks [see e.g., , ].
In practice, the minimum-volume simple assumption ensures that ${\bm{Q}}_{t}$ encloses the noisy conditional data distribution at time $t$: $p(\tilde{{\textnormal{y}}}_{t}\mid{\mathbf{x}}_{1:t})$. Here containment ensures that the estimated noise matrix could have generated each point in the noisy dataset – i.e., so that the corresponding noisy probabilities $p(\tilde{{\textnormal{y}}}_{t}\mid{\mathbf{x}}_{1:t})$ obey Eq. 1. Our use of this assumption guarantees the identifiability of ${\bm{Q}}_{t}$ when the posterior distribution is sufficiently-scattered over the unit simplex [see also , for details].

#### Implementation

The formulation above applies to any generic matrix-valued function according to Def. 1 with parameters $\omega$. Because time series classification tasks can admit many types of temporal label noise functions (Table 1), we must ensure $\omega$ has sufficient representational capacity to handle many noise functions. Therefore in practice, we instantiate our solution as a fully connected neural network with parameters $\omega$, ${\bm{Q}}_{\omega}(\cdot):\mathbb{R}\rightarrow[0,1]^{c\times c}$, adjusted to meet Def. 1 (see Section D.2).

We can now model any temporal label noise function owing to the universal approximation properties of neural networks. Provided the function space $\Theta$ defines autoregressive models of the form $p(y_{t}|{\bm{x}}_{1:t})$ (e.g., RNNs, Transformers, etc.), we can solve Eq. 2 using an augmented Lagrangian method for equality-constrained optimization problems  []:

$$ $\displaystyle\mathcal{L}(\theta,\omega)=\frac{1}{T}\sum_{t=1}^{T}\left[\left\| {\bm{Q}}_{\omega}(t)\right\|_{F}+\lambda R_{t}(\theta,\omega)+\frac{c}{2}|R_{t }(\theta,\omega)|^{2}\right]$ (3) $$

Here:
$\left\|{\bm{Q}}_{\omega}(t)\right\|_{F}$ denotes the Frobenius norm of ${\bm{Q}}_{\omega}(t)$, which acts as a convex surrogate for $\textrm{Vol}({\bm{Q}}_{\omega}(t))$.
Likewise, $R_{t}(\theta,\omega)=\frac{1}{n}\sum_{i=1}^{n}\ell_{t}(y_{t,i},{\bm{Q}}_{
\omega}(t)^{\top}{\bm{h}}_{\theta}({\bm{x}}_{1:t,i}))$ denotes the violation of the equality constraint for each $t=1\ldots T$. $\lambda\in\mathbb{R}_{+}$ is the Lagrange multiplier and $c>0$ is a penalty parameter. Both are initially set to a default value of $1$, we gradually increase the penalty parameter until the constraint holds and $\lambda$ converges to the Lagrangian multiplier of Eq. 2 []. This approach recovers the best-fit parameters to the optimization problem in Eq. 2. Additional details on our implementation can be found in Appendix C.

### 3.2 Joint Discontinuous Estimation

An alternative strategy to learning the noise function, is to assume that there is no temporal relationship between each ${{\bm{Q}}_{t}}$ across time and treat each time step independently. This is realistic in situations where time steps are unevenly spaced or there are long periods of time between subsequent labels (e.g., some clinical data involves labels collected over years or decades). We can carry out this *discontinuous* estimation by adopting a similar objective as above. Denoting $R_{t}(\theta)=\frac{1}{n}\sum_{i=1}^{n}\ell_{t}(y_{t,i},\hat{{\bm{Q}}}_{t}^{
\top}{\bm{h}}_{\theta}({\bm{x}}_{1:t,i}))$:

$$ $\displaystyle\mathcal{L}(\theta,[\hat{{\bm{Q}}}_{t}]_{t=1}^{T})=\frac{1}{T} \sum_{t=1}^{T}\left[R_{t}(\theta)+\lambda\cdot\log\det(\hat{{\bm{Q}}}_{t})\right]$ (4) $$

The objective in Eq. 4, denoted as VolMinTime, minimizes the volume of $\bm{Q}$ using the $\log\det$ of a square matrix []. In contrast to Eq. 3, each $\hat{{\bm{Q}}}_{t}$ is parameterized with a *separate* set of trainable real-valued weights, which are learned independently with the data from time $t$ using a standard convex optimization algorithm (e.g., gradient descent).
This provides a direct time-series modification of a state-of-the-art technique for noise transition matrix estimation in the static setting [].

### 3.3 Plug-In Estimation

It is also advantageous to have a simple, plug-in estimator of the temporal noise function. Plug-in estimators are model-agnostic, can flexibly be deployed to other models, and can be efficient to estimate. We can construct a plug-in estimator of temporal label noise using *anchor points*, instances whose labels are known to be correct. Empirical estimates of the class probabilities of anchor points can be used to estimate the noise function. This estimate can then be plugged into any of the aforementioned loss correction methods in Section 3. Formally, in a sequential setting, anchor points [, , ] are instances that maximize the probability of belonging to class $i$ at time step $t$:

$$ $\displaystyle\bar{{\bm{x}}}_{t}^{i}=\operatorname*{arg\,max}_{{\bm{x}}_{t}}p({ \tilde{{\textnormal{y}}}_{t}=i\mid{{\bm{x}}}_{1:t}})$ (5) $$

Since $p({{\textnormal{y}}_{t}=i\mid\bar{{\bm{x}}}_{1:t}^{i}})\approx 1$ for the clean label, we can express each entry of the label noise matrix as:

$$ $\displaystyle{\hat{\bm{Q}}}(t)_{i,j}=p({\tilde{{\textnormal{y}}}_{t}=j\mid\bar {{\bm{x}}}^{i}_{1:t}}).$ (6) $$

We can use a two-step approach to get a plug-in estimate of $\hat{{\bm{Q}}}(t)$. We denote this approach as AnchorTime, where we identify anchor points for each class $y\in\mathcal{Y}$ and $t=1,\dots,T$ and set each entry of $\hat{\bm{Q}}(t)_{i,j}$ according to Eq. 6 (see Appendix B for details). This is analogous to the approach in static prediction tasks by Patrini et al. [].

### 3.4 Discussion

All three methods improve performance in sequential classification tasks by accounting for temporal label noise (see e.g., Fig. 1 and Section 4). However, they each have their own strengths and limitations. Here we provide practical guidance for users to discriminate between methods:

- •
Joint Continuous Estimation (TENOR) imposes continuity across time steps – assuming that nearby points likely have similar noise levels. This assumption can improve reconstruction, and thus performance, in settings with multiple time steps as it reduces the effective number of model parameters. Conversely, it may also lead to misspecification in settings that exhibit discontinuity.
- •
Joint Discontinuous Estimation (VolMinTime) can handle discontinuous temporal noise processes, but requires fitting more parameters, which scales according to $T$ – this can lead to computational challenges and overfitting, especially for long sequences.
- •
Plug-In estimation (AnchorTime) has a simpler optimization problem. This is useful in cases where separate datasets are used to estimate the noise and to learn the classifier. However, the existence of anchor points is difficult to verify [].

## 4 Experiments

We benchmark our methods on a collection of sequential classification tasks from real-world applications. Our goal is to evaluate methods in terms of how robust they are to temporal label noise, and characterize when it is actually important to consider such noise. We provide additional details on our setup and results in Appendix D.

### 4.1 Setup

#### Datasets

We use four real-world (binary and multiclass) datasets from healthcare, each representing a time-series classification task with sequential labels over a complex feature space that is likely to exhibit temporal label noise. These datasets include sequential accelerometer data for human activity recognition (har [], har70 []) and continuous EEG signals for sleep detection (eeg_sleep []) and blink detection (eeg_eye []). Refer to Appendix D for more details.

To study our approach under the exact data-generating assumptions (1 and 2), we also consider a synthetic dataset, synth. Synthetic data allows us to confidently identify the causes of variance in model performance.
All datasets were treated as binary classification tasks. We also include multiclass results for eeg_sleep, har, and synth in Section E.7.

#### Noise Models

We consider labels in the training sample as “ground truth” clean labels and corrupt them using one of five different temporal label noise functions as well as a time-independent function at various noise levels. Three functions are shown in Fig. 2, and complete descriptions are provided in Section D.4.

#### Methods

We highlight the key differences between TENOR and existing approaches by training a sequential classifier for each dataset using different Algorithms grouped as follows:

- *Static.* We consider an Uncorrected learning baseline (NLL loss), which assumes no noise exists and two static methods: Anchor [, , ] and VolMinNet [].
- *Temporal.* We consider TENOR and temporal variations of the static methods that we develop as described in Section 3. We refer to these as AnchorTime, VolMinTime, respectively.

For each dataset, we fit a RNN with *Gated Recurrent Units* (GRU) [], which is often used in classifying sequential labels [, ]. TENOR uses an additional fully-connected neural network with $10$ hidden layers to estimate ${\bm{Q}}(t)$. Additional details on network architecture are in Section D.2.

#### Evaluation

We split each dataset into a *noisy* training sample (80%, used to train the models and correct for label noise) and a *clean* test sample (20%, used to compute unbiased estimates of out-of-sample performance). We evaluate each model in terms of the *test accuracy* on test data and characterize how well each method learns the temporal noise function, using the *Mean Absolute Error* $\textrm{MAE}({\bm{Q}}_{t},\hat{{\bm{Q}}}_{t}):=\frac{1}{T}\sum_{t=1}^{T}\|{\bm
{Q}}_{t}-\hat{{\bm{Q}}}_{t}\|$ between the true ${\bm{Q}}_{t}$ and estimated $\hat{{\bm{Q}}}_{t}$ for all $t$.

### 4.2 Results and Discussion

**Table 2: Clean test set accuracy (%) and reconstruction error $\textrm{MAE}(\hat{{\bm{Q}}}(t))$ for all methods and datasets with 30% label noise on average. We report the mean value $\pm$ st.dev for each metric over 10 runs and highlight methods that achieve the best results in green. As shown, TENOR achieves superior performance to all baselines. We show results for *Mixed* noise function, which denotes class-conditional noise where one class has increasing noise and one class has decreasing noise over time, and include other results in Appendix E.**
|  |  | synth | har | har70 | eeg_eye | eeg_sleep |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Accuracy $\uparrow$ | MAE $\downarrow$ | Accuracy $\uparrow$ | MAE $\downarrow$ | Accuracy $\uparrow$ | MAE $\downarrow$ | Accuracy $\uparrow$ | MAE $\downarrow$ | Accuracy $\uparrow$ | MAE $\downarrow$ |
| Static | Uncorrected | 79.2$\pm$1.2 | – | 70.6$\pm$3.0 | – | 77.3$\pm$1.7 | – | 71.3$\pm$.8 | – | 68.2$\pm$2.0 | – |
| Anchor | 80.8$\pm$0.7 | .16$\pm$.00 | 79.1$\pm$2.6 | .13$\pm$.01 | 79.3$\pm$.01 | .11$\pm$.01 | 75.1$\pm$1.1 | .10$\pm$.01 | 65.9$\pm$2.3 | .11$\pm$.00 |  |
| VolMinNet | 85.6$\pm$0.9 | .10$\pm$.00 | 85.1$\pm$2.7 | .11$\pm$.00 | 81.0$\pm$0.7 | .11$\pm$.00 | 73.2$\pm$1.4 | .13$\pm$.00 | 70.3$\pm$2.2 | .11$\pm$.00 |  |
| \cdashline2-12<br>Temporal | AnchorTime | 85.3$\pm$0.8 | .10$\pm$.01 | 80.0$\pm$1.8 | .11$\pm$.01 | 81.2$\pm$1.1 | .08$\pm$.00 | 79.5$\pm$1.8 | .06$\pm$.01 | 70.4$\pm$2.8 | .06$\pm$.00 |
|  | VolMinTime | 85.8$\pm$0.5 | .08$\pm$.00 | 82.4$\pm$2.7 | .10$\pm$.00 | 82.0$\pm$1.2 | .08$\pm$.00 | 76.9$\pm$1.8 | .08$\pm$.00 | 70.3$\pm$3.9 | .09$\pm$.01 |
|  | TENOR | 94.4$\pm$1.0 | .03$\pm$.01 | 95.8$\pm$2.2 | .03$\pm$.01 | 89.0$\pm$0.3 | .02$\pm$.00 | 83.7$\pm$0.5 | .01$\pm$.00 | 70.4$\pm$2.3 | .04$\pm$.01 |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: 1.	What is the meaning of variable d in line 126? Is that mean a multivariate time series with d variables?
2.	When performing a train-test split on the datasets, were individuals also split?
soundness: 3
strengths: 1.	The problem proposed by the authors is novel and interesting, and they evaluate it on real-world datasets.
2.	The method introduced in the manuscript is cleverly crafted and has a theoretical foundation.
summary: The manuscript introduced temporal label noise in time series classification tasks and proposed a novel framework that are robust to it. Experiments were conducted on 4 datasets with synthetic noise injection and a real-world noisy labelled dataset, in which the method proposed in the manuscript is superior to other methods. The authors also visualized the dynamic evolution of label noise over time.
weaknesses: 1.	In many settings, the performance of Plug-In method is inferior to that of static methods.
2.	Only one of the five datasets in the manuscript contains both clean and noisy labels. In the remaining real-world scenarios, there is no experimental evidence to confirm that “the label noise evolves over time” actually exists.
3.	The experimental scenarios are focused on healthcare; validating the work in more diverse scenarios would broaden its applicability.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: See also weakness.

The model does not use the noisy label in the prediction (the predictor h_{\theta} only depends on x_{1:t}).  I believe it would be interesting to include the noisy label in the prediction of the true label (i.e., if the noisy label is always accurate, can we use it for prediction? I understand this is a slightly different setting, as the goal of the paper is to learn a predictor).   

The fact that we learn a noisy label structure Q_w(t) that is “continual” over time seems implicit in Eq. (3). However, it is not clear what the difference is between Eq(3) and the model discussed in “discontinuous estimation”. It seems to me that $t$ is an integer, and $Q_w(t)$ could be completely different than $Q_w(t+1)$ in principle (what is the temporal relationship discussed in line 274)

It is a bit unclear how Section 2, in particular section 2.2, is related to the proposed method in Section 3. It seems to me, that the “loss of the classifier” is embedded in the constraint of Equation 2. Lines 233-234 says that Eq(2) minimizes the forward temporal loss as in Def 2, but this loss does not actually appear on Equation 2.   

Typos: 277 continuuity
soundness: 2
strengths: Learning from time series data is an interesting problem. The paper addresses a challenging problem, where the label noise can change over time, and it is well-motivated. Overall, the introduction of the paper and the experimental sections are well-written and easy to follow. I found the use of the minimum-volume simplex assumption as an objective to solve the problem to be intriguing.
summary: The paper discusses the problem of learning from time series classification task, where the label noise can vary over time. Each instance is a sequence over T time steps, where at iteration $t$ we have access to the features at time $t$, and a noisy label $\tilde y_t$ obtained from the true label $y_t$. Each of this instance is i.i.d., and the assumption is that the true label $y_t$ at time $t$ only depends on the past features $x_{1:t}$, and the noisy labels are conditionally independent to the features given by the true labels. In particular, the authors assume there exists a noise mapping $Q_t$ that describes the noise process at time $t$. They propose a novel method to learn a classifier that maps sequences $x_{1:t}$ to a label $y$. This model simultaneously approximates the noisy mapping $Q_t$ (Eq 3), that is used to approximate the true labels, which are used during training
weaknesses: (**important**). The appendix of the paper is only a draft and it looks like it was not finished to be written. It also seems to be dissociated to the main paper.
- Appendix A is self contained and not referenced in the main paper. It is not discussed what the content of this section adds as a contribution.
- Appendix B looks dissociated to the main paper. First of all, three assumptions are introduced (they seems related to the 2 assumptions used in the main paper. In this case, why re-introduce them, and why do we use 3 assumptions rather than 2?). Most of the proofs are just a sequence of mathematical equations with text. Lines 810 to 840 are just a sequence of equations without text (maybe do a table?). The proof of Proposition 1 does not appear, which is only the theoretical result in the main paper (is Theorem 3 the proof of Proposition 1? Why does it have  different notation?).
- Section G.2 is empty. Page 25 to 47 include a sequence of a lot of figure without almost no text. I recommend the authors to only include the figures that are used to “say” something, together with a text explaination.

 The technical section is also sometimes unclear.  

Notations is sometimes unclear. What is q_t in line 140? According to the notation defined in the paper $\mathcal{X} \subseteq \mathbb{R}^{d \times T}$, but the function $h_{\theta}$ that has domain $\mathcal{X}$ can also have an input matrix $d \times t$. 
 
There is no comment on the assumptions 1 and 2 (except that they are two standard assumptions). I believe a few lines commenting on those assumptions would be helpful.

In lines 259-261, why is the Frobenius norm a convex surrogate for the volume? (A citation is also probably needed).  

In Line 260, should $R_t$ be defined over $\tilde{y}$ rather than $y$?

 In lines 262-264, my understanding is that $\lambda$ is the Lagrange multiplier of the constraint of Equation~2. I would be clearer on this, since $\lambda$ does not appear in Eq 2.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 4
questions: (1) Is there an assumption that all data instances have equal number of time steps?
soundness: 4
strengths: 1) The formulation of the proposed methodology is clear and concise.
2) The motivation and subsequent real-world example is well demonstrated.
summary: The paper proposes a model to incorporate varying noise rates for time-series classification. The method specifies a matrix-valued function indicating label-noise distribution. This noise function is incorporated into a forward temporal loss over noisy labels that is used to learn a classifier (using a neural net) minimizing the loss. The authors describe three approaches: continuous, discontinuous, and plug-in. They empirically evaluate these approaches in various benchmark datasets. They also demonstrate the approach in a real-world dataset within the healthcare domain in which the authors claim that the problem of temporal noisy labels is prevalent in time-series classification. The empirical results clearly indicate that the proposed approach works well compared to SOTA static approaches that treat noisy labels as static rather than temporal.
weaknesses: (1) It is not clear how is learning achieved when weights are initialized for each time step in discontinuous estimation. It looks like the dataset is small for the learning models to converge as all the datasets largely seem to have less than 1000 samples.
(2) Can the estimator in the discontinuous case be replaced by a simpler model (with lesser number of parameters)? If so, can such an alternative be used as baseline mechanisms to compare for temporal noise (as all alternative mechanisms shown in the paper are for static cases)? 
(3) It is interesting that both continuous and discontinuous estimations perform quite similarly on the real-world stress detection example. Does this mean that the proposed approach works best when there is large variance in noise rates?

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: See Weaknesses above
soundness: 3
strengths: 1. The problem of Temporal Label Noise is both interesting and important, and this paper is the first to formally define it.

2. The paper is well-organized, clearly written, and easy to follow.

3. The mathematical symbols and equations are well-defined and easy to understand.
summary: The paper introduces the problem of temporal label noise, where label quality fluctuates over time due to time-dependent noise. This type of noise may improve, deteriorate, or change periodically, impacting the accuracy of predictions. The authors define this problem and highlight that existing methods fail to handle the temporal aspect of label noise effectively. They propose methods to estimate the time-varying noise function directly from data, allowing the development of noise-tolerant classifiers. Experiments on real-world datasets validate their proposal, demonstrating its effectiveness under various types of temporal label noise.
weaknesses: 1. My primary concern is with the problem formulation, or more specifically, the goal of the problem. In Section 2.1, the paper aims to "estimate parameters $\hat{\theta}$ for a model robust to noise," which implicitly assumes the existence of a **fixed and static optimal model** $\theta^\star$—i.e., that p(y_t|x_t) remains consistent across all timestamps. However, in time-series problems, the environment is constantly changing, and the optimal model for different timestamps should also be changing.

2. The other major concern relates to the methodology. The paper appears to assume that the function Q is known to the learner, and focuses only on optimizing the parameter w. However, I think that **obtaining an appropriate function Q is the most challenging aspect of this problem**. Although in Section 3.2, the authors select Q as a fully connected neural network as a general form for the practical scenarios, it seems to be inconsistent with the earlier discussions in Table 1.

3. Another concern is about the experiments. The authors seem to only conduct experiments on low-dimensional datasets. I think that including experiments on larger-sized datasets would strengthen the results and make the findings more convincing.

Please let me know if I have misunderstood any part of the paper.

Meta-review: A fairly good paper that should be accepted for publication at ICLR. I hope this paper can advance the research area label-noise learning.

My only comment is about related work. The standard non-temporal label noise can be regarded as a type of distribution shift, where the test distribution is clean and the training distribution changes to some noisy version. Then, the temporal label noise is actually a type of continuous distribution shift, where the change from test to training is also time-dependent. Even though continuous distribution shift focused on continuous covariate shift rather than continuous class-posterior shift (i.e., label noise), it is still related to temporal label noise as a bigger topic covering the problem under consideration. However, the term distribution shift didn't appear at all. I hope you can acknowledge that your temporal label noise is a special case of continuous distribution shift, so that you have also contributed to distribution shift research in addition to label-noise research.

Decision: Accept (Poster)