Title: Con4m: Context-aware Consistency Learning Framework for Segmented Time Series Classification
Venue: NeurIPS 2024
Accepted: yes
Keywords: Time series classification

Segmented time series

Label consistency learning
Abstract: Time Series Classification (TSC) encompasses two settings: classifying entire sequences or classifying segmented subsequences. The raw time series for segmented TSC usually contain Multiple classes with Varying Duration of each class (MVD). Therefore, the characteristics of MVD pose unique challenges for segmented TSC, yet have been largely overlooked by existing works. Specifically, there exists a natural temporal dependency between consecutive instances (segments) to be classified within MVD. However, mainstream TSC models rely on the assumption of independent and identically distributed (i.i.d.), focusing on independently modeling each segment. Additionally, annotators with varying expertise may provide inconsistent boundary labels, leading to unstable performance of noise-free TSC models. To address these challenges, we first formally demonstrate that valuable contextual information enhances the discriminative power of classification instances. Leveraging the contextual priors of MVD at both the data and label levels, we propose a novel consistency learning framework Con4m, which effectively utilizes contextual information more conducive to discriminating consecutive segments in segmented TSC tasks, while harmonizing inconsistent boundary labels for training. Extensive experiments across multiple datasets validate the effectiveness of Con4m in handling segmented TSC tasks on MVD. The source code is available at https://github.com/MrNobodyCali/Con4m.

Con4m : Context-aware Consistency Learning Framework for Segmented Time Series Classification

Abstract

Time Series Classification (TSC) encompasses two settings: classifying entire sequences or classifying segmented subsequences. The raw time series for segmented TSC usually contain M ultiple classes with V arying D uration of each class ( MVD ). Therefore, the characteristics of MVD pose unique challenges for segmented TSC, yet have been largely overlooked by existing works. Specifically, there exists a natural temporal dependency between consecutive instances (segments) to be classified within MVD . However, mainstream TSC models rely on the assumption of independent and identically distributed ( i.i.d. ), focusing on independently modeling each segment. Additionally, annotators with varying expertise may provide inconsistent boundary labels, leading to unstable performance of noise-free TSC models. To address these challenges, we first formally demonstrate that valuable contextual information enhances the discriminative power of classification instances. Leveraging the contextual priors of MVD at both the data and label levels, we propose a novel consistency learning framework Con4m , which effectively utilizes contextual information more conducive to discriminating consecutive segments in segmented TSC tasks, while harmonizing inconsistent boundary labels for training. Extensive experiments across multiple datasets validate the effectiveness of Con4m in handling segmented TSC tasks on MVD . The source code is available at https://github.com/MrNobodyCali/Con4m .

1 Introduction

Time Series Classification (TSC) is one of the most challenging problems in the field of machine learning. TSC aims to assign labels to a series of temporally ordered data points. These points either form a complete sequence or are subsequences (segments) resulting from the segmentation of a long time series. Existing works [50, 20] largely focus on the assumption of independent and identically distributed ( i.i.d. ), in which case each sequence or segment is regarded as an independent instance to be classified, not differentiating between these two settings. In fact, for many practical applications, the raw time series before segmentation for segmented TSC tasks contain M ultiple classes with

Figure 1: (a) Reasonable model predictions exhibit coherence across consecutive segments rather than repeated interruptions. (b) In the healthcare domain, different physicians have varying annotations regarding the start and end times of seizure waves. (c) Based on the proximity to the boundary, we divide each class sequence into 5 levels, from which an equal number of segments are sampled. A one-layer MLP is trained on the segments from each level respectively for the same number of epochs. (d) We visualize the predicted probability of the trained MLP for each level. We observe that as the segments approach the boundaries, the model finds it increasingly challenging to make correct classifications, resulting in more extreme wrong predictions. This strongly underscores the significance of handling boundary segments.

V arying D uration of each class ( MVD ). For example, in the healthcare domain, the brain signals of epileptic patients often record over several days, encompassing multiple seizure onsets, each with varying durations and intervals. In the field of activity recognition, sensors continuously record users' behavior data, including walking, riding, and running, among other activities, each with varying durations. Therefore, the characteristics of the raw MVD lead to the uniqueness of segmented TSC tasks. Given this, our research concentrates on effectively modeling segmented TSC tasks based on MVD , presenting distinctive challenges.

(1) Leveraging contextual information. In contrast to TSC tasks for complete sequences, in which classified sequences are relatively independent, there exist natural temporal dependencies between consecutive classified segments for segmented TSC. We take the seizure detection task as an example, in which given the brain signals of epileptic patients, the model should identify whether a segment includes seizure waves or not. As illustrated in Figure 1(a), given the sustained nature of seizure onsets, the model predictions for consecutive segments should exhibit coherence, with seizure and normal predictions appearing in continuous and concentrated patterns. However, mainstream TSC models [58, 71, 61] focus on the i.i.d. assumption and model the internal context within each segment to be classified, largely overlooking the dependencies between consecutive segments.

In the domain of video analysis, works in temporal action segmentation (TAS) [17] have modeled the temporal dependency between different video frames and made frame-wise predictions. However, unlike the I3D features [8] used as input in these works, time series lack a unified pretrained model for feature extraction, and the dependency between segments is more variable and ambiguous. Furthermore, TAS works focus on modeling the dependency of instances from a data perspective, without explicitly leveraging contextual label information. Therefore, how to leverage contextual information in segmented TSC tasks to make more reasonable classifications is crucial and challenging.

(2) Inconsistent boundary labels. In domains with precious labels, different annotators collaboratively contribute annotations. The raw annotations of MVD typically include the start and end times for each class. However, due to inherent ambiguity and a lack of unified quantification standards, for MVD , the boundaries between states are not clearly defined, or the transitional state itself represent a mixed state. Consequently, behind inconsistent labels, there is no artificially defined true label. Therefore, our work aims to harmonize this inconsistency as much as possible to reduce the instability

of model training and enhance its performance. Returning to Figure 1(b), in the seizure detection task, owing to the natural fuzzy transition from seizure onset to completely normal, different physicians have varying experiences regarding when seizure waves terminate.

Furthermore, inconsistent boundary labeling causes that boundary segments with similar patterns may have opposite labels, leading to unstable model training. To validate the detrimental impact, as shown in Figure 1(c), we divide each class sequence in the seizure detection task into 5 levels, where higher levels indicate proximity to the boundary. We then sample an equal number of balanced binary segments for each level. Subsequently, a one-layer MLP is trained for the same number of epochs on the segments from each level respectively. Figure 1(d) visualizes the results after training. We observe that as the level increases (closer to the boundary), the model's accuracy steadily decreases, and erroneous predictions become more extreme. The results highlight the significant detrimental impact of inconsistent boundary labels on noise-free model performance.

Noisy label learning (NLL) [56] aims to learn robust models from data containing corrupted labels. While the inconsistent labels in MVD are not intentionally corrupted but rather stem from implicit discrepancies due to experiential differences, NLL remains the most relevant approach to address such discrepancies. To the best of our knowledge, Scale-teaching [47] and SREA [9] are the only NLL works specifically designed for time series and are thus the most relevant to our work. However, they also face the issue of overlooking contextual dependencies across consecutive time segments, posing the challenge of handling inconsistent boundary labels using context during training.

To overcome the challenges above, we propose Con4m -a label Con sistency learning framework, which leverages effective Con textual information, achieving Co here n t predictions and Con tinuous representations for seg m ented TSC tasks, while harmonizing inconsistent boundary labels for training. Specifically, we first formally demonstrate that valuable contextual information enhances the discriminative power of classification instances. Based on the insights, by incorporating prior knowledge of data locality and label coherence, we guide and constrain the model to focus on contextual information more conducive to discriminating consecutive segments in segmented TSC tasks. Meanwhile, leveraging model predictions that thoroughly encompass contextual information, Con4m progressively changes the training labels in an adaptive manner to harmonize inconsistent labels across consecutive segments. This leads to a more robust model. Our contributions are summarized as follows:

(1) We are the first to propose a practical consistency learning framework Con4m for the segmented TSC based on the raw MVD . (2) By comprehensively integrating prior knowledge from the data and label perspectives, we guide the model to focus on effective contextual information. Based on context-aware predictions, a progressive harmonization approach for handling inconsistent training labels is designed to yield a more robust model. (3) Extensive experiments on three public and one private MVD datasets demonstrate the superior performance of Con4m . The Con4m 's ability to harmonize inconsistent labels is further verified by the label substitution experiment and case study.

2 Theoretical Analysis

In this section, we aim to formally demonstrate the benefit of contextual information for classification tasks, and to establish the existence of an upper bound for this benefit. Consequently, by introducing prior knowledge, we can guide the model to focus on valuable contextual information more conducive to improving the benefit for segmented TSC tasks.

Assuming that the random variables of the instances to be classified and the corresponding labels are denoted as x t and y t . A t represents the contextual instance set introduced for x t . x A t denotes the random variable for the contextual instance set. Mutual information measures the correlation between two random variables. In a classification task, a higher correlation between instances and labels indicates that the instances are more easily distinguishable by the labels. This benefits the classification task, making it more readily addressable. Therefore, from an information-theoretic perspective, we elucidate the benefit of contextual information through the following theorem.

Theorem 2.1. The more the introduced contextual instance set enhance the discriminative power of the target instance, the greater the benefit for the classification task.

Proof. Firstly, we establish that the introduction of contextual information does not compromise classification tasks, i.e. , it does not diminish the correlation between instances and labels.

The inequality holds due to the non-negativity of conditional mutual information.

According to (1), the increase in I ( y t ; x A t | x t ) determines the extent to which the introduction of contextual information can be beneficial for classification tasks. Expanding I ( y t ; x A t | x t ) , we have:

Given a fixed instance x t and the inherent distribution p ( y t | x t ) of the data, the KL divergence is a convex function for x A t that attains its minimum at p ( y t | x t , x A t ) = p ( y t | x t ) . As p ( y t | x t , x A t ) approaches the boundary of the probability space, where the predictive probability of one class approaches 1 and the rest approach 0 , the value of KL divergence increases. A stronger discriminative power regarding x t implies less uncertainty regarding y t , which is equivalent to approaching the boundary of the probability space.

Due to the convexity of the KL divergence and the boundedness of p ( y t | x t , x A t ) , there exists a contextual instance set in the data that maximizes D KL ( p ( y t | x t , x A t ) ∥ p ( y t | x t )) . We denote the instance set as A ∗ t and the maximum value of KL divergence as D ∗ t . Besides, we note that ∑ x A t p ( x A t | x t ) = 1 . Hence, we can obtain the upper bound for the information gain I ( y t ; x A t | x t ) ≤ ∑ x t p ( x t ) ∑ x A t p ( x A t | x t ) D ∗ t ≤ ∑ x t p ( x t ) D ∗ t . The convexity of the KL divergence also implies monotonicity, indicating that as A t approaches A ∗ t , the KL divergence increases, leading to a greater information gain for the classification task.

According to Theorem 2.1, valuable contextual information enhances the discriminative power of the instances. While the optimal instance set A ∗ t is challenging to directly obtain or optimize, focusing the model on contextual instances more likely to be included in A ∗ t is beneficial for enhancing the performance of the classification task. Furthermore, x A t not only contains information at the data level but also encompasses information at the label level (which can be replaced with y A t ). Therefore, we can guide the model to focus on contextual information more conducive to segmented TSC tasks by simultaneously introducing prior knowledge from both the data and label perspectives.

3 The Con4m Method

In this section, we introduce the details of Con4m . Based on the insights of Theorem 2.1, we introduce contextual prior knowledge of data locality (Sec. 3.1) and label coherence (Sec. 3.2) to guide the model to focus on contextual information more conducive to discriminating consecutive segments in segmented TSC tasks. In Sec. 3.3, inspired by the idea of noisy label learning, we propose a label harmonization framework to achieve a more robust model. Before delving into the details of Con4m , we provide the formal definition of the segmented TSC task in our work.

Definition 3.1. Given a time interval comprising of T consecutive time points and labels, denoted as ( X,Y ) = { ( X 1 , Y 1 ) , ( X 2 , Y 2 ) , . . . , ( X T , Y T ) } , a w -length sliding window with stride length r is employed for segmentation. ( X,Y ) is partitioned into L time segments, represented as ( x, y ) = { ( x i , y i ) = ( { X ( i -1) × r +1 , . . . , X ( i -1) × r + w } , Majority ( { Y ( i -1) × r +1 , . . . , Y ( i -1) × r + w } )) | i = 1 , . . . , L } . The model is tasked with predicting segmented labels y i for each time segment x i .

3.1 Continuous Contextual Representation Encoder

Local continuity is an inherent attribute of MVD , meaning each class should be locally continuous and only change at its actual boundary. Smoothing with a Gaussian kernel [18, 16, 66] promotes the continuity of representations of time segments in a local temporal window. This not only helps the model make similar predictions of consecutive segments within the same class but also aligns with the gradual nature of class transitions. Furthermore, for graph neural networks based on the

homophily assumption, aggregating neighbor information belonging to the same class can improve the discriminative power of the target instance [49, 73]. Therefore, we introduce the Gaussian prior to guide the model to focus on contextual instances A t proximate to the target instance.

Vanilla self-attention [15] with point-wise attention computations often fail to obtain continuous representations after aggregation. Therefore, we use the Gaussian kernel Φ( x, y | σ ) as prior weights to aggregate neighbors to obtain smoother representations. Since the neighbors of boundary segments may belong to different classes, we allow each segment to learn its own scale parameter σ . Formally, as Figure 2(a) shows, the two-branch Con-Attention in the l -th layer is:

where L is the number of consecutive segments, d is the dimension of hidden representations, c l -1 ∈ R L × d is the output representations of the l -1 -th layer, and W l ∗ ∈ R d × d are all learnable matrices. Rescale ( · ) refers to row normalization by index i . To distinguish between two computational branches, we use g/G to represent the branch based on Gaussian prior, and s/S to represent the branch based on self-attention. S l and G l are the aggregation weights. We use the conventional attention mechanism [4] to adaptively fuse z l s and z l g . Finally, as illustrated in Figure 2(a), by stacking the multi-head version of Con-Attention layers, we construct Con-Transformer, which serves as the backbone of the continuous encoder of Con4m to obtain final representations c . We employ learnable absolute positional encoding (APE) [23] for the input representations.

3.2 Context-aware Coherent Class Prediction

In the segmented TSC task of MVD , consecutive time segments not only provide contextual information at the data level but also possess their own class information. As depicted in Figure 1(a), considering the persistence of each class and the gradual nature of class transitions, the model's predictions should exhibit more coherence and concentration, rather than being interspersed. Therefore, we integrate and constrain the model's predictions from both the individual and holistic perspectives to achieve more coherent predictions.

Neighbor Class Consistency Discrimination. In graphs, label propagation algorithms [30, 33] are often utilized to refine and smooth the predictions of neighbor instances, thereby enhancing their discrimination. Drawing inspiration from this, by weightedly aggregating predictions from similar time segments, the model can focus on contexts A t more likely to belong to the same class as the target segment. Although there is no explicit graph structure between time segments, we can train a discriminator to determine whether two segments belong to the same class. The model then aggregates the contextual class predictions based on the discriminator's outputs, thus making more robust predictions. As the right part of Figure 2(b) shows, we formalize this process as follows:

where ˆ R ∈ R L × L × 2 is the probability of whether two segments in the same time interval belong to the same class and ( ·∥· ) denotes tensor concatenation. ˆ p represents the model's independent prediction for a segment, while ˜ p denotes the context-aware prediction that incorporates the results from neighboring segments. We then define the two training losses as ℓ 1 = CrossEntropy (ˆ p, y ) and ℓ 2 = CrossEntropy ( ˆ R, ˜ Y ) , where ˜ Y = [ 1 y i =y j ] i,j ∈{ 1 ,...,L } . Given that ℓ 1 and ℓ 2 are of the same magnitude, we equally sum them as the final loss.

Prediction Behavior Constraint. Unlike graphs, there exists a holistic temporal relationship between consecutive time segments. Therefore, we should further constrain the overall predictive behavior along the time axis. For MVD , as Figure 1(a) shows, within a suitably chosen time interval, consecutive segments almost span at most two classes. Therefore, we ensure the monotonicity of predictions across consecutive segments through hard constraints, thereby utilizing contextual label information y A t to integrate and refine predictions across these segments.

As shown in the middle part of Figure 2(b), for each class in the predictions, there are only four prediction behaviors for consecutive segments, namely high confidence , low confidence , confidence decreasing , and confidence increasing . To constrain the behavior, we use function fitting to integrate ˜ p . Considering the wide applicability, we opt for the hyperbolic tangent function ( i.e. , Tanh) as our basis. Formally, we introduce four tunable parameters to exactly fit the monotonicity as:

where parameter a constrains the range of the function's values, k controls the slope of the transition of the function, b and h adjust the symmetry center of the function, and x is the given free vector in the xcoordinate. We use the MSE loss to fit the contextual predictions ˜ p as ℓ 3 = ∥ Tanh ( x | a, k, b, h ) -˜ p ∥ 2 . It deserves to be emphasized that ˜ p in the process has no gradient and therefore does not affect the parameters of the encoder. Please see Appendix B for more fitting details.

After function fitting, we obtain independent predictions ˆ p for each segment and constrained predictions ¯ p that leverage contextual label information. For the inference stage, we use the average of them as the final coherent predictions, i.e. , ˆ y = arg max (ˆ p + ¯ p ) / 2 .

3.3 Label Consistency Training Framework

Due to inherent ambiguity, the annotation of MVD often lacks quantitative criteria, resulting in experiential differences across individuals. Such discrepancies are detrimental to models and we propose a training framework to enable Con4m to adaptively harmonize inconsistent labels.

Learning from easy to hard. We are based on the fact that although people may have differences in the fuzzy transitions between classes, they tend to reach an agreement on the most significant core part of each class. In other words, the empirical differences become more apparent when approaching the transitions. Therefore, we adopt curriculum learning techniques to help the model learn instances

from the easy (core) to the hard (transition) part. Formally (see the diagram in Figure 1(b)), for a continuous K -length class, we divide it into N l = 5 equally sized levels as follows:

Then we sample the same number of time intervals from each level. The higher the level, the more apparent the inconsistency. Therefore, as the left part of Figure 2(b) shows, during the training stage, Con4m learns the time intervals in order from low to high levels, with a lag gap of E g = 5 epochs.

Harmonizing inconsistent labels. Inspired by the idea of noisy label learning, we gradually change the raw labels to harmonize the inconsistency. The model preferentially changes the labels of the core segments that are easier to reach a consensus, which can avoid overfitting of uncertain labels. Moreover, the model will consider both the independent and constrained predictions to robustly change inconsistent labels. Specifically, given the initial label y 0 , we update the labels y e = arg max p e for the e -th epoch, where p e is obtained as follows:

where ω e = Rescale ([ exp (( e -m ) / 2)] m ∈{ 0 ,..., 4 } ) is the exponentially averaged weight vector to aggregate the predictions of the latest 5 epochs to achieve a more robust label update. ˆ p e -m and ¯ p e -m are the independent and constrained predictions in the e -m -th epoch respectively and · denotes the dot product. The dynamic weighting factor, η , is used to adjust the degree of label update. As the left part of Figure 2(b) shows, η linearly increases from 0 to 1 with E η epochs, gradually weakening the influence of the original labels. Besides, in the initial training stage, the model tends to improve independent predictions. As the accuracy of independent predictions increases, the model assigns a greater weight to the constrained predictions. See the hyperparameter analysis for E η in Appendix C.

4 Experiment

4.1 Experimental Setup

Datasets. In this work, we use three public [31, 7, 37] and one private MVD data to measure the performance of models. Specifically, the Tufts fNIRS to Mental Workload [31] data ( fNIRS ) contains brain activity recordings from adult humans performing controlled cognitive workload tasks. The HHAR (Heterogeneity Human Activity Recognition) dataset [7] captures sensor data from multiple smart devices to explore the impact of device heterogeneity on human activity recognition. The SleepEDF [37] data ( Sleep ) contains PolySomnoGraphic sleep records for subjects over a whole night. The private SEEG data records brain signals indicative of suspected pathological tissue within the brain of epileptic patients. More detailed descriptions can be found in Table 1 and Appendix D.

Label disturbance. We introduce a novel disturbance method to the raw labels Y of the public datasets to simulate scenarios where labels are inconsistent. Specifically, we first look for the boundary points between different classes in a complete long MVD data. Then, we randomly determine with a 0 . 5 probability whether each boundary point should move forward or backward. Finally, we randomly select a new boundary point position from r % of the length of the class in the direction of the boundary movement. In this way, we can interfere with the boundaries and simulate label inconsistency. Meanwhile, a larger value of r % indicates a higher degree of label inconsistency. For SEEG dataset, inconsistent labels already exist in the raw data and we do not disturb it.

Baselines. We compare Con4m with state-of-art models from various domains, including two noisy label learning (NLL) models for time series classification (TSC): SREA [9] and Scale-teaching [47]

(Scale-T), three image classification models with noisy labels: SIGUA [28], UNICON [36] and Sel-CL [43], three supervised TSC models: MiniRocket [13], TimesNet [64] and PatchTST [51], and three temporal action segmentation (TAS) models: MS-TCN2 [42], ASFormer [68] and DiffAct [45]. See more detailed descriptions of the baselines in Appendix E.

Implementation details. We use cross-validation [39] to evaluate the model's generalization ability by partitioning the subjects in the data into non-overlapping subsets for training and testing. As shown in Table 1, for fNIRS and SEEG, we divide the subjects into 4 groups and follow the 2 training1 validation1 testing ( 2 -1 -1 ) setting to conduct experiments. We divide the HHAR and Sleep datasets into 3 groups and follow the 1 -1 -1 experimental setting. Notice that SEEG data is derived from real clinical datasets and annotated by multiple experts, resulting in naturally inconsistent labels. We employ a voting mechanism which brings annotators together to collectively decide the boundaries to minimize discrepancies in test labels. Considering the high cost of this approach, we do not apply it to the training and validation sets. Therefore, we leave the test group aside and only change the training and validation groups to conduct cross-validation. Finally, we only report the mean values of cross-validation results in the main context. See more details and the full results in Appendix G.

4.2 Label Disturbance Experiment

The average results over all cross-validation experiments are presented in Table 2. Overall, Con4m outperforms almost all baselines across all datasets and all disturbance ratios.

Results of different methods. For fNIRS, TAS models achieve competitive performance compared to Con4m , demonstrating the advantage in modeling contextual data dependency among segments. For HHAR, Sleep and SEEG data with more ambiguous boundaries, the performance of TAS models deteriorates significantly, and TSC and NLL models slightly outperform TAS models. Benefiting from multi-scale modeling, Scale-T exhibits significantly better performance on the Sleep and SEEG data compared to SREA. Nevertheless, Con4m that fully consider contextual information demonstrate a notable performance improvement (HHAR-0%: 3.24%; Sleep-0%: 7 . 15% ; SEEG: 6 . 45% ) in more complex and ambiguous data.

Results of different r % . NLL methods demonstrate close performance degradation as r % increases from 0% to 20% compared with Con4m . However, with a higher ratio from 20% to 40% , SIGUA, UNICON, Sel-CL, SREA, and Scale-T show averaged 3 . 01% , 5 . 23% , 1 . 92% , 3 . 34% , and 3 . 22% decrease across fNIRS and Sleep data, while Con4m shows 2 . 37% degradation. For TSC models, non-deep learning-based MiniRocket shows a more robust performance compared to other TSC models. The performance of PatchTST on fNIRS data exhibits significant instability, possibly due to its tendency to overfit inconsistent labels too quickly. DiffAct in TAS models shows the most sensitive performance to boundary perturbations from 0% to 40% across three public data ( 13 . 23% decrease). The stable performance of Con4m indicates that our proposed training framework can effectively harmonize inconsistent labels.

Results of symmetric disturbance. We also corrupt the labels with symmetric disturbance based on segmented labels y rather than raw MVD labels, which is commonly employed in the NLL works [62, 43, 32] of the image classification domain. As shown in Figure 3(a), compared to our novel boundary disturbance, Con4m exhibits stronger robustness to symmetric disturbance. Even with the 20% disturbance ratio, Con4m treats it as a form of data augmentation, resulting in improved performance. This indicates that overcoming more challenging boundary disturbance aligns better with the nature of time series data.

iOriginal

65

1.68%

63.44

Consistent

4.11%

60.85

60

F1

7.53%

54.83

55

50

MiniRocket

PatchTST

TimesNet

4.3 Label Substitution Experiment

Since ambiguous boundaries are inherent to SEEG data and the majority voting procedure is costly, we limit this procedure to only one high-quality testing group in the label disturbance experiment. Besides, on the SEEG data, Con4m modifies approximately 10% of the training labels, which is a significant proportion. Therefore, it is necessary to further evaluate the effectiveness of our label harmonization process on SEEG data. Specifically, we train the TSC baselines based on the harmonized labels generated by Con4m and observe to what extent the performance of TSC models is improved. As shown in Figure 3(b), PatchTST and TimesNet, employing deep learning architectures, are more susceptible to label inconsistency, so they obtain more significant performance improvement ( 4 . 11% and 7 . 53% ). Unlike modified PatchTST that considers the contextual data information across consecutive segments, TimesNet only focuses on the independent segments, thus having a more dramatic improvement. In contrast, MiniRocket achieves only a 1 . 68% increase, indicating that MiniRocket is more robust with a non-deep learning-based simple random feature mapping.

4.4 Ablation Experiment

We introduce two types of model variations. (1) Preserve only one module. We preserve only the Con-Transformer (Con-T), Coherent Prediction (Coh-P), or Curriculum Learning (Cur-L) module separately. (2) Remove only one component. In addition to removing the above three modules, we also remove the function fitting component (-Fit) and η ( E η = 0 ) to verify the necessity of prediction behavior constraint and progressively updating labels.

As shown in Table 3, when keeping one module, +Coh-P achieves the best performance with an averaged 2 . 78% decrease in F 1 score, indicating that introducing the contextual label information are most effective for MVD . The utility of each module varies across datasets. For example, for Sleep data, the Con-T contributes more to performance improvement compared to the Cur-L module, while the opposite phenomenon is observed for SEEG data. As for removing one component, even when we only remove the Tanh function fitting, the F 1 score significantly decreases 1 . 72% on average. On

the Sleep20% and SEEG data, the drop caused by -Fit is more significant than that caused by some other modules. Moreover, the model variation η achieves the worst results ( 9 . 23% decrease in F 1 ). The results imply that during early training stages, the model tends to learn the consistent parts of the raw labels. Premature use of unreliable predicted labels as subsequent training supervision signals leads to model poisoning and error accumulation.

4.5 Case Study

We present a case study to provide a specific example that illustrates how Con4m works for MVD in Figure 4. We show comparative visualization results for the predictions in a continuous time interval in the SEEG testing set. In SEEG data, we assign the label of normal segments as 0 and that of seizures as 1 . As the figure shows, Con4m demonstrates a more coherent narrative by constraining the prediction behavior and aligning with the contextual data information. In contrast, Scale-T, Sel-CL and MiniRocket exhibit noticeably interrupted and inconsistent predictions. MS-TCN2 fails to identify normal segments. More impressively, Con4m accurately identifies the consistent boundary within the time interval spanning across two classes. We also utilize the C-score proposed by the ClaSP model [19] to assess the segmentation capability of models. The C-score reflects the ability of models to recognize segmentation boundaries by measuring the trade-off between precision (correctly identified change points) and recall (finding all true change points), ensuring the model captures meaningful transitions without over-segmenting or missing important splits. We compute the scores for each model across three sets of experiments on SEEG data and take the average. Notably, Con4m outperforms the other models significantly, while MS-TCN2, specifically designed for segmentation tasks, also achieves impressive scores. This verifies that the label consistency framework can harmonize the boundaries more effectively. Refer to Appendix H for more cases.

5 Conclusion and Discussion

In this work, we focus on the raw time series MVD for segmented time series classification (TSC) tasks, demonstrating unique challenges that are overlooked by existing mainstream TSC models. We first formally demonstrate that valuable contextual information enhances the discriminative power of classification instances. Based on the insights, we introduce contextual prior knowledge of data locality and label coherence to guide the model to focus on contextual information more conducive to discriminating consecutive segments in segmented TSC tasks. Leveraging effective contextual information, a label consistency learning framework Con4m is proposed to progressively harmonize inconsistent labels during training. Extensive experiments validate the superior performance achieved by Con4m and highlight the effectiveness of the proposed consistent label training framework. Our work still has some limitations. We have solely focused on analyzing and designing end-to-end supervised models. Further exploration of large-scale models would be challenging yet intriguing. Con4m is a combination of segmentation and classification, both of which are fully supervised. Exploring its application in unsupervised segmentation tasks is worthwhile. When faced with more diverse label behaviors, the function fitting module needs to engage in more selection and design of basis functions. Nevertheless, our work brings new insights to the TSC domain, re-emphasizing the importance of the inherent temporal dependence of time series.

Acknowledgments and Disclosure of Funding

This work was partially supported by National Natural Science Foundation of China (No. 62322606, No. 62441605).

References

Dana Angluin and Philip Laird. Learning from noisy examples. Mach. Learn. , 2(4):343--370, 1988.

Samaneh Azadi, Jiashi Feng, Stefanie Jegelka, and Trevor Darrell. Auxiliary image regularization for deep cnns with noisy labels. In International Conference on Learning Representations (ICLR) , 2016.

Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint arXiv:1811.00075 , 2018.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR) , 2015.

Peter L Bartlett, Michael I Jordan, and Jon D McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association , 101(473):138-156, 2006.

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML) , pages 41--48, 2009.

Henrik Blunck, Sourav Bhattacharya, Thor Prentow, Mikkel B. Kjrgaard, and Anind K. Dey. Heterogeneity Activity Recognition. UCI Machine Learning Repository, 2015. DOI: https://doi.org/10.24432/C5689X.

Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017.

Andrea Castellani, Sebastian Schmitt, and Barbara Hammer. Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noise. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD) , pages 469-484, 2021.

Ranak Roy Chowdhury, Xiyuan Zhang, Jingbo Shang, Rajesh K. Gupta, and Dezhi Hong. Tarnet: Task-aware reconstruction for time-series transformer. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) , pages 212--220, 2022.

Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The ucr time series archive. IEEE/CAA Journal of Automatica Sinica , 6(6):1293-1305, 2019.

Angus Dempster, François Petitjean, and Geoffrey I. Webb. Rocket: Exceptionally fast and accurate time series classification using random convolutional kernels. Data Min. Knowl. Discov. , 34(5):1454--1495, 2020.

Angus Dempster, Daniel F. Schmidt, and Geoffrey I. Webb. Minirocket: A very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD) , pages 248--257, 2021.

Don Dennis, Durmus Alp Emre Acar, Vikram Mandikal, Vinu Sankar Sadasivan, Venkatesh Saligrama, Harsha Vardhan Simhadri, and Prateek Jain. Shallow rnn: Accurate time-series classification on resource constrained devices. In Advances in Neural Information Processing Systems (NeurIPS) , 2019.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , pages 4171-4186, 2019.

Guodong Ding and Angela Yao. Temporal action segmentation with high-level complex activity labels. Trans. Multi. , 25:1928--1939, 2023.

Guodong Ding, Fadime Sener, and Angela Yao. Temporal action segmentation: An analysis of modern techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 46(2):1011--1030, 2023.

Zexing Du, Xue Wang, Guoqing Zhou, and Qing Wang. Fast and unsupervised action boundary detection for action segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3313-3322, 2022.

Arik Ermshaus, Patrick Schäfer, and Ulf Leser. Clasp: parameter-free time series segmentation. Data Min. Knowl. Discov. , 37(3):1262-1300, 2023.

Navid Mohammadi Foumani, Lynn Miller, Chang Wei Tan, Geoffrey I Webb, Germain Forestier, and Mahsa Salehi. Deep learning for time series classification and extrinsic regression: A current survey. arXiv preprint arXiv:2302.02515 , 2023.

Jean-Christophe Gagnon-Audet, Kartik Ahuja, Mohammad-Javad Darvishi-Bayazi, Pooneh Mousavi, Guillaume Dumas, and Irina Rish. Woods: Benchmarks for out-of-distribution generalization in time series. arXiv preprint arXiv:2203.09978 , 2022.

Vivien Sainte Fare Garnot and Loic Landrieu. Lightweight temporal self-attention for classifying satellite images time series. In Advanced Analytics and Learning on Temporal Data: 5th ECML PKDD Workshop (AALTD) , pages 171--181, 2020.

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning (ICML) , pages 1243--1252, 2017.

Tieliang Gong, Qian Zhao, Deyu Meng, and Zongben Xu. Why curriculum learning & selfpaced learning work in big/noisy data: A theoretical perspective. Big Data and Information Analytics , 1(1):111-127, 2016.

Alex Graves, Marc G. Bellemare, Jacob Menick, Rémi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. In Proceedings of the 34th International Conference on Machine Learning (ICML) , pages 1311-1320, 2017.

Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision. In Advances in Neural Information Processing Systems (NeurIPS) , 2018.

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems (NeurIPS) , 2018.

Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang, and Masashi Sugiyama. SIGUA: Forgetting may make learning with noisy labels more robust. In Proceedings of the 37th International Conference on Machine Learning (ICML) , pages 4006-4016, 2020.

Bo Han, Quanming Yao, Tongliang Liu, Gang Niu, Ivor W. Tsang, James T. Kwok, and Masashi Sugiyama. A survey of label-noise representation learning: Past, present and future. arXiv preprint arXiv:2011.04406 , 2021.

Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, and Austin Benson. Combining label propagation and simple models out-performs graph neural networks. In International Conference on Learning Representations (ICLR) , 2021.

Zhe Huang, Liang Wang, Giles Blaney, Christopher Slaughter, Devon McKeon, Ziyu Zhou, Robert Jacob, Robert Jacob, and Michael Hughes. The tufts fnirs mental workload dataset benchmark for brain-computer interfaces that generalize. In Proceedings of the Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks , 2021.

Zhizhong Huang, Junping Zhang, and Hongming Shan. Twin contrastive learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11661-11670, 2023.

Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019.

Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and PierreAlain Muller. Deep learning for time series classification: A review. Data Min. Knowl. Discov. , 33(4):917--963, 2019.

Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. arXiv preprint arXiv:1712.05055 , 2018.

Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. Unicon: Combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9676-9686, 2022.

B. Kemp, A.H. Zwinderman, B. Tuk, H.A.C. Kamphuisen, and J.J.L. Oberye. Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the eeg. IEEE Transactions on Biomedical Engineering , 47(9):1185-1194, 2000.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) , 2015.

Ron Kohavi. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI) , pages 1137--1143, 1995.

M. Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems (NeurIPS) , 2010.

Neil D. Lawrence and Bernhard Schölkopf. Estimating a kernel fisher discriminant in the presence of label noise. In Proceedings of the Eighteenth International Conference on Machine Learning (ICML) , pages 306--313, 2001.

Shijie Li, Yazan Abu Farha, Yun Liu, Ming-Ming Cheng, and Juergen Gall. Ms-tcn++: Multistage temporal convolutional network for action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 45(6):6647--6658, 2023.

Shikun Li, Xiaobo Xia, Shiming Ge, and Tongliang Liu. Selective-supervised contrastive learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 316-325, 2022.

Jason Lines, Sarah Taylor, and Anthony Bagnall. Time series classification with hive-cote: The hierarchical vote collective of transformation-based ensembles. ACM Trans. Knowl. Discov. Data , 12(5), 2018.

Daochang Liu, Qiyue Li, Anh-Dung Dinh, Tingting Jiang, Mubarak Shah, and Chang Xu. Diffusion action segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 10139-10149, 2023.

Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by overparameterization. In Proceedings of the 39th International Conference on Machine Learning (ICML) , pages 14153-14172, 2022.

Zhen Liu, ma peitian, Dongliang Chen, Wenbin Pei, and Qianli Ma. Scale-teaching: Robust multi-scale training for time series classification with noisy labels. In Advances in Neural Information Processing Systems (NeurIPS) , pages 33726-33757, 2023.

Tambet Matiisen, Avital Oliver, Taco Cohen, and John Schulman. Teacher-student curriculum learning. IEEE Transactions on Neural Networks and Learning Systems (TNNLS) , 31(9): 3732-3740, 2020.

Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a feather: Homophily in social networks. Annual review of sociology , 27(1):415-444, 2001.

Matthew Middlehurst, Patrick Schäfer, and Anthony Bagnall. Bake off redux: A review and experimental evaluation of recent time series classification algorithms. arXiv preprint arXiv:2304.13029 , 2023.

Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations (ICLR) , 2023.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS) , 2019.

Giorgio Patrini, Alessandro Rozza, Aditya Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: a loss correction approach. arXiv preprint arXiv:1609.03683 , 2017.

Deepta Rajan and Jayaraman J. Thiagarajan. A generative modeling approach to limited channel ecg classification. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , pages 2571-2574, 2018.

Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) , pages 1567--1577, 2022.

Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems (TNNLS) , 34(11):8135-8153, 2023.

Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080 , 2015.

Wensi Tang, Guodong Long, Lu Liu, Tianyi Zhou, Michael Blumenstein, and Jing Jiang. Omniscale CNNs: a simple and effective kernel size configuration for time series classification. In International Conference on Learning Representations (ICLR) , 2022.

Heng Wang and Cordelia Schmid. Action recognition with improved trajectories. In IEEE International Conference on Computer Vision (ICCV) , pages 3551-3558, 2013.

Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 44(9):4555-4576, 2021.

Yucheng Wang, Yuecong Xu, Jianfei Yang, Min Wu, Xiaoli Li, Lihua Xie, and Zhenghua Chen. Graph-aware contrasting for multivariate time-series classification. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) , pages 15725-15734, 2024.

Hongxin Wei, Lue Tao, RENCHUNZI XIE, and Bo An. Open-set label noise can improve robustness against inherent label noise. In Advances in Neural Information Processing Systems (NeurIPS) , pages 7978-7992, 2021.

Daphna Weinshall, Gad Cohen, and Dan Amir. Curriculum learning by transfer learning: Theory and experiments with deep networks. In Proceedings of the 35th International Conference on Machine Learning (ICML) , pages 5238-5246, 2018.

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations (ICLR) , 2023.

Kai Wu, Kaixin Yuan, Yingzhi Teng, Jing Liu, and Licheng Jiao. Broad fuzzy cognitive map systems for time series classification. Appl. Soft Comput. , 128(C):109458, 2022.

Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In International Conference on Learning Representations (ICLR) , 2022.

Chao-Han Huck Yang, Yun-Yun Tsai, and Pin-Yu Chen. Voice2series: Reprogramming acoustic models for time series classification. In Proceedings of the 38th International Conference on Machine Learning (ICML) , pages 11808-11819, 2021.

Fangqiu Yi, Hongyu Wen, and Tingting Jiang. Asformer: Transformer for action segmentation. In The British Machine Vision Conference (BMVC) , 2021.

Chong You, Zhihui Zhu, Qing Qu, and Yi Ma. Robust recovery via implicit bias of discrepant learning rates for double over-parameterization. arXiv preprint arXiv:2006.08857 , 2020.

Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? arXiv preprint arXiv:1901.04215 , 2019.

Bowen Zhao, Huanlai Xing, Xinhan Wang, Fuhong Song, and Zhiwen Xiao. Rethinking attention mechanism in time series classification. Inf. Sci. , 627(C):97--114, 2023.

Evgenii Zheltonozhskii, Chaim Baskin, Avi Mendelson, Alex M. Bronstein, and Or Litany. Contrast to divide: Self-supervised pre-training for learning with noisy labels. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , pages 1657-1667, 2022.

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Advances in Neural Information Processing Systems (NeurIPS) , pages 7793-7804, 2020.

A Details of Related Works

Time series classification (TSC). TSC has become a popular field in various applications with the exponential growth of available time series data in recent years. In response, researchers have proposed numerous algorithms [34]. High accuracy in TSC is achieved by classical algorithms such as Rocket and its variants [12, 13], which use random convolution kernels with relatively low computational cost, as well as ensemble methods like HIVE-COTE [44], which assign weights to individual classifiers.

Moreover, the flourishing non-linear modeling capacity of deep models has led to an increasing prevalence of TSC algorithms based on deep learning. Various techniques are utilized in TSC: RNN-based methods [54, 14] capture temporal changes through state transitions; MLP-based methods [22, 65] encode temporal dependencies into parameters of the MLP layer; and the latest method TimesNet [64] converts one-dimensional time series into a two-dimensional space, achieving state-of-the-art performance on five mainstream tasks. Furthermore, Transformer-based models [67, 10] with attention mechanism have been widely used.

The foundation of our work lies in these researches, including the selection of the backbone and experimental setup. However, mainstream TSC models [50, 20] are often designed for publicly available datasets [3, 11] based on the i.i.d. samples, disregarding the inherent contextual dependencies between classified segments in MVD . Although some time series models [55, 51] use patch-by-patch technique to include contextual information, they are partially context-aware since they only model the data dependencies within each time segment, ignoring the dependencies between consecutive segments.

Noisy label learning (NLL). NLLis an important and challenging research topic in machine learning, as real-world data often rely on manual annotations prone to errors. Early works focus on statistical learning [1, 41, 5]. Researches including Sukhbaatar et al. [57] launch the era of noise-labeled representation learning.

The label noise transition matrix, which represents the transition probability from clean labels to noisy labels [29], is an essential tool. Common techniques for loss correction include forward and backward correction [53], while masking invalid class transitions with prior knowledge is also an important method [26]. Adding an explicit or implicit regularization term in objective functions can reduce the model's sensitivity to noise, whereas re-weighting mislabeled data can reduce its impact on the objective [2, 69, 46]. Other methods involve training on small-loss instances and utilizing memorization effects. MentorNet [35] pretrains a secondary network to choose clean instances for primary network training. Co-teaching [27] and Co-teaching+ [70], as sample selection methods, introduce two neural networks with differing learning capabilities to train simultaneously, which filter noise labels mutually. The utilization of contrastive learning has emerged as a promising approach for enhancing the robustness in the context of classification tasks of label correction methods [43, 72, 32].

These works primarily focus on handling noisy labels. And ensuring overall label consistency by modifying certain labels is crucial for MVD . To the best of our knowledge, Scale-teaching [47] and SREA [9] are the only NLL works specifically designed for time series. Scale-teaching designs a fine-to-coarse cross-scale fusion mechanism for learning discriminative patterns by utilizing time series at different scales to train multiple DNNs simultaneously. SREA trains a classifier and an autoencoder with a shared embedding representation, progressively self-relabeling mislabeled data samples in a self-supervised manner. However, they still face the issue of overlooking contextual dependencies across consecutive time segments.

Curriculum learning (CL). Bengio et al. [6] propose CL, which imitates human learning by starting with simple samples and progressing to complicated ones. Based on this notion, CL can denoise noisy data since learners are encouraged to train on easier data and spend less time on noisy samples [24, 60]. Current mainstream approaches include Self-paced Learning [40], where students schedule their learning, Transfer Teacher [63], based on a predefined training scheduler; and RL Teacher [25, 48], which incorporates student feedback into the framework. The utilization of CL proves to be particularly advantageous in situations involving changes in the training labels. Hence, this technique is utilized to enhance the harmonization process of boundary labels from MVD in a more stable manner.

Temporal action segmentation (TAS). TAS is a critical task in video understanding and analysis. It involves segmenting an untrimmed video sequence into meaningful temporal segments and assigning a

predefined action label to each segment [17]. The majority works on TAS typically take visual feature vectors, either hand-crafted (IDT) [59] or extracted from an off-the-shelf CNN backbone (I3D) [8], as input for each frame. TAS uses sequential modeling to incorporate temporal dependencies and sequential context for improved TAS accuracy.

While TAS shares some similarities with segmented TSC, there still exist some problems. Time series data often have high sampling rates, however, it is impossible to input excessively long sequences into TAS models designed for video data. Moreover, unlike the I3D features [8] used as input in these works, time series lack a unified pretrained model for feature extraction. Besides, the changes between adjacent video frames are smooth and continuous, whereas time series are more variable and exhibit more ambiguous boundaries. Also, TAS works focus on modeling the dependency of instances from a data perspective without explicitly leveraging contextual label information.

B Implementation Details of Prediction Behavior Constraint

To fit the hyperbolic tangent function (Tanh), we use the mean squared error (MSE) loss function. In practice, we use the Adam optimizer with a learning rate of 0 . 1 to optimize the trainable parameters. The maximum number of iterations is set to 100 , and the tolerance value for stopping the fitting process based on loss change is set to 1 e -6 . Sequences belonging to one minibatch are parallelized to fit their respective Tanh functions. To adapt to the value range of the standard Tanh function, we rescale the sequential predictions to [ -1 , 1] before fitting.

However, it can be difficult to achieve a good fit when fitting with the Tanh function. Specifically, random initialization may fail to fit the sequential values properly when a long time series undergoes a state transition near the boundary. For example, as Figure 5(a) shows, we fit a sequence in which only the last value is 1 . We set all default initial parameters as 1 and fit it. It can be observed that the fitting function cannot properly fit the trend and will mislabel the last point.

Appropriate parameter initialization is needed to avoid excessive bias. After careful observation, we find that parameter k controls the slope at the transition part of Tanh, and parameter b controls the abscissa at the transition point. In the process, all fitting values are assigned with uniform abscissa values. Therefore, we calculate the maximum difference between adjacent values and the corresponding position in the entire sequence. And these two values are assigned to parameters k and b , respectively. This allows us to obtain suitable initial parameters and avoid getting trapped in local optima or saddle points during function fitting. Formally, given the L -length input sequence ˜ p , we initialize parameters k and b as follows:

where Abs ( · ) and Sign ( · ) denote the absolute value function and sign function respectively. di is the difference vector. After proper initialization, as Figure 5(b) shows, we can obtain more accurate fitting results to reduce the probability of mislabeling. We also show some other cases (Figure 5(c)(d)) for the fitting results to verify the effectiveness of the fitting process we propose.

C Hyperparameter Analysis

The dynamic weighting factor η is introduced to progressively update the labels, preventing the model from overly relying on its own predicted labels too early. To validate the utility of η and determine an appropriate linear growth epoch E η , we conduct the hyperparameter search experiment on SEEG data. As shown in Figure 6(b), with smaller E η (corresponding to a higher growth rate), there is a significant improvement in model performance. This aligns with our motivation that during the early stage of model training, the primary objective is to better fit the original labels. At this stage, the model's own predictions are unreliable. If the predicted results are used as training labels too early in subsequent epochs, the model would be adversely affected by its own unreliability. On the other hand, excessively large E η leads to a slower rate of label updates, making it more challenging for the model to timely harmonize inconsistent labels. Nonetheless, considering the impact of variance, the model exhibits robustness to slightly larger E η . In this work, we uniformly use E η = 30 as the default value.

D Details of Datasets

fNIRS. All signals are sampled at a frequency of 5 . 2 Hz. At each time step, they record 8 real-valued measurements, with each measurement corresponding to 2 concentration changes (oxyhemoglobin and deoxyhemoglobin), 2 types of optical data (intensity and phase), and 2 spatial positions on the forehead. Each measurement unit is a micromolar concentration change per liter of tissue (for oxy-/deoxyhemoglobin). They label each part of the active experiment with one of four possible levels of n -back working memory intensity ( 0 -back, 1 -back, 2 -back, or 3 -back). More specifically, in an n -back task, the subject receives 40 numbers in sequence. If a number matches the number n steps back, the subject is required to respond accordingly. There are 16 rounds of tasks, with a 20 -second break between each task. Following Huang et al. [31], we only apply classification tasks for 0 -back and 2 -back tasks in our work. Therefore, we only extract sequences for 0 -back and 2 -back tasks and concatenate them in chronological order.

HHAR. The HHAR (Heterogeneity Human Activity Recognition) dataset [7] captures sensor data from multiple smart devices to explore the impact of device heterogeneity on human activity recognition. It involves measurements from accelerometers and gyroscopes, which are two common motion sensors found in smartphones and smartwatches. The dataset includes six types of human activities: 'Biking', 'Sitting', 'Standing', 'Walking', 'Stairs Up', and 'Stairs Down'. For each activity, both 3-axis accelerometer and 3-axis gyroscope readings are recorded, resulting in six key features

per time step. The data was collected from five different device types, consisting of four smartwatches and eight smartphones, across nine participants. Each device samples data at its highest possible frequency, generally around 50 Hz. For data preprocessing, the accelerometer and gyroscope signals are aligned based on their timestamps, normalized within each channel, and divided by the devices, following the procedure outlined in Gagnon-Audet et al. [21]. To ensure balanced samples, we combined data from the Galaxy S3 Mini, LG watch, and Gear watch into a single group.

Sleep. The Sleep-EDF database records PolySomnoGraphic sleep data from 197 subjects, including EEG, EOG, chin EMG, and event markers. Some data also includes respiration and temperaturerelated signals. The database contains two studies: the Sleep Cassette study and the Sleep Telemetry study. The former records approximately 40 hours of sleep from two consecutive nights, while the latter records around 18 hours of sleep. Well-trained technicians manually score the corresponding sleep graphs according to the Rechtschaffen and Kales manual. The data is labeled in intervals of 30 seconds, with each interval being marked as one of the eight possible stages: W, R, 1, 2, 3, 4, M, or ?. In our work, we utilize only the data from the Sleep Cassette study, and retain only the signals from the EEG Fpz-Cz channel and EOG horizontal channel. The EEG and EOG signals were sampled at a frequency of 100 Hz. Following Kemp et al. [37], we remove the labels for stages ? and M from the data, and merge stages 3 and 4, resulting in a 5 -classification task.

SEEG. The private SEEG data records brain signals indicative of suspected pathological tissue within the brains of seizure patients. They are anonymously collected from a top hospital we cooperate with. For a patient suffering from epilepsy, 4 to 11 invasive electrodes with 52 to 153 channels are used for recording signals. In total, we have collected 847 hours of SEEG signals with a high frequency ( 1 , 000 Hz or 2 , 000 Hz) and a total capacity of 1 . 2 TB. Professional neurosurgeons help us label the seizure segments for each channel. Before sampling for the database, we remove the bad channels marked by neurosurgeons. Then we uniformly downsample the data to 250 Hz and use a low-pass filter to process the data with a cutoff frequency of 30 Hz. Finally, we normalize and sample the intervals for each channel respectively.

E Implementation Details of Baselines

SREA [9]: This time series classification model with noisy labels jointly trains a classifier and an autoencoder with shared embedding representations. It gradually corrects the mislabelled data samples during training in a self-supervised fashion. We use the default model architecture from the source code provided by the author (https://github.com/Castel44/SREA).

Scale-teaching [47]: This work designs a fine-to-coarse cross-scale fusion mechanism for learning discriminative patterns by utilizing time series at different scales to train multiple DNNs simultaneously. It uses well-learned multi-scale time series embeddings for noise label correction at sample feature levels. We modify the code to match our datasets based on the code provided by the author (https://github.com/qianlima-lab/Scale-teaching).

SIGUA [28]: This model adopts gradient descent on good data as usual, and learning-rate-reduced gradient ascent on bad data, thereby trying to reduce the effect of noisy labels. We modify the network for time series data based on the open source code provided by SREA, using the code from the author (https://github.com/bhanML/SIGUA).

UNICON [36]: UNICON introduces a Jensen-Shannon divergence-based uniform selection mechanism and uses contrastive learning to further combat the memorization of noisy labels. We modify the model for time series data according to the code provided by the author (https://github.com/nazmul-karim170/UNICON-Noisy-Label)

Sel-CL [43]: Selective-Supervised Contrastive Learning (Sel-CL) is a latest baseline model in the field of computer vision. It selects confident pairs out of noisy ones for supervised contrastive learning (Sup-CL) without knowing noise rates. We modify the code for time series data, based on the source code provided by the author (https://github.com/ShikunLi/Sel-CL)

MiniRocket [13]: Rocket [12] achieves state-of-the-art accuracy for time series classification by transforming input time series using random convolutional kernels, and using the transformed features to train a linear classifier. MiniRocket is a variant of Rocket that improves processing time, while offering essentially the same accuracy. We use the code interface from the sktime package (https://github.com/sktime/sktime).

TimesNet [64]: This model focuses on temporal variation modeling. With TimesBlock, it can discover the multi-periodicity adaptively and extract the complex temporal variations from transformed 2D tensors by a parameter-efficient inception block. We use the code from the TSlib package (https://github.com/thuml/Time-Series-Library).

PatchTST [51]: This is a self-supervised representation learning framework for multivariate time series by segmenting time series into subseries level patches, which are served as input tokens to Transformer with channel-independence. We modify the code to achieve classification for each patch, based on the source code from the Time Series Library (TSlib) package (https://github.com/thuml/Time-Series-Library).

MS-TCN2 [42]: This work proposes two multi-stage architectures for the temporal action segmentation task. While the first stage generates an initial prediction, this prediction is iteratively refined by the higher stages. Instead of the commonly used temporal pooling, they use dilated convolutions to increase the temporal receptive field. We modify the code for time series data, based on the source code provided by the author (https://github.com/sj-li/MS-TCN2).

ASFormer [68]: ASFormer is a Transformer-based model for action segmentation tasks. It explicitly brings in the inductive priors of local connectivity and applies a pre-defined hierarchical representation pattern to handle long input sequences. The decoder is also carefully designed to refine the initial prediction from the encoder. We modify the code for time series data, based on the source code provided by the author (https://github.com/ChinaYi/ASFormer).

DiffAct [45]: In this work, action predictions are iteratively generated from random noise with input video features as conditions. It also devises a unified masking strategy for the conditioning inputs to enhance the modeling of striking characteristics of human actions. We modify the code for time series data, based on the source code provided by the author (https://github.com/Finspire13/DiffAct).

F Implementation Details of Con4m

The non-linear encoder g enc used in Con4m is composed of three 1-D convolution layers. The number of kernels vary across different data and you can find corresponding parameters in the default config file of our source code. We construct the Con-Transformer based on the public codes implemented by HuggingFace 1 . We set d = 128 and the dimension of intermediate representations in FFN module as 256 for all experiments. The number of heads and dropout rate are set as 8 and 0 . 1 respectively. Since we observe that one-layer Con-Attention can fit the data well, we do not stack more layers to avoid overfitting. Note that Con4m consists of two Con-Transformers and we use two Con-Attention layers. The model is optimized using Adam optimizer [38] with a learning rate of 1 e -3 and weight decay of 1 e -4 , and the batch size is set as 64 . We build our model using PyTorch 2.0.0 [52] with CUDA 11.8. And the model is trained on a workstation (Ubuntu system 20.04.5) with 2 CPUs (AMD EPYC 7H12 64-Core Processor) and 8 GPUs (NVIDIA GeForce RTX 3090).

G Full Results

The full results of the label disturbance experiment are listed in Table 4, 5, 6 and 7. For fNIRS, we first divide the data into 4 groups by subjects and follow the 2 training1 validation1 testing ( 2 -1 -1 ) setting to conduct cross-validation experiments. Therefore, there are C 2 4 × C 1 2 = 12 experiments in total. Similarly, we divide the HHAR and Sleep data into 3 groups and follow the 1 -1 -1 experimental setting. Therefore, we carry out C 1 3 × C 1 2 = 6 experiments. For SEEG data, we follow the same setting as fNIRS. Notice that for SEEG data, inconsistent labels already exist in the raw data. We obtain a high-quality testing group by using a majority voting procedure to determine the boundaries. Then we leave the testing group aside and only change the validation group to report the mean value of C 2 3 = 3 experiments. All the experimental results are listed in lexicographical order according to the group name composition. We also report the mean value and standard derivation of all experiments. Specifically, we use the STDEVA to estimate standard deviation based on a sample of data.

H Case Study

As shown in Figure 7, we present four cases to compare and demonstrate the differences between our proposed Con4m and other baselines. The first two cases involve transitions from a seizure state of label 1 to a normal state of label 0 . The third case consists of entirely normal segments, while the fourth case comprises entirely seizure segments. As illustrated in the figure, Con4m exhibits more coherent narratives by constraining the predictions to align with the contextual information of the data. Moreover, it demonstrates improved accuracy in identifying the boundaries of transition states. In contrast, other baselines exhibit fragmented and erroneous predictions along the time segments. This verifies that Con4m can achieve clearer recognition of boundaries, and it can also make better predictions on the consecutive time segments belonging to the same class.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: In this paper, abstract, introduction and conclusion accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of the work are discussed in the conclusion (Section 5).

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Theoretical analysis is in Section 2.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: SEEG is a private dataset so one may cannot reproduce the experiments on the SEEG dataset. However, we also use another three public datasets, and the code of our work is fully provided, which could help readers understand and reproduce our work.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: SEEG is a private dataset so one may not be able to reproduce the experiments on the SEEG dataset. We provide detailed experimental setups and code to assist readers in reproducing the results on publicly available datasets.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We give a detailed description of our experiment setting in Section 4.1 and Appendix F. We also conduct a hyperparameter analysis in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The standard derivation is shown in the full experimental results in Appendix G.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide information about CPU, GPU, etc. in Appendix F.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: All authors reviewed and conducted the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a time-series classification method which exploits temporal consistency (implemented by contextual information). Also, the proposed method can handle noisy class boundaries.\n\nSTRENGTHS:\nS1. This paper presents a novel problem formulation, time-series classification with noisy class boundaries.\n\nS2. The proposed method takes full advantage of temporal consistency, which is an inherent property of time series.\n\nS3. The authors created their own proprietary dataset, SEEG, for extensive experiments.\n\nWEAKNESSES:\nW1. First of all, the writing in Introduction and Figure 1 are very misleading. \"**Inconsistent** boundary labels\" (with \"different annotators\") and Figure 1(a) indicate that multiple inconsistent annotations coexist for the same time-series instance. However, after reading a few pages, I found a discrepancy between my understanding and the presentation in this paper. Subsequently, I realized that the authors were referring to scenarios in which the class boundaries could be inaccurate. That is, there is **one** possibly erroneous boundary label for each transition. This presentation issue really affected my negative impression on this paper.\n\nW2. I do not agree that existing studies largely focus on the assumption of i.i.d. It is very easy to find the existing work that considers temporal consistency. For example, please see https://cs.stanford.edu/people/jure/pubs/ticc-kdd17.pdf. More recently, please see https://openreview.net/pdf?id=gjNcH0hj0LM. That is, many existing methods consider that consecutive timestamps tend to have the same class label. \n\nW3. Con-Attention is the same as Anomaly-Attention. Please see Equation (2) in https://arxiv.org/pdf/2110.02642. Therefore, along with W2, the technical novelty for the first contribution is not very significant.\n\nW4. Section 2 is not tightly connected to subsequent sections. In my opinion, Theorem 2 is too obvious, and it does not need to be formulated as a theorem. In fact, it does not look like a formal theorem. \n\nW5. The Tanh function fitting seems to be needed for each segment boundary and is not straightforward (involved with many parameters). Thus, the training efficiency could be an issue. It would be good to report the training cost of the proposed method.\n\nW6. The authors assume that the center of a class interval has the highest confidence. This assumption could be true or false depending on the domain. As in the motivating example, if the annotators have a difficulty in identifying the end times of seizure waves, the highest confidence will appear earlier than the center.\n\nW7. The authors made several strong assumptions or heuristics. For example, it is assumed that consecutive segments almost span at most two classes; a class interval is divided into five levels; a training cap is given as five epochs. Thus, this paper lacks rigorousness and generalizability. \n\nW8. It is not trivial to set up the hyperparameter value, especially, $E_\\eta$. The default value is not guaranteed to work well for other datasets. A detailed guideline is necessary.\n\nW9. The degree of noisy labels in each dataset should be analyzed in detail. First, it is not clear why there is no noise in fNIRS and Sleep, while there is real noise in SEEG. Is the annotation strategy different in these datasets? How do you know the ground-truth labels in SEEG? Second, can you measure the degree of real noise just like the noise rate in the image classification domain? Can you approximate the corresponding value of $r$ for this degree of noisy labels in SEEG? Overall, without detailed information on noisy labels, it would be difficult to analyze the effect of noisy label learning.\n\n---\n\nI have adjusted my rating to 5 during the discussion period.\n\nQUESTIONS:\nSee W1~W9.\n\nLIMITATIONS:\nIn my opinion, the generalizability of the proposed method is not clearly verified. In this regard, the authors did not properly discuss the limitations of the proposed method. Please see my comments to further improve the manuscript.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of segmented time series classification (TSC) for Multiple classes with Varying Duration (MVD) data. The authors propose Con4m, a consistency learning framework that leverages contextual information with a focus on inconsistent boundary labels. The method incorporates continuous contextual representation encoding, context-aware coherent class prediction, and a label consistency training framework. Experiments on 3 datasets demonstrate Con4m's superior performance compared to state-of-the-art baselines, especially in handling inconsistent labels.\n\nSTRENGTHS:\n- Originality: The paper introduces a new approach to segmented TSC for MVD data, addressing challenges often overlooked in existing TSC models. The Con4m framework creatively combines ideas from curriculum learning, noisy label learning, and temporal action segmentation.\n\n- Quality: The theoretical analysis in Section 2 and throughout provides a solid foundation for the proposed method. The experiments are comprehensive, including comparisons with various baselines, label disturbance experiments, and ablation studies.\n\n- Clarity: The paper is well-structured and clearly written. The figures, especially Figure 2 and Figure 3, effectively illustrate the proposed method's architecture and workflow.\n\n- Significance: The work has potential implications for various domains dealing with MVD data, such as healthcare and activity recognition. The label harmonization approach could be particularly valuable in scenarios where obtaining consistent labels is challenging or costly.\n\nWEAKNESSES:\n- Limited exploration of hyperparameters: The paper doesn't thoroughly discuss the sensitivity of the method to key hyperparameters, such as N_l = 5 in the label consistency framework or E_g = 5 for curriculum learning.\n\n- Scalability concerns: The paper doesn't address how the method would scale to larger datasets or longer time series. This could be a limitation for practical applications with high-dimensional or long-duration data.\n\n- Comparison with semi-supervised methods: Given that the method deals with inconsistent labels, it might be beneficial to compare it with semi-supervised learning approaches that are designed to handle partially labeled or noisy data.\n\n- Generalization to other domains: While the method is tested on three datasets, they are all from the healthcare domain (all 3 of them focusing on brain signal analysis). It would be valuable to see how well the approach generalizes to other domains with MVD data, such as activity recognition or financial time series. For example, initiatives like WOODS (https://arxiv.org/abs/2203.09978/) offer diverse timeseries data for evaluations.\n\nQUESTIONS:\n1. In the label consistency framework, why is N_l = 5? Was this value determined through hyperparameter search? How sensitive is the method to this choice?\n\n2. Why was fNIRS not used in the ablation studies? Is there a specific reason for excluding this dataset from the detailed analysis?\n\n3. How does the computational complexity of Con4m compare to the baseline methods? Is there a significant increase in training time or memory requirements?\n\n4. Have you explored the potential of applying Con4m to semi-supervised or few-shot learning scenarios, where only a portion of the data has reliable labels?\n\n5.  The paper mentions that Con4m modifies approximately 10% of the training labels in the SEEG data. How does this percentage vary across different datasets or disturbance ratios? Is there a way to estimate the optimal percentage of labels that should be modified?\n\n6.  Could you provide more insights into the choice of the hyperbolic tangent function for prediction behavior constraint? Have you experimented with other monotonic functions, and if so, how do they compare?\n\nLIMITATIONS:\nSee the questions above. I would especially emphasize the narrow benchmarks and tasks. Learning across segments is a broader problem within the timeseries domain.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Con4m, a novel framework for time-series classification and temporal action segmentation that leverages contextual information. The framework is designed to improve the prediction accuracy by incorporating context from surrounding data segments.\nThe proposed method combines time-series classification (TSC) and temporal action segmentation (TAS) in a unified framework, making it versatile for different types of sequential data. It tries to address the issue of common time series classification methods which overlook the dependencies between consecutive segments.\nThe method demonstrates robustness to label disturbances, showing significant improvements over baseline models across different datasets (fNIRS, Sleep, and SEEG).\n\nSTRENGTHS:\n-\tThe paper proposes context-aware boundary detection and classification for long multi-class time series.  The topic is very interesting and focuses on a real challenge in processing time-series data. \n- Con4m integrates contextual information to improve the coherence and accuracy of predictions. This approach helps in better recognizing boundaries and transitions in time-series data.\n-\tThe paper is well-written and presented with several example illustrations.\n-\tThe model is Compared against different time-series classification baselines. The evaluation shows great performance across Sleep and SEEG datasets and demonstrates robustness to label disturbances\n\nWEAKNESSES:\n1.\tWhat if combining other models with a change-detection model to detect segment boundaries first and then classify the windows within the segments? That would be great to see the impact of Con4m in comparison with other baselines in this setup. \n2.\tSince Con4m is capable of doing segmentation and classification together, it will be necessary to evaluate and compare the impact of each with other SOTA.\nMinor change:\n-\tFigure 4(b) has been referred to before Figure 4(a). Please fix that\n3. The experiments are limited, and somewhat unfair. The chosen baselines do not claim ability in segment detection. There are several works that have done segment or change point detection, and they need to be included in baselines. Also, evaluations are limited to three datasets.\n-\tMinor suggestion: It would be nice to visualize the whole framework to show each module.\n\nQUESTIONS:\nSee the initial two comments (no 1 and no 2) in weaknesses. In addition:\n\n3.\tIn Label Disturbance Experiments: It is not clear how the authors evaluated the models. Is it based on the new disturbed boundaries or the original boundaries? What is the evaluation setup for the other baselines as they are not considering the labelled boundaries in their calculations (I assume they should be boundary-agnostic).\n4.\tWhile section 4.3 shows an interesting set of experiments, I believe it only measures the degree of agreement between the baseline models and Con4m about the modified labels. Unless authors can ensure all the labels modified by Con4m are correct and accurate\n\nLIMITATIONS:\nThe following limitations are not considered/discussed:\n\n-\tAlthough I appreciate the novelty of the method, the final improvement is subtle compared to other baselines given they have not claimed any ability in segment detection. Also, the evaluations are limited to 3 datasets.\n-\tAs the authors mentioned in the conclusion the model heavily relies on the availability of labels even for the segmentation part. While most segmentation models are unsupervised, I suggest including comparison by combining other baselines with unsupervised segmentation methods.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors proposed a learning framework called $\\textit{Con}4\\textit{n}$ that leverages contextual prior of Multiple classes with Varying Duration (MVD) to enhance the discriminative power of consecutive time series segments while harmonizing inconsistent labels associate to these later. The authors stated and shown through extensive experiment that their framework enables to encode underlying temporal dependence between consecutive time series segments and mitigate the data annotation error that may be due to discrepancy of expertise.\n\nSTRENGTHS:\n- Originality: The authors propose a sophisticated architecture for encoding the temporal dependency between successive time series segments during a classification task. The originality of this work lies in the fact that the classification process is reinforced by a prior contextual information encoded by a Gaussian kernel. Their approach is more realistic than previous work, which assumes an independent and identical distribution of successive time series segments.\n\n- Quality: The article is well-written and structured. The experiment is consistent enough;\n\n- Clarity:  Although the document is fairly clear overall, there are still a number of areas for improvement (please refer to weaknesses and questions);\n\n- Significance: The proposed solution, once mature, will undoubtedly benefit many domains in which time series classification can be very challenging due to the complexity of data structure and divergence of experiences.\n\nWEAKNESSES:\n- Figure 2 is not intuitive enough to understand the overall architecture of the model;\n\n- $\\hat{p}$ and $\\tilde{p}$ are mentioned without explanation. Even though for $\\hat{p}$  it is quite obvious, it must be explained clearly (see paragraph $\\textbf{Neighbor Class Consistency Discrimination}$.\n\n- Space and Time complexity: Although efficient, the proposed model may require significant memory and computational time in large-scale applications.\n\n- The model may fail when confronted with seasonal time series data. This is because the Gaussian kernel used is not suitable for capturing the periodic pattern of the data;\n\n- Execution time is not included in the results. This is an important measure to report, as it plays a crucial role in sensitive areas such as medicine, where accuracy and prediction time are just as important.\n\nQUESTIONS:\n- What features do you use in positional encoding? Is it the order index of time series segments or timestamps? Please mention it in the main text;\n\n- Can the model work with multivariate time series data, i.e. using the observation of various features in a time segment to perform a classification task? If so, how will you encode the multivariate time series segments?\n\n- Do the authors think that their model will still work correctly when faced with seasonal time series data, given that the Gaussian Kernel used is not appropriate for capturing the periodic pattern of the data?  \n\n- Why did the authors choose the attention mechanism to merge $z_s^l$ and $z_g^l$? Although probably less efficient, have they tried simple techniques such as concatenation, Hadamard product or addition, which have the advantage of reducing spatial and temporal complexity?\n\nLIMITATIONS:\nAlthough the authors have discussed the limitations of their work, the space and time complexity of the model is an additional limitation. Furthermore, unless the reviewer has missed some points, it appears that the proposed model has only been evaluated on univariate time series data (please correct the reviewer if he is wrong). The reviewer suggests investigating these points in future work.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}