Con4m: Context-aware Consistency Learning Framework for Segmented Time Series Classification
Authors: Junru Chen
Tianyu Cao
Jing Xu
Jiahe Li
Zhilong Chen
Tao Xiao
Yang Yang
Venue: NeurIPS 2024 poster (2024)
Keywords: Time series classification
Segmented time series
Label consistency learning
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Time Series Classification (TSC) encompasses two settings: classifying entire sequences or classifying segmented subsequences. The raw time series for segmented TSC usually contain Multiple classes with Varying Duration of each class (MVD). Therefore, the characteristics of MVD pose unique challenges for segmented TSC, yet have been largely overlooked by existing works. Specifically, there exists a natural temporal dependency between consecutive instances (segments) to be classified within MVD. However, mainstream TSC models rely on the assumption of independent and identically distributed (i.i.d.), focusing on independently modeling each segment. Additionally, annotators with varying expertise may provide inconsistent boundary labels, leading to unstable performance of noise-free TSC models. To address these challenges, we first formally demonstrate that valuable contextual information enhances the discriminative power of classification instances. Leveraging the contextual priors of MVD at both the data and label levels, we propose a novel consistency learning framework Con4m, which effectively utilizes contextual information more conducive to discriminating consecutive segments in segmented TSC tasks, while harmonizing inconsistent boundary labels for training. Extensive experiments across multiple datasets validate the effectiveness of Con4m in handling segmented TSC tasks on MVD. The source code is available at https://github.com/MrNobodyCali/Con4m.

Con4m: Context-aware Consistency Learning Framework for Segmented Time Series Classification
Junru Chen Zhejiang University edu. on
Tianyu Cao
Zhejiang University ty- cao:zju. edu, en
Jing Xu
Jiahe Li
Zhilong Chen Zhejiang University zhi 1 lolooohennezuuu.. edu. cn
State Grid Power Supply Co. Ltd. Ltxu1111@mail.com com
Zhejiang University
j jiahelizzju. edu. on
Tao Xiao
Yang Yang
State Grid Power Supply Co. Ltd. (txxxtu@163.comm com
Zhejiang University yangyauzju ju. edu. cn
Abstract
Time Series Classification (TSC) encompasses two settings: classifying entire se- quences classifying segmented subsequences. The raw time series for segmented TSC usually contain Multiple classes with Varying Duration of each class (MVD). Therefore, the characteristics of MVD pose unique challenges for segmented TSC, yet have been largely overlooked by existing works. Specifically, there exists a natural temporal dependency between consecutive instances (segments) be clas- sified within MVD. However, mainstream TSC models rely on the assumption independent and identically distributed (i..d.), focusing on independently modeling each segment. Additionally, annotators with varying expertise may provide incon- sistent boundary labels, leading to unstable performance of noise-free TSC models. To address these challenges, we first formally demonstrate that valuable contex- tual information enhances the discriminative power of classification instances. Leveraging the contextual priors of MVD at both the data and label levels, we propose novel consistency learning framework Con4m, which effectively utilizes contextual information more conducive to discriminating consecutive segments in segmented TSC tasks, while harmonizing inconsistent boundary labels for train- ing. Extensive experiments across multiple datasets validate the effectiveness of Con4m in handling segmented TSC tasks on MVD. The source code available htttp::/ggttieii hitgggtttbbb... httpa///...ommooooooooaaaoo/ooo
Introduction
Time Series Classification (TSC) is one of the most challenging problems in the field of machine learning. TSC aims to assign labels to series of temporally ordered data points. These points either form a complete sequence or are subsequences (segments) resulting from the segmentation of a long time series. Existing works [50 20 largely focus on the assumption of independent and identically distributed (i.i.d),, in which case each sequence segment is regarded be classified, not differentiating between these two settings. fact, for many practical applications, the raw time series before segmentation for segmented TSC tasks contain Multiple classes with
independent instance to
Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Correct
Wrong
Seizure
Normel
Seizre
Seizure
Nomal
Normal
Seizure
Normal
1,0
Seizure
Seizure
Seizure
Seizure
Normal
Nomaal
Normal
Normal
(a) Model predictions exhibiting coherence are more reasonable.
Segmentation Prediction
088
Setzure
Predicted Prohability
Subject
04
More Extreme
www (oppa
mm
Sazzure
0.4
Subjert
mwl Imn
02
33%
Seizure
Subject
00
Led
1x3
Lx.4
(d) Predicted probabilitics from MLP for training samples for each level.
Illustration of annotation inconsistencies among three physicians.
Ix5
lv
Iv3
Iv2
lv1
Iv3
lv4
lvs
Iv5
Ivs
Iv3
Iv2
Iv3
lv4
Seizure
Boundary
Nomma
(c) Schemattc diagram of the division of each class sequence in (d), with higher-level data closer to the boundary.
Figure 1: (a) Reasonable model predictions exhibit coherence across consecutive segments rather than repeated interruptions. (b) In the healthcare domain, different physicians have varying annotations regarding the start and end times of seizure waves. (c) Based on the proximity the boundary, divide each class sequence into levels, from which an equal number of segments are sampled. one-layer MLP is trained on the segments from each level respectively for the same number of epochs.
(d) We visualize predicted probability of the trained MI.P for each level. We observe that as the segments approach the boundaries, the model finds it increasingly challenging make correct classifications, resulting in more extreme wrong predictions. This strongly underscores the significance of handling boundary segments.
Varying Duration of each class (MVD). For example, in the healthcare domain, the brain signals of epileptic patients often record over several days, encompassing multiple seizure onsets, each with varying durations and intervals. In the field of activity recognition, sensors continuously record users' behavior data, including walking, riding, and running, among other activities, each with varying durations. Therefore, the characteristics of the MVD lead to the uniqueness of segmented TSC tasks. Given this, our research concentrates on effectively modeling segmented TSC tasks based on MVD, presenting distinctive challenges.
(1) Leveraging contextual information. contrast to TSC tasks for complete sequences, which classified sequences are relatively independent, there exist natural temporal dependencies between consecutive classified segments for segmented TSC. We take the seizure detection task as an example, in which given the brain signals of epileptic patients, the model should identify whether a segment includes seizure waves or not. As illustrated in Figure [a), given the sustained nature of seizure onsets, the model predictions for consecutive segments should exhibit coherence, with seizure and normal predictions appearing in continuous and concentrated patterns. However, mainstream TSC models [58 77 61 focus on the i.i.d. assumption and model the internal context within each segment to be classified, largely overlooking the dependencies between consecutive segments.
In the domain of video analysis, works in temporal action segmentation (TAS) [1 have modeled the temporal dependency between different video frames and made frame-wise predictions. However, unlike the 13D features [ used as input in these works, time series lack a unified pretrained model for feature extraction, and the dependency between segments more variable and ambiguous. Further- more, TAS works focus modeling the dependency of instances from data perspective, without explicitly leveraging contextual label information. Therefore, how to leverage contextual information in segmented TSC tasks make more reasonable classifications is crucial and challenging.
(2) Inconsistent boundary labels. In domains with precious labels, different annotators collabora- tively contribute annotations. The raw annotations of MVD typically include the start and end times for each class. However, due to inherent ambiguity and a lack of unified quantification standards, for MVD, the boundaries between states are not clearly defined, or the transitional state itself represent a mixed state. Consequently, behind inconsistent labels, there no artificially defined true label. Therefore, work aims to harmonize this inconsistency much as possible to reduce the instability

---

of model training and enhance its performance. Returning to Figure bb), in the seizure detection task, owing to the natural fuzzy transition from seizure onset to completely normal, different physicians have varying experiences regarding when seizure waves terminate.
Furthermore, inconsistent boundary labeling causes that boundary segments with similar patterns may have opposite labels, leading to unstable model training. To validate the detrimental impact, as shown in Figure we divide each class sequence in the seizure detection task into 5 levels, where higher levels indicate proximity to the boundary. We then sample an equal number of balanced binary segments for each level. Subsequently, a one-layer MLP is trained for the same number of epochs on the segments from each level respectively. Figure [d) visualizes the results after training. We observe that as the level increases (closer the boundary), the model's 's accuracy steadily decreases, and erroneous predictions become more extreme. The results highlight the significant detrimental impact of inconsistent boundary labels on noise-free model performance.
Noisy label learning (NLL) [56] aims to learn robust models from data containing corrupted labels. While the inconsistent labels in MVD are not intentionally corrupted but rather stem from implicit discrepancies due to experiential differences, NLL remains the most relevant approach to address such discrepancies. To the best of our knowledge, Scale-teaching 47 and SREA ( are the only NLL works specifically designed for time series and are thus the most relevant to our work. However, they also face the issue of overlooking contextual dependencies across consecutive time segments, posing the challenge of handling inconsistent boundary labels using context during training.
To overcome the challenges above, we propose Con4m-a label Consistency learning framework, which leverages effective Contextual information, achieving Coherent predictions and Continuous representations for segmented TSC tasks, while harmonizing inconsistent boundary labels for training. Specifically, we first formally demonstrate that valuable contextual information enhances the discrim- inative power of classification instances. Based on the insights, by incorporating prior knowledge of data locality and label coherence, we guide and constrain the model to focus on contextual information more conducive to discriminating consecutive segments in segmented TSC tasks. Meanwhile, lever- aging model predictions that thoroughly encompass contextual information, Con4m progressively changes the training labels in an adaptive manner to harmonize inconsistent labels across consecutive segments. This leads to a more robust model. Our contributions are summarized as follows:
(1) We are the first propose a practical consistency learning framework Con4m for the segmented TSC based on the raw MVD. (2) By comprehensively integrating prior knowledge from the data and label perspectives, we guide the model to focus on effective contextual information. Based on context-aware predictions, a progressive harmonization approach for handling inconsistent training labels is designed yield a more robust model. (3) Extensive experiments on three public and one private MVD datasets demonstrate the superior performance of Con4m. The Con4m's 's ability to harmonize inconsistent labels is further verified by the label substitution experiment and case study.
2 Theoretical Analysis
In this section, we aim to formally demonstrate the benefit of contextual information for classification tasks, and to establish the existence of upper bound for this benefit. Consequently, by introducing prior knowledge, we can guide the model to focus on valuable contextual information more conducive to improving the benefit for segmented TSC tasks.
Assuming that the random variables of the instances to be classified and the corresponding labels are denoted as x; and ye Ar represents the contextual instance set introduced for Xt. I', denotes the random variable for the contextual instance set. Mutual information measures the correlation between two random variables. In classification task, higher correlation between instances and labels indicates that the instances are more easily distinguishable by the labels. This benefits the classification task, making it more readily addressable. Therefore, from an information -thrraation-theorttiic perspective, we elucidate the benefit of contextual information through the following theorem.
Theorem 2.1. The more the introduced contextual instance set enhance the discriminative power of the target instance, the greater the benefit for the classification task.
Proof. Firstly, we establish that the introduction of contextual information does not compromise classification tasks, Le., it does not diminish the correlation between instances and labels.
Iy Xe, Xx.) 1(y Xx, (xx) + I(y,; xx) 2 1(y,;x)).
(I)
3

---

The inequality holds due to the non-negativity of conditional mutual information.
According ① the increase in T(y;; Ks, (xe) determines the extent which the introduction of contextual information can be beneficial for classification tasks. Expanding T(y;; Xs, (xx), we have:
log ply,, (x,
T(y;sA,|xx)) = Σ p(x))
ΣΣ
p(y, XA, XA,
p(y,/xx)p(xx,ll
b1
Ahe
p(ye|xe,XA.)
Ep(x) E
Σ (l. xx)) log ply, xx))
Ahe Y+
Σ
p(x))
p(xa. (x) Xx.
Xt
Xhe
Given a fixed instance X₂ and the inherent distribution ply, xx) of the data, the KL divergence is a convex function for Xs, that attains its minimum at Xx,) = p(y,/x)). As p(y, Xx,) approaches the boundary of the probability space, where the predictive probability of one class approaches 1 and the rest approach O, the value of KI. divergence increases. A stronger discriminative power regarding X, implies less uncertainty regarding y:: which is equivalent to approaching the boundary of the probability space.
Due the convexity of the KL divergence and the boundedness of p(y, (Xe, KA.). there exists a contextual instance set the data that maximizes DKI, (pI, X2, Ilppy,, (x))) We denote the instance set áj and the maximum value of KL divergence that Ex, p(XA, x = 1. Hence, we can obtain the upper bound for the information gain I((y,i Xx) Σx, Ex,, DT) I Ex, Df The convexity of the KL diver- gence also implies monotonicity, indicating that as A, approaches Ai, the KL divergence increases. leading to a greater information gain for the classification task.
note DJ. Besides,
According to Theorem 2.1 valuable contextual information enhances the discriminative power of the instances. While the optimal instance set A7 is challenging to directly obtain or optimize, focusing the model on contextual instances more likely be included in A is beneficial for enhancing the performance of the classification task. Furthermore, XA, not only contains information at the data level but also encompasses information at the label level (which can be replaced with ya,). Therefore, we can guide the model to focus on contextual information more conducive to segmented TSC tasks by simultaneously introducing prior knowledge from both the data and label perspectives.
The Con4m Method
In this section, we introduce the details of Con4m. Based on the insights of Theorem 2. we introduce contextual prior knowledge of data locality (Sec. (3TT)
(STc.B.1) and label coherence (Sec. 3.2 to guide the model to focus contextual information more conducive discriminating consecutive segments in segmented TSC tasks. In Sec. 3.3 inspired by the idea of noisy label learning, we propose label harmonization framework achieve more robust model. Before delving into the details of Con4m, we provide the formal definition of the segmented TSC task in our work.
Definition 3.1. Given a time interval comprising Tf consecutive time points and labels, denoted as (X,Y) {(Xi, (X2, Y2),..., (XT,, Yr) a w-length sliding window with stride length r is em- ployed for segmentation. (X,Y) partitioned into L time segments, represented as (x, y) (10,, yi) = (1X ((X(-I)xr+t,--- X(i-11xr+w>
Y (1-1)xr+ul b =
1...., T.). The model is tasked with predicting segmented labels ya for each time segment Tt.
3.1
Continuous Contextual Representation Encoder
Local continuity is an inherent attribute of MVD, meaning each class should be locally continuous
and only change at its actual boundary. Smoothing with a Gaussian kernel [88] 16 66 promotes
the continuity of representations of time segments in local temporal window. This not only helps the model make similar predictions of consecutive segments within the same class but also aligns
with the gradual nature of class transitions. Furthermore, for graph neural networks based on the

---

homophily assumption, aggregating neighbor information belonging to the same class can improve the discriminative power the target instance 499 73 Therefore, we introduce the Gaussian prior to guide the model to focus on contextual instances A, proximate to the target instance.
Vanilla self-attention 15 with point-wise attention computations often fail obtain continuous representations after aggregation. Therefore, we use the Gaussian kernel (Ix ylo) as prior weights aggregate neighbors to obtain smoother representations. Since the neighbors of boundary segments may belong to different classes, we allow each segment to learn its own scale parameter a. Formally, as Figure a) shows, the two-branch Con-Attention in the I-th layer is:
Q, K, V. Vg =d -w c..- Wk.c' c' -e---WW, - -W W
QK vd
(Iv exp (- [| [-1--1p i13
1.SEEL.....) )
St SoftMax
G²
Rescale
203
d/ ='V.' by GVVg. s' Fusion( 2/ og)),
where L is the number of consecutive segments, is the dimension of hidden representations, d-11 RLxd is the output representations of the / - 1-th layer, and W idxdd all learnable matrices. Rescale(-) refers to row normalization by index i. To distinguish between two computational branches, we use g/ to represent the branch based on Gaussian prior, and s/S to represent the branch based self-attention. S and G' are the aggregation weights. We the conventional attention mechanism 4 to adaptively fuse = and =g- Finally, as illustrated Figure 2 the multi-head version of Con-Attention layers, we construct Con- Non-Transformer which serves as the backbone of the continuous encoder of Con4m to obtain final representations We employ Icarnable absolute positional encoding (APE) 23 for the input representations.
a), by stacking
((I)fe))) 4/2,1m))
Adid le ayee-NNom
Rescale
Con-Iranffomer2 (CT2)
FFN
MatMul
Add &i Layar Novt
Pooling
Pooling
V.
MatMul
X
Attention
cn
Matiuul Scale
SoftVia
Posiitonal Emtloddingg
0
(a) Overview of continuous contextual representation encoder in Con4m.
27
Behavior Constraint
confidence
confidence,
0.6
Level Level
f
MUPP
MUPP
MiLP2
ML
MLP2
0.2
10 20 30 40 60
spocin
Pv (1 n/w
81 +
MUP
MUPP
MILPI
MI
MLPPI
Training
Canfidence docroasing
incressing
SonMaa
SoltMaa
SoltMaa
SonMaaa
SollMaa
Inference mno
(b) Overview context-aware coherent class prediction and consistent label training framework in Con4m.
Figure 2: Overview of Con4m. (a) Overview continuous contextual representation encoder Con4m. The leftmost part shows the details of Con-Attention. The right part of the figure shows the architecture of Con-Transformer and the whole encoder of Con4m. (b) Overview of context-aware coherent class prediction and consistent label training framework in Con4m. The right part deseribes the neighbor class consistency discrimination task and the prediction behavior constraint. The leftmost part presents the training and inference details for label harmonization.

---

3.2 Context-aware Coherent Class Prediction
In the segmented TSC task of MVD, consecutive time segments not only provide contextual infor- mation at the data level but also possess their own class information. As depicted in Figure Ta), considering the persistence of each class and the gradual nature of class transitions, the model's pre- dictions should exhibit more coherence and concentration, rather than being interspersed. Therefore, we integrate and constrain the model's predictions from both the individual and holistic perspectives to achieve more coherent predictions.
Neighbor Class Consistency Discrimination. In graphs, label propagation algorithms 30 are often utilized to refine and smooth the predictions of neighbor instances, thereby enhancing their discrimination. Drawing inspiration from this, by weightedly aggregating predictions from similar time segments, the model can focus on contexts A, more likely to belong to the same class as the target segment. Although there is no explicit graph structure between time segments, we can train a discriminator to determine whether two segments belong to the same class. The model then aggregates the contextual class predictions based on the discriminator's 's outputs, thus making more robust predictions. As the right part of Figure 2 shows, we formalize this process follows:
MSotMaxx (MLP2 (cil (cill1s)),, (cills)))eee.......
SoftMax (MLP) (c)), p= R
where R E R is the probability of whether two segments in the same time interval belong to the same class and (-Il-) denotes lensor concatenation. á represents the model's independent prediction for a segment, while denotes the context-aware prediction that incorporates the results from neighboring segments. We then define the two training losses as l CrossEntropy(0, y) and
la CrossEntropy R, Y), where Y = [1 (yy=syll.dtt......) Given that l and Ez are of the same magnitude, we equally sum them as the final loss.
Prediction Behavior Constraint. Unlike graphs, there exists holistic temporal relationship between consecutive time segments. Therefore, we should further constrain the overall predictive behavior along the time axis. For MVD, as Figure aa) shows, within a suitably chosen time interval, consecutive segments almost span most two classes. Therefore, we ensure the monotonicity of predictions across consecutive segments through hard constraints, thereby utilizing contextual label information YA, to integrate and refine predictions across these segments.
As shown in the middle part of Figure 2 b), for each class in the predictions, there are only four prediction behaviors for consecutive segments, namely high confidence, low confidence, confidence decreasing, and confidence increasing. To constrain the behavior, we use function fitting to integrate p. Considering the wide applicability, we opt for the hyperbolic tangent function (i.e., Tanh) as our basis. Formally, we introduce four tunable parameters to exactly fit the monotonicity as:
F Tanh r/a, k b, h) = a x Tanh (k x (r + b)) + h,
where parameter a constrains the range of the function's 's values, k controls the slope of the transition of the function, b and adjust the symmetry center of the function, and x the given free vector in the x- coordinate. We use the MSE loss to fit the contextual predictions deserves be emphasized that ý in the process has no gradient and therefore does not affect the parameters of the encoder. Please see Appendix B for more fitting details.
la (Taah((za,, k. b, h) 2
After function fitting, we obtain independent predictions for each segment and constrained predic- tions ğ that leverage contextual label information. For the inference stage, we use the average of them as the final coherent predictions, i.e., ÿ arg max (p + /2.
3.3 Label Consistency Training Framework
Due to inherent ambiguity, the annotation of MVD often lacks quantitative criteria, resulting in experiential differences across individuals. Such discrepancies are detrimental to models and we propose training framework to enable Con4m to adaptively harmonize inconsistent labels.
Learning from easy to hard. We are based on the fact that although people may have differences in the fuzzy transitions between classes, they tend to reach an agreement on the most significant core part of each class. In other words, the empirical differences become more apparent when approaching the transitions. Therefore, we adopt curriculum learning techniques to help the model learn instances

---

from the easy (core) to the hard (transition) part. Formally (see the diagram in Figure Tb)), for continuous K Kllength class, we divide it into N; equally sized levels as follows:
K (( 1, L, + 1) 2N, 222,
K
L. [1 2N,
K 1U(12N--
2N, I.K.
K
(2)
Then we sample the same number of time intervals from each level. The higher the level, the more apparent the inconsistency. Therefore, the left part of Figure b)) shows, during the training stage, Con4m learns the time intervals in order from low to high levels, with a lag gap of Egg = 5 epochs.
Harmonizing inconsistent labels. Inspired by the idea of noisy label learning, we gradually change the raw labels to harmonize the inconsistency. The model preferentially changes the labels of the core segments that are easier to reach consensus, which can avoid overfitting of uncertain labels. Moreover, the model will consider both the independent and constrained predictions to robustly change inconsistent labels. Specifically, given the initial label yo, we update the labels y/ = arg max Pe for the e-th epoch, where Pe is obtained as follows:
6 We' Pe- Peemmmmmfo.....
Pe We me
Pe P== - + (1- pe "E 2
"
where We = Rescale([cxp(((e (())(((em))2))))))))) me (3) is the exponentially averaged weight vector to aggregate the predictions of the latest 5 epochs to achieve more robust label update. Pe-m and Pe-m are the independent and constrained predictions in the e m-th epoch respectively and denotes the dot product. The dynamic weighting factor, 17, is used to adjust the degree of label update. As the left part of Figure 2 b) shows, linearly increases from 0 to with E,, epochs, gradually weakening the influence of the original labels. Besides, in the initial training stage, the model tends improve independent predictions. As the accuracy of independent predictions increases, the model assigns greater weight the constrained predictions. See the hyperparameter analysis for E.y in Appendix
4 Experiment
4.1 Experimental Setup
Datasets. In this work, we use three public [31 37 and one private MVD data measure the performance of models. Specifically, the Tufts fNIRS to Mental Workload [31] data (fNIRS) contains brain activity recordings from adult humans performing controlled cognitive workload tasks. The HHAR (Heterogeneity Human Activity Recognition) dataset [7] captures sensor data from multiple smart devices to explore the impact of device heterogeneity on human activity recognition. The SleepEDF (37) data (Sleep) contains PolySomnoGGraphic sleep records for subjects over whole night. The private SEEG data records brain signals indicative of suspected pathological tissue within the brain of epileptic patients. More detailed descriptions can be found in Table [ and Appendix
Table 1: Overview of MVD datasets used in this work.
Data
Sample Frequency
of Features
Subjects
Groups
Cross Validation
Total Intervals
Interval Length
Window Length
Total
Classes
Length
Segments
ENIRS
5.2Hz
8
2
68
4
12
4,080
38.46s
4.81s
0.96s
146,880
HHAR
50Hz
5,400
4s
156,600
Sleep 100Hz
154
6,000
40s
2.5s
1.25s
186,000
SEEG
250Hz
2
8
4
3
8,000
16s
1s
0.5s
248,000
Label disturbance. We introduce a novel disturbance method to the raw labels Y of the public datasets to simulate scenarios where labels are inconsistent. Specifically, we first look for the boundary points between different classes in a complete long MVD data. Then, we randomly determine with 0.5 probability whether each boundary point should move forward or backward. Finally, randomly select a new boundary point position from r% of the length of the class in the direction of the boundary movement. In this way, can interfere with the boundaries and simulate label inconsistency. Meanwhile, a larger value of 7% indicates a higher degree of label inconsistency. For SEEG dataset, inconsistent labels already exist in the raw data and do not disturb it.
Baselines. We compare Con4m with ofate-of-art models from various domains, including two noisy label learning (NLL) models for time series classification (TSC): SREA [9] and Scale-teaching [47
7

---

(Scale-T), T), three image classification models with noisy labels: SIGUA (28], UNICON 36 and
Sel-CL (3] three supervised TSC models: MiniRocket [13], TimesNet [64 and PatchTST [TT and
three temporal action segmentation (TAS) models: MS-TCN2 [4]. ASFormer 68 and DiffAct Act
45
See more detailed descriptions of the baselines in Appendix
Implementation details. We use cross-validation 39 to evaluate the model's generalization ability by partitioning the subjects in the data into non-overlapping subsets for training and testing. As shown in Table for INIRS and SEEG, we divide the subjects into 4 groups and follow the 2 training-l validation-1 testing (2-1-1) setting to conduct experiments. We divide the HHAR and Sleep datasets into 3 groups and follow the 1-1-1 experimental setting. Notice that SEEG data is derived from real clinical datasets and annotated by multiple experts, resulting in naturally inconsistent labels. We employ voting mechanism which brings annotators together to collectively decide the boundaries to minimize discrepancies in test labels. Considering the high cost of this approach, we do not apply it to the training and validation sets. Therefore, we leave the test group aside and only change the training and validation groups to conduct cross- volss-vaiiatiion. Finally, we only report the mean values of cross-validation results in the main context. See more details and the full results in Appendix G
4.2 Label Disturbance Experiment
The average results over all cross-validation experiments are presented in Table Overall, Con4m outperforms almost all baselines across all datasets and all disturbance ratios.
Table Comparison with baseline methods in the testing F1 score (%) on three datasets. The best results are in bold and we underline the second best results. The worst results are denoted in italics.
5%
INIRS 31
HHAR 7
Sleep [37
SEEG
Model
0%
20%
40%
0%
20%
40%
0%
20%
40%
raw
MS-TCN2
42
71.48
70.99
69.40
69.79
66.72
62.29
60.07
59.03
56.17
61.88
TAS
ASFormer
68
71.69
70.75
69.18
62.52
60.92
60.77
59.09
55.52
53.89
56.71
DiffAct
45
71.15
69.72
65.45
56.76
53.86
50.63
49.12
43.32
38.86
60.62
MiniRocket
13
61.28
60.41
57.87
70.34
63.32
59.25
62.00
61.75
58.38
62.39
TSC
TimesNet
64
67.47
65.39
63.45
72.07
70.19
66.76
59.50
57.72
55.73
50.99
PatchTST 51
51.79
55.38
52.67
52.00
45.46
45.69
58,40
56.16
53.05
58.45
SIGUA 28
67.37
65.24
63.47
68.94
68.47
67.60
$4.28
53.07
51.32
53.19
NLL
UNICON 36
61.15
60.45
57.35
62.26
61.63
58.34
62.26
61.63
58.34
60.53
Sel-CL 43
63.86
62.45
61.75
73.00
72.28
72.81
63.48
63.45
61.72
60.50
TSC
SREA 9
70.10
69.65
69.40
68.64
66.02
65.67
48.81
48.80
45.72
55.21
Scale-T
47
70:40
68.06
66.51
77.77
76.71
75.97
63.21
63.40
60.77
67.64
NLL
Con4m
71.28
71.27
70.04
80.29
78.59
75.52
68.02
66.31
64.31
72.00
Results of difffernt methods. For INIRS, TAS models achieve competitive performance compared to Con4m, demonstrating the advantage in modeling contextual data dependency among segments. For HHAR, Sleep and SEEG data with more ambiguous boundaries, the performance of TAS models deteriorates significantly, and TSC and NLL models slightly outperform TAS models. Benefiting from multi-scale modeling. Scale-T exhibits significantly better performance on the Sleep and SEEG data compared to SREA. Nevertheless, Con4m that fully consider contextual information demonstrate a notable performance improvement (HHAR-0%: 3. 3.24%; Sleep-0%: 7.15%; SEEG: 6. 6.45%) in more complex and ambiguous data.
Results of different r%. NLL methods demonstrate close performance degradation as 7% increases from 0% to 20% compared with Con4m. However, with higher ratio from 20% to 40%, SIGUA, UNICON, Sel-CL, SREA, and Scale-T show averaged 3.01 9. 5.23%. 1.92%. 3.34%, and 3.22% decrease across fNIRS and Sleep data, while Con4m shows 2.37% degradation. For TSC models. non-deep learning- based MiniRocket shows a more robust performance compared to other TSC models. The performance of PatchTST on fNIRS data exhibits significant instability, possibly due to its tendency to overfit inconsistent labels too quickly. DiffAct TAS models shows the most sensitive performance to boundary perturbations from 0% to 40% across three public data 233.23% decrease). The stable performance of Con4m indicates that our proposed training framework can effectively harmonize inconsistent labels.
8

---

Results of symmetric disturbance. We also corrupt the labels with symmetric disturbance based on segmented labels y rather than raw MVD labels, which is commonly employed the NLL of the image classification domain. As shown in Figure Daa a), compared to our novel boundary disturbance, Con4m exhibits stronger robustness to symmetric disturbance. Even with the 20% disturbance ratio, Con4m treats it as form of data augmentation, resulting in improved performance. This indicates that overcoming more challenging boundary disturbance aligns better with the nature of time series data.
works 62 43 32
73
69
INIRS
Sleep
Original
72
68
Consistent
4.11%)
60.85
(960
60
71
66
753% $4,83
70
Symmetric
Symmetric
Boundary
Boundary
MiniRocket
PatchTST
TimesNet
Disturbance Ratio (+%)
Disturhance Ratio (r %6)
Symmetric disturbance fNIRS and Sleep datasets
(b) Comparison before and after label substitution
Figure 3:
Comparison results of symmetric disturbance and label substitution experiments.
4.3 Label Substitution Experiment
Since ambiguous boundaries are inherent to SEEG data and the majority voting procedure is costly, we limit this procedure to only one high-quality testing group in the label disturbance experiment. Besides, on the SEEG data, Con4m modifies approximately 10% of the training labels, which significant proportion. Therefore, it is necessary to further evaluate the effectiveness of our label harmonization process on SEEG data. Specifically, we train the TSC baselines based on the harmonized labels generated by Con4m and observe to what extent the performance of TSC models is improved. As shown in Tigure3b) PatchTST and TimesNet, employing deep learning architectures, are more susceptible to label inconsistency, so they obtain more significant performance improvement (4.11% and 7.53%). Unlike modified PatchTST that considers the contextual data information across consecutive segments, TimesNet only focuses on the independent segments, thus having a more dramatic improvement. In contrast, MiniRocket achieves only 1.68% increase, indicating that MiniRocket is more robust with a non-deep learning-hase hased simple random feature mapping.
4.4 Ablation Experiment
We introduce two types of model variations. (1) Preserve only one module. We preserve only the Con- Transformer (Con-T), T), Coherent Prediction (Coh-P), or Curriculum Learning (Cur-L) module separately. (2) Remove only one component. In addition to removing the above three modules, we also remove the function fitting component (-Fit) and r (E, = 0) to verify the necessity of prediction behavior constraint and progressively updating labels.
Table 3: Comparison with model ablations in the Fi score (%) in inconsistent scenarios. The best results are in bold and we underline the second best results. The worst results are denoted in italics.
Model
Preserve one
Remove one
Con-T T
Coh-P
L
Con-T
Coh-P
Cur-L
Fit
7
Convin
Dataset
5% Acc.
Fi
Acc.
Fi
Acc.
F.
Asc.
Fi
Acc.
Acc.
Fi
Acc.
Fi
Acc.
Fi Acc. Fi
Sleep
65.97 65.05 65.76 65.10 65.31 64.76 65.73
65.53
65.84
65.07 65.85 65.43 66.06 65.28 62.02 59.97
66.61 66.31
63.94
62.67 64.42 62.76 63.69
62.23
64.44
63.05
64,23
63.03
64.89
63.07
64.69 63.22 61.93
57.98
65.34 64.31
SEEG
71.68 67.85 71.69 69.04
71.32 67,22 73.85 70.59
72.41
68.26
74.17
71.18 73.47 70.63
70.70 66.04
74.60 72.00
As shown in Table 3 when keeping one module, +Coh-P achieves the best performance with an averaged 2.78% decrease in Fi score, indicating that introducing the contextual label information are most effective for MVD. The utility of each module varies across datasets. For example, for Sleep data, the Con-T contributes more to performance improvement compared to the Cur-L module, while the opposite phenomenon is observed for SEEG data. As for removing one component, even when we only remove the Tanh function fitting, the Fi score significantly decreases 1.72% average. On

---

the Sleep-20% and SEEG data, the drop caused by -Fit is more significant than that caused by some other modules. Moreover, the model variation -7) achieves the worst results (9.23% decrease Fi). The results imply that during early training stages, the model tends to learn the consistent parts of the raw labels. Premature use of unreliable predicted labels as subsequent training supervision signals leads to model poisoning and error accumulation.
4.5 Case Study
True Positive
False Positive
False Negative
Boundary
True Negative
C-score: 0.930
Con4m
C-score: 0.406
Scale-T
C-score: 0.557
Sel-CL
C-score: 0.386
(c) MiniRocket
C-score: 0.902
MS-TCN2
12s
14s
Figure 4: Casc study for a continuous time interval in SEEG testing set. The C-score, introduced by the ClaSP model [19], assessing the ability of models to recognize segmentation boundaries by measuring the trade-off between brececnnpreciionn (correctly identified change points) and recall (finding all true change points).
We present a case study to provide a specific example that illustrates how Con4m works for MVD in Figure 4 We show comparative visualization results for the predictions in a continuous time interval in the SEEG testing set. In SEEG data, we assign the label of normal segments as and that of seizures as 1. As the figure shows, Con4m demonstrates a more coherent narrative by constraining the prediction behavior and aligning with the contextual data information. In contrast, Scale-T, Sel-CL and MiniRocket exhibit noticeably interrupted and inconsistent predictions. MS-TCN2 fails to identify normal segments. More impressively, Con4m accurately identifies the consistent boundary within the time interval spanning across two classes. We also utilize the C-score proposed by the ClaSP model [9] to assess the segmentation capability of models. The C-score reflects the ability of models to recognize segmentation boundaries by measuring the trade-off between precision (correctly identified change points) and recall (finding all true change points), ensuring the model captures meaningful transitions without over-segmenting or missing important splits. We compute the scores for each model across three sets of experiments on SEEG data and take the average. Notably, Con4m outperforms the other models significantly, while MS-TCN2, specifically designed for segmentation tasks, also achieves impressive scores. This verifies that the label consistency framework can harmonize the boundaries more effectively. Refer Appendix H for more cases.
Conclusion and Discussion
In this work, we focus on the raw time series MVD for segmented time series classification (TSC) tasks, demonstrating unique challenges that are overlooked by existing mainstream TSC models. We first formally demonstrate that valuable contextual information enhances the discriminative power of classification instances. Based on the insights, we introduce contextual prior knowledge of data locality and label coherence guide the model focus on contextual information more conducive to discriminating consecutive segments in segmented TSC tasks. Leveraging effective contextual information, a label consistency learning framework Con4m is proposed to progressively harmonize inconsistent labels during training. Extensive experiments validate the superior performance achieved by Con4m and highlight the effectiveness of the proposed consistent label training framework. Our work still has some limitations. We have solely focused on analyzing and designing end-to-end supervised models. Further exploration of large-scal, models would be challenging yet intriguing. Con4m is a combination of segmentation and classification, both of which are fully supervised. Exploring its application unsupervised segmentation tasks worthwhile. When faced with more diverse label behaviors, the function fitting module needs to engage in more selection and design of basis functions. Nevertheless, our work brings new insights to the TSC domain, re-emphasizing the importance of the inherent temporal dependence of time series.
10

---

Acknowledgments and Disclosure of Funding
This work was partially supported by National Natural Science Foundation of China (No. 62322606, No. 62441605).
References
[1] Dana Angluin and Philip Laird. Learning from noisy examples. Mach. Learn., 3(4):333-93) 1988.
[2] Samaneh Azadi, Jiashi Feng, Stefanie Jegelka, and Trevor Darrell. Auxiliary image regulariza- tion for deep cnns with noisy labels. In International Conference on Learning Representations (ICLR), 2016.
[3] Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprini arXiv: 1a0::811100072 2018.
[4] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR), 2015.
[5] Peter L Bartlett, Michael I Jordan, and Jon D McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, (01(473):138-1 2006.
[6] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning (ICMI.), pages 41 48, 2009.
[7] Henrik Blunck, Sourav Bhattacharya, Thor Prentow, Mikkel B. Kjrgaard, and Anind K. Dey. Heterogeneity Activity Recognition. UCI Machine Learning Repository, 2015. DOI: https://oo..rg/1.2245///666
[8] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2017.
[9] Andrea Castellani, Sebastian Schmitt, and Barbara Hammer. Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noise. In Joint European Conference Machine Learning and Knowledge Discovery in Databases (ECML PKDD), pages 469-484, 2021.
[10] Ranak Roy Chowdhury, Xiyuan Zhang, Jingbo Shang, Rajesh K. Gupta, and Dezhi Hong. Tarnet: Task-aware reconstruction for time-series transformer. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 212-220, 2022.
[11] Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The uer time series archive. IEEE/CAA Journal of Automatica Sinica, 1293-13333 2019.
[12] Angus Dempster, François Petitjean, and Geoffrey I. Webb. Rocket: Exceptionally fast and accurate time series classification using random convolutional kernels. Data Min. Knowl. Discov, 34(5):1454 1495, 2020.
[13] Angus Dempster, Daniel Schmidt, and Geoffrey I. Webb. Minirocket: very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD), pages 248 257, 2021.
[14] Don Dennis, Durmus Alp Emre Acar, Vikram Mandikal, Vinu Sankar Sadasivan, Venkatesh Saligrama, Harsha Vardhan Simhadri, and Prateek Jain. Shallow rnn: Accurate time-series classification on resource constrained devices. Advances in Neural Information Processing Systems (NeurIPS), 2019.

---

[15] Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL.), pages 4171-4180 2019.
[16] Guodong Ding and Angela Yao. Temporal action segmentation with high-level complex activity labels. Trans. Multi., 25:1928- -1939, 2023.
1939,,
[17] Guodong Ding. Fadime Sener, and Angela Yao. Temporal action segmentation: An analysis of modern techniques. IEFE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), (6(2):1011-10333 2023.
[18) Zexing Du, Xue Wang, Guoqing Zhou, and Qing Wang. Fast and unsupervised action boundary detection for action segmentation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3313-3322, 2022.
[19]
Arik Ermshaus, Patrick Schäfer, and Ulf Leser. Clasp: parameter-free time series segmentation. Data Min. Knowl. Discov., 37(3): 222-2222222222222 1300, 2023.
[20] Navid Mohammadi Foumani, Lynn Miller, Chang Wei Tan, Geoffrey Webb, Germain Forestier, and Mahsa Salehi. Deep learning for time series classification and extrinsic regression: A current survey. arXiv preprint arXiv:2302.02515, 2023.
[21] Jean-Christophe Gagnon-Audet, Kartik Ahuja, Mohammad farvishi-Bayazi, Pooneh Mousavi, Guillaume Dumas, and Irina Rish. Woods: Benchmarks for out-of-distribution generalization in time series. arXiv preprint arXiv:2203.09978 2022.
[22] Vivien Sainte Fare Garnot and Loic Landrieu. Lightweight temporal self-attention for classifying satellite images time series. Advanced Analytics and Learning on Temporal Data: 5th ECML PKDD Workshop (AALTD), pages 171--81,, 2020.
[23] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convolu- tional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning (ICML), pages 1143-1255. 2017.
[24] Tieliang Gong, Qian Zhao, Deyu Meng, and Zongben Xu. Why curriculum learning & self- paced learning work in big/noisy data: theoretical perspective. Big Data and Information Analytics, (1):111-122) 2016.
[25] Alex Graves, Mare G. Bellemare, Jacob Menick, Rémi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. In Proceedings of the 34th International Conference on Muchine Learning (ICML), pages 1311-1320, 2017
[26] Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision. In Advances in Neural Information Processing Systems (NeurIPS), 2018
[27] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems (NeurIPS), 2018
[28] Bo Han, Gang NIu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang, and Masashi Sugiyama. SIGUA: Forgetting may make learning with noisy labels more robust. In Proceedings of the 37th International Conference on Machine Learning (ICML), pages 4006-4016, 2020.
[29] Bo Han, Quanming Yao, Tongliang Liu, Gang Niu, Ivor W. Tsang, James T. Kwok, and Masashi Sugiyama. A survey of label-noise representation learning: Past, present and future. arXiv preprint 20 I. /.04066, 2021.
[30] Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, and Austin Benson. Combining label propagation and simple models perfoorss graph neural networks. In International Conference on Learning Representations (ICLR), 2021.
12

---

[31] Zhe Huang, Liang Wang, Giles Blaney, Christopher Slaughter. Devon McKeon, Ziyu Zhou, Robert Jacob, Robert Jacob, and Michael Hughes. The tufts fnirs mental workload dataset bench- mark for brain-computer interfaces that generalize. In Proceedings of the Neural Information Processing Systems (NeurIPS) Truck on Datasets and Benchmarks, 2021.
[32] Zhizhong Huang, Junping Zhang, and Hongming Shan. Twin contrastive learning with noisy la- bels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition (CVPR), pages I 11661-118 1670, 2023.
[33] Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi-supervised learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[34] Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre- Alain Muller. Deep learning for time series classification: A review. Data Min. Knowl. Discov., 33(4):117-9633 2019.
[35] Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. arXiv preprint arXiv:1712.05055, 2018.
[36] Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. Unicon: Combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9676 6676-9686, 2022.
[37] B. Kemp, A.H. Zwinderman, B. Tuk, H.A.C. Kamphuisen, and J.J.L. Oberye. Analysis of deeendeeendent neuronal feedback loop: The slow-wave microcontinuity of the eeg. IEEE Transactions on Biomedical Engineering, 47(9):1185- 2000.
[38] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations (ICLR), 2015.
[39] Ron Kohavi. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference Artificial Intelligence (IJCAI), pages 1137 1143, 1995.
[40] M. Kumar, Benjamin Packer, and Daphne Koller, Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems (NeurIPS), 2010.
[41] Neil D. Lawrence and Bernhard Schölkopf. Estimating a kernel fisher discriminant in the presence of label noise. In Proceedings of the Eighteenth International Conference on Machine Learning (ICML), pages 306-3313 2001.
[42] Shijie Li, Yazan Abu Farha, Yun Liu, Ming-Ming Cheng, and Juergen Gall. Ms-tcn++: Multi- stage temporal convolutional network for action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(6):6647 66588 2023.
[43] Shikun Li, Xiaobo Xia, Shiming Ge, and Tongliang Liu. Selective- contrastive learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 316-325, 2022.
[44] Jason Lines, Sarah Taylor, and Anthony Bagnall, Time series classification with hive-cote: The hierarchical vote collective of cansformation-bacee ensembles. ACM Trans. Knowl. Discov. Data, 12(5), 2018
[45] Daochang Liu, Qiyue Li, Anh-Dung Dinh, Tingting Jiang, Mubarak Shah, and Chang Xu. Diffusion action segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages
2023.
[46] Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over- parameterization. Proceedings of the 39th International Conference on Machine Learning (ICML), pages 14153-14172, 2022.
13

---

[47] Zhen Liu, ma peitian. Dongliang Chen, Wenbin Pei, and Qianli Ma. Scale-teaching: Robust multi-scale training for time series classification with noisy labels. In Advances in Neural Information Processing Systems (NeurIPS), pages 33726-337577, 2023.
[48] Tambet Matiisen, Avital Oliver, Taco Cohen, and John Schulman. learning. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 31(9): 3732-3740, 2020.
curriculum
[49] Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of feather: Homophily in social networks. Annual review of sociology, 27(1):415-444, 2001.
[50] Matthew Middlehurst, Patrick Schäfer, and Anthony Bagnall. Bake off redux: A review and experimental evaluation of recent time series classification algorithms. arXiv preprint arXiv:2304. 13029, 2023.
[51] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations (ICLR), 2023.
[52] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, hegf-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[53] Giorgio Patrini, Alessandro Rozza, Aditya Menon, Richard Nock, and Lizhen Qu Mak- ing deep neural networks robust to label noise: loss correction approach. arXiv preprint arXiv: 00660000.0000..... 2017.
[54] Deepta Rajan and Jayaraman J. Thiagarajan. A generative modeling approach to limited channel eeg classification. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 2571-2574, 2018.
[55] Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial- -peporlltemporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 1567-115 2022.
[56] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae- Lee. Learning from noisy labels with deep neural networks: A survey. IÉEE Transactions on Neural Networks and Leurning Systems (TNNLS), 34(11):8135 11-881555513,,, 2023.
[57] Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdey, and Rob Fergus. Training convolutional networks with noisy labels. arXiv preprint arXiv: 2015.
[58] Wensi Tang, Guodong Long, Lu Liu, Tianyi Zhou, Michael Blumenstein, and Jing Jiang. Omni- scale CNNs: simple and effective kernel size configuration for time series classification. In International Conference Learning Representations (ICLR), 2022.
[59] Heng Wang and Cordelia Schmid. Action recognition with improved trajectories. In IEEE International Conference on Computer Vision (ICCV), pages 3551-3558, 2013.
[60] Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEFE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(5):4555545777 2021.
[61] Yucheng Wang, Yuecong Xu, Jianfei Yang, Min Wu, Xiaoli Li, Lihua Xie, and Zhenghua Chen. Graph-aware contrasting for multivariate time-series classification. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 15725-15734, 2024.
[62] Hongxin Wei, Lue Tao, RENCHUNZI XIE, and Bo An. Open-set label noise can improve robustness against inherent label noise. Advances in Neural Information Processing Systems (NeurIPS), pages 7978-7992, 2021.
14

---

[63] Daphna Weinshall, Gad Cohen, and Dan Amir. Curriculum learning by transfer learning: Theory and experiments with deep networks. In Proceedings of the 35th International Conference on Machine Learning (ICML), pages 2238-2246, 2018.
[64] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d 2d-variation modeling for general time series analysis. In The Eleventh International Conference Learning Representations (ICLR), 2023.
[65] Kai Wu, Kaixin Yuan, Yingzhi Teng, Jing Liu, and Licheng Jiao. Broad fuzzy cognitive map systems for time series classification. Appl. Soft Comput., 128(C):109458, 2022.
[66] Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In International Conference on Learning Representations (ICLR), 2022.
[67] Chao-Han Huck Yang, Yun-Yur Tsai, and Pin- Yu Chen. Voice2series: Reprogramming acoustic models for time series classification. Proceedings of the 38th International Conference on Machine Learning (ICML), pages 1188--1889, 2021.
[68] Fangqiu Yi, Hongyu Wen, and Tingting Jiang. Asformer: Transformer for action segmentation. In The British Machine Vision Conference (BMVC), 2021.
[69] Chong You, Zhihui Zhu, Qing Qu, and Yi Ma. Robust recovery via implicit bias of discrepant learning rates for double over-paramettrrizatonn. arXiv preprini SkKi::20060888577 2020.
[70] Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? arXiv preprint 1rXiv:1901.0420 2019.
[71] Bowen Zhao, Huanlai Xing, Xinhan Wang, Fuhong Song, and Zhiwen Xiao. Rethinking attention mechanism in time series classification. Inf. Sci., 62797)177- 2023.
[72] Evgenii Zheltonozhskii, Chaim Baskin, Avi Mendelson, Alex M. Bronstein, and Or Litany. Contrast to divide: Self-supervised pre-training for learning with noisy labels. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pages 1657-1667, 2022.
[73] Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu. and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Advances in Neural Information Processing Systems (NeurIPS), pages 7793-7804, 2020.
15

---

A Details of Related Works
Time series classification (TSC), TSC has become a popular field in various applications with the exponential growth of available time series data in recent years. In response, researchers have proposed numerous algorithms [4]. High accuracy in TSC is achieved by classical algorithms such as Rocket and its variants [2 13 which use random convolution kernels with relatively low computational cost, as well as ensemble methods like HIVE-COTE 44 which assign weights to individual classifiers.
Moreover, the flourishing non-linear modeling capacity of deep models has led to an increasing preva- Ience of TSC algorithms based on deep learning. Various techniques are utilized in TSC: RNNed methods [54] [4] capture temporal changes through state transitions; MLP-based methods [22 65
encode temporal dependencies into parameters of the MLP layer, and the latest method TimesNet [64 converts one- nne-dimensional time series into two-dimensional space, achieving state-of-the-ar per- formance on five mainstream tasks. Furthermore, Transformer- based models 66 [0] with attention mechanism have been widely used.
The foundation of work lies in these researches, including the selection of the backbone and experimental setup. However, mainstream TSC models (50] 20 are often designed for publicly avail- able datasets 3 || based on the i.i.d. samples, disregarding the inherent contextual dependencies between classified segments in MVD. Although some time series models [55] [1]] use patch-by-pate technique to include contextual information, they are partially context-aware since they only model the data dependencies within each time segment, ignoring the dependencies between consecutive segments.
Noisy label learning (NLL). NLL is an important and challenging research topic in machine learning, real-world data often rely on manual annotations prone to errors. Early works focus statistical
learning [ 41 5 Researches including Sukhbaatar et al. [57] launch the era of noise-labeled representation learning.
The label noise transition matrix, which represents the transition probability from clean labels to noisy labels [29], is an essential tool. Common techniques for loss correction include forward and backward correction (53), while masking invalid class transitions with prior knowledge is also an important method [26). Adding explicit implicit regularization term in objective functions can reduce the model's s sensitivity to noise, whereas re-weighting mislabeled data can reduce its impact on the objective (2, 69 (6). Other methods involve training on small-loss instances and utilizing memorization effects. MentorNet (35 pretrains a secondary network to choose clean instances for primary network training. Co-teaching 277 and Co-teaching+ [70]. sample selection methods, introduce two neural networks with differing learning capabilities to train simultaneously. which filter noise labels mutually. The utilization of contrastive learning has emerged promising approach for enhancing the robustness in the context of classification tasks of label correction methods (43 72 32)
These works primarily focus handling noisy labels. And ensuring overall label consistency by modifying certain labels is crucial for MVD. To the best of our knowledge, Scale-tcaching [47] and are the only NLL works specifically designed for time series. Scale-tcaching designs fine-to-coarse cross-scale fusion mechanism for learning discriminative patterns by utilizing time series at different scales train multiple DNNs simultaneously. SREA trains classifier and an autoencoder with a shared embedding representation, progressively self-relabeling mislabeled data samples in a self-supervised manner. However, they still face the issue of overlooking contextual dependencies across consecutive time segments.
SREA p
Curriculum learning (CL.. Bengio et al. 66] propose CL., which imitates human learning by starting with simple samples and progressing to complicated ones. Based on this notion, CL can denoise noisy data since learners are encouraged to train on easier data and spend less time on noisy samples [24] (60) Current mainstream approaches include Self-paced Learning [40], where students schedule their learning, Transfer Teacher [63]. based on predefined training scheduler; and RL Teacher 25 48 which incorporates student feedback into the framework. The utilization of CL proves to be particularly advantageous in situations involving changes in the training labels. Hence, this technique is utilized to enhance the harmonization process of boundary labels from MVD in a more stable manner.
Temporal action segmentation (TAS). TAS is critical task video understanding and analysis. involves segmenting an untrimmed video sequence into meaningful temporal segments and assigning
16

---

predefined action label to each segment 17] The majority works on TAS typically take visual feature vectors, either nand-crafted (IDT) 599 extracted from an off-the-shelf CNN backbone (I3D) ( as input for each frame. 'AS uses sequential modeling to incorporate temporal dependencies and sequential context for improved TAS accuracy.
While TAS shares some similarities with segmented TSC, there still exist some problems. Time series data often have high sampling rates, however, it impossible to input excessively long sequences into TAS models designed for video data. Moreover, unlike the I3D features [ used as input in these works, time series lack a unified pretrained model for feature extraction. Besides, the changes between adjacent video frames are smooth and continuous, whereas time series are more variable and exhibit more ambiguous boundaries. Also, TAS works focus on modeling the dependency of instances from a data perspective without explicitly leveraging contextual label information.
B Implementation Details of Prediction Behavior Constraint
To fit the hyperbolic tangent function (Tanh), practice, we use the Adam optimizer with learning rate of 0.1 to optimize the trainable parameters. The maximum number of iterations is set to 100, and the tolerance value for stopping the fitting process based on loss change is set to 1 - 6. Sequences belonging to one minibatch are parallelized to fit their respective Tanh functions. To adapt the value range of the standard Tanh function, rescale the sequential predictions to [-1,17 1] before fitting.
the mean squared error (MSE) loss function. In
However, it can be difficult to achieve good fit when fitting with the Tanh function. Specifically, random initialization may fail to fit the sequential values properly when a long time series undergoes state transition near the boundary. For example, Figure a) shows, we fit sequence in which only the last value 1. We set all default initial parameters as and fit it. It can be observed that the fitting function cannot properly fit the trend and will mislabel the last point.
0.50 Tamh(-0.02
0.02 x (=- 0.56)) 0.94
1.48 x Tanh x (= 14.86) 0.48
0.5
loss 3.519
loss 0.002
-15
-10
0
10
15
-15
-10
-5
10
(a) Fit with all initialization
(b) with designed initialization
1.13 Tanh(-0.69 (x 17.28)) 0.22
0.67
Tanh(0.57 9.92))
0.08
0.5
0.5
0.009
loss 0.945
-15
-10
0
10 15
-15
-10
-5
10
(c) Fil with designed initialization
(d) Fit with designed initialization
Figure 5: Cases for Tanh fitting.
Appropriate parameter initialization is needed to avoid excessive bias. After careful observation, find that parameter controls the slope the transition part of Tanh, and parameter controls the abscissa at the transition point. In the process, all fitting values are assigned with uniform abscissa values. Therefore, corresponding position in the entire sequence. And these two values are assigned to parameters k and respectively. This allows to obtain suitable initial parameters and avoid getting trapped in local optima or saddle points during function fitting. Formally, given the L-length input sequence ý, we initialize parameters and b as follows:
calculate the maximum difference between adjacent values and the
di p.41 P.lkE(L...LL1)
17

---

k, b max (ibs arg max (Abs(di)),
Sign(di(b))
b 0= -
+ +
where Abs(.) and Sign(-) denote the absolute value function and sign function respectively. di is the difference vector. After proper initialization, as Figure(5{b) shows, we can obtain more accurate fitting results to reduce the probability of mislabeling. We also show some other cases (F)gure((c))d)) for the fitting results to verify the effectiveness of the fitting process we propose.
C Hyperparameter Analysis
28
K
(N
2)
(M
K 2N.
(N
R IN.
(3t
K IN
...
(2N)
1)
INN
K
K
73
Con4m
69
Level
Level
Con4m-17
65
Level Ni
E
(a) Visualization illustration of equation (2) in curriculum learning
(b) Hyperparameter analysis
Figure 6: Visualization of data division in curriculum learning and hyperparameter analysis of Egg-
The dynamic weighting factor 7 is introduced to progressively update the labels, preventing the model from overly relying its own predicted labels too early. To validate the utility of r and determine an appropriate linear growth epoch E,,, we conduct the hyperparameter search experiment on SEEG data. As shown in Figure [bb. with smaller E., (corresponding to higher growth rate), there is significant improvement in model performance. This aligns with our motivation that during the early stage of model training, the primary objective is to better fit the original labels. At this stage, the model's own predictions are unreliable. If the predicted results are used as training labels too early in subsequent epochs, the model would be adversely affected by its own unreliability. On the other hand, excessively large E, leads to a slower rate of label updates, making it more challenging for the model timely harmonize inconsistent labels. Nonetheless, considering the impact of variance, the model exhibits robustness to slightly larger Ex, In this work, we uniformly use E. 30 as the default value.
D Details of Datasets
INIRS. All signals are sampled at frequency of 5.2 At each time step, they record 8 real-valued measurements, with each measurement corresponding to concentration changes (oxyhemoglobin and deoxyhemoglobin) 2 types of optical data (intensity and phase), and 2 spatial positions on the forehead. Each measurement unit a micromolar concentration change per liter of tissue (for pxy-/deoxyhemoglobin
They label each part of the active experiment with one of four possible levels of backack working memory intensity (0-back, 1-back, 2-back, or 3-back). More specifically, in an n-back task, the subject receives 40 numbers in sequence. If number matches the number n steps back, the subject is required to respond accordingly. There are 16 rounds of tasks, with a 20-second break between each task. Following Huang et al. [31], we only apply classification tasks for O-back and 2-back tasks in our work. Therefore, we only extract sequences for O-back and 2-back tasks and concatenate them in chronological order.
The HHAR (Heterogeneity Human Activity Recognition) dataset 7 captures sensor data from multiple smart devices to explore the impact of device heterogeneity on human activity recognition. It involves measurements from accelerometers and gyroscopes, which are two common motion sensors found in smartphones and smartwatches. The dataset includes six types of human activities: 'Biking', 'Sitting', 'Standing'. 'Walking', 'Stairs Up', and 'Stairs Down'. For each activity, both 3-axis accelerometer and 3-axis gyroscope readings are recorded, resulting in six key features
HHAR.
18

---

per time step. The data was collected from five different device types, consisting of four smartwatches and eight smartphones, across nine participants. Each device samples data at its highest possible frequency, generally around 50Hz. For data preprocessing, the accelerometer and gyroscope signals are aligned based on their timestamps, normalized within each channel, and divided by the devices, following the procedure outlined in Gagnon-Audet et al. [21. To ensure balanced samples, we combined data from the Galaxy S3 Mini, LG watch, and Gear watch into a single group.
Sleep. The Sleep-EDF database records Graphicii sleep data from 197 subjects, including EEG, EOG, chin EMG, and event markers. Some data also includes respiration and temperature- related signals. The database contains two studies: the Sleep Cassette study and the Sleep Telemetry study. The former records approximately 40 hours of sleep from two consecutive nights, while the latter records around 18 hours of sleep. Well-trained technicians manually score the corresponding sleep graphs according to the Rechtschaffen and Kales manual. The data is labeled in intervals of 30 seconds, with each interval being marked as one of the eight possible stages: W, R, 1, 2, 3, 4, M, or ?. In our work, we utilize only the data from the Sleep Cassette study, and retain only the signals from the EEG Fpz-Cz channel and EOG horizontal channel. The EEG and EOG signals were sampled at frequency of 100Hz. Following Kemp et al. [7 we remove the labels for stages and M from the data, and merge stages 3 and 4, resulting in 5-classification task.
The private SEEG data records brain signals indicative of suspected pathological tissue within the brains of seizure patients. They are anonymously collected from a top hospital we cooperate with. For patient suffering from epilepsy, 4 to 11 invasive electrodes with 52 153 channels are used for recording signals. In total, we have collected 847 hours of SEEG signals with high frequency (1, 000Hz or 2, (00Hz) and a total capacity of 1.2TB. Professional neurosurgeons help us label the seizure segments for each channel. Before sampling for the database, we remove the bad channels marked by neurosurgeons. Then we uniformly downsample the data to 250Hz and use a low-pass filter to process the data with a cutoff frequency of 30Hz. Finally, we normalize and sample the intervals for each channel respectively.
SEEG.
E Implementation Details of Baselines
SREA 9
This time series classification model with noisy labels jointly trains classifier and an autoencoder with shared embedding representations. It gradually corrects the mislabelled data samples during training in sull-super fashion. We use the default model architecture from the source code provided by the author
This work designs tie-to-coarse cross scale fusion mechanism for learning discriminative patterns by utilizing time series at different scales to train multiple DNNs simultane- ously. It uses well-learned multi-scale time series embeddings for noise label correction at sample feature levels. We modify the code to match our datasets based on the code provided by the author
Scale-teaching [7]:
https://iitubbcoo/qaaaiiaaa hawaiiaaaaaaaslatteeeeaiii
This model adopts gradient descent on good data as usual, and learning-rate--rduuce gradient ascent on bad data, thereby trying to reduce the effect of noisy labels. We modify the network for time series data based on the open source code provided by SREA, using the code from the author (https:/github.com/bii htt../o.//mammm//GAAA.
SIGUA (28):
UNICON [6]: UNICON introduces a Lensen-Shannov divergence- based uniform selection mechanism and uses contrastive learning to further combat the memorization of noisy la- bels. We modify the model for time series data according the code provided by the author https://giiubbccom//aammlilee wtt.aaaaaiimiiiooooooo... 700NNCNN-NNiss-LLbee)
Selective-Supervisce Contrastive Learning (Sel-CL) is a latest bascline model in the field of computer vision. It selects confident pairs out of noisy ones for supervised contrastive learning (Sup-CL) without knowing noise rates. We modify the code for time series data, based on the source code provided by the author
Sel-CL 43
MiniRocket [13] Rocket [12 achieves state-of- ofute-of-the-arr the-art accuracy for time series classification by transforming input time series using random convolutional kernels, and using the transformed features to train linear classifier. MiniRocket is a variant of Rocket that improves processing time, while offering essentially the same accuracy. We use the code interface from the sktime package htttss//gthhbb.co//kiimm/tiiii
19

---

This model focuses on temporal variation modeling, With TimesBlock, it can discover the multi- periiiitrio adaptively and extract the complex temporal variations from trans- formed 2D tensors by parameter-effiiient inception block. We use the code from the TSlib package httpss//gihubh..om//um/// Sattrfuummmmmmm--See----iaaaaaaa Series-Library).
TimesNet [64]]
This is a self-supervised representation learning framework for multivariate time series by segmenting time series into subseries level patches, which are served as input tokens to Transformer with channel-indepe iineeeeedddeeeeencee.. We modify the code to achieve classifica- tion for each patch, based on the source code from the Time Series Library (TSlib) package httts::/gituub.co//uum//// Saapam////iimmSSee---irrrrre
PatchTST [51:
This work proposes two multi-stage architectures for the temporal action segmen- tation task. While the first stage generates an initial prediction, this prediction is itcratively refined by the higher stages. Instead of the commonly used temporal pooling, they use dilated convolutions to increase the temporal receptive field. We modify the code for time series data, based on the source code provided by the author
MS-TCN2 42
ASFormer [68]: ASFormer
Transformer- model for action segmentation tasks. It explicitly brings in the inductive priors of local connectivity and applies pre-defined hierarchical representation pattern to handle long input sequences. The decoder is also carefully designed to refine the initial prediction from the encoder. We modify the code for time series data, based on the source code provided by the author htttss:/github.com/// com/China Yannnhhhhaai/SSoeeeeee Yi/ASFormer).
DiffAct [45]: In this work, action predictions are iteratively generated from random noise with input video features conditions. It also devises unified masking strategy for the conditioning inputs to enhance the modeling of striking characteristics of human actions. We modify the code for time se- ries data, based on the source code provided by the author (https://github.com// cowm://ppir/////eeeeee /3/Dif.Act).
F Implementation Details of Con4m
The non-linear encoder Jenc used in Con4m is composed of three 1-D convolution layers. The number of kemness vary across different data and you can find corresponding parameters in the default config file of our source code. We construct the Con-Transformer based on the public codes implemented by HuggingFace" We set d=128 and the dimension of intermediate representations in FFN module as 256 for all experiments. The number of heads and dropout rate are set as and 0.1 respectively. Since we observe that one-layer avoid overfitting. Note that Con4m consists of two Con-Tr Transformers and we use two Con-Attention layers. The model is optimized using Adam optimizer (38) with a learning rate of 1e-3 and weight decay of 1e-4, and the batch size is set as 64. We build our model using Py 2.0.0 [52) with CUDA 11.8. And the model is trained on workstation (Ubuntu system 20.04.5) with 2 CPUs (AMD EPYC 7H12 64-Core Processor) and GPUs (NVIDIA GeForce RTX 3090).
can fit the data well, we do not stack more layers to
G Full Results
The full results of the label disturbance experiment are listed in Table 4 5 6 and 7 For INIRS, we first divide the data into groups by subjects and follow the 2 training- validation-1 testing (2-1-1) setting to conduct cross-validation experiments. Therefore, there are C2 x C2 = 12 experiments in total. Similarly, we divide the HHAR and Sleep data into 3 groups and follow the 1-1-1 experimental setting. Therefore, we carry out C₁2 C2 6 experiments. For SEEG data, we follow the same setting as fNIRS. Notice that for SEEG data, inconsistent labels already exist in the raw data. We obtain a high-quality testing group by using majority voting procedure to determine the boundaries. Then we leave the testing group aside and only change the validation group to report the mean value of C3 = 3 experiments. All the experimental results are listed in lexicographical order according to the group name composition. We also report the mean value and standard derivation of all experiments. Specifically, we use the STDEVA to estimate standard deviation based sample of data.
bert.py
20

---

TAS
TSC
NLL
TSC&NLL
Exp
MS-
Mini Rocket
Times
Parch
UNI
Scale-
TCN2
Former
Net
TST
GUA
CON
Cl.
68.61
51.07
63.85
68.55
2
70.50
71.85
68.62
62.45
68.25
48.21
67.55
53.17
65.49
70.29
74.26
71.64
70.21
70.08
71.42
60.96
66.38
54.23
65.56
61.71
61.44
67.14
68.99
70.51
71.89
73.38
72.25
62.24
69.73
55.07
68.83
60.74
63.23
70.23
71.64
72.65
5 71.87
72.91
72.80
61.35
66.46
57.11
67.96
54.87
63.21
70.
75.85
68.55
72.53
73.12
7
69.
68.08
70.77
60.11
62.64
50.46
64.03
62.54
60.13
70.55
62.42
70.63
8
72.29
72.44
72.13
62,90
69.57
49.75
69.15
63.05
65.31
70.90
66.81
73.36
72.52
72.22
69.47
58.78
67.23
48.86
68.41
66.77
63.45
70.29
71.88
72.60
10 74,48
70.20
73.02
60.75
71.17
55.63
68.24
$9.40
65.47
71.59
70.60
39
70.71
70.09
12
73.08
75.02
70.81
61.93
71.30
45.40
67.89
66.88
64.70
72.17
68.95
75.14
Avg
71.48
71.69
71.15
61.28
67.47
51.79
67.37
61.15
63.86
70.10
70.40
71.28
Std
1.91
1.32
1.13
3.91
3.72
1.28
4.14
2.11
20.19
59.22
62.74
49.81
64.38
68.32
65.55
69.48
2
71 26
70.42
69.61
60.10
67.31
58.02
63.38
63.30
64.00
70.40
68.36
72.94
72.18
66.46
71.39
59.52
60.19
54,99
64.27
51.67
57.25
70.16
64.44
72.06
71.19
71.74
71.53
63.15
66.04
62.28
67.91
62.25
61.45
69.78
66.18
73.56
71 85
72.38
69.18
62.04
67.66
55,29
66.07
56.26
64.22
68.45
66.59
71.41
72.03
57.22
70.34
72.08
69 94
66.99
68.32
59.15
59.02
53.13
60,97
49.65
59.
67.69
71.55
62.66 68
20
69.19
72.09
64.24
60.33
66.27
56.57
63.72
66.42
63.28
70.85
75.03
71.59
74.44
72.21
69.70
59.41
66.73
52.13
67.11
62.28
61.54
68.15
66.94
71.84
10
72.15
72.79
69.91
60.77
69.07
56.96
68.27
59.87
68.18
71.29
64.83
71.72
12
74,10
70.41
72.16
60.79
66.22
58.49
67.87
62.81
63.62
70.93
69.68
73.76
Ave
70.99
70.75
69.72
60.41
65,39
55,38
65.24
60.45
62.45
69.65
68.06
71.27
2.45
2.17
1.32
3.71
$.22
1.22
2.92
68.74
65.53
65.47
57.21
62.93
51.39
60.63
52.63
61.98
69.37
62.03
.90
2
67.96
20.88
64.39
58.85
62.10
50.27
59.84
45.74
62.50
69.43
68.10
71.91
70,76
71.54
66,20
59.23
68.56
58.48
67.18
63.46
64.25
68.84
66.90
70.68
72.62
66.59
64.44
$7.05
59.96
54.44
64.60)
63.00
58.31
68.49
67.67
71.55
71.36
72.64
61.00
58.43
66.76
53.18
64,20
59.95
69) 85.
68.12
72.75
7
67.31
67.94
68.49
56.25
60.06
49.93
61.58
$2.33
60.33
69.19
$9.47
66.69
40
66.67
66.67
53.20
67.27
60.76
63.91
70.49
66.59
67.74
66.30
56.95
63.86
61.90
61,80
65.20
62.82
68.42
67.82
69.88
10 69.56
66.67
65.55
55.78
62.01
49.37
63.19
55.84
61.04
70.16
66.68
69.00)
71.66
72.80
62.43
58.34
62.78
$7.77
62.12
57.92
61.68
68.36
68.64
70.59
12
71.32
72.74
66.53
59,81
63.96
48.00
64,21
56.73
62.93
71.05
69.38
72.02
Avg Std
69. 40.40
69.18
65.45
57.87
63.45
52.67
63.47
$7.35
61.75
69.40
66.51
70.04
2.10
2.75
2.22
1.22
4.95
2.38
5.57
0.85
2.14
Table 4: Full results of the label disturbance experiment on INIRS data. The best results are in bold and we underline the second best results, The worst results are denoted in italics.
H
Case Study
As shown in Figure 7 we present four cases to compare and demonstrate the differences between our proposed Con4m and other baselines. The first two cases involve transitions from seizure state of label to a normal state of label 0. The third case consists of entirely normal segments, while the fourth case comprises entirely seizure segments. As illustrated in the figure, Con4m exhibits more coherent narratives by constraining the predictions to align with the contextual information of the data. Moreover, demonstrates improved accuracy in identifying the boundaries of transition states. In contrast, other baselines exhibit fragmented and erroneous predictions along the time segments. This verifies that Con4m can achieve clearer recognition of boundaries, and it can also make better predictions on the consecutive time segments belonging to the same class.
21

---

TAS
TSC
NLL
TSC&NLL
+%
Exp
TCN2
Former
DiffAct
Rocket
Mini
Net
Patch
TST
GUA
CON
UNI
CL.
SREA
Con-tnn
41.39
34.82
41.92
52.89
45.80
30.88
46.79
52.71
47.69
47.40
45.80
52.37
77.19
55.28
53.24
75.76
89.33
52.88
80.2]
98.20
93.87
76.48
95.08
3
43.71
42.61
45.85
54.11
44.75
42.92
46.89
47.60
46.38
48.14
49.43
48.45
0
81.92
82.37
65.32
75.00
92.82
55.04
77.44
81.12
81.04
93.65
98.78
90.67
79.96
70.57
94.37
88.03
70.61
94.05
93.02
87.18
91.70
94,63
94,09
83.89
80.07
63.73
69.91
71.70
59.66
68.27
83.61
81.73
67.06
88.05
93.02
Avg
69.79
62.52
56.76
70.34
72.07
52.00
78.61
73.00
77:77
$0.29
Std
21.56
01.08
11.51
15:47
22.00
13.74
19.01
22.67
20.63
18.01
23.52
23.26
42,39
44,86
38.64
47.97
45,
30.52
47.59
51.25
46.59
45.54
48.
54.82
68.54
64.63
54.97
71.37
83.16
39.71
81.73
97.72
90.03
70.52
95.35
93.31
46.72
47.95
38.36
49.72
48.64
43,00
$1.86
50.40
47.72
53.70
51.06
45.06
4
87.95
77.25
66.51
69.95
85.91
41.34
76.36
91.20
86.85
73.44
93.33
96.37
20
82,97
68.53
69.22
86.79
87.36
68.77
88.61
91.35
89.62
85.33
90.48
90.61
71,76
62.32
55.47
54,14
70.13
49.41
64.65
79.70
72.84
67.55
81.81
91.38
Avg
66.72
60.92
53.86
63.32
70.19 19
45.46
68, 68,447
76.94
72.28
66.02
76.71
78.59
12.38
15.26
12.95
21.05
14.29
22.49
39.17
42,50
37.82
65.6
44.86
30.87
52.92
48.83
48.60
46.
50.21
50.78
2
59.62
59.00
51.47
66.27
73.60
42.93
72.53
92.05
94.60
67.78
94.41
91.36
54.82
46.54
42.00
48.52
43.99
52.23
49.13
47.87
55.59
4%
53.33
4
72.94
78.15
60.46
62.46
87.26
46.10
76.04
75.99
80.98
74.54
93,49
91.57
40
75.12
75.85
57.81
77.54
79.70
62.93
88.42
93.46
86.63
85.48
87.43
82.62
72.08
62.57
54.23
53.03
71.13
51.42
63.43
79.79
78.15
64.05
81.56
83.48
Avg Std
62,29
60.77
50.63
59.25
66,76
45.69
67.60
73.21
72.81
65.67
75.97
75.52
13.94
14.64
8.95
11.68
18.18
10.87
14.13
19.95
19.85
13.74
21.06
18.58
Table Full results of the label disturbance experiment on HHAR data. The best results are bold
and we underline the second best results. The worst results are denoted in italics.
TAS
TSC
NLL
TSC&NLL
Patch
UNL
+%
Exp
TCN2
Former
DiffAct
Rocket
Net
TST
GUA
CON
CL
SRFA
Condo
59.46
55.89
51.79
62.16
58.73
58.42
54.79
62.41
48.95
63.05
68.80
59.47
58.37
39.55
61.14
59.72
58.60
52.69
62.49
62.87
46.93
62.70
67.63
3
60.72
59.60
52.65
62.74
60.76
59.44
56.19
62.62
65.47
49,38
64.91
69.29
0
47.42
59.22
53.51
61.01
62.85
48.5.
66.66
5
57.26
58.17
50.62
62.20
60.80
57.45
54,36
62.89
63,82
43. 82.82
63.07
66.61
62.12
63.86
52.68
62.10
58.76
57.25
54.12
62.11
62.37
50.24
62.61
69.11
Avg
59.09
49.12
62.00
62.26
48.81
68.02
Std
1.73
2.64
5.08
0.55
1.10
0.90
1.19
0.66
1.10
1.09
0.85
1.22
59.81
57.52
37.81
61. 86
58.07
$6.82
53.73
62.75
64.44
49, 80)
62.24
67.077
58.06
51.43
48.61
61.51
57.44
55,50
51.04
62.68
63.58
47.56
63.78
64.25
61.31
55.38
42.49
62.35
56.10
57.03
54.51
61.44
64.58
49.30
64,28
68.50
20
56.09
57.82
44.68
61.61
57.23
56.77
53.12
59.39
62.33
47.65
63.43
65.25
59.07
55.79
42.49
61.75
58.99
54.78
52.83
61.92
63.28
48.18
63.82
65.90
59.85
55.21
43.84
61.43
58.47
56.05
53,18
61.60
62.51
50.28
62.81
66.86
Avg
59.03
55.52
43.32
61.75
57.72
56.16
53.07
61.63
63.45
48.80
63.40
66.31
Std
2.29
0.33
0.89
1.22
1.16
1.50
56.51
56.67
37.69
58.62
57.20
$5.98
52.10
58.17
61.54
47.23
61.15
65.38
2
54.02
54.82
38.80
57.96
55.26
52.60
50.08
58, 12
61.64
44,56
60.65
64.27
56.53
52.12
65.36
4
55.76
56.15
39.68
58, 68
54.36
54,31
52.21
57.58
61.59
45.53
61.09
65.69
40
57.14
52.19
41.95
57.47
56.80
50,80
49.48
57.16
61.28
44.03
60.53
61.82
57.06
52.31
38.13
58.36
55.44
52.50
50.18
59.40
61.01
45.00
59.08
63.33
Avg Std
56.17
53.89
38.86
58.38
55.73
53.05
51.32
58.34
61.72
45.72
60.77
64.31
1.16
2.29
1.78
0.60
1.06
1.82
1.68
0.99
0.79
1.56
0.99
1.51
Table Full results of the label disturbance experiment on Sleep data. The best results are bold and we underline the second best results. The worst results are denoted in italics.
22

---

NLJL
1%
Exp
TCN2
MS-
Former
AS
Mini Rocket
Times
Patch
TST
SI-
CON
UNI
Sel-
SREA
Scale-
Conóm
65.86
51.41 .1
64.82
62.11
50.25
58.51
52.09
60.64
62.57
53.09
66.80
72.26
2
63,74
59.68
60.63
61.12
50.55
60.60
52.27
64.43
$1.65
57.57
66.85
73.21
56.03
63.92
56.22
56.51
54.99
70.52
Avg Std
61.88
56.71
60.62
62.39
50.99
58.45
53.19
60.53
60.50
55.21
67.64
72.00
5.17
4.60
4.21
144
1.03
2.19
1.75
3.96
8.01
2.25
1.42
1.36
Table Full results of the label disturbance experiment on SEEG data. The best results are in bold
and underline the second best results. The worst results are denoted in italics.
True Positive
False Positive
Boundary
False Negative
True Negative
(a) Con4m
(b) Scale-T
(c) Sel-CL
(c) MiniRocket
(e) MS-TCN2
Positive
False Positive
False Negative
Negative
Boundary
Con4m
MAL M
mM
(b) Scale-T
MAL
v
aM
1/4
Sel-CL
MA
1M
1/4
(d) MiniRocket
n
mm
1/4
mmp
MM
1/
MS-TCN2
True Positive
False Positive
False Negative
True Negative
(a) Con4m
(b) Scale-T
(c) Sel-CL
(d) MiniRocket
(e) MS-TCN2
4s
66
8s
10s
12s
14s
True Positive
False Positive
False Negative
True Negative
(a) Con4m
www.
Scale-T
(c) M
MiniRocket
www.
MS-TCN2
Figure 7: More cases for continuous time intervals SEEG testing set.
23

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In this paper, abstract, introduction and conclusion accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work discussed in the conclusion (Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification:
Theoretical analysis is in Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
SEEG is private dataset one may cannot reproduce the experiments on the SEEG dataset. However, we also use another three public datasets, and the code of our work is fully provided, which could help readers understand and reproduce our work.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
25

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: SEEG is private dataset may not be able to reproduce the experiments on the SEEG dataset. We provide detailed experimental setups and code to assist readers in reproducing the results publicly available datasets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
y /nips. /nips.cc//ppp http:/ccccccccguudeeeeeeeeeeeeee wwaicc//uudss//ee haabll/c/guideesodeeuuubiiis
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/ettails
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We give detailed description of our experiment setting in Section 4. and Appendix F
We also conduct hyperparameter analysis Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
The standard derivation is shown in the full experimental results in Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide information about CPU, GPU, etc. in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp:://eeurpp:: httpp///ppppppcccccccccctieeeeee
Answer: [Yes]
Justification:
All authors reviewed and conducted the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
27

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "eZgNFlyY3F", "reviewer": "Reviewer_gPgT", "summary": "The authors proposed a learning framework called $\\textit{Con}4\\textit{n}$ that leverages contextual prior of Multiple classes with Varying Duration (MVD) to enhance the discriminative power of consecutive time series segments while harmonizing inconsistent labels associate to these later. The authors stated and shown through extensive experiment that their framework enables to encode underlying temporal dependence between consecutive time series segments and mitigate the data annotation error that may be due to discrepancy of expertise.", "questions": "- What features do you use in positional encoding? Is it the order index of time series segments or timestamps? Please mention it in the main text;\n\n- Can the model work with multivariate time series data, i.e. using the observation of various features in a time segment to perform a classification task? If so, how will you encode the multivariate time series segments?\n\n- Do the authors think that their model will still work correctly when faced with seasonal time series data, given that the Gaussian Kernel used is not appropriate for capturing the periodic pattern of the data?  \n\n- Why did the authors choose the attention mechanism to merge $z_s^l$ and $z_g^l$? Although probably less efficient, have they tried simple techniques such as concatenation, Hadamard product or addition, which have the advantage of reducing spatial and temporal complexity?", "limitations": "Although the authors have discussed the limitations of their work, the space and time complexity of the model is an additional limitation. Furthermore, unless the reviewer has missed some points, it appears that the proposed model has only been evaluated on univariate time series data (please correct the reviewer if he is wrong). The reviewer suggests investigating these points in future work.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Originality: The authors propose a sophisticated architecture for encoding the temporal dependency between successive time series segments during a classification task. The originality of this work lies in the fact that the classification process is reinforced by a prior contextual information encoded by a Gaussian kernel. Their approach is more realistic than previous work, which assumes an independent and identical distribution of successive time series segments.\n\n- Quality: The article is well-written and structured. The experiment is consistent enough;\n\n- Clarity:  Although the document is fairly clear overall, there are still a number of areas for improvement (please refer to weaknesses and questions);\n\n- Significance: The proposed solution, once mature, will undoubtedly benefit many domains in which time series classification can be very challenging due to the complexity of data structure and divergence of experiences.", "weaknesses": "- Figure 2 is not intuitive enough to understand the overall architecture of the model;\n\n- $\\hat{p}$ and $\\tilde{p}$ are mentioned without explanation. Even though for $\\hat{p}$  it is quite obvious, it must be explained clearly (see paragraph $\\textbf{Neighbor Class Consistency Discrimination}$.\n\n- Space and Time complexity: Although efficient, the proposed model may require significant memory and computational time in large-scale applications.\n\n- The model may fail when confronted with seasonal time series data. This is because the Gaussian kernel used is not suitable for capturing the periodic pattern of the data;\n\n- Execution time is not included in the results. This is an important measure to report, as it plays a crucial role in sensitive areas such as medicine, where accuracy and prediction time are just as important.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for recognizing our work.\n\nW1. We will integrate Figures 2 and 3 into a single illustration to improve clarity.\n\n---\n\nW2. We will explicitly clarify the meanings of `p^` and `p~` in the main text: `p^` represents the model's independent prediction for a sample, while `p~` denotes the context-aware prediction that incorporates the results from neighboring samples.\n\n---\n\nW3. The time complexity of Con4m can be divided into two main components. The primary computational cost is on the same order as vanilla Transformer. Assuming the number of consecutive input time segments is $L$, the hidden representation dimension is $D$, the number of classes in the classification task is $C$, and the local iteration count of the function fitting module is $I$.\n\n- Con-Transformer: The time complexity of vanilla self-attention is $\\mathcal{O}(LD^2+L^2D)$, and the time complexity of the Gaussian kernel branch is $\\mathcal{O}(LD^2+L^2)$.\n- Coherent class prediction: The time complexity of Neighbor Class Consistency Discrimination is $\\mathcal{O}(LDC)$, and the time complexity of the Tanh function fitting is $\\mathcal{O}(ICL)$.\n\nTherefore, the computational cost and bottlenecks of Con4m are similar to vanilla Transformer.\n\n---\n\nW4&Q3. Thank you for the insightful question. Con4m is primarily designed for classification tasks, focusing on extracting structures and patterns within time segments. When dealing with datasets that have periodic patterns, we believe Con4m can still effectively capture these periodicities, provided that a complete cycle is included within a single classification instance. In this context, the Gaussian kernel acts as a one-dimensional Gaussian smoothing filter. We believe that classification tasks are typically less sensitive to periodic patterns compared to forecasting tasks. Furthermore, we have included a human activity recognition dataset HHAR, where sensor data naturally exhibit periodic changes as people walk or run. Con4m performs well on the six-class HHAR dataset, demonstrating its ability to capture periodic patterns.\n\n---\n\nW5. The execution time of Con4m on SEEG data is as follows: (1) Offline training (124,000 training samples + 62,000 validation samples): approximately 30s/epoch; (2) Batch inference (batch size of 64 with 62,000 test samples): approximately 14s. These times fully meet the practical requirements for inference.\n\n---\n\nQ1. We employ learnable absolute positional encoding. The absolute positional encoding is equivalent to the order index of time series segments, as the fundamental unit of our model is a time segment. This clarification will be addressed in the primary text of the paper.\n\n---\n\nQ2. Con4m is capable of handling multivariate time series data. As shown in Table 1, the fNIRS and Sleep datasets have 8 and 2 features, respectively. We employ a multi-channel CNN to jointly extract information from these features (see Figure 2), as the primary focus of this paper is not on multi-channel modeling. However, Con4m can be seamlessly integrated with existing multi-channel modeling approaches by using their output representations as inputs to Con4m.\n\n---\n\nQ4. Summation is the most direct and straightforward approach to achieving smoother representations, as concatenation and the Hadamard product do not explicitly provide this effect. Additionally, we enable the model to adaptively learn the necessary degree of smoothness, as sample representations in transition regions should not be continuous with those from different classes.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pUwQPfhj1F", "reviewer": "Reviewer_7tXB", "summary": "The paper proposes Con4m, a novel framework for time-series classification and temporal action segmentation that leverages contextual information. The framework is designed to improve the prediction accuracy by incorporating context from surrounding data segments.\nThe proposed method combines time-series classification (TSC) and temporal action segmentation (TAS) in a unified framework, making it versatile for different types of sequential data. It tries to address the issue of common time series classification methods which overlook the dependencies between consecutive segments.\nThe method demonstrates robustness to label disturbances, showing significant improvements over baseline models across different datasets (fNIRS, Sleep, and SEEG).", "questions": "See the initial two comments (no 1 and no 2) in weaknesses. In addition:\n\n3.\tIn Label Disturbance Experiments: It is not clear how the authors evaluated the models. Is it based on the new disturbed boundaries or the original boundaries? What is the evaluation setup for the other baselines as they are not considering the labelled boundaries in their calculations (I assume they should be boundary-agnostic).\n4.\tWhile section 4.3 shows an interesting set of experiments, I believe it only measures the degree of agreement between the baseline models and Con4m about the modified labels. Unless authors can ensure all the labels modified by Con4m are correct and accurate", "limitations": "The following limitations are not considered/discussed:\n\n-\tAlthough I appreciate the novelty of the method, the final improvement is subtle compared to other baselines given they have not claimed any ability in segment detection. Also, the evaluations are limited to 3 datasets.\n-\tAs the authors mentioned in the conclusion the model heavily relies on the availability of labels even for the segmentation part. While most segmentation models are unsupervised, I suggest including comparison by combining other baselines with unsupervised segmentation methods.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tThe paper proposes context-aware boundary detection and classification for long multi-class time series.  The topic is very interesting and focuses on a real challenge in processing time-series data. \n- Con4m integrates contextual information to improve the coherence and accuracy of predictions. This approach helps in better recognizing boundaries and transitions in time-series data.\n-\tThe paper is well-written and presented with several example illustrations.\n-\tThe model is Compared against different time-series classification baselines. The evaluation shows great performance across Sleep and SEEG datasets and demonstrates robustness to label disturbances", "weaknesses": "1.\tWhat if combining other models with a change-detection model to detect segment boundaries first and then classify the windows within the segments? That would be great to see the impact of Con4m in comparison with other baselines in this setup. \n2.\tSince Con4m is capable of doing segmentation and classification together, it will be necessary to evaluate and compare the impact of each with other SOTA.\nMinor change:\n-\tFigure 4(b) has been referred to before Figure 4(a). Please fix that\n3. The experiments are limited, and somewhat unfair. The chosen baselines do not claim ability in segment detection. There are several works that have done segment or change point detection, and they need to be included in baselines. Also, evaluations are limited to three datasets.\n-\tMinor suggestion: It would be nice to visualize the whole framework to show each module.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "W1&2. \n\n1. Our setup differs significantly from segmentation models (FLOSS[1], ESPRESSO[2], ClaSP[3]) in that they are able to identify change points but are unable to determine the specific classes before and after these points, particularly in multi-class tasks.\n2. Our public datasets are multivariate, which presents a challenge for the performance of most segmentation models (ClaSP[3]), which are designed to handle univariate time series.\n3. Our use case may be impractical and complex due to the necessity of setting/learning  parameters, including the number of segments and the length of subsequences, in segmentation models (FLOSS[1], ESPRESSO[2]).\n\nNevertheless, we conduct an exploratory analysis of ClaSP on the SEEG dataset (the only univariant dataset) and present the results with two metrics for reference. These results are solely intended for illustration purposes, as unsupervised segmentation models are not well-suited to our scenario and setup.\n\n[1] Gharghabi, Shaghayegh, et al. \"Domain agnostic online semantic segmentation for multi-dimensional time series.\" (2019).\n\n[2] Deldari, Shohreh, et al. \"Espresso: Entropy and shape aware time-series segmentation for processing heterogeneous sensor data.\" (2020).\n\n[3] Ermshaus, Arik, Patrick Schäfer, and Ulf Leser. \"ClaSP: parameter-free time series segmentation.\" (2023).\n\n---\n\nW3. Thank you for your valuable questions and suggestions. We will reorder Figures 4(a) and 4(b) and attempt to combine Figures 2 and 3 into a single illustration. However, we would like to emphasize that Con4m was not designed specifically for segmentation. Our modeling is consistently based on Problem Definition 3.1, which involves making independent predictions for each segment and then integrating them into coherent predictions. Our work aims to highlight a point often overlooked by current mainstream supervised TSC approaches: the temporal dependency and coherence of contextually classified samples. Experiment 4.3 further illustrates the contributions of our work.\n\nOn the other hand, all the baselines related to temporal action segmentation (TAS) are equipped with segmentation capabilities, as they are specifically designed for segmentation tasks. Since Con4m is a supervised learning model, we selected representative supervised learning models in TAS for comparison. Therefore, the selection and comparison of baselines are fair. Nevertheless, we have considered your recommendation to incorporate unsupervised segmentation models for comparison in W1&2.\n\n---\n\nQ3. Our experimental setup is consistently aligned with Problem Definition 3.1 (lines 136-140). All models are trained on the disturbed data and tested on the original noise-free data. We introduce disturbances at the timestamp level of the original data (lines 236-243), then sample time intervals based on these disturbances and segment them. All models treat each time segment as an instance for prediction and evaluation, which aligns with the TSC task setup. The evaluation setup itself is independent of whether a model can detect boundaries. Con4m does not explicitly perform boundary detection, whereas TAS models are specifically designed for boundary segmentation.\n\n---\n\nQ4. Section 4.3 effectively demonstrates Con4m’s prediction consistency. Moreover, Fig.5 and Fig.8 use color coding to visually highlight the alignment between the model’s predictions and the actual labels. It is evident that, unlike Con4m, other models fail to provide predictions that are both coherent and accurate. In practical applications, Con4m offers substantial improvements in usability, significantly outperforming other models in overall classification metrics. Dispersed and inconsistent predictions can hinder doctors from quickly identifying seizure regions without examining large amounts of raw data.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HRI3Vr8YLg", "reviewer": "Reviewer_uhXL", "summary": "This paper addresses the challenge of segmented time series classification (TSC) for Multiple classes with Varying Duration (MVD) data. The authors propose Con4m, a consistency learning framework that leverages contextual information with a focus on inconsistent boundary labels. The method incorporates continuous contextual representation encoding, context-aware coherent class prediction, and a label consistency training framework. Experiments on 3 datasets demonstrate Con4m's superior performance compared to state-of-the-art baselines, especially in handling inconsistent labels.", "questions": "1. In the label consistency framework, why is N_l = 5? Was this value determined through hyperparameter search? How sensitive is the method to this choice?\n\n2. Why was fNIRS not used in the ablation studies? Is there a specific reason for excluding this dataset from the detailed analysis?\n\n3. How does the computational complexity of Con4m compare to the baseline methods? Is there a significant increase in training time or memory requirements?\n\n4. Have you explored the potential of applying Con4m to semi-supervised or few-shot learning scenarios, where only a portion of the data has reliable labels?\n\n5.  The paper mentions that Con4m modifies approximately 10% of the training labels in the SEEG data. How does this percentage vary across different datasets or disturbance ratios? Is there a way to estimate the optimal percentage of labels that should be modified?\n\n6.  Could you provide more insights into the choice of the hyperbolic tangent function for prediction behavior constraint? Have you experimented with other monotonic functions, and if so, how do they compare?", "limitations": "See the questions above. I would especially emphasize the narrow benchmarks and tasks. Learning across segments is a broader problem within the timeseries domain.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Originality: The paper introduces a new approach to segmented TSC for MVD data, addressing challenges often overlooked in existing TSC models. The Con4m framework creatively combines ideas from curriculum learning, noisy label learning, and temporal action segmentation.\n\n- Quality: The theoretical analysis in Section 2 and throughout provides a solid foundation for the proposed method. The experiments are comprehensive, including comparisons with various baselines, label disturbance experiments, and ablation studies.\n\n- Clarity: The paper is well-structured and clearly written. The figures, especially Figure 2 and Figure 3, effectively illustrate the proposed method's architecture and workflow.\n\n- Significance: The work has potential implications for various domains dealing with MVD data, such as healthcare and activity recognition. The label harmonization approach could be particularly valuable in scenarios where obtaining consistent labels is challenging or costly.", "weaknesses": "- Limited exploration of hyperparameters: The paper doesn't thoroughly discuss the sensitivity of the method to key hyperparameters, such as N_l = 5 in the label consistency framework or E_g = 5 for curriculum learning.\n\n- Scalability concerns: The paper doesn't address how the method would scale to larger datasets or longer time series. This could be a limitation for practical applications with high-dimensional or long-duration data.\n\n- Comparison with semi-supervised methods: Given that the method deals with inconsistent labels, it might be beneficial to compare it with semi-supervised learning approaches that are designed to handle partially labeled or noisy data.\n\n- Generalization to other domains: While the method is tested on three datasets, they are all from the healthcare domain (all 3 of them focusing on brain signal analysis). It would be valuable to see how well the approach generalizes to other domains with MVD data, such as activity recognition or financial time series. For example, initiatives like WOODS (https://arxiv.org/abs/2203.09978/) offer diverse timeseries data for evaluations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "W1&Q1. We consider $N_l$ and $E_g$ as auxiliary hyperparameters that work together rather than core hyperparameters of the model. When selecting these values, we focus on ensuring that the model’s loss approaches convergence for the newly added level within $E_g$ epochs. In contrast, $E_\\eta$ has a more significant impact on the model’s performance. See more details in global rebuttal G2.\n\n---\n\nW2. From an architectural perspective, Con4m is built on the Transformer framework, allowing for scalability by adjusting the hidden layer dimensions or stacking layers. From a data perspective, time series data differs from NLP sequences as it consists of continuous numerical values rather than discrete tokens. By adjusting the duration of each time segment (patch), we can control the sequence length L. Consequently, Con4m can be scaled to handle longer time series by appropriately tuning the patch length, the number of CNN layers, and the size of the convolutional kernels.\n\n---\n\nW3&Q4. Our method differs from semi-supervised TSC methods that assume reliable labels on partially labeled datasets. In contrast, we begin with initial labels for the entire dataset, but their reliability remains unclear. Our model is specifically designed to evaluate label reliability as part of its learning process, addressing inconsistencies not typically covered by semi-supervised methods. Therefore, we cannot consider semi-supervised or few-shot methods as a baseline or apply them to our task due to these differences in task settings.\n\n---\n\nW4. Thank you for your valuable suggestion. Based on the link you provided for WOODS, we identified that the Human Activity Recognition (HHAR) dataset aligns with our experimental setup to some extent. Therefore, we incorporated the HHAR dataset in our experiments. Please refer to the global rebuttal G1 for more details.\n\n---\n\nQ2. Table 2 demonstrates that fNIRS has relatively clear boundaries, as TAS models perform well on this dataset even without considering label consistency. Therefore, to more effectively highlight Con4m’s advantages in handling inconsistent labels, we decided not to include the fNIRS dataset in the ablation studies.\n\n---\n\nQ3. We believe that due to the significant differences in architecture and domain among various baselines, directly comparing computational complexity would be unfair. For example, models based on the Transformer architecture have a significantly higher time complexity than those based on CNNs. Therefore, we only compare the time complexity of Con4m with that of a standard Transformer.\n\nThe time complexity of Con4m can be divided into two main parts. The primary computational cost is on the same order as vanilla Transformer. Assuming the number of consecutive input time segments is $L$, the hidden representation dimension is $D$, the number of classes in the classification task is $C$, and the local iteration count of the function fitting module is $I$.\n\n- Con-Transformer: The time complexity of vanilla self-attention is $\\mathcal{O}(LD^2+L^2D)$, and the time complexity of the Gaussian kernel branch is $\\mathcal{O}(LD^2+L^2)$.\n- Coherent class prediction: The time complexity of Neighbor Class Consistency Discrimination is $\\mathcal{O}(LDC)$, and the time complexity of the Tanh function fitting is $\\mathcal{O}(ICL)$.\n\nTherefore, the computational cost and bottlenecks of Con4m are similar to vanilla Transformer.\n\n---\n\nQ5. Since label inconsistency is difficult to precisely define, it is challenging to determine an optimal percentage of label modifications. However, during training, one of our criteria for stopping is when the model no longer changes additional labels. You can find this implementation in our code. Generally, datasets with higher noise levels tend to have a higher percentage of label changes. For example, the fNIRS-0 dataset has a change rate of 3%, while SEEG has a change rate of 13%. Similarly, for the Sleep dataset with disturbance ratios of 0%, 20%, and 40%, the change rates are 10%, 12%, and 18%, respectively.\n\n---\n\nQ6. The Tanh function is particularly well-suited to encapsulate the four behaviors depicted in Figure 3. As a widely used activation function, it is easier to understand and manipulate. In contrast, functions like arctangent and arcsine are more complex and harder to control. Appendix B demonstrates Tanh’s ease of optimization and its precise fitting capabilities.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0qgbL2Q74U", "reviewer": "Reviewer_Gi9s", "summary": "This paper proposes a time-series classification method which exploits temporal consistency (implemented by contextual information). Also, the proposed method can handle noisy class boundaries.", "questions": "See W1~W9.", "limitations": "In my opinion, the generalizability of the proposed method is not clearly verified. In this regard, the authors did not properly discuss the limitations of the proposed method. Please see my comments to further improve the manuscript.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "S1. This paper presents a novel problem formulation, time-series classification with noisy class boundaries.\n\nS2. The proposed method takes full advantage of temporal consistency, which is an inherent property of time series.\n\nS3. The authors created their own proprietary dataset, SEEG, for extensive experiments.", "weaknesses": "W1. First of all, the writing in Introduction and Figure 1 are very misleading. \"**Inconsistent** boundary labels\" (with \"different annotators\") and Figure 1(a) indicate that multiple inconsistent annotations coexist for the same time-series instance. However, after reading a few pages, I found a discrepancy between my understanding and the presentation in this paper. Subsequently, I realized that the authors were referring to scenarios in which the class boundaries could be inaccurate. That is, there is **one** possibly erroneous boundary label for each transition. This presentation issue really affected my negative impression on this paper.\n\nW2. I do not agree that existing studies largely focus on the assumption of i.i.d. It is very easy to find the existing work that considers temporal consistency. For example, please see https://cs.stanford.edu/people/jure/pubs/ticc-kdd17.pdf. More recently, please see https://openreview.net/pdf?id=gjNcH0hj0LM. That is, many existing methods consider that consecutive timestamps tend to have the same class label. \n\nW3. Con-Attention is the same as Anomaly-Attention. Please see Equation (2) in https://arxiv.org/pdf/2110.02642. Therefore, along with W2, the technical novelty for the first contribution is not very significant.\n\nW4. Section 2 is not tightly connected to subsequent sections. In my opinion, Theorem 2 is too obvious, and it does not need to be formulated as a theorem. In fact, it does not look like a formal theorem. \n\nW5. The Tanh function fitting seems to be needed for each segment boundary and is not straightforward (involved with many parameters). Thus, the training efficiency could be an issue. It would be good to report the training cost of the proposed method.\n\nW6. The authors assume that the center of a class interval has the highest confidence. This assumption could be true or false depending on the domain. As in the motivating example, if the annotators have a difficulty in identifying the end times of seizure waves, the highest confidence will appear earlier than the center.\n\nW7. The authors made several strong assumptions or heuristics. For example, it is assumed that consecutive segments almost span at most two classes; a class interval is divided into five levels; a training cap is given as five epochs. Thus, this paper lacks rigorousness and generalizability. \n\nW8. It is not trivial to set up the hyperparameter value, especially, $E_\\eta$. The default value is not guaranteed to work well for other datasets. A detailed guideline is necessary.\n\nW9. The degree of noisy labels in each dataset should be analyzed in detail. First, it is not clear why there is no noise in fNIRS and Sleep, while there is real noise in SEEG. Is the annotation strategy different in these datasets? How do you know the ground-truth labels in SEEG? Second, can you measure the degree of real noise just like the noise rate in the image classification domain? Can you approximate the corresponding value of $r$ for this degree of noisy labels in SEEG? Overall, without detailed information on noisy labels, it would be difficult to analyze the effect of noisy label learning.\n\n---\n\nI have adjusted my rating to 5 during the discussion period.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "W1. We apologize for any confusion caused. Fig. 1 is intended to emphasize that, despite the similarity in seizure patterns, different annotators can still provide inconsistent labels for the same type of seizure and across different recordings from the same patient.  For simplicity, we only illustrated a segment of brain signals to represent a seizure pattern type, rather than a single instance. This will be explicitly stated in the revised version. Additionally, inconsistent labels do not equate to erroneous labels, as there is a lack of unified quantitative metrics to define boundaries between different classes.\n\n---\n\nW2. Our focus is on supervised TSC, which assigns each segment a single label, as stated in the problem definition in Sec. 3. We acknowledge that the studies you referenced consider temporal consistency. However, the first paper focuses on clustering, which is fundamentally distinct because it involves setting/learning the number of clusters within a sequence. The second paper concentrates on active learning. While these studies leverage temporal consistency, they do not align directly with the supervised TSC models we discuss. And our work explicitly integrates temporal consistency into the modeling of supervised TSC from both the data and label perspectives.\n\n---\n\nW3. The only similarity between the two is the parallel computation of $G^l$. Anomaly Transformer uses the similarity between inter-sequence and intra-sequence distributions to measure anomaly scores, whereas our approach aims to achieve smoother representations through fusion. We have cited this work in the main text (line 143) and have not referred to its code.\n\n---\n\nW4. While Theorem 2 may seem intuitively obvious, we provide a more formal proof from an information-theoretic perspective (lines 109-121). Based on this theorem, we designed our model from both the data and label perspectives, as detailed in Sec. 3.1 (lines 142-149) and 3.2 (lines 170-173, 182-187).\n\n---\n\nW5. Assuming the number of consecutive input time segments is $L$, the hidden representation dimension is $D$, the number of classes is $C$, and the internal iteration steps of the Tanh function fitting are $I$, the time complexity of the function fitting module is $\\mathcal{O}(ICL)$. The complexity of the encoder’s final prediction layer is $\\mathcal{O}(DCL)$. In practice, keeping $I < D$ helps manage the time complexity of the function fitting module. In our experiments, we set $I=100$ and $D=128$, with average training times reported per epoch.\n\nThe additional computational cost is higher for the Sleep dataset due to its classification as a five-class problem. SEEG, like fNIRS, is a binary classification problem. However, the additional overhead from function fitting is relatively smaller due to the larger input sample window length and the total number of training samples. As a result, the function fitting module has a higher computational overhead on datasets with more classes, which is a limitation of our model design. However, this overhead becomes less significant with larger datasets.\n\n|             | fNIRS        | Sleep        | SEEG         |\n| ----------- | ------------ | ------------ | ------------ |\n| Con4m       | 11.45s       | 37.73s       | 24.75s       |\n| w/o fitting | 1.71s        | 3.92s        | 6.78s        |\n|             | $\\times$6.70 | $\\times$9.63 | $\\times$3.65 |\n\n---\n\nW6. Our assumption is that annotators “tend to reach an agreement on the most significant core part of each class,” which is common in temporal action segmentation. We apologize for any confusion caused by Fig. 1; we used real data, so the illustration does not depict an entire seizure episode due to its length. In practice, doctors often disagree on both the exact start and end times of seizures. While domain knowledge can aid in sample selection, our design remains general when such prior knowledge is unavailable.\n\n---\n\nW7. We acknowledge in lines 334-335 that modeling transitions for a single class is a limitation. However, based on domain knowledge and statistical information from the datasets, we can select an appropriate window length that meets this condition. For the auxiliary parameters $N^l$ and $E_g$, we do not consider them core hyperparameters of the model. We ensure that the model’s loss approaches convergence for the new level data within $E_g$ epochs. See more details in global rebuttal G2.\n\n---\n\nW8. Although we tuned $E_\\eta$ on the SEEG dataset, we used the same hyperparameter across all 4 datasets, demonstrating Con4m’s robustness. According to Appendix C, $E_\\eta$ should not be set too low. For practical use, start with a value of 10 and adjust in intervals of 10 until the validation set performance peaks or slightly declines.\n\n---\n\nW9. The fNIRS and Sleep datasets are publicly available with curated labels, so they are considered noise-free. In contrast, SEEG data is derived from real clinical datasets and annotated by multiple experts, resulting in naturally inconsistent labels. We employ a voting mechanism (Appendix G, lines 746-749) to minimize discrepancies in test labels. However, due to the lack of a unified standard, inconsistencies in SEEG do not imply “incorrect” labels but rather reflect differences in experiences. Unlike video segmentation, where single-frame semantics are clearer, SEEG annotations can vary significantly.\n\nOur goal is to harmonize, not correct, these labels using methods inspired by noisy label learning. Therefore, we cannot directly calculate an r value for SEEG but instead use indirect label substitution experiments (Sec. 4.3) to validate Con4m’s effectiveness in label harmonization. Comparisons with other noisy label learning baselines and ablation studies further demonstrate Con4m’s ability to handle inconsistent labels.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
G1 (Results for a new dataset).

Based on the suggestion from reviewer uhXL, we conducted a search of the WOODS dataset and found that the Human Activity Recognition (HHAR) subset aligns well with our scenario and setup. Consequently, we include the HHAR dataset in our experiments. Due to time constraints, we only report the results of the best-performing model from each category of baselines, and we plan to include the remaining results in the revised version.

We follow the preprocessing steps outlined in WOODS (https://arxiv.org/pdf/2203.09978), dividing the HHAR dataset according to the device. To ensure balanced samples, we combined data from the Galaxy S3 Mini, LG watch, and Gear watch into a single group. A 6-fold cross-validation experiment is conducted in accordance with the Sleep dataset's configuration. As Table 1 in the rebuttal pdf shows, Con4m still achieves competetive or superior performance in HHAR dataset. Due to the distribution discrepancies among devices, there are significant variations in experimental results across different groups.

---

G2 (Results for hyperparameter analysis).

We consider $N_l$ and $E_g$ as auxiliary hyperparameters that work in tandem rather than core hyperparameters of the model. When selecting these values, we empirically observe that the model’s loss converges for the newly added levels within $E_g$ epochs.

As shown in Figure 1 in the rebuttal pdf, smaller values for either $N_l$ or $E_g$ can hinder the model’s ability to fit data at newly introduced levels, leading to a decline in performance. However, when $N_l$ and $E_g$ are set within the range of 4 to 7, Con4m demonstrates stable performance, indicating that the model is not particularly sensitive to these hyperparameters. In contrast, the parameter $E_\eta$ has a more significant impact on the model’s performance, and careful tuning of this hyperparameter is more crucial.

Meta-review: This paper should be accepted as a poster because it introduces a novel framework, Con4m, for time-series classification and temporal action segmentation that leverages contextual information to enhance prediction accuracy. The proposed method creatively combines time-series classification and segmentation into a unified framework, which is valuable for dealing with complex sequential data with inconsistent labels. The originality of the work lies in using a Gaussian kernel to encode contextual prior information, which is more realistic than the assumption of independent and identically distributed time series segments commonly made by other models. The authors also conducted extensive experiments on various datasets, demonstrating the framework's effectiveness in improving coherence and robustness against label disturbances. While the results are promising, they also highlight several areas for future improvement.

However, the paper does have some limitations that should be addressed. Some parts of the paper lack clarity, such as the need for clearer explanations of specific terms and the overall architecture. Moreover, there are concerns regarding the model's scalability and efficiency, especially in large-scale applications where computational and memory resources could be a constraint. The evaluation setup could also be more comprehensive, including comparisons with models that specifically address change-point detection and segmentation, which would provide a more robust assessment of Con4m's capabilities. Despite these limitations, the paper presents a sound approach with a solid theoretical foundation and holds potential for future research and application, making it a suitable candidate for a poster presentation.
Decision: Accept (poster)