Paper: Con4m: Unleashing the Power of Consistency and Context in Classification for Blurred-Segmented Time Series (ICLR-2024)

Reviewer comment (Experiments):
Model degradation by label inconsistency in transition is not validated. The number of timestamp where transition occurs is very small comparing to the length of a whole time series. Does it really harm the model performance significantly? Plus, when annotating SleepEDF, multiple doctors are already recruited to make an agreement in their annotations, which can reduce inconsistency in state transition regions.

Author response:
Solving the state transition problem holds significant practical significance.

- **Harmonizing inconsistent boundary labels is an important means of reducing annotation costs.** As you mentioned, SleepEDF, being a widely used publicly available dataset, requires highly accurate labels. Achieving this level of precision necessitates the labor-intensive process of expert annotation, which is time-consuming. In the era of large-scale models, data and labels have become the core fuel for model performance. Unlike the field of NLP, medical annotations are precious and scarce. To reduce annotation costs and enable multiple institutions to collaborate on building the foundational model, harmonizing the annotations of different doctors is an essential step.
- **Inconsistent boundary labels can also lead to a decrease in model performance.** Samples near the boundaries also appear throughout the entire time series. For instance, in SEEG data, there exists waveform overlap between the seizure waves and the interictal waves. As demonstrated in the case study presented in the paper, the repeated occurrence of samples near the boundaries confuses the model during training. Learning label consistency with contextual information can reduce the interference caused by inconsistent labels, allowing the model to converge more consistently on training samples.
- **Accurately identifying boundaries is necessary for certain application scenarios.** For example, in the seizure detection task based on SEEG data, doctors need to determine the lesion areas by considering the temporal order of seizure waves generated by different channels (corresponding to different brain tissues). This is because the seizure waves will propagate from the lesion areas to surrounding brain tissues. The spread occurs rapidly, making it crucial to finely identify the boundaries of seizure onsets. Inconsistent labels prevent the model from providing precise and consistent predictions at the boundaries.

Reviewer comment (Novelty):
Methods seem to be a heuristic without enough justification and not novel. In neighbor class consistency discrimination, there could be so many ways to achieve it but there is no explanation on the design choice the authors made. Also, the theory does not support the reason why $\ell_2$ loss should be used.

Author response:
- The core insight of our theory lies in the selection of valuable contextual information. While people often focus solely on contextual information at the data level, **our theoretical framework allows for the incorporation of contextual information at the label level into the model's design**. In Section 3.2, lines 134-144, we mention that the theoretical variable $x_{\mathbb{A}\_t}$ can be replaced by $y_{\mathbb{A}\_t}$, benefiting from the analytical framework of information theory.
- Based on theoretical analysis, the design in Section 3.2 revolves around incorporating $y_{\mathbb{A}_t}$ into the model's predictions. Introducing the neighbor class consistency discrimination task is the most intuitive approach. We explicitly discriminate whether contextual samples belong to the same category through supervised learning. Based on the output weights of the discriminator, **we directly incorporate the label information of contextual samples into the final predictions through a linear weighted sum**. This design is straightforward, concise, and effective.

Reviewer comment (Experiments):
Experiment setting is not convincing. The labels are disturbed synthetically and one of three datasets is a private dataset, which cannot be reproducible.

Author response:
- As you mentioned, publicly available datasets like SleepEDF require highly accurate labels. The absence of BST datasets is due to previous works ignoring the issue of label inconsistency. It is precisely because of this absence that we propose a novel label disturbance method for time series data (lines 219-226). Our experiments (lines 266-272) demonstrate that the label disturbance method we propose is more essential and challenging for time series data. **We have also included the codes in the attachment on how to disturb the publicly available datasets, which can be used by others.**
- To validate the model's performance on more realistic datasets with inconsistent labels, we conducted experiments on SEEG. Apart from the data of the subjects in the test group, the data of other subjects were not manually harmonized (lines 239-242). **These experimental results can more realistically reflect the impact of label inconsistency on the model.** Due to strict ethical review, SEEG is highly sensitive medical data that cannot be publicly released at the moment. However, we are actively working towards collaborations with hospitals and subjects, hoping to share this data in a safer and restricted manner.

Reviewer comment (Writing):
What is the dimension of $x_t$? Is it different from $x_1,\ldots,x_L$? Is $x_t \in \mathbb{R}^{L \times d}$ where $d$ is the number of feature?

Author response:
- In the expressions related to mutual information and probability, $x_t$ represents the random variable of the input raw data. We have adopted the `\displaystyle \r` command as suggested in the ICLR template.
- In Figure 1, $x \in \mathbb{R}^{L \times W \times d}=\{ x_1, \dots, x_L \}$, where $L$ represents the number of consecutive time segments, $W$ represents the window size (number of time points) in each segment, and $d$ represents the dimensionality of the input features.
- All of $x_1,\dots,x_L$ are realizations of $x_t$.

Reviewer comment (Theory):
The function fitting incurs more computations in training loop. Can you elaborate on computational complexity?

Author response:
- Function fitting does introduce additional computational overhead. However, **since we are fitting a Tanh function and the gradients do not backpropagate to the parameters of the encoder, such overhead is acceptable within the entire model**.
- Let's assume the number of consecutive time segments in the input is $L$, the hidden representation dimension is $D$, and the number of classification categories is $C$. The Tanh function fitting involves $I$ internal iterative steps. The function fitting module fits the predictions for each category separately, where the prediction for each category in each time segment is a constant. Therefore, the time complexity of the function fitting module is $\mathcal{O}(ICL)$. If we only consider the final prediction linear layer in the encoder, which maps the hidden representation to the prediction logits, its complexity is $\mathcal{O}(DCL)$.
- Hence, we suggest controlling $I<D$ in practical usage to effectively manage the time complexity of the function fitting module. In our experiments, we set the default parameters to $I=100$ and $D=128$.

Reviewer comment (Reproducibility):
At which layer $\ell_1$ and $\ell_2$ is applied?

Author response:
As depicted on the left side of Figure 2, both $\ell_1$ and $\ell_2$ jointly guide the learning process of the entire encoder.

Reviewer comment (Writing):
The paper is not written well and is a bit difficult to follow. To begin with, the term "blurred segmented time series" is not concretely defined throughout the manuscript. To the best of my knowledge, this term is not ubiquitous in the ML community. Further, the introduction does not clearly define the problem formulation. In particular, it is not clear whether the end result is to classify the individual segments or classify an entire time series which comprises multiple segments. The problem formulation is not mathematically defined even in subsequent sections, which keeps a reader busy guessing. Further, several terms like samples vs segments vs timestamps, and state vs labels are used confusingly at several places, which makes it super difficult to understand the exact problem formulation. It is also not clear whether a segment is of fixed length or varying length.

Author response:
Apologies for any confusion caused. Allow us to provide the following clarifications:

- Blurred-Segmented Time Series (BST) refers to a type of time series data that exhibits distinct characteristics different from traditional Time Series Classification (TSC) data. In practical applications. BST data often arises, and it is characterized by two main aspects—Blurred Transitions and Prolonged State Durations. We establish this definition in lines 23-29 within the main context.

- We divide a time interval into several consecutive segments and mainly focus on classification for each individual segment. We appreciate your feedback and we've incorporated a formal problem definition at the beginning of Section 3 as follows:

  >**Definition 1.** Given a time interval comprising of $T$ consecutive time points, denoted as $s=\\{ s_1,s_2,\dots,s_T \\}$, a sliding window length of $w$ and slide length $r$ is employed for segmentation. The time interval is partitioned into $L$ time segments, represented as $x=\\{ x_i=\\{ s_{(i-1) \times r + 1},\dots,s_{(i-1) \times r + w} \\} | i=1,\dots,L \\}$. The model is tasked with predicting labels for each time segment $x_i$.

- In our paper, we aim to employ 'segment' and 'state' when referring to specific scenarios, and in the context of machine learning, we prefer the terms 'sample' and 'label'. According to the problem definition above, our model focuses on predicting labels for each time segment. Therefore, 'sample' is equivalent to 'segment'. Also, the terms 'state' and 'label' are synonymous in our context, with each state having a corresponding label. Thank you for your recommendation and we will standardize and streamline the use of these terms in our paper.

- According to the problem definition, each time segment maintains a fixed length in the same dataset, as indicated in Table 1.

Reviewer comment (Experiments):
The motivation regarding too much noise in the labels in the segments due to label inconsistencies on boundary segments is also not super convincing. For instance, why can't one simply get rid of such boundary segments and train the model only on cleaner samples?

Author response:
- The nature of BST data inherently lacks a clear definition of boundaries. As described in lines 30-37, due to the absence of standardized quantification criteria, different doctors exhibit experiential variations in annotating boundaries. Moreover, this variation itself is ambiguous and lacks a precise definition. Therefore, the removal of boundary samples is not feasible in practice.
- Employing a fixed length to remove boundary data is a crude and uncontrollable approach. Likewise, eliminating uncertain boundary samples through expert consensus would be prohibitively costly and unsustainable.
- Some scenarios require precise boundary labels. For instance, in the seizure detection task based on SEEG data, doctors need to consider the temporal sequence of seizure occurrences in different brain regions to devise subsequent surgical strategies. If the model does not account for boundary labels during the training phase, it will struggle to accurately identify the class of samples at the boundaries during the inference phase.

Reviewer comment (Writing):
The description of the proposed approach is also quite difficult to follow. Several key notations are not well defined (e.g., what are V_s and V_t), and I had to read the papers in related work (e.g., Xu et al. 2022) in quite detail, from where the ideas are borrowed. Even then, certain components of the approach, such as neighbor class consistency discrimination, are still not clear to me.

Author response:
- As shown in the leftmost part of Figure 1, we implement the two-branch Con-Attention. $Q$, $K$ and $V$ vectors represent the query, key and value of the self-attention mechanism respectively. To distinguish between different computational branches, we use $s/S$ to represent the branch based on Gaussian prior, and $t/T$ to represent the branch based on vanilla self-attention. $\sigma$ is the learnable scale parameter of the Gaussian kernel function. $T^l$ and $S^l$ are the aggregation coefficients of the two branches in the $l$-th layer.
- For the neighbor class consistency discrimination part, our framework allows for the incorporation of contextual information at the label level into the model's design. According to Theorem 1, we aim to identify a set of contextual samples that maximizes the model's predictive capability at the label level. Since directly optimizing the aggregation at the label level is challenging, we adopt the approach of aggregating predictions of samples belonging to the same class to enhance the model's predictive capability. This approach is inspired by the observation that for graph neural networks based on the homophily assumption, aggregating neighbor information belonging to the same class can improve predictive performance [1,2]. Therefore, we explicitly train a discriminator to determine whether two samples belong to the same class. The model selects a contextual sample set based on the discriminator's output probabilities and weights their predictions based on the probability of being classified as the same class.

[1] McPherson, Miller, Lynn Smith-Lovin, and James M. Cook. "Birds of a feather: Homophily in social networks." *Annual review of sociology* 27.1 (2001): 415-444.

[2] Zhu, Jiong, et al. "Beyond homophily in graph neural networks: Current limitations and effective designs." *Advances in neural information processing systems* 33 (2020): 7793-7804.

Reviewer comment (Theory):
Page 3, line 86-87: This statement doesn't sound quite valid to me. What does it mean to say that we need to increase p(x_{A_t}|x_t)? We aren't talking about one specific value of x_{A_t} here, it's a distribution, right? And ultimately all the terms are being summed up over all possible values of x_{A_t}. Similar concern for KL divergence argument. Basically, the justification given in support of the design of the proposed approach is not convincing and needs more rigor.

Author response:
1. In simple terms, increasing predictive ability refers to reducing the uncertainty of the label variable $y_{t}$ given the input sample variable $x_{A_{t}}$. In other words, we aim to make the distribution $p(y_{t}|x_{A_{t}},x_{t})$ approach a one-hot encoding form for the true labels. Initially, our intention was to present the theorem in a more natural language format to enhance reader comprehension. We also considered using mathematical notation, but it is indeed challenging to express it concisely in a formula. Do you have any suggestions on how to approach this?

2. Due to the convexity of the KL divergence, it exhibits monotonicity. As a result, the KL divergence attains its maximum value within the bounded space of $p(y_{t}|x_{A_{t}},x_{t})$ distributions, with the distribution that maximizes the KL divergence located at the boundary of the feasible space, i.e., the one-hot encoding distribution. This is equivalent to maximizing the predictive ability for the label variable. Even in practical scenarios, it may not be possible to achieve a $p(y_{t}|x_{A_{t}},x_{t})$ distribution that exactly matches the one-hot encoding form. However, the monotonicity property of the KL divergence still holds, indicating the existence of distributions $p(y_{t}|x_{A_{t}},x_{t})$ that maximize the KL divergence, with a larger value as the distribution approaches the boundary of the one-hot encoding. There can be multiple choices for $x_{A_{t}}$ that yield the maximum KL divergence, but this does not affect the conclusion of the theorem.

3. Your understanding is correct. Greater predictive ability corresponds to reduced uncertainty of $y_t$, i.e., the distribution $p(y_{t}|x_{A_{t}},x_{t})$ approaching the one-hot encoding form. We will make this more explicit in the main text. Alternatively, you can also view this problem from the perspective of information entropy. For discrete random variables with finite domains, as the distribution approaches the boundary, the information entropy decreases, indicating reduced uncertainty.

4. To achieve the upper bound on information gain, both of these conditions are necessary and can be simultaneously satisfied. Among them, finding the context set $A_{t}^{\*}$ that maximizes the KL divergence is crucial for the first inequality to hold with equality, $\mathbb{I}(y_t;x_{A_{t}}|x_t) = \sum_{x_t} p(x_t) \sum_{x_{A_{t}}} p(x_{A_{t}}|x_t) D_{t}^{\*}$. Furthermore, the optimal strategy distribution $p(x_{A_{t}^{\*}}|x_t)=1$ and $p(x_{A_{t}}|x_t)=0$ (for $A_t \neq A_{t}^{\*}$) ensures that the second inequality holds with equality, $\sum_{x_t} p(x_t) \sum_{x_{A_{t}}} p(x_{A_{t}}|x_t) D_{t}^{\*} = \sum_{x_t} p(x_t) D_{t}^{\*}$.

Reviewer comment (Writing):
Page 3, lines 87-88: What do you exactly mean by "easier to predict"? Do you mean adding small noise to the samples? Perhaps being more specific here along with some citations will help.

Author response:
1. In simple terms, increasing predictive ability refers to reducing the uncertainty of the label variable $y_{t}$ given the input sample variable $x_{A_{t}}$. In other words, we aim to make the distribution $p(y_{t}|x_{A_{t}},x_{t})$ approach a one-hot encoding form for the true labels. Initially, our intention was to present the theorem in a more natural language format to enhance reader comprehension. We also considered using mathematical notation, but it is indeed challenging to express it concisely in a formula. Do you have any suggestions on how to approach this?

2. Due to the convexity of the KL divergence, it exhibits monotonicity. As a result, the KL divergence attains its maximum value within the bounded space of $p(y_{t}|x_{A_{t}},x_{t})$ distributions, with the distribution that maximizes the KL divergence located at the boundary of the feasible space, i.e., the one-hot encoding distribution. This is equivalent to maximizing the predictive ability for the label variable. Even in practical scenarios, it may not be possible to achieve a $p(y_{t}|x_{A_{t}},x_{t})$ distribution that exactly matches the one-hot encoding form. However, the monotonicity property of the KL divergence still holds, indicating the existence of distributions $p(y_{t}|x_{A_{t}},x_{t})$ that maximize the KL divergence, with a larger value as the distribution approaches the boundary of the one-hot encoding. There can be multiple choices for $x_{A_{t}}$ that yield the maximum KL divergence, but this does not affect the conclusion of the theorem.

3. Your understanding is correct. Greater predictive ability corresponds to reduced uncertainty of $y_t$, i.e., the distribution $p(y_{t}|x_{A_{t}},x_{t})$ approaching the one-hot encoding form. We will make this more explicit in the main text. Alternatively, you can also view this problem from the perspective of information entropy. For discrete random variables with finite domains, as the distribution approaches the boundary, the information entropy decreases, indicating reduced uncertainty.

4. To achieve the upper bound on information gain, both of these conditions are necessary and can be simultaneously satisfied. Among them, finding the context set $A_{t}^{\*}$ that maximizes the KL divergence is crucial for the first inequality to hold with equality, $\mathbb{I}(y_t;x_{A_{t}}|x_t) = \sum_{x_t} p(x_t) \sum_{x_{A_{t}}} p(x_{A_{t}}|x_t) D_{t}^{\*}$. Furthermore, the optimal strategy distribution $p(x_{A_{t}^{\*}}|x_t)=1$ and $p(x_{A_{t}}|x_t)=0$ (for $A_t \neq A_{t}^{\*}$) ensures that the second inequality holds with equality, $\sum_{x_t} p(x_t) \sum_{x_{A_{t}}} p(x_{A_{t}}|x_t) D_{t}^{\*} = \sum_{x_t} p(x_t) D_{t}^{\*}$.

Reviewer comment (Writing):
Section 3.1: What is meant by a "smoother representation"? Perhaps you meant to say that the representation function should be "temporally smooth" so that the neighboring segments get embedded close-by in the embedding space?

Author response:
Thank you for seeking clarification. And your insight aligns exactly with what we mean. We will incorporate this clarification into our paper.

Reviewer comment (Experiments):
Section 4.2, In lines 266-272: Is the noise coming due to challenging boundary disturbances similar to random noise as introduced in this experiment?

Author response:
Detailed implementation of the boundary disturbance is shown in lines 219-226 under the heading "Label disturbance". There exists a fundamental distinction between boundary disturbance and random noise. The boundary disturbance is employed to simulate scenarios where labels are inconsistent. The disturbance is introduced by disturbing the boundary points between different states. It is noted that only the results depicted in Figure 3(b) involve in random noise, which shows that boundary disturbance poses a more intrinsic and challenging problem compared to random noise for time series data. All of the other experiments are based on boundary disturbance.

Reviewer comment (Theory):
The clarity of the paper can be improved. Proposition 1 is a basic mutual information inequality. It is unclear how the mutual information \(I(y_t;x_t,x_{\mathbb{A}_t})\) relates to the performance of a model.

Author response:
- Specifically, mutual information measures the intrinsic properties of a dataset. For example, $\mathbb{I}(y_t;x_t)$ quantifies the correlation between sample and label distributions in the dataset from a probabilistic perspective. $\mathbb{I}(y_t;x_t,x_{A_{t}})$, on the other hand, describes the correlation between sample and label distributions when contextual samples are introduced. The theoretical framework is built upon the assumption that the model can perfectly capture these correlations (line 79). In other words, we seek the theoretical upper bound of the model's performance.
- A larger value of mutual information indicates stronger correlation between variables. For instance, in the inequality $\mathbb{I}(y_t;x_t,x_{A_{t}}) \ge \mathbb{I}(y_t;x_t)$, introducing contextual samples $x_{A_{t}}$ enhances the correlation between the target sample $x_t$ and its label. In classification tasks, a higher correlation between input samples and labels implies that input samples are more easily distinguishable by their labels. In an ideal scenario, the model also possesses a higher upper bound on its performance to discriminate between samples.

Reviewer comment (Theory):
The proof of Theorem 1 mismatches with the claim. The proof only analyzes in what cases \(I(y_t;x_{\mathbb{A}_t}|x_t)\) can be increased. How the predictive capability for the labels is defined? How do we know the contextual information enhances the predictive capability? And what is the connection between predictive capability and the mutual information gain?

Author response:
Based on the previous answer, the increase in $I(y_t;x_{A_t}|x_t)$ determines the extent to which the upper bound of the model's performance improves. The purpose of Theorem 1 is to elucidate the specific contextual sample set that can maximize the correlation between the target sample and its label (i.e., $I(y_t;x_{A_t}|x_t)$). The following explanation is based on our latest version of Theorem 1 (Please see https://openreview.net/forum?id=EvBx5whpzJ&noteId=ZprzVTtrvl). In our proof, predictive capability is equivalent to the uncertainty of the conditional probability distribution of the label given the input sample. As the distribution $p(y_t|x_t,x_{A_{t}})$ approaches the boundary of the probability space, meaning the predicted probability for a certain class approaches 1 (and the probabilities for other classes approach 0), the uncertainty decreases. According to the proof, lower uncertainty corresponds to larger values of KL divergence. The upper bound of $I(y_t;x_{A_t}|x_t)$ is achieved when the KL divergence is maximized, indicating the lowest uncertainty. Therefore, we conclude that introducing contextual samples that maximally enhance predictive capability for the label results in the maximum information gain.

Reviewer comment (Theory):
The motivation for the proposed method is not clear. For example, why using a Gaussian kernel function can better align with \(p(x_{\mathbb{A}_t}|x_t)\) and \(p(y_t|x_t, x_{\mathbb{A}_t})\)?

Author response:
- According to Theorem 1, the model needs to find the optimal contextual sample set that enhances the predictive ability of each sample's label. In this paper, we utilize learnable weights to allow the model to adaptively select potential contextual sample sets. After aggregating contextual sample information, we train the model using sample labels. Through explicit supervised learning, the model can enhance its predictive ability for samples end-to-end while optimizing the selected contextual sample set.
- The Gaussian kernel function serves as our prior knowledge for selecting the contextual sample set. BST data exhibits temporal persistence for each state (line 38). By paying closer attention to and aggregating neighboring samples, the model can acquire temporally smoother representations of time segments. Smoother representations lead to smoother predictive probabilities. This benefits not only the prediction of consecutive time segments belonging to the same category with the same label but also aligns with the gradual nature of class transitions (lines 110-114). According to the conclusions of the latest version of Theorem 1, the use of the Gaussian kernel function allows for a more targeted selection of the contextual sample set, thereby enhancing the model's predictive capability. We will revise these statements in the new version accordingly.

Reviewer comment (Theory):
In Proposition 1, how the mutual information \(I(y_t;x_t,x_{\mathbb{A}_t})\) relates to the performance of a model.

Author response:
- Specifically, mutual information measures the intrinsic properties of a dataset. For example, $\mathbb{I}(y_t;x_t)$ quantifies the correlation between sample and label distributions in the dataset from a probabilistic perspective. $\mathbb{I}(y_t;x_t,x_{A_{t}})$, on the other hand, describes the correlation between sample and label distributions when contextual samples are introduced. The theoretical framework is built upon the assumption that the model can perfectly capture these correlations (line 79). In other words, we seek the theoretical upper bound of the model's performance.
- A larger value of mutual information indicates stronger correlation between variables. For instance, in the inequality $\mathbb{I}(y_t;x_t,x_{A_{t}}) \ge \mathbb{I}(y_t;x_t)$, introducing contextual samples $x_{A_{t}}$ enhances the correlation between the target sample $x_t$ and its label. In classification tasks, a higher correlation between input samples and labels implies that input samples are more easily distinguishable by their labels. In an ideal scenario, the model also possesses a higher upper bound on its performance to discriminate between samples.

Reviewer comment (Theory):
In Theorem 1, how the predictive capability for the labels is defined? How do we know the contextual information enhances the predictive capability? And what is the connection between predictive capability and the mutual information gain?

Author response:
Based on the previous answer, the increase in $I(y_t;x_{A_t}|x_t)$ determines the extent to which the upper bound of the model's performance improves. The purpose of Theorem 1 is to elucidate the specific contextual sample set that can maximize the correlation between the target sample and its label (i.e., $I(y_t;x_{A_t}|x_t)$). The following explanation is based on our latest version of Theorem 1 (Please see https://openreview.net/forum?id=EvBx5whpzJ&noteId=ZprzVTtrvl). In our proof, predictive capability is equivalent to the uncertainty of the conditional probability distribution of the label given the input sample. As the distribution $p(y_t|x_t,x_{A_{t}})$ approaches the boundary of the probability space, meaning the predicted probability for a certain class approaches 1 (and the probabilities for other classes approach 0), the uncertainty decreases. According to the proof, lower uncertainty corresponds to larger values of KL divergence. The upper bound of $I(y_t;x_{A_t}|x_t)$ is achieved when the KL divergence is maximized, indicating the lowest uncertainty. Therefore, we conclude that introducing contextual samples that maximally enhance predictive capability for the label results in the maximum information gain.

Reviewer comment (Theory):
Why using a Gaussian kernel function can better align with \(p(x_{\mathbb{A}_t}|x_t)\) and \(p(y_t|x_t,x_{\mathbb{A}_t})\)?

Author response:
- According to Theorem 1, the model needs to find the optimal contextual sample set that enhances the predictive ability of each sample's label. In this paper, we utilize learnable weights to allow the model to adaptively select potential contextual sample sets. After aggregating contextual sample information, we train the model using sample labels. Through explicit supervised learning, the model can enhance its predictive ability for samples end-to-end while optimizing the selected contextual sample set.
- The Gaussian kernel function serves as our prior knowledge for selecting the contextual sample set. BST data exhibits temporal persistence for each state (line 38). By paying closer attention to and aggregating neighboring samples, the model can acquire temporally smoother representations of time segments. Smoother representations lead to smoother predictive probabilities. This benefits not only the prediction of consecutive time segments belonging to the same category with the same label but also aligns with the gradual nature of class transitions (lines 110-114). According to the conclusions of the latest version of Theorem 1, the use of the Gaussian kernel function allows for a more targeted selection of the contextual sample set, thereby enhancing the model's predictive capability. We will revise these statements in the new version accordingly.

Reviewer comment (Theory):
Some assumption of the proposed method seems a little strong. In Section 3.2, for the prediction behavior constraint, it is assumed that consecutive time segments span at most 2 classes within a suitably chosen time interval. The time interval may have a big impact on the model performance. However, it is not clear how to choose a suitable time interval for each dataset. The authors also need to perform experiments studying the impacts of the time interval on different datasets.

Author response:
Thank you for your valuable suggestion.

- With domain knowledge, given the prior visual window size of manual annotation, we recommend selecting a slightly larger window size as the time interval length for two reasons:

  - Typically, the annotation window does not contain data spanning more than two classes.
  - Opting for a slightly larger window allows the model to leverage more contexts.

  For instance, in the case of SEEG data, the doctors tend to use a 15-seconds visual window to identify the seizure waves. And based on empirical evidence, the duration of seizures is generally not shorter than 15 seconds. Hence, we chose a 16-second time interval to try to avoid the occurrence of multiple label transitions.

- We are currently conducting experiments with time intervals scaled at 0.5, 1.5, and 2 times the initially selected length. Upon completion of the experiments, we will promptly provide the results and analysis.

  To ensure the comparison to existing experimental parameters as fair as possible:

  - We maintain identical experiment groups.

  - We sample the same number of time intervals based on three different time interval lengths respectively. We proportionally scale both the window length and slide length, ensuring consistency in the number of time segments across all experiments.

  - The hyperparameters of the main model and baselines remain consistent.

Reviewer comment (Experiments):
The experimental analysis is not sufficient. Compared with existing methods, one advantage of the proposed method is to exploit the prior information at both the data and class levels. The authors are suggested to perform experiments studying the performance of the proposed method with only considering the prior information at data level and class level respectively.

Author response:
We have conducted relevant experimental analyses. In the ablation experiments, - Con-T refers to replacing the Con-Transformer module with vanilla Transformer, thereby removing the prior information at the data level. - Coh-P indicates the simultaneous removal of the Neighbor Class Consistency Discrimination module and the function fitting module, meaning the elimination of the contextual information at the label level. - Fit represents the removal of only the function fitting module, implying the exclusion of prior trend information at the label level. Conversely, + Con-T and + Coh-P denote the retention of data-level and label-level contextual information, respectively.

Reviewer comment (Experiments):
As discussed in Section 3.2, the time interval may have a big impact on the model performance. How to choose a suitable interval for each dataset?

Author response:
Thank you for your valuable suggestion.

- With domain knowledge, given the prior visual window size of manual annotation, we recommend selecting a slightly larger window size as the time interval length for two reasons:

  - Typically, the annotation window does not contain data spanning more than two classes.
  - Opting for a slightly larger window allows the model to leverage more contexts.

  For instance, in the case of SEEG data, the doctors tend to use a 15-seconds visual window to identify the seizure waves. And based on empirical evidence, the duration of seizures is generally not shorter than 15 seconds. Hence, we chose a 16-second time interval to try to avoid the occurrence of multiple label transitions.

- We are currently conducting experiments with time intervals scaled at 0.5, 1.5, and 2 times the initially selected length. Upon completion of the experiments, we will promptly provide the results and analysis.

  To ensure the comparison to existing experimental parameters as fair as possible:

  - We maintain identical experiment groups.

  - We sample the same number of time intervals based on three different time interval lengths respectively. We proportionally scale both the window length and slide length, ensuring consistency in the number of time segments across all experiments.

  - The hyperparameters of the main model and baselines remain consistent.