Partial Channel Dependence with Channel Masks for Time Series Foundation Models

Authors: Seunghan Lee, Taeyoung Park, Kibok Lee
Primary area: learning on time series and dynamical systems
Keywords: Time Series, Foundation Model, Channel Dependence, Transformer

Abstract:
Recent advancements in foundation models have been successfully extended to the time series (TS) domain, facilitated by the emergence of large-scale TS datasets. However, previous efforts have primarily focused on designing model architectures to address explicit heterogeneity among datasets such as various numbers of channels, while often overlooking implicit heterogeneity such as varying dependencies between channels. In this work, we introduce the concept of partial channel dependence (PCD) for models capturing channel dependencies (CDs) via attention, which enables a more sophisticated adjustment of CDs based on dataset-specific information. To achieve PCD, we propose a channel mask that captures the relationships between channels within a dataset using two key components: 1) a correlation matrix that encodes relative dependencies between channels, and 2) domain parameters that learn the absolute dependencies specific to each dataset, refining the correlation matrix. We validate the effectiveness of PCD across four tasks in TS including forecasting, classification, imputation, and anomaly detection, under diverse settings, including few-shot and zero-shot scenarios with both TS foundation models and single-task models.

Decision: reject
Reviewer scores: 3, 6, 6, 3, 5

Reviews and discussion:
<thread>
creation date:
1730707109391

thread_type:
official_review

<thread_object>
creator: reviewer_jkmr

<content>
summary:
This work proposes a time-series foundation model designed to capture varying channel dependencies across different datasets. By using a correlation matrix as a prior, the model incorporates a learnable domain parameter to construct a domain-specific channel mask, effectively capturing global channel dependencies. In conjunction with local channel dependency masks (as introduced in prior works, such as iTransformer), the proposed model accommodates diverse channel dependencies across datasets, leading to performance improvements in tasks such as forecasting, imputation, and anomaly detection (in full-shot, few-shot, and zero-shot settings).

strengths:
- The paper presents a straightforward algorithm with clear explanations.
- The manuscript covers a broad range of experiments across various time-series tasks (anomaly detection, forecasting, imputation, etc.) in different settings (supervised, few-shot/zero-shot domain/task) with extensive analyses (e.g., robustness to missing values) that support its claimed advantages.
- The newly introduced experiment, "masked channel prediction," is particularly novel and promising, though it is only briefly discussed in the manuscript.

weaknesses:
Despite the demonstrated performance gains across multiple tasks and settings, some of the motivation and design choices remain insufficiently addressed in the manuscript. Additionally, certain reported experimental results differ from what appears in the paper, and key experimental details are not fully explained.

First, I would appreciate clarification from the authors regarding the principles underlying channel dependency in the proposed time-series foundation model:

- Why is it crucial to account for varying channel dependencies? To what extent is this heterogeneity in channel dependency necessary for constructing a robust foundation model for time-series data? Prior research has highlighted its importance for forecasting tasks both empirically [1][2] and theoretically [3], yet the rationale for its role in the foundation model is not fully studied. What is the anticipated impact of the time-series foundation model, and how does it address channel dependencies? (Simply stating "inherent heterogeneity" may be insufficient in this context.)
- The global mask encapsulates correlation across various domains during training, which is then reused during testing without further adjustment. Is it sufficient to rely on the global correlation matrix alone? Since local correlations may vary over time, the correlation matrix at test time might differ significantly from that during training. How do you ensure its stability under these conditions?
- Is correlation an appropriate metric for capturing channel dependencies? Some studies in forecasting suggest a "causal relationship" between channels, while others tackle spurious correlations (where channels appear correlated but are not causally related).

Furthermore, several design choices would benefit from clarification:

- What motivated the choice to use domain-specific global attention while sharing local attention across multiple domains? What specific roles do global and local attention play in the model's functioning?
- Is the "pair-wise dot product" between global and local attention sufficient to achieve the intended effect? Under extreme test-time scenarios where variables v1 and v2 exhibit low global correlation (approaching zero) but high local correlation, this design might yield low attention scores, potentially failing to capture abrupt correlation increases under certain test-time conditions.

In addition, some minor experimental inconsistencies and essential experimental details require clarification:

- Some scores of baselines appear lower than what has been reported in the original paper. In Table 3, the MSE/MAE score on iTransformer is way higher than it has been reported (Appendix F, Table 8 in [4]) Have these results been reproduced, and if so, what could account for the discrepancies?
- How is the global correlation matrix defined? Does it only consider the correlation matrix over the training period?
- In zero-shot experiments on new datasets, how is the domain parameter handled? As the domain parameter cannot be directly learned for unseen domains, is it substituted with that of a similar domain?

[1] Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators (ICLR 2024) 

[2] Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series (arXiv 2024) 

[3] Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning (ICML 2024)

[4] ITRANSFORMER: INVERTED TRANSFORMERS ARE EFFECTIVE FOR TIME SERIES FORECASTING (ICLR 2024)

questions:
In conclusion, while the paper presents an extensive array of experimental evidence, the motivation for addressing channel dependency heterogeneity is not entirely convincing in terms of its necessity for building a foundational model. Although channel dependencies may be relevant to specific tasks in the time-series domain (e.g., forecasting), the underlying rationale and its influence on design choices are not fully explained in the manuscript. Additionally, certain key experimental results appear inconsistent, which is problematic given the paper's emphasis on empirical performance gains over theoretical justification. Nevertheless, the proposed architecture is straightforward and appears tailored to address varying channel dependencies, and it has the potential to be impactful if the dependencies are better elucidated and some of the reported results are clarified.

If the authors address these points and resolve potential experimental issues in the main results table, I would be inclined to raise my rating.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730719103379

thread_type:
official_review

<thread_object>
creator: reviewer_saes

<content>
summary:
This work introduces the concept of partial channel dependence (PCD) to address implicit heterogeneity in time series (TS) data. By utilizing a channel mask that incorporates a correlation matrix to encode relative dependencies between channels and domain parameters to learn dataset-specific absolute dependencies, the authors refine the correlation matrix for better channel dependency adjustments. The effectiveness of PCD is validated across four TS tasks—forecasting, classification, imputation, and anomaly detection.

strengths:
1. This work focuses on a highly valuable research direction and highlights the importance of Partial Channel Dependence in Time Series analysis.
2. This work proposes a concise approach to adjust the correlation matrix obtained from prior knowledge.

weaknesses:
1. The concept of Partial Channel Dependence (PCD) is already discussed in the CCM[1] where the authors employ a clustering approach to capture latent PCD. The current content lacks a detailed comparison and discussion with this work.
2. It is important to note that not all foundation models are based on attention mechanisms (TTM [2]), and not all methods that utilize attention mechanisms effectively capture the attention between channels like UniTS (TimesFM[3], Timer[4], MOIRAI[5], MOMENT[6]). As a plugin for foundation models, the generality of the CM method is insufficient.
3. The paper does not specify how the correlation matrix mentioned was constructed, and references [7] and [8] primarily focus on capturing correlation relationships with lag properties. In contrast, this work does not explore lag properties but instead utilizes the complete sequences. The construction method of the correlation matrix needs to be explained in detail.
4. Domain parameters highly correlated with the construction method of the correlation matrix, If use other methods, they may not be effective. The effectiveness of domain parameters in the experiments lacks further validation.
5. In the experiments, there is a lack of comparison with other plugins, such as LIFT[8], and each task only validates the plugin's improvement on a few models, making its generality difficult to confirm. More importantly, a large number of foundation models have not been considered in the experiments, such as [2-6]. The existence of these issues raises serious doubts about the effectiveness and generality of the plugin. 

[1] From Similarity to Superiority: Channel Clustering for Time Series Forecasting.

[2] Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series.

[3] A decoder-only foundation model for time-series forecasting.

[4] Timer: Transformers for Time Series Analysis at Scale.

[5] Unified Training of Universal Time Series Forecasting Transformers.

[6] MOMENT: A Family of Open Time-series Foundation Models.

[7] Vcformer: Variable correlation transformer with in-herent lagged correlation for multivariate time series forecasting.

[8] Rethinking channel dependence for multivariate time series  forecasting:Learning from leading indicators.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730719812210

thread_type:
official_review

<thread_object>
creator: reviewer_bw9c

<content>
summary:
This paper presents Partial Channel Dependence (PCD), a method designed to capture the varying dependencies between channels across different datasets. PCD achieves this by applying channel-wise attention multiplied by the corresponding dataset mask. Experimental results demonstrate that PCD yields an average performance improvement across various time series tasks.

strengths:
* The method is straightforward, and the motivation is clearly articulated.
* Extensive experiments across various scenarios validate that PCD enhances the performance of Transformer-based multivariate time series models.

weaknesses:
* The results for iTransformer and PatchTST presented in Table 3 differ significantly from those reported in the original papers. Additionally, the ETTh2, ETTm1, and ETTm2 datasets are well-known multivariate time series datasets. Could you please provide a comprehensive comparison of the same baselines on these datasets across all prediction lengths?
* While PCD does enhance the performance of Transformer-based models for multivariate time series forecasting, I recommend including recent MLP-based and CNN-based models in the baselines, such as RLinear and ModernTCN. Additionally, GNN-based models like CrossGNN are also adept at capturing multivariate relationships. Including these would strengthen the performance comparisons.
* The length of the input time series is an important factor influencing experimental results. Therefore, I would like to see the impact of varying sequence lengths on the results w CM and w/o CM.
* There are several literature on channel dependency modeling, such as [1-2]. Detailed discussion is needed.

[1] Zhao et al., Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators, ICLR 2024

[2] Qi et al., Enhancing Multivariate Time Series Forecasting with Mutual Information-driven Cross-Variable and Temporal Modeling

questions:
1.	Could the authors apply a similar masking approach to CI models like PatchTST or PITS, and compare their performance across different datasets? If possible, please provide performance comparisons for the original settings w CM and w/o CM.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730742877861

thread_type:
official_review

<thread_object>
creator: reviewer_bsuw

<content>
summary:
The paper introduces a novel concept called Partial Channel Dependence (PCD) to address implicit heterogeneity in time series data, specifically focusing on varying dependencies between channels. The authors propose a channel mask mechanism that combines correlation matrices (for relative dependencies) with learned domain parameters (for absolute dependencies). The approach is evaluated across multiple time series tasks (forecasting, classification, imputation, and anomaly detection) in both few-shot and zero-shot settings, demonstrating its versatility across different foundation models and single-task models.

strengths:
- Introduces a novel methodology for handling channel relationships specifically designed for pretraining foundation models in time series
- Proposes a systematic framework for incorporating dataset-specific channel dependencies into the pretraining process
- Demonstrates superior performance in challenging scenarios (few-shot and zero-shot learning)

weaknesses:
- **Dataset Identification Requirement**: The method requires knowing exactly which samples belong to which dataset during pretraining. This is a strong assumption that may not hold in real-world applications where data sources might be mixed or unclear.
- **Lack of Fixed vs. Variable Dependency Analysis**: The paper assumes variable channel dependencies are necessary but doesn't justify why a simpler fixed dependency structure wouldn't work equally well. Without this comparison, the added complexity of variable dependencies might be unnecessary.

- **Missing Critical Baseline Comparison**

    - No comparison against the same architecture with full channel dependence
    - No comparison against the same architecture with full channel independence

questions:
- Can you provide comparisons of your method against the same architecture with full channel dependence and full channel independence?
- What theoretical guarantees or analysis can you provide to show when partial dependence would outperform the extreme cases?
- How does your method handle scenarios where dataset boundaries are ambiguous or when data comes from multiple unknown sources?
- Can you quantify the computational overhead and memory requirements of your approach compared to full dependence and independence cases?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730745951371

thread_type:
official_review

<thread_object>
creator: reviewer_21yw

<content>
summary:
This paper addresses the limitation that time series models often consider only explicit heterogeneity among datasets, such as varying sequence lengths and numbers of channels, while overlooking implicit heterogeneity like channel dependencies within the data. The authors propose a module called the Channel Mask (CM) to enable models to reflect channel dependence (CD) by adjusting the degree of CD based on dataset characteristics, achieving Partial Channel Dependence (PCD). By integrating CM into existing time series models, they demonstrate improved performance across various time series tasks.

strengths:
1. The paper critiques the limitations of using correlation coefficients for measuring inter-channel relationships and proposes a new way to measure channel dependence through the CD ratio. The CM introduces very few parameters (α and β from domain parameters) yet effectively learns the implicit inter-channel relationships, leading to performance improvements in time series models.

2. The authors validate their approach through various experiments, including few-shot and zero-shot settings, and provide thorough ablation studies and analyses to demonstrate the effectiveness and suitability of the CM structure and its relationship with CD.

weaknesses:
1. The paper lacks theoretical grounding or in-depth analysis to explain why the proposed method leads to performance improvements. A deeper understanding of the underlying mechanisms would clarify and strengthen the contribution.
1.1. For example, how can (or how should) we define the concept of “implicit heterogeneity”? Why do we need this concept? While there are studies on channel dependence in time series machine learning, is the concept in this work different from existing studies? Furthermore, how is the rigorous definition of implicit heterogeneity connected to the proposed CM method?  What is the rationale behind the use of correlation information between different real-world time series datasets in different domains to achieve better performance?
1.2. For another example, while the use of correlation matrices primarily captures linear relationships between channels, the application of a sigmoid function in the CM introduces nonlinearity to the model. The authors mention that static correlations (global CD) are reflected through the CM, while dynamic, local correlations are captured by the attention mechanism. This design may help address some aspects of nonlinearity and temporal variation in channel dependencies. However, it remains a question whether this approach is fully sufficient to reflect the complex and changing characteristics inherent in time series data. A clear explanation of the rationale behind how the CM models these dynamic changes, or further investigation into its effectiveness in this regard, would strengthen our understanding. These considerations might be related to the rigorous definition of implicit heterogeneity in time series.
1.3. Meanwhile, the authors state that “However, most previous works have focused on the model architecture to either capture or disregard CD, often overlooking the potential differences in CD across datasets.” However, they do not show specific theoretical analysis results on why and how existing studies on the model architecture are limited in capturing the differences in CD across datasets.

2. The technical contribution is also limited.
2.1. Following the above comment, the claim that existing CD models overlook differences in CD between datasets may not be fully substantiated. Since models like Crossformer, iTransformer, and TimeSiam learn attention patterns specific to each dataset, it's unclear whether they truly neglect dataset-specific CD differences.
2.2. While the authors introduce PCD as an intermediary concept between channel independence (CI) and channel dependence (CD), the structure of CM, which uses a channel correlation matrix, cannot be applied to CI models. The study applies CM to CD models (iTransformer, UniTS) and shows performance improvements but does not verify whether applying CM to CI models can enhance performance. Thus, the proposed PCD cannot be extended to CI settings, limiting its utility in models that assume channel independence.
2.3. The CM seems to be applicable only to Transformer models that apply attention along the channel axis and cannot be applied to models that apply attention along the time axis. For instance, existing studies like ST-MEM [1] and UniTST [2], which learn CD by applying attention along the time axis after channel flattening, cannot utilize CM. This limitation reduces the general applicability of the proposed method to other architectures.
- [1] Na, Y., Park, M., Tae, Y., & Joo, S. (2024). Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram. arXiv preprint arXiv:2402.09450.
- [2] Liu, J., Liu, C., Woo, G., Wang, Y., Hooi, B., Xiong, C., & Sahoo, D. (2024). UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting. arXiv preprint arXiv:2406.04975.

2.4. The CD ratio proposed in the study is calculated after combining CM with the time series model and training, and its value may vary depending on the model used. The paper does not provide sufficient evidence to demonstrate that the CD ratio is consistent across different models for the same dataset, raising concerns about its adequacy as a metric for measuring channel dependence.
2.5. Meanwhile, although the authors started their argument from the emergence of time series foundation models (TSFMs), they do not provide sufficient validation of this work for different TSFMs.

questions:
Please see the weaknesses. Some other (related) questions are as follows.
1. In the experiments involving TimeSiam, which encoder was used? The TimeSiam paper proposes both PatchTST (CI properties) and iTransformer (CD properties) as encoders. Since they have different characteristics, specifying the encoder used is essential. Additionally, since TimeSiam is utilized for classification tasks in your paper, it should be categorized appropriately, and the performance of CM + TimeSiam in classification tasks should also be evaluated.
2. The CI framework does not attend between channels but attends with timestamps or patches in each channel. Then, how is the formulation of A in equation (1) justified as the identity matrix in equation (1) indicates the channel relationship? For example, how does PatchTST match to this formulation?
3. Is the CD ratio consistent across different models for the same dataset? To establish the CD ratio as a reliable metric for measuring dataset CD, it should be verified whether CD ratios computed using different models (e.g., iTransformer vs. UniTS) yield consistent results.
4. Since the CM has been validated on datasets with significant CD, it would be beneficial to test its performance on synthetic datasets with uncorrelated channels to verify that CM yields a low CD ratio and does not introduce unnecessary dependencies.
5. Time series data often exhibit non-stationarity, which can affect correlation measures. How does your method handle non-stationary data, and does the CM adjust for changes in channel dependencies over time?
6. Have you considered other measures that can capture nonlinear or more complex dependencies between channels, such as mutual information? This could potentially enhance the CM's ability to model complex inter-channel relationships.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732360882318

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
### **7. Inconsistent results with the original paper (bW9C, JKMr)**

We reproduced experiments using the official iTransformer [A] and UniTS [B] code repositories without modifications, but found that iTransformer exhibited similar issues reported on GitHub, and UniTS required setting the hidden dimension to $D=32$ to resolve convergence issues, which resulted in degraded performance, with further details in **Appendix B**.

&nbsp;

### **8. Minor Issues**
- **Varying Size of Lookback Window $L$ (bW9C)**: We set $L$ to match the backbone method for a fair comparison. Following the reviewer’s feedback, we conducted an ablation study (**Appendix F**) with different values of $L$ across multiple datasets. The results demonstrate that the impact of CM remains consistent regardless of $L$, validating its robustness.

- **Additional Related works (bW9C)**: While LIFT [A] and CDAM [B] are related to our work in handling CD, no plug-in methods specifically capture varying CD across multiple datasets for TSFMs. Nonetheless, we attempted to compare our method with them in a single-task model, but LIFT lacks support for recent CD models, and CDAM does not provide code. We have included both methods in the **2.Related Works** section (**L87--89**).

&nbsp;

For your convenience, we have uploaded a PDF that includes both the revised manuscript and appendices, with key changes highlighted in **green**. We believe the discussions and empirical results will strengthen our contribution. If there are any points we may have missed or if you have further questions or suggestions, please do not hesitate to share them with us. We would be happy to address them.

&nbsp;

Thank you very much.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732360909922

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
# General Comment
&nbsp;

Dear AC and reviewers,

First of all, we deeply appreciate your time and effort in reviewing our paper. Our work introduces the concept of **partial channel dependence (PCD)**, which adjusts channel dependencies (CD) captured by the model based on dataset-specific information. The approach uses a **channel mask (CM)** composed of two components: a correlation matrix for relative dependencies and domain parameters for absolute dependencies, refining the correlation matrix for each dataset.

As highlighted by the reviewers, our work introduces a **novel approach** addressing implicit heterogeneity across TS datasets (21yw, BSUW). It is **straightforward and well-motivated** (bW9C, JKMr), supported by **extensive experiments** across various tasks (all). The method demonstrates improved performance in challenging scenarios, including few-shot and zero-shot settings (BSUW, JKMr), while effectively capturing dataset-specific dependencies (21yw, BSUW).

&nbsp;

In our responses, we have addressed the concerns raised by all reviewers and strengthened our claims with additional analyses. Below, we present key highlights to support your post-rebuttal discussion (all sections/figures/tables labeled according to the revised version):

&nbsp;

### **1. Heterogeneity across TS datasets (21yw, JKMr)**

Reviewers (21yw, JKMr) asked to further clarify the **definition of heterogeneity** across TS datasets. As stated in **L37--43**, explicit heterogeneity refers to observable differences like sequence length and number of channels, while implicit heterogeneity involves unobservable factors, such as varying channel dependencies (CD) across datasets. To enhance clarity, we have revised the **1.Introduction** section.

&nbsp;

### **2. Necessity of capturing varying CD across datasets (21yw, JKMr)**
Reviewers (21yw, JKMr) questioned the necessity of capturing varying CDs across datasets. As noted in **L43--46**, addressing implicit heterogeneity is essential for TSFMs, which are trained on **multiple** datasets with varying CDs which **require different approaches** (e.g., CI or CD). We show that considering this heterogeneity improves performance, as shown in **Table 7** (CD vs. PCD).

&nbsp;

### **3. Generality of our method (21yw, bW9C, saeS)**
Reviewers (21yw, bW9C, saeS) raised concerns about the generality of our method. However, PCD is proposed to enhance TSFMs that **capture CDs with attention mechanism** by leveraging dataset-specific characteristics, rather than being a universal solution for all TSFMs, as noted in **L60--61** and **L69--70**. We focus on TSFMs capturing CDs due to the growing emphasis on attention mechanisms for capturing CDs, and our approach builds on this trend and its strengths. Additionally, CD models, with their higher capacity, are better suited for large or multiple datasets, aligning with the use of CM.

&nbsp;

### **4. Appropriateness of correlation metric (21yw, saeS, JKMr)**
Reviewers (21yw, saeS, JKMr) questioned the appropriateness of using correlation as a metric. While our primary focus is on utilizing **dataset-specific information** rather than the correlation metric itself, we acknowledge the reviewers' concerns regarding the choice of metrics for capturing channel relationships. As detailed in **Table 10**, we have already conducted an analysis using four different metrics -- Correlation, Euclidean, Cosine, and Dynamic Time Warping (DTW) -- showing consistent performance improvements across all metrics.

&nbsp;

### **5. Design choice of CM and attention matrix (21yw, JKMr)**
Reviewers (21yw, JKMr) raised concerns about our design choice of the global CM and attention matrix. The (global) CM is designed to capture stable characteristics across time, focusing on global CD rather than adapting to local changes. Adjusting the CM for local variations would undermine its stability. To capture local variations, we use the attention matrix, which dynamically adjusts to the input time series. This approach ensures both stability and adaptability, as demonstrated in experiments on 13 real-world datasets with significant distribution shifts (**Table 12**). Additional experiments with the Weather dataset showed minimal changes in the global CM during local fluctuations (L2 distance of 0.0002), confirming its stability.

&nbsp;

### **6. Effectiveness of CM under non-stationarity (21yw)**
As noted by a reviewer (21yw), TS data can exhibit non-stationarity. However, we observed CM's effectiveness under such conditions through experiments on 13 real-world datasets with distribution shifts (**Tables 2, 3, E.1**) and datasets with varying levels of missing data (10%, 25%, 50%, 75%), as detailed in **L471--475**. Results in **Figure 10** show that CM remains robust in both forecasting performance and the CD ratio even under these non-stationary scenarios.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732609890381

thread_type:
official_comment

<thread_object>
creator: area_chair_6qsb

<content>
title:
Encouragement to Actively Participate in the Discussion Phase

comment:
Dear Reviewers,

Thank you for your valuable contributions to the review process so far. As we enter the discussion phase, I encourage you to actively engage with the authors and your fellow reviewers. This is a critical opportunity to clarify any open questions, address potential misunderstandings, and ensure that all perspectives are thoroughly considered.

Your thoughtful input during this stage is greatly appreciated and is essential for maintaining the rigor and fairness of the review process.

Thank you for your efforts and dedication.
</content>
</thread_object>

</thread>


Metareview:
metareview:
(a) Summary of Scientific Claims and Findings
The submission introduces Partial Channel Dependence, a concept designed to handle varying channel dependencies in multivariate time series datasets by using a proposed Channel Mask. This mask integrates two components: (1) a correlation matrix to encode relative dependencies and (2) domain-specific parameters for dataset-specific absolute dependencies. The method is implemented into Transformer-based time series foundation models and is evaluated across tasks such as forecasting, classification, imputation, and anomaly detection, with special emphasis on few-shot and zero-shot scenarios.

(b) Strengths of the Paper
Novelty: The introduction of PCD to address implicit heterogeneity across datasets is novel and demonstrates improved adaptability compared to existing methods focused solely on explicit heterogeneity.
Empirical Evaluation: Extensive experiments are conducted across diverse tasks and settings. The inclusion of ablation studies helps clarify the role of the CM components in improving performance.
Simplicity and Scalability: The CM design introduces minimal additional parameters, making it computationally efficient and scalable.

(c) Weaknesses of the Paper
Limited Generalization: The method is restricted to Transformer-based models that use attention mechanisms for channel dependencies. It is inapplicable to many existing TSFMs that adopt channel-independent (CI) strategies or alternate CD formulations.
Theoretical Gaps: The paper lacks a rigorous theoretical analysis to explain the observed performance gains or validate the CD ratio as a universal metric for channel dependence across models.
Inadequate Comparison: Comparisons with concurrent state-of-the-art models are either missing or insufficient, particularly for newly proposed TSFMs such as MOIRAI, Timer, and TTMs.
Validation Across Architectures: The experiments are limited to a subset of TSFMs and specific datasets. Broader architectural compatibility and robustness under non-standard scenarios, such as extreme non-stationarity, are insufficiently validated.
Reviewer Critiques: Persistent misunderstandings in the rebuttal stage, such as inadequate generalization or misalignment with prior works, indicate potential issues in communication and framing.

(d) Reasons for Rejecting the Paper
Scope Limitation: The method’s applicability is confined to attention-based CD models, leaving out a significant portion of TSFM research.
Lack of Robust Benchmarking: While the authors conducted comprehensive experiments, the absence of meaningful comparisons with concurrent models diminishes the credibility of the claimed state-of-the-art performance.
Incomplete Theoretical Contributions: The lack of a rigorous framework for implicit heterogeneity and the CD ratio metric weakens the foundational significance of the proposed approach.
Feedback from Reviewers: Significant concerns from reviewers about limited generalizability, insufficient theoretical insights, and inadequate comparisons were not fully addressed in the rebuttal phase.