From Similarity to Superiority: Channel Clustering for Time Series Forecasting

Authors: jialin  chen
jan eric lenssen
aosong  feng
weihua  hu
matthias  fey
leandros  tassiulas
jure  leskovec
rex  ying
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 07:39:18

Abstract: Time series forecasting has attracted significant attention in recent decades.
    Previous studies have demonstrated that the Channel-Independent (CI) strategy improves forecasting performance by treating different channels individually, while it leads to poor generalization on unseen instances and ignores potentially necessary interactions between channels. Conversely, the Channel-Dependent (CD) strategy mixes all channels with even irrelevant and indiscriminate information, which, however, results in oversmoothing issues and limits forecasting accuracy.
    There is a lack of channel strategy that effectively balances individual channel treatment for improved forecasting performance without overlooking essential interactions between channels. Motivated by our observation of a correlation between the time series model's performance boost against channel mixing and the intrinsic similarity on a pair of channels, we developed a novel and adaptable \textbf{C}hannel \textbf{C}lustering \textbf{M}odule (CCM). CCM dynamically groups channels characterized by intrinsic similarities and leverages cluster information instead of individual channel identities, combining the best of CD and CI worlds. Extensive experiments on real-world datasets demonstrate that CCM can (1) boost the performance of CI and CD models by an average margin of 2.4% and 7.2% on long-term and short-term forecasting, respectively; (2) enable zero-shot forecasting with mainstream time series forecasting models; (3) uncover intrinsic time series patterns among channels and improve interpretability of complex time series models.

Main paper: 
 introduction: 
 Time series forecasting has attracted a surge of interest across diverse fields, ranging from economics, energy [1, 2] , weather [3, 4] , to transportation planning [5, 6] . The complexity of the task is heightened by factors including seasonality, trend, noise in the data, and potential cross-channel information. Despite the numerous deep learning time series models proposed recently [7, 8, 9, 10, 11, 12, 13, 14] , an unresolved challenge persists in the effective management of channel interaction within the forecasting framework [15, 16] . Previous works have explored two primary channel strategies: Channel-Independent (CI) and Channel-Dependent (CD) strategies. The Channel-Independent (CI) strategy has shown promise in better forecasting performance by having individual models for each channel. However, a critical drawback is its limited generalizability and robustness on unseen channels [17] . Besides, it tends to overlook potential interactions between various channels. Conversely, the Channel-Dependent (CD) strategy models all channels as a whole and captures intricate channel relations, while they tend to show oversmoothing and have trouble fitting to individual channels, especially when the similarity between channels is very low. Moreover, existing , which are layer weights assigned to K clusters, obtaining weights θ i for the i-th channel. The learned prototypes retain pre-trained knowledge, enabling zero-shot forecasting on unseen samples in both univariate and multivariate scenarios. models typically treat univariate data in a CI manner, neglecting the interconnections between time series samples, even though these dependencies are commonly observed and beneficial in real-world scenarios, such as stock market or weather forecasting [18, 19, 20] .Proposed work. To address the aforementioned challenges, we propose a Channel Clustering Module (CCM) that balances individual channel treatment and captures necessary cross-channel dependencies simultaneously. CCM is motivated by the key observations that CI and CD models typically rely on channel identity information. The level of reliance is anti-correlated with the similarity between channels (see Sec. 4 .1 for an analysis). This intriguing phenomenon alludes to the model's analogous behavior on similar channels. The proposed CCM thereby involves the strategic clustering of channels into cohesive clusters, where intra-cluster channels exhibit a higher degree of similarity. To capture the underlying time series patterns within these clusters, we employ cluster-aware Feed Forward to assign independent weights to each cluster and replace individual channel treatment with individual cluster treatment. Moreover, CCM learns expressive prototype embeddings in training, which enables zero-shot forecasting on unseen samples by grouping them into appropriate clusters.CCM is a plug-and-play solution that is adaptable to most mainstream time series models. We evaluate the effectiveness of CCM on four different time series backbones (aka. base models): TSMixer [7] , DLinear [8] , PatchTST [21] , and TimesNet [13] . It can also be applied to other state-ofthe-art models for enhanced performance. Extensive experiments verify the superiority of CCM in long-term and short-term forecasting benchmarks, achieving an average margin of 2.4% and 7.2%, respectively. Additionally, we collect stock data from a diverse range of companies to construct a new stock univariate dataset. Leveraging information from intra-cluster samples, CCM consistently shows a stronger ability to accurately forecast stock prices in the dynamic and intricate stock market. Moreover, CCM enhances zero-shot forecasting capacities of time series backbones in cross-domain scenarios, which further highlights the robustness and versatility of CCM.The contributions of this paper are: (1) We propose a novel and unified channel strategy, i.e., CCM, which is adaptable to most mainstream time series models. CCM explores the optimal trade-off between channel individual treatment and cross-channel modeling, (2) CCM demonstrates superiority in improving performance on long-term and short-term forecasting, and (3) through learning prototypes from clusters, CCM enables zero-shot forecasting on unseen samples in both univariate and multivariate scenarios.2 Related WorkQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: Clearly state the contribution in the introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
time series forecasting models: 
 Traditional machine learning methods such as Prophet [22, 23] , ARIMA [24] capture the trend component and seasonality in time series [25] . As data availability continues to grow, deep learningQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: Discussed the limitations in conclusion.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community.Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
theory assumptions and proofs: 
 Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
motivation for channel similarity: 
 To motivate our similarity-based clustering method, we conduct the following toy experiment. We select four recent and popular time series models with different backbones. TSMixer [7] and DLinear are linear models. PatchTST [21] is a transformer-based model with a patching mechanism and TimesNet [13] is a convolutional network that captures multi-periodicity in data. Among these, TSMixer and TimesNet utilize a Channel-Dependent strategy while DLinear and PatchTST adopt the Channel-Independent design. We train a time series model across all channels and evaluate the channel-wise Mean Squared Error (MSE) loss on the test set. Then, we repeat training while randomly shuffling channels in each batch. Note that for both CD and CI models, this means channel identity information will be removed. We report the average performance gain in terms of MSE loss across all channels based on the random shuffling experiments (denoted as ∆L(%)) in Table 1 .Table 1 : Averaged performance gain from channel identity information (∆L(%)) and Pearson Correlation Coefficients (PCC) between {∆L ij } i,j and {SIM(X i , X j )} i,j . The values are averaged across all test samples. We attribute the models' performance decrease in the random shuffling experiments to the loss of channel identity information. We see that all models rely on channel identity information to achieve better performance. Next, we define channel similarity based on radial basis function kernels [66] asEQUATION)where σ is a scaling factor. Note that the similarity is computed on the standardized time series to avoid scaling differences. More details are discussed in Appendix A.1. The performance difference in MSE from the random shuffling experiment for channel i is denoted as ∆L i . We define∆L ij := |∆L i -∆L j | and calculate the Pearson Correlation Coefficients (PCC) between {∆L ij } i,jand {SIM(X i , X j )} i,j , as shown in Table 1 . The toy example verifies the following two assumptions:(1) Existing forecasting methods heavily rely on channel identity information.(2) This reliance clearly anti-correlates with channel similarity: for channels with high similarity, channel identity information is less important. Together, these two assumptions motivate us to design an approach that provides cluster identity instead of channel identity, combining the best of both worlds: high capacity and generalizability.Channel Clustering. Motivated by the above observations, we first initialize a set ofK cluster embeddings {c 1 , • • • , c K }, where c k ∈ R d , dis the hidden dimension and K is a hyperparameter. Given a multivariate time series X ∈ R T ×C , each channel in the input X i is transformed into a d-dimensional channel embedding h i through an MLP. The probability that a given channel X i is associated with the k-th cluster is the normalized inner-product of the cluster embedding c k and the channel embedding h i , which is computed asEQUATIONThe normalization operator ensures that k p i,k = 1 and validates the clustering probability distribution across k clusters. We utilize reparameterization trick [67] to obtain the clustering membership matrix M ∈ R C×K where M ik ≈ Bernoulli(p i,k ). Higher probability p i,k results in M ik close to 1, leading to the deterministic existence of certain channels in the corresponding cluster.Prototype Learning. The cluster assigner also creates a d-dimensional prototype embedding for each cluster in the training phase.Let C = [c 1 , • • • , c K ] ∈ R K×d denote the cluster embedding, and H = [h 1 , • • • , h C ] ∈ R C×d denotethe hidden embedding of the channels. To emphasize the intra-cluster channels and remove interference from out-of-cluster channel information, we design a modified cross-attention as follows,EQUATIONwhere the clustering membership matrix M is an approximately binary matrix to enable sparse attention on intra-cluster channels specifically. W Q , W K and W V are learnable parameters. The prototype embedding C ∈ R K×d serves as the updated cluster embedding for subsequent clustering probability computing in Eq. 2.Cluster Loss. We further introduce a specifically designed loss function for the clustering quality, termed ClusterLoss, which incorporates both the alignment of channels with respective clusters and the distinctness between different clusters in a self-supervised context. Let S ∈ R C×C denote the channel similarity matrix S ij = SIM(X i , X j ) defined in Eq. 1. The ClusterLoss is formulated as:EQUATIONwhere Tr indicates a trace operator. Tr M ⊤ SM maximizes the channel similarities within clusters, which is a fundamental requirement for effective clustering. Tr I -MM ⊤ S instead encourages separation between clusters, which further prevents overlap and ambiguity in clustering assignments. L C captures meaningful time series prototypes without relying on external labels or annotations. The overall loss function thereby becomes L = L F + βL C , where L F is the general forecasting loss such as MSE loss; and β is a regularization parameter for a balance between forecasting accuracy and cluster quality.Cluster-aware Feed Forward. Instead of using individual Feed Forward per channel in a CI manner or sharing one Feed Forward across all channels in a CD manner, we assign a separate Feed Forward to each cluster to capture the underlying shared time series patterns within the clusters. In this way, we use cluster identity to replace channel identity. Each Feed Forward is parameterized with a single linear layer due to its efficacy in time series forecasting [8, 15, 7] . Let h θ k (•) represent the linear layer for the k-th cluster with weights θ k . Z i represents the hidden embedding of the i-th channel before the last layer. The final forecast is thereby averaged across the outputs of all cluster-aware Feed Forward with {p i,k } as weights, e.g.,Y i = k p i,k h θ k (Z i ) for the i-th channel. For computational efficiency, it is equivalent to Y i = h θ i (Z i ) with averaged weights θ i = k p i,k θ k .Univariate Adaptation. In the context of univariate time series forecasting, we extend the proposed method to clustering on samples. We leverage the similarity between two univariate time series as defined in Eq. 1, and classify univariate time series with comparable patterns into the same cluster. This univariate adaptation allows it to capture interrelation within samples and extract valuable insights from analogous time series. This becomes particularly valuable in situations where meaningful dependencies exist among various univariate samples, such as the stock market.Zero-shot Forecasting. Zero-shot forecasting is useful in time series applications where data privacy concerns restrict the feasibility of training models from scratch for unseen samples. The prototype embeddings acquired during the training phase serve as a compact representation of the pre-trained knowledge and can be harnessed for seamless knowledge transfer to unseen samples or new channels in a zero-shot setting. The pre-trained knowledge is applied to unseen instances by computing the clustering probability distribution on the pre-trained clusters, following Eq. 2, which is subsequently used for averaging cluster-aware Feed Forward. The cross-attention is disabled to fix the prototype embeddings in zero-shot forecasting. It is worth noting that zero-shot forecasting is applicable to both univariate and multivariate scenarios. We refer to Appendix B for detailed discussion.CCM effectively strikes a balance between the CI and CD strategies. On originally CI models, CCM introduces strategic clustering on channels, which not only reduces the model complexity but also enhances their generalizability. Simultaneously, CCM increases the model complexity on originally CD models with negligible overhead for higher capacities. We refer to Figure 5 for empirical analysis. Theoretically, the computational complexity of clustering probability computation (Eq. 2) and the cross-attention (Eq. 3) are O(KCd), where K, C are the number of clusters and channels, respectively, and d is the hidden dimension. One may also use other attention mechanisms [68, 69, 70] for efficiency. The complexity of cluster-aware Feed Forward scales linearly in C, K, and the forecasting horizon H. 
experiments: 
 CCM consistently improves performance based on CI or CD models by significant margins across multiple benchmarks and settings, including long-term forecasting on 9 public multivariate datasets (Sec. 5.2); short-term forecasting on 2 univariate datasets (Sec. 5.3); and zero-shot forecasting in cross-domain and cross-granularity scenarios (Sec. 5.4).Datasets. For long-term forecasting, we experiment on 9 popular benchmarking datasets across diverse domains [11, 12, 71] , including weather, traffic and electricity. M4 dataset [72] is used in short-term forecasting, which is a univariate dataset that covers time series across diverse domains and various sampling frequencies from hourly to yearly. We further provide a new stock time series dataset with 1390 univariate time series. Each time series records the price history of an individual stock spanning 10 years. Due to the potential significant fluctuations in stock performance across different companies, this dataset poses challenges for capturing diverse and evolving stock patterns in financial markets. The statistics of long-and short-term datasets are shown in Table 2 and Table 3 . We follow standard protocols [11, 12, 13] for data splitting on public benchmarking datasets. As for the stock dataset, we divide the set of stocks into train/validation/test sets with a ratio of 7:2:1. Therefore, validation/test sets present unseen samples (i.e., stocks) for model evaluation. This evaluation setting emphasizes the data efficiency aspect of time series models for scenarios where historical data is limited or insufficient for retraining from scratch given unseen instances. More details on datasets are provided in Appendix C.1.Base Models and Experimental Details. CCM is a model-agnostic channel strategy that can be applied to arbitrary time series forecasting models for improved performance. We meticulously select four recent state-of-the-art time series models as base models: TSMixer [7] , DLinear [8] ,PatchTST [21] and TimesNet [13] , which mainly cover three mainstream paradigms, including linear models, transformer-based and convolutional models. For fair evaluation, we use the optimal experiment configuration as provided in the official code to implement both base models and the enhanced version with CCM. All the experiments are implemented with PyTorch on a single NVIDIA RTX A6000 48GB GPU. Experiment configurations and implementations are detailed in Appendix C.3. Experimental results in the following sections are averaged on five runs with different random seeds. Refer to Appendix C.6 for standard deviation results.We report the mean squared error (MSE) and mean absolute error (MAE) on nine real-world datasets for long-term forecasting evaluation in Comparison with Regularization Solution. PRReg [17] recently displays to improve the performance of CD models, by predicting residuals with a regularization term in the training objective. We evaluate the effectiveness of PRReg and our proposed CCM on long-term forecasting performance enhancement based on CI and CD models. Following the previous training setting [17] , we develop CI and CD versions for Linear [8] and Transformer [41] and report MSE loss. Table 5 shows that CCM surpasses PRReg in most cases, highlighting its efficacy compared with regularization solutions. See full results in Appendix C.4. In both M4 and stock datasets, we follow the univariate forecasting setting. For M4 benchmarks, we adopt the evaluation setting in prior works [42] and report the symmetric mean absolute percentage error (SMAPE), mean absolute scaled error (MASE), and overall weighted average (OWA). As for the stock dataset, we implement MAE and MSE as metrics in Table 6 .See more details on metrics in Appendix C.2.Remarkably, the efficacy of CCM is consistent across all M4 sub-datasets with different sampling frequencies. Specifically, CCM outperforms the state-of-the-art linear model (DLinear) by a significant margin of 11.62%, and outperforms the best convolutional method TimesNet by 8.88%. In Table 6 , we also observe a significant performance improvement on the stock dataset, achieved by applying CCM. In the stock dataset, we test on new samples (i.e., univariate stock time series) that the model has not seen during Existing time series models tend to be rigidly tailored to a specific dataset, leading to poor generalization on unseen data. In contrast, CCM leverages learned prototypes to capture cluster-specific knowledge. This enables meaningful comparisons between unseen time series and pre-trained knowledge, facilitating accurate zero-shot forecasting. Following prior work [73] , we adopt ETT collections [11] , where ETTh1 and ETTh2 are hourly recorded, while ETTm1 and ETTm2 are minutely recorded. "1" and "2" indicate two different regions where the datasets originated. Table 7 shows MSE and MAE results on test datasets. CCM consistently improves the zero-shot forecasting capacity of base time series models in 48 scenarios, including generalization to different regions and different granularities. Specifically, based on the results, we make the following observations. (1) CCM exhibits more significant performance improvement with longer forecasting horizons, highlighting the efficacy of memorizing and leveraging pre-trained knowledge in zero-shot forecasting scenarios.(2) CCM demonstrates a better effect on originally CI base models. For instance, the averaged improvement rates on two CI models, i.e., DLinear and PatchTST, are 10.48% and 11.13% respectively, while the improvement rates on TSMixer and TimesNet are 5.14% and 9.63%. Overall, the experimental results verify the superiority and efficacy of CCM in enhancing zero-shot forecasting and the practical value of generalization within closely related domains under varying conditions.Channel Clustering Visualization. The similarity matrix in the lower left further corroborates these findings. Clustering is also observable in ETTh2 dataset, particularly among channels 0, 4, and 5, as well as channels 2, 3, and 6. Comparatively, channel 1 shows a dispersion among clusters, partly due to its capturing of unique or diverse aspects of the data that do not closely align with the features represented by any clusters. The clustering results demonstrate that CCM not only elucidates the intricate relationships and potential redundancies among the channels but also offers critical insights for feature analysis and enhancing the interpretability of time series models.Weight Visualization of Cluster-aware Projection. For instance, in the ETTm1 dataset, Cluster 0 shows bright diagonal striping patterns, which may suggest that it is primarily responsible for capturing the most dominant periodic signals in the corresponding cluster. In contrast, Cluster 1 exhibits denser stripes, indicating its role in refining the representation by capturing more subtle or complex periodicities that the first layer does not. The visualization implies the model's ability to identify and represent periodicity in diverse patterns, which is crucial for time-series forecasting tasks that are characterized by intricate cyclic behaviors. Figure 4 shows an ablation study on cluster ratios, which is defined as the ratio of the number of clusters to the number of channels. 0.0 means all channels are in a single cluster. We observe that the MSE loss slightly decreases and then increases as the cluster ratio increases, especially for DLinear, PatchTST, and TimesNet. Time series models with CCM achieve the best performance when the cluster ratio is in the range of [0.2, 0.6].It is worth noticing that DLinear and PatchTST, two CI models among four base models, benefit consistently from channel clustering with any number of clusters. Additional ablation studies on the look-back window length and clustering step are provided in Appendix D.We evaluate the model size and runtime efficiency of the proposed CCM with various numbers of clusters on ETTh1 dataset, as shown in Figure 5 . The batch size is 32, and the hidden dimension is 64.We keep all other hyperparameters consistent to ensure fair evaluation. It is worth noting that CCM reduces the model complexity based on Channel-Independent models (e.g., PatchTST, DLinear), since CCM essentially uses cluster identity to replace channel identity. The generalizability of CI models is thereby enhanced as well. When it comes to Channel-Dependent models, CCM increases the model complexity with negligible overhead, considering the improved forecasting performance.  
conclusion: 
 This work introduces a novel Channel Clustering Module (CCM) to address the challenge of effective channel management in time series forecasting. CCM strikes a balance between individual channel treatment and capturing cross-channel dependencies by clustering channels based on their intrinsic similarity. Extensive experiments demonstrate the efficacy of CCM in multiple benchmarks, including long-term, short-term, and zero-shot forecasting scenarios. Refinement of the CCM clustering and domain-specific similarity measurement could potentially improve the model performance further. Moreover, it would be valuable to investigate the applicability of CCM in other domains beyond time series forecasting in future works. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: Provided error bars in Appendix C.6.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: Provided details in Appendix C.3.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Discussed the social impacts in Appendix F.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: the paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Cited the original paper that produced the dataset used in this work.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: Essentially, the similarity between two time series X i and X j is defined as SIM(X i , X j ) = exp(EQUATION), where d(•, •) can be any distance metric [74, 75] , such as Euclidean Distance (L p ), Editing Distance (ED) and Dynamic Time Warping (DTW) [76] . One may also use other similarity definitions, such as Longest Common Subsequence (LCSS) and Cross-correlation (CCor).Firstly, the choice of Euclidean distance in this work is motivated by its efficiency and low computational complexity, especially in the case of large datasets or real-time applications. Let H denote the length of the time series. The complexity of the above similarity computation is shown in Table 8 . O(H) O(H 2 ) O(H 2 ) O(H 2 ) O(H 2 )Secondly, it's worth noting that while there are various similarity computation approaches, studies have demonstrated a strong correlation between Euclidean distance and other distance metrics [77] . This high correlation suggests that, despite different mathematical formulations, these metrics often yield similar results when assessing the similarity between time series. This empirical evidence supports the choice of Euclidean distance as a reasonable approximation of similarity for practical purposes. In our implementation, we select σ = 5 in Eq. 1 to compute the similarities based on Euclidean distance.The definitions of Channel Dependent (CD) and Channel Independent (CI) settings are pivotal to this work. The fundamental difference lies in whether a model captures cross-channel information. There are slightly varied interpretations of Channel Independent (CI) in previous works and we summarize as follows.1. In some works [15, 54] , CI is broadly defined as forecasting each channel independently, where cross-channel dependencies are completely ignored. For linear models [78, 16] , CI is specifically defined as n individual linear layers for n channels in previous works. Each linear layer is dedicated to modeling a univariate sequence, with the possibility of differing linear weights across channels. 2. In previous work [17] , CI also means all channels being modeled independently yet under a unified model.All the above works acknowledge that CI strategies often outperform CD approaches, though this comparison is not the focal point of our work. It's also recognized that the specific CI strategy employed in DLinear and PatchTST contributes significantly to their performance. The CI setting in [17] represents a specific instance within the broader CI setting in other works [15, 54] . To avoid ambiguity, we use f (i) to represent the model for the i-th channel specifically, aligning with previous definitions without conflict.We provide pseudocodes for training time series models enhanced with CCM in Algorithm 1. Algorithm 2 displays pseudocodes for the inference phase, where both the training and test sets have the same number of channels. The components in the pretrained model F, highlighted in blue, remain fixed during the inference phase. It's important to note that zero-shot forecasting in Algorithm 2 is adaptable to various scenarios. Let's discuss these scenarios:• Training on a univariate dataset and inferring on either univariate or multivariate samples:In this case, the model learns prototypes from a vast collection of univariate time series in the training set. As a result, the model can effortlessly adapt to forecasting unseen univariate time series in a zero-shot manner. To forecast unseen multivariate time series, we decompose each multivariate sample into multiple univariate samples, where each univariate sample can be processed by the pretrained model. The future multivariate time series can be obtained by stacking multiple future univariate time series.• Training on a multivariate dataset and inferring on either univariate or multivariate samples: For Channel-Dependent models, test samples should have the same number of channels as the training samples, as seen in sub-datasets within ETT collections [11] . In contrast, for Channel-Independent models, zero-shot forecasting can be performed on either univariate or multivariate samples, even when they have different numbers of channels.Algorithm 1 Forward function of time series models with channel clustering module. C is the number of channels in the dataset. K is the number of clusters. T is the length of historical data. H is the forecasting horizon.Input: Historical time series X ∈ R T ×C Output: Future time series Y ∈ R H×C ; Prototype embedding C ∈ R K×d Initialize the weights of K linear layerθ k for k = 1, • • • , K Initialize K cluster embedding c k ∈ R d for k = 1, • • • , K. ▷ Cluster Embedding C X ← Normalize(X) S i,j ← exp( -∥Xi-Xj ∥ 2 2σ 2).▷ Compute Similarity Matrix S h i ← MLP(X i ) for each channel i.▷ Channel Embedding H via MLP Encoder in the Cluster Assigner p i,k ← Normalize(c ⊤ k hi ∥c k ∥∥hi∥ ) ∈ [0, 1]. ▷ Compute Clustering Probability Matrix P M ← Bernoulli(P). ▷ Sample Clustering Membership Matrix M C ← Normalize exp( (W Q C)(W K H) ⊤ √ d ) ⊙ M ⊤ W V H. ▷ Update Cluster Embedding C via Cross Attention Ĥ = Temporal Module(H). ▷ Update via Temporal Modules for channel i in {1, 2, • • • , C} do Y i ← h θ i ( Ĥi ) where θ i = k p i,k θ k .▷ Weight Averaging and Projection end for Algorithm 2 Zero-shot forecasting of time series models with channel clustering module. C is the number of channels in both the training and test datasets. K is the number of clusters. T is the length of historical data. H is the forecasting horizon.Input: Historical time series X ∈ R T ×C ; Pretrained Models F Output: Future time series Y ∈ R H×C ; Load the weights of K linear layerθ k for k = 1, • • • , K from F Load K cluster embedding c k ∈ R d for k = 1, • • • , K from F. ▷ Cluster Embedding C X ← Normalize(X) S i,j ← exp( -∥Xi-Xj ∥ 2 2σ 2 ). ▷ Compute Similarity Matrix S h i ← MLP(X i ) for each channel i. ▷ Channel Embedding H via MLP Encoder in the Cluster Assigner p i,k ← Normalize( c ⊤ k hi ∥c k ∥∥hi∥ ) ∈ [0, 1]. ▷ Compute Clustering Probability Matrix P M ← Bernoulli(P). ▷ Sample Clustering Membership Matrix M Ĥ = Temporal Module(H). ▷ Update via Temporal Modules for channel i in {1, 2, • • • , C} do Y i ← h θ i ( Ĥi ) where θ i = k p i,k θ k .▷ Weight Averaging and Projection end forC.1 Datasets Public Datasets. We utilize nine commonly used datasets for long-term forecasting evaluation. Firstly, ETT collection [11] , which documents the oil temperature and load features of electricity transformers over the period spanning July 2016 to July 2018. This dataset is further subdivided into four sub-datasets, ETThs and ETTms, with hourly and 15-minute sampling frequencies, respectively. s can be 1 or 2, indicating two different regions. Electricity dataset [79] encompasses electricity consumption data from 321 clients from July 2016 to July 2019. Exchange [80] compiles daily exchange rate information from 1990 to 2016. Traffic dataset contains hourly traffic load data from 862 sensors in San Francisco areas from 2015 to 2016. Weather dataset offers a valuable resource with 21 distinct weather indicators, including air temperature and humidity, collected every 10 minutes throughout the year 2021. ILI documents the weekly ratio of influenza-like illness patients relative to the total number of patients, spanning from 2002 to 2021. Dataset statistics can be found in Table 9 .We adopt M4 dataset for short-term forecasting evaluation, which involves 100,000 univariate time series collected from different domains, including finance, industry, etc.The M4 dataset is further divided into 6 sub-datasets, according to the sampling frequency. We implement short-term forecasting on the stock dataset, which is more focused on market sentiment, and short-term events that can cause stock prices to fluctuate over days, weeks, or months. Thereby, we set the forecasting horizon as 7 and 24.Following standard evaluation protocols [13] , we utilize the Mean Absolute Error (MAE) and Mean Square Error(MSE) for long-term and stock price forecasting. The Symmetric Mean Absolute Percentage Error (SMAPE), Mean Absolute Scaled Error (MASE), and Overall Weighted Average (OWA) are used as metrics for M4 dataset [72, 42] . The formulations are shown in Eq. 5. Let y t and y t denote the ground-truth and the forecast at the t-th timestep, respectively. H is the forecasting horizon. In M4 dataset, MASE is a standard scale-free metric, where s is the periodicity of the data (e.g., 12 for monthly recorded sub-dataset) [72] . MASE measures the scaled error w.r.t. the naïve predictor that simply copies the historical records of s periods in the past. Instead, SMAPE scales the error by the average between the forecast and ground truth. Particularly, OWA is an M4-specific metric [81] that assigns different weights to each metric.MAE = 1 H H t=1 |y t -y t | , MSE = 1 H H i=1 (y t -y t ) 2 , SMAPE = 200 H H i=1 |y t -y t | |y t | + | y t | , MASE = 1 H H i=1 |y t -y t | 1 H-s H j=s+1 |y j -y j-s | , OWA = 1 2 SMAPE SMAPE Naïve2 + MASE MASE Naïve2 (5)regularization objective, is proposed to improve the robustness of CD methods as follows.EQUATIONwhere the superscript j indicates the sample index. L F is MSE loss. A (j) = X j L,: represents the last values of each channel in X (j) . Therefore, the objective encourages the model to generate predictions that are close to the nearby historical data and keep the forecasts smooth and robust. The regularization term Ω (L 2 norm in practice) further restricts the complexity of the model and ensures smoothness in the predictions. It was demonstrated that PRReg can achieve better performance than original CD and CI strategies [17] . We conduct extensive experiments on long-term forecasting benchmarks to compare PRReg and CCM. The full results are shown in Table 11 . The numbers in parentheses next to the method represent the forecasting horizon. We set the length of the look-back window to 36 for ILI and 96 for other datasets for consistency with previous works [17] . The base models are linear model [8] and basic transformer [41] . The values in the PRReg column are the best results with any λ, collected from [17] . We observe from Table 11 that CCM successfully improves forecasting performance on original CI/CD strategies (or reached comparable results) in 13 out of 16 settings, compared with PRReg method.We report the degree of multivariate correlation across multiple channels (measured by the average Pearson correlation coefficient) in Table 12 and Table 13 . r denotes the degree of multivariate correlation. Then the Pearson correlation coefficient between r and the performance improvement rate is 0.258 in long-term forecasting, indicating a weak correlation. It demonstrates that CCM tends to achieve a greater boost on datasets that are intrinsically correlated within channels. Compared with datasets used in long-term benchmarks, M4 demonstrates more significant correlations between time series. Therefore, capturing the correlation within the dataset in short-term cases potentially leads to greater improvement in the forecasting performance than in long-term cases. Experimental results in this paper are averaged from five runs with different random seeds. We report the standard deviation for base models and CCM-enhanced versions on long-term forecasting datasets in Table 14 , M4 dataset in Table 15 and stock dataset in Table 16 .The number of clusters is an important hyperparameter in the CCM method. To verify the effectiveness of our design, we conduct an ablation study for all base models on four long-term forecasting datasets.The full results are shown in Table 17 . We tune different cluster ratios, defined as the ratio of the number of clusters to the number of channels. Original means the original base model without any channel clustering mechanism. 0.0 indicates grouping all channels into the same cluster. We make the following observations. (1) For most cases, the channel clustering module (CCM) with any number of clusters greater than 1 consistently improves the forecasting performance upon base models. (2) For Channel-Independent base models, such as DLinear and PatchTST, grouping all channels into one cluster results in a channel fusion at the last layer, leading to a degradation in forecasting performance compared to the original CI models. (3) For most cases, the cluster ratio in the range of [0.2, 0.6] typically results in the best performance.In this section, we conduct additional ablation experiments to investigate the effect of different lookback window lengths in the newly collected stock dataset, which determines how much historical information the time series model uses to make short-term forecasts. Specifically, the ablation study helps identify the risk of overfitting or underfitting based on the chosen look-back window length. An overly long window may lead to overfitting, while a short window may cause underfitting. Table 18 display the forecasting performance on forecasting horizon 7 and 24. The length of the look-back window ranges from two to four times the forecasting horizon. From Table 18 , we make the following observations. (1) CCM effectively improves the base models' forecasting performance in 24 cases across different base models, forecasting horizons, and look-back window lengths consistently. (2) Especially, CCM achieves better enhancement when the look-back window is shorter.We conducted an ablation study on different clustering steps to investigate its effect on downstream performance, reported in Table 19 . The ablation study follows the setting in Table 5 . Random means we randomly assign each channel to a cluster, leading to worse clustering quality. K-Means means using the k-means algorithm to replace our clustering step, leading to suboptimal prototype embedding. The proposed CCM is essentially an advanced variant of K-Means with learnable prototype embedding and cross-attention mechanism. Results show that both clustering quality and We use the model's performance difference on the original dataset and the randomly shuffled dataset as the model's fitting ability on a specific channel. Note that MSE loss is computed on channels that have been standardized, which means that any scaling differences between them have been accounted for. Figure 6 highlights a noteworthy pattern: when two channels exhibit a higher degree of similarity, there tends to be a corresponding similarity in the performance on these channels. This observation suggests that channels with similar characteristics tend to benefit similarly from the optimization. It implies a certain level of consistency in the improvement of MSE loss when dealing with similar channels. This paper presents the Channel Clustering Module (CCM) for enhanced performance of time series forecasting models, aiming to balance the treatment of individual channels while capturing essential cross-channel interactions. Despite its promising contributions, there still exist limitations and directions for future work that warrant consideration.Limitation. While CCM shows improvements in forecasting, its scalability to extremely large datasets remains to be tested. Moreover, the clustering and embedding processes in CCM introduce additional computational overhead. The efficiency of CCM in real-time forecasting scenarios, where computational resources are limited, requires further optimization.Future Works. Future research can focus on adapting CCM to specific domains, such as biomedical signal processing or geospatial data analysis, where external domain-specific knowledge can be involved in the similarity computation. Furthermore, exploring alternative approaches to develop a dynamical clustering mechanism within CCM could potentially enhance forecasting efficacy. It is also worth examining the effectiveness of CCM in contexts beyond time series forecasting.Social Impact. The Channel Clustering Module (CCM) presented in this paper holds significant potential for positive social impact. By improving the accuracy and efficiency of forecasting, CCM can benefit a wide range of applications critical to society. For instance, in healthcare, CCM could enhance the prediction of patient outcomes, leading to better treatment planning and resource allocation. Additionally, in financial markets, CCM could aid in predicting market trends, supporting informed decision-making and potentially reducing economic risks for individuals and organizations. Overall, the development and refinement of CCM could potentially enhance the quality of life and promote societal well-being.NeurIPS Paper ChecklistJustification: well conform with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).The code is available at https://github.com/Graph-and-Geometric-Learning/TimeSeriesCCMhttps://github.com/yuqinie98/PatchTSThttps://github.com/cure-lab/LTSF-Linearhttps://github.com/google-research/google-research/tree/master/tsmixerhttps://github.com/thuml/TimesNet 


Decision: This paper proposed to model the correlation between channels to improve  time series forecasting via an attention-based channel clustering method. Experimental results have demonstrated the effectiveness of the proposed channel clustering approach. Although the channel clustering approach is new, the channel dependency modeling  is not the first time explored in literature, e.g., [1-2]. A detailed discussion is necessary.

[1] Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators

[2] Enhancing Multivariate Time Series Forecasting with Mutual Information-driven Cross-Variable and Temporal Modeling

Some reviewers have concerns on the computation complexity and scaling ability. Authors have provided responses to these questions.

Reviews: review: paper_summary: paper_summary: The paper proposes a new Channel Clustering Module and a corresponding Cluster Loss to group similar channels using a cross-attention mechanism. This creates a hybrid between channel independent and channel dependent approaches. Experiments in the paper compare both with and without the proposed module on a variety of existing state of the art methods and are run on time-series datasets in different domains on both short and long-term time horizons and demonstrate that this approach is generally applicable regardless of the domain. These experiments show an improvement in performance over prior work while also enabling zero-shot forecasting. The clustering produced by this module surfaces relationships between channels which are useful for feature analysis and interpretability.
strength_weakness: strength_weakness: The paper is evaluated on a variety of real-world datasets but evaluation on some synthetic data to highlight scenarios where existing methods produce a higher error but adding the clustering module reduces the error would help solidify the claims of the paper.
questions: questions: - In section 4.1, why shuffle channels in each batch? Channel independent implies each channel is handled individually while the motivating toy experiment here changes the channel every batch. What happens when not shuffling the channels?
- How does the prediction error change when the prediction time horizon is changed between training and inference? Is it possible to modify the time horizon after training?
- In section D.3, how do you use random and k-means for cluster assignment? Additional details about these experiments would help clarify quality improvement described in the table.
- How do you expect the performance of the CCM to change when the channel count is on the order of millions?
- What does a cluster ratio of 1.0 mean?
limitations: limitations: The authors list that the work needs to be tested on large datasets and that there is an additional performance overhead of the proposed CCM module.
paper_summary: paper_summary: The paper presents a new Channel Clustering Module (CCM) for time series forecasting, which dynamically groups channels based on intrinsic similarities to balance the strengths of Channel-Independent (CI) and Channel-Dependent (CD) strategies. CCM improves forecasting accuracy by enhancing model generalization and capturing essential cross-channel interactions, achieving significant performance gains in both long-term and short-term forecasting. The module also supports zero-shot forecasting and improves the interpretability of complex time series models. Extensive experiments demonstrate CCM's effectiveness and adaptability across various mainstream time series models.
strength_weakness: strength_weakness: 1. The introduction of CCM increases the model's complexity, particularly for original CD models, which may result in higher computational overhead.

2. The paper acknowledges that the scalability of CCM to extremely large datasets remains untested, which could be a limitation for practical applications requiring the processing of large-scale data.
questions: questions: 1. How can the similarity metric be further refined to enhance the clustering quality and address performance variations across different domains?

2. What strategies can be employed to test and ensure the scalability of CCM for extremely large datasets?

3. How does CCM perform in real-world applications with diverse and dynamic datasets, and what adjustments are necessary to optimize its performance in such scenarios?

4. What techniques can be implemented to mitigate the increased computational overhead introduced by CCM, especially for models with high channel and cluster counts?
limitations: limitations: 1. CCM does not outperform CI/CD strategies in a few cases, possibly due to the underlying channel relationships in certain real-world domains not aligning well with the similarity metric used by CCM.

2. The increased model complexity introduced by CCM may lead to higher computational costs, particularly for models with numerous channels and clusters.
paper_summary: paper_summary: Time series forecasting has been a topic of interest, with previous studies exploring different strategies. The Channel-Independent (CI) strategy treats channels individually, improving forecasting performance but lacking generalization and ignoring channel interactions. On the other hand, the Channel-Dependent (CD) strategy combines channels indiscriminately, leading to oversmoothing and reduced accuracy. A channel strategy is needed that balances individual treatment and essential interactions. Based on the correlation between performance and channel mixing, a novel Channel Clustering Module (CCM) was developed. CCM groups channels with intrinsic similarities and utilizes cluster information, combining the advantages of CD and CI. Experimental results show that CCM enhances the performance of CI and CD models, enables zero-shot forecasting, and improves interpretability of complex models by uncovering intrinsic patterns among channels.
strength_weakness: strength_weakness: 1. The experimental section involves a limited number of baseline methods, for example, SOTA LLM-based time series prediction models [1, 2] were not selected.
2. I noticed that CCM introduces additional model complexity, as it has an independent Feed Forward layer for each cluster. When the value of K is large, this may result in an excessive number of Feed Forward layers, leading to a significant increase in space complexity. On the other hand, the time complexity of CCM is linearly related to K and C. For certain datasets with a higher number of channels (e.g., Traffic), CCM may noticeably increase the time complexity of the base model. Some of the results in Figure 5 of the Appendix hint at this issue.
3. According to Table 14, the improvements brought by CCM to the base model are sometimes overshadowed by the perturbation caused by randomness. This may indicate that CCM has certain limitations.

[1] Zhou, Tian, et al. "One fits all: Power general time series analysis by pretrained lm." Advances in neural information processing systems 36 (2023): 43322-43355.
[2] Jin, Ming, et al. "Time-llm: Time series forecasting by reprogramming large language models." arXiv preprint arXiv:2310.01728 (2023).
questions: questions: 1. Can the author provide the percentage increase in runtime and memory usage after integrating CCM for each method on the Traffic dataset?
2. Can a certain hyperparameter analysis be conducted for the parameter β (i.e., β ∈ {0, 0.25, 0.5, 0.75, 1.0}) to demonstrate the impact of the Cluster Loss on model performance?
3.  Can the author provide a significance test (i.e., p-value test) for the experimental results, especially for cases where the values in Table 4 are close? As in some cases, the improvements are minimal, and the observed performance gains might be due to randomness rather than the effectiveness of CCM.
limitations: limitations: The author elaborates on the limitations of their work. The main limitations are as follows: CCM does not outperform CI/CD in a few cases in the long-term forecasting benchmark; CCM’s scalability to extremely large datasets remains to be tested.
paper_summary: paper_summary: The paper introduces the Channel Clustering Module (CCM), a novel approach to enhance time series forecasting models. CCM addresses the limitations of traditional Channel-Independent (CI) and Channel-Dependent (CD) strategies by dynamically clustering channels based on their intrinsic similarities. This approach allows the model to balance individual channel treatment with capturing essential cross-channel dependencies, leading to improved forecasting performance. CCM is adaptable to various time series models and demonstrates its effectiveness through extensive experiments on multiple real-world datasets.
strength_weakness: strength_weakness: - The paper could benefit from more detailed discussions on the selection of similarity metrics and the impact of hyperparameters on performance.
- The computational efficiency of CCM, especially in large-scale applications, is not extensively discussed.
questions: questions: - Where is the official code?
- In terms of thinking, the principal component analysis method is actually similar to the proposed method. Please provide a detailed explanation.
limitations: limitations: The limitation of the Channel Clustering Module (CCM) outlined in the paper includes its scalability to extremely large datasets and the computational overhead introduced by the clustering and embedding processes. While CCM shows improvements in forecasting, its efficiency in real-time forecasting scenarios with limited computational resources remains to be optimized. Additionally, the clustering and embedding processes in CCM introduce additional computational overhead, which could be a concern in scenarios where computational efficiency is critical.

Comments: title: We would like to hear from Reviewer SRyH
We would like to hear from Reviewer rxdY
We would like to hear from Reviewer yXcm
We would like to hear from Reviewer 9puR
Rebuttal Summary by Authors
comment: Thank you for the rebuttal. These have addressed most of my concerns.
Thank you for the rebuttal. The clarifications provided have mitigated some concerns.
Thanks for the authors' rebuttal. Since the authors well addressed my concerns, I will raise my score.
I will complete my response promptly. Thank you for the reminder.
Dear Reviewer SRyH,

As the discussion period is close to end, we would like to follow up and ensure that our responses have adequately addressed your concerns. 

We sincerely appreciate your comments and suggestions, which have significantly contributed to improving our paper. In response to your valuable feedback, we have added more discussion on the model efficiency and clarified the difference between PCA and our proposed CCM. We would be more than happy to further discuss if there are any remaining questions. Thanks again for your time and consideration.

Regards,\
Authors
Dear Reviewer rxdY,

As the discussion period is close to end, we would like to follow up and ensure that our responses have adequately addressed your concerns. We sincerely appreciate your comments and suggestions, which have significantly contributed to improving our paper. In response to your valuable feedback, we conducted additional experiments on significance test, complexity evaluation and hyperparameter ablation study. We would be more than happy to further discuss if there are any remaining questions. Thanks again for your time and consideration.

Regards,\
Authors
Dear Reviewer yXcm,

We sincerely appreciate your comments and suggestions.  As the discussion period is close to end, we would like to follow up and ensure that our responses have adequately addressed your concerns. 

We would like to emphasize that **most of the issues raised were minor and had already been addressed or discussed in the original submission and appendix.** Nevertheless, we have taken this opportunity to clarify these points and further improve the revision. Specifically, we have added more discussion on the model efficiency and potentially improved similarity metric (please refer to the author rebuttal and global comments). 

We would be more than happy to further discuss if there are any remaining questions. Thanks again for your time and consideration.

Regards, \
Authors
Dear Reviewer 9puR,

As the discussion period is close to end, we would like to follow up and ensure that our responses have adequately addressed your concerns. We sincerely appreciate your comments and suggestions, which have significantly contributed to improving our paper. We would be more than happy to further discuss if there are any remaining questions. Thanks again for your time and consideration.

Regards,
Authors
Dear Reviewers,

We sincerely appreciate your thorough and valuable feedback on our manuscript. We have provided detailed clarifications and additional experimental results to address all concerns raised. 

## I. Contributions
We are pleased that many reviewers have offered positive comments, and we would like to highlight our key contributions:

- **Generalization and Comprehensive Evaluation:** CCM is a model-agnostic channel strategy that is adaptable to most mainstream time series models. Experiments are thoroughly conducted across multiple different models, datasets, forecasting scenarios. [Reviewers 9puR, rxdY, SRyH]
- **Superior Performance Improvement:** Our extensive experiments demonstrate CCM's effectiveness in significantly enhancing base model performance for time series forecasting across multiple benchmarks. [Reviewers 9puR,  rxdY]
- **Zero-shot Capability:**  CCM supports zero-shot forecasting by leveraging learned prototypes, which is crucial for handling challenging scenarios with scarce or limited training data. [Reviewers yXcm, rxdY]
- **Clear Motivation and Novelty:** CCM addresses limitations of existing strategies (CD and CI) on suboptimal performance or insufficient channel interactions, through a novel approach to channel clustering for time series forecasting. We appreciate the reviewers' acknowledgment of our manuscript's clarity and readability. [Reviewers 9puR, SRyH]

## II. Common Question
We also recognize that there are some common questions from reviewers. We conclude the important clarifications as below.

> “CCM’s scalability to extremely large datasets remains to be tested” as mentioned in the Limitation section.

- We included **theoretical analysis (Sec 4.3)** and **empirical test (Appendix C.6)** for CCM’s complexity and efficiency. The complexity is linear w.r.t. number of channels. The modest computational overhead on CD models is reasonable and acceptable, as justified by the significant performance improvements achieved. It is worth noting that CCM consistently reduces the model complexity of CI models.
- Our current research prioritizes time series benchmarks used in prior works [1,2,3,4]. Our experiments include a traffic dataset comprising 15,122,928 observations and a stock dataset comprising 13,900,000 observations. These datasets represent large-scale real-world datasets widely used in previous research within this domain. 
- Scalability should not be an issue of CCM in real-world deployment due to several empirical evidence:  
    * Consistent reduction in model complexity for CI models
    * Linear scaling with respect to the number of channels, enabling efficient handling of high-dimensional data
    * Maintained performance gains across different dataset sizes


## III. Main modifications in our revised version
According to reviewers’ suggestions, we have made the following improvements in our revised version:
- We added implementation details of the k-means algorithm in ablation studies in Appendix D.3.
- We provided significance tests with p-value for cases where values are close before/after applying CCM.
- We conducted ablation studies on the parameter $\beta$ in the loss function to demonstrate CCM’s robustness.

We hope these modifications and clarifications address your concerns while highlighting the contributions and significance of our work. We are confident that CCM represents a valuable advancement in time series forecasting and look forward to further constructive discussions. Thank you for your time and consideration.


Sincerely,\
Authors



[1] TSMixer: An All-MLP Architecture for Time Series Forecasting\
[2] Are Transformers Effective for Time Series Forecasting?\
[3] A Time Series is Worth 64 Words: Long-term Forecasting with Transformers\
[4] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis