From Similarity to Superiority: Channel Clustering for Time Series Forecasting
Authors: Jialin Chen
Jan Eric Lenssen
Aosong Feng
Weihua Hu
Matthias Fey
Leandros Tassiulas
Jure Leskovec
Rex Ying
Venue: NeurIPS 2024 poster (2024)
Keywords: Deep Learning
Time Series Forecasting
Primary area: deep_learning_architectures

Abstract: Time series forecasting has attracted significant attention in recent decades.
    Previous studies have demonstrated that the Channel-Independent (CI) strategy improves forecasting performance by treating different channels individually, while it leads to poor generalization on unseen instances and ignores potentially necessary interactions between channels. Conversely, the Channel-Dependent (CD) strategy mixes all channels with even irrelevant and indiscriminate information, which, however, results in oversmoothing issues and limits forecasting accuracy.
    There is a lack of channel strategy that effectively balances individual channel treatment for improved forecasting performance without overlooking essential interactions between channels. Motivated by our observation of a correlation between the time series model's performance boost against channel mixing and the intrinsic similarity on a pair of channels, we developed a novel and adaptable \textbf{C}hannel \textbf{C}lustering \textbf{M}odule (CCM). CCM dynamically groups channels characterized by intrinsic similarities and leverages cluster information instead of individual channel identities, combining the best of CD and CI worlds. Extensive experiments on real-world datasets demonstrate that CCM can (1) boost the performance of CI and CD models by an average margin of 2.4% and 7.2% on long-term and short-term forecasting, respectively; (2) enable zero-shot forecasting with mainstream time series forecasting models; (3) uncover intrinsic time series patterns among channels and improve interpretability of complex time series models.

From Similarity to Superiority: Channel Clustering for Time Series Forecasting
Jialin Chen', Jan Eric Lenssen Aosong Feng', Weihua Hu², Matthias Fey?, Leandros Tassiulas' Jure Leskovec2*, Rex Ying!
Yale University, AK Max Planck Institute for Informatics, *Stanford University
Abstract
Time series forecasting has attracted significant attention in recent decades. Previ- ous studies have demonstrated that the Channel-Independent (CI) strategy improves forecasting performance by treating different channels individually, while leads poor generalization on unseen instances and ignores potentially necessary in- teractions between channels. Conversely, the Dhannel-Dependent (CD) strategy mixes all channels with even irrelevant and indiscriminate information, which, however, results in oversmoothing issues and limits forecasting accuracy. There is lack of channel strategy that effectively balances individual channel treatment for improved forecasting performance without overlooking essential interactions between channels. Motivated by our observation of correlation between the time series model's performance boost against channel mixing and the intrinsic similarity on a pair of channels, we developed a novel and adaptable Channel Clustering Module (CCM). CCM dynamically groups channels characterized by intrinsic similarities and leverages cluster information instead of individual channel identities, combining the best of CD and CI worlds. Extensive experiments on real-world datasets demonstrate that CCM can (1) boost the performance of CI and CD models by an average margin of 2.4% and 7.2% on long-term and short-term forecasting, respectively; (2) enable zero-shot forecasting with mainstream time series forecasting models; (3) uncover intrinsic time series pattemss among channels and improve interpretability of complex time series models
Introduction
Time series forecasting has attracted a surge of interest across diverse fields, ranging from economics, energy [1, 2], weather [3,4], to transportation planning [5, 6]. The complexity of the task is heightened by factors including seasonality, trend, noise in the data, and potential cross-channel information.
Despite the numerous deep learning time series models proposed recently [7, 8, 9, 10, 11, 12, 13, 14], an unresolved challenge persists in the effective management of channel interaction within the forecasting framework [15, 16]. Previous works have explored two primary channel strategies: Channel-Independeen (CI) and Channel-Dependent (CD) strategies. The Grtanne--Indpendden (CD) strategy has shown promise in better forecasting performance by having individual models for each channel. However, critical drawback is its limited generalizability and robustness on unseen channels [17]. Besides, it tends to overlook potential interactions between various channels. Conversely. the Channel-Dependent (CD) strategy models all channels
whole and captures intricate channel relations, while they tend to show oversmoothing and have trouble fitting to individual channels, especially when the similarity between channels is very low. Moreover, existing
The code is available at http:://s..ttt.cooccc/.....ccatt
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

[b) The General Framework
[b) Time Series Forecasting Framework Enhanced by Channel Clustering Module
Triniing
Zero-shot Forecasting
future forecasting V
Claster-aware Feed Forward
Feed Forward
Wisgts Avcouiig
Temporal Modules
Temporal Modules
Assigner
Temporal Modules
(prcccdpe curseen scet. (om))222..0)
1ID Hieetto
Nomalization
Normaliatiion
Normaliztiion
unseen time series whYh
The pipeline of applying Channel Clustering Module (CCM) to general time series models. (a) is the general framework of most time series models. (b) illustrates two modified modules when applying CCM: Cluster Assigner and Cluster-aware Feed Forward. Cluster Assigner learns channel clustering based intrinsic similarities and creates prototype embeddings for each cluster via cross-attention mechanism. The clustering probabilities { are subsequently used in Cluster-aware Feed Forward to average {0x, k= 10x)==1> which are layer weights assigned to clusters, obtaining weights 8¹ for the i-th channel. The learned prototypes retain pre-trained knowledge, enabling zero-shot forecasting on unseen samples in both univariate and multivariate scenarios.
Figure I:
models typically treat univariate data in a CI manner, neglecting the interconnections between time series samples, even though these dependencies are commonly observed and beneficial in real-world scenarios, such as stock market or weather forecasting [18, 19, 20].
Proposed work. To address the aforementioned challenges, we propose a Channel Clustering Module (CCM) that balances individual channel treatment and captures necessary cross-channel dependencies simultancously. CCM motivated by the key observations that CI and CD models typically rely on channel identity information. The level of reliance is anti-correlated with the similarity between channels (see Sec. 4.1 for an analysis). This intriguing phenomenon alludes to the model's 's analogous behavior on similar channels. The proposed CCM thereby involves the strategic clustering of channels into cohesive clusters, where intra-cluster channels exhibit higher degree of similarity. To capture the underlying time series patterns within these clusters, employ cluster-aware Feed Forward to assign independent weights to each cluster and replace individual channel treatment with individual cluster treatment. Moreover, CCM learns expressive prototype embeddings in training, which enables zero-shot forecasting on unseen samples by grouping them into appropriate clusters.
CCM is a blug-and-play solution that is adaptable to most mainstream time series models. We evaluate the effectiveness of CCM on four different time series backbones (aka. base models): TSMixer [7], DLinear [8], PatchTST [21], and TimesNet [13]. It can also be applied to other state-of- the-art models for enhanced performance. Extensive experiments verify the superiority of CCM in long-term and short-term forecasting benchmarks, achieving an average margin of 2.4% and 7.2%, respectively. Additionally, we collect stock data from a diverse range of companies to construct new stock univariate dataset. Leveraging information from intra-cluster samples, CCM consistently shows a stronger ability to accurately forecast stock prices in the dynamic and intricate stock market. Moreover, CCM enhances zero-shot forecasting capacities of time series backbones in cross-domain scenarios, which further highlights the robustness and versatility CCM.
The contributions of this paper are: (1) We propose a novel and unified channel strategy, i.e., CCM, which is adaptable to most mainstream time series models. CCM explores the optimal trade-off between channel individual treatment and cross-channel modeling, (2) CCM demonstrates superiority in improving performance on long-term and short-term forecasting, and (3) through learning prototypes from clusters, CCM enables zero-shot forecasting on unseen samples in both univariate and multivariate scenarios.
2
Related Work
2.1 Time Series Forecasting Models
Traditional machine learning methods such as Prophet [22, 23), ARIMA [24] capture the trend component and scasonality in time series [25]. As data availability continues to grow, deep learning

---

methods revolutionized this field, introducing more complex and efficient models [26, 27]. Convo- lutional Neural Networks (CNNs) [13, 14, 28, 29, 30), have been widely adopted to capture local temporal dependencies. Recurrent Neural Networks (RNNs) [31,32, 33, 34, 28] excel in capturing sequential information, yet they often struggle with longer sequences. Transformer-haan based mod- els [11,35, 12, 21, 36, 37, 38, 10, 39, 40], typically equipped with self-attention mechanisms [41], demonstrate their proficiency in handling long-range dependencies, although they require substantial computational resources. Recently, linear models [42, 43, 44], e.g., DLinear [8], TSMixer [7], have gained popularity for their simplicity and effectiveness in long-term time series forecasting, but they may underperform with non-linear and complex patterns. Besides, traditional tricks, including seand-saasonal decomposition [8, 45, 46] and multi multi-periodicity analysis [47, 48, 13, 49, 50, 51, 52] continue to play crucial role in aiding in the preprocessing stage for advanced models.
2.2 Channel Strategies in Time Series Forecasting
Most deep learning models [12, 39, 10] adopt the Channel-Dependerg (CD) strategy, aiming to harness the full spectrum of information across channels. Conversely, the Channel- Indeeeedendeeeenntt (CI) approaches [21, 8] build forecasting models for each channel independently. Prior works on CI and CD strategy [17, 15, 54, 16] present that CI leads to higher capacity and lower robustness, whereas CD is the opposite. Predicting residuals with regularization (PRReg) [17] is thereby proposed to incorporate a regularization term in the objective to encourage smoothness in future forecasting, However, the essential challenge from the model design perspective has not been solved and it remains challenging to develop a balanced channel strategy. Prior research has explored effective clustering of channels to improve the predictive capabilities in diverse applications, including image classification [55], natural language processing (NLP) [56, 57], anomaly detection [58, 59, 60]. For instance, in traffic prediction [61, 62]. clustering techniques have been proposed to group relevant traffic regions to capture intricate spatial patterns. Despite the considerable progress in these areas, the potential and effect of channel clustering in time series forecasting remain under-explored.
3 Preliminaries
Time Series Forecasting. Formally, let X =[x ær] E TxCC be time series, where I is the length of historical data, 2, E R2C represents the observation at time t. C' denotes the number of variates (i.e., channels). The objective is to construct predictive model that estimates the future values of the series, Y = [#r+1..... #r E IR #R#xC where is the forecasting horizon. We use [(.4] E R (X, for simplicity) denote the i-th channel in the time series.
Channel Dependent (CD) and Channel Independent (CI). The CI strategy models each channel X, separately and ignores any potential cross-channel interactions. This approach is typically denoted as f()) RRT IR for i 1,.... , C, where f() is specifically dedicated to the i-th channel. Refer to Appendix A.2 for more details. In contrast, the CD strategy models all the channels as a whole with function f TT HxC This strategy is essential in scenarios where channels not just parallel data streams but are interrelated, such as in financial markets or traffic flows.
4 Proposed Method
In this work, we propose a Channel Clustering Module (CCM), a model-agnostic method that is adaptable to most mainstream time series models. The pipeline of applying CCM is visualized in Figure 1. General time series models, shown in Figure I(a), typically consist of three core components [15, 63]: an optional normalization layer (e.g., RevIN [64]. SAN [65]), temporal modules including linear layers, transformer-basee that forecasts the future values. Motivated by the empirical observation discussed in Sec. 4.1, CCM presents with a cluster assigner preceding the temporal modules, followed by a cluster-aware Feed Forward (Sec. 4.2). The cluster assigner implements channel clustering based on intrinsic similarities and employs cross-attention mechanism to generate prototypes for each cluster, which stores the knowledge from the training set and endows the model with zero-shot forecasting capacities.
or convolutional backbones, and a ferwaror layer
3

---

4.1 Motivation for Channel Similarity
To motivate our similarity- -bsedd clustering method, we conduct the following toy experiment. We select four recent and popular time series models with different backbones. TSMixer [7] and DLinear are linear models. PatchTST [21] is a transformer-
model with patching mechanism and TimesNet [13] is convolutional network that captures multi multi-periodicity in data. Among these, TSMixer and TimesNet utilize a Channel-Dependent strategy while DLinear and PatchTTST adopt the Graannel-Independent design. We train time series model across all channels and evaluate the -hannel-wise Mean Squared Error (MSE) loss on the test set. Then, we repeat training while randomly shuffling channels in each batch. Note that for both CD and CI models, this means channel identity information will be removed. We report the average performance gain in terms of MSE loss across all channels based on the random shuffling experiments (denoted as AL(%)) in Table 1. We attribute the models' performance decrease in the random shuffling experiments to the loss of channel identity information. We see that all models rely on channel identity information to achieve better performance. Next, we define channel similarity based on radial basis function kernels [66] as
Table Averaged performance gain from channel identity information (AC(%)) and Pearson Corre- lation Coefficients (PCC) between {ACij Ji.j and (SIM(X,, X,)Ji.- The values are averaged across
all test samples.
Model
TSMixer CD
DLincar C/
PatchTST C/
TimesNot
(I,(X,, exp( || - 13
Channel Strategy
CD
(1)
(C()
2.67
1,10
11.30
18.90
20
PCC
-.67
0.66
-0.61
-0.66
where a a scaling factor. Note that the similar- ity is computed on the standardized time series to avoid scaling differences. More details are discussed in Appendix A.I. The performance difference in MSE from the random shuffling experiment for channel is denoted as AC,. We define AC = AC, Agy and calculate the Pearson Correlation Coefficients (PCC) between (ACi, J and (X,(X, X,) as shown in Table 1. The toy example verifies the following two assumptions: (1) Existing forecasting methods heavily rely on channel identity information.
ETTml
A.(%)
PCC
4.41
-0688
-067
5.55
6.83
-0.88
14.98
-0.67
Exchange
AC())
PCC
16.43
19.34
27.98
24.57
(2) This reliance clearly anti-correlates with channel similarity: for channels with high similarity, channel identity information is less important. Together, these two assumptions motivate us to design an approach that provides cluster identity instead of channel identity, combining the best of both worlds: high capacity and generalizability.
4.2 CCM: Channel Clustering Module
Channel Clustering. Motivated by the above observations. we first initialize set of K cluster embeddings (c1.... . where ck Idd d is the hidden dimension and K a hyperparameter. Given a multivariate time series X E TCC each channel in the input X, is transformed into d-dimensional channel embedding h; through an MIP. The probability that a given channel X; is associated with the k-th cluster is the normalized inner-product of the cluster embedding and the channel embedding h4, which computed as
CK hi |c bb]ll
Pi, Normalize
E [0.
The normalization operator ensures that Σx Pt.k = 1 and validates the clustering probability distribu- tion actoss & clusters. We utilize reparamcterization trick [67] to obtain the clustering membership matrix M E RCX where M, x Bernoulli (p.,)). Higher probability pt,x k results in M, close to I, leading to the deterministic existence of certain channels in the corresponding cluster.
Prototype Learning. The cluster assigner also creates a d-dimensional prototype embedding for each cluster in the training phase. Let C [ci, ,ck] R denote the cluster embedding, and H .h..... hc] CC'd denote the hidden embedding of the channels. To emphasize the intra-cluster channels and remove interference from out-of-cluster channel information, we design a modified cross-attention follows,
( (WQC)(WHH) C M WvH.
Č = Normalize (cxp(
(3)
vd

---

where the clustering membership matrix M is an approximately binary matrix to enable sparse attention on intra-cluster channels specifically. Wo. Wx and Wr are learnable parameters. The prototype embedding Č E Ig* serves as the updated cluster embedding for subsequent clustering probability computing in Eq. 2.
Cluster Loss. We further introduce specifically designed loss function for the clustering quality, termed ClusterLooss, which incorporates both the alignment of channels with respective clusters and the distinctness between different clusters in self-supervised context. Let S E RCCC denote the channel similarity matrix S = SIM(X, X;) defined in Eq. 1. The ClusterLoss is formulated as:
Lc = Tr (M' SM) + Tr ((I MM") S),
where Tr indicates a trace operator. Tr (M' SM) maximizes the channel similarities within clusters, which is a fundamental requirement for effective clustering. Tr (MMM) S) instead encourages separation between clusters, which further prevents overlap and ambiguity in clustering assignments. Le captures meaningful time series prototypes without relying on external labels or annotations. The overall loss function thereby becomes & = L + fLc, where LF is the general forecasting loss such as MSE loss; and /3 is regularization parameter for a balance between forecasting accuracy and cluster quality.
Cluster-aware Feed Forward. Instead of using individual Feed Forward per channel in CI manner or sharing one Feed Forward across all channels CD manner, we assign a separate Feed Forward to each cluster to capture the underlying shared time series patterns within the clusters. In this way, we use cluster identity to replace channel identity. Each Feed Forward parameterized with a single linear layer due to its efficacy in time series forecasting [8, 15, 7J. Let hag (-) represent the linear layer for the k-th cluster with weights 0g. Z; represents the hidden embedding of the i-th channel before the last layer. The final forecast is thereby averaged across the outputs of all cluster-awand Feed Forward with {Pr,e) as weights, e.g., Y; = Σn Pi, (Z,) for the i-th channel. For computational efficiency, is equivalent to Y; has (Z.) with averaged weights 0 = Σx Pi,KOE.
Univariate Adaptation. In the context of univariate time series forecasting, extend the proposed method to clustering on samples. We leverage the similarity between two univariate time series as defined in Eq. 1, and classify univariate time series with comparable patterns into the same cluster. This univariate adaptation allows it to capture interrelation within samples and extract valuable insights from analogous time series. This becomes particularly valuable in situations where meaningful dependencies exist among various univariate samples, such as the stock market.
Zero-shot Forecasting, Zero-shot forecasting useful in time series applications where data privacy concerns restrict the feasibility of training models from scratch for unseen samples. The prototype embeddings acquired during the training phase serve as compact representation of the pre-trained knowledge and can be harnessed for seamless knowledge transfer to unseen samples or new channels in zero-shot setting. The pre-trained knowledge is applied to unseen instances by computing the clustering probability distribution on the pre-trained clusters, following Eq. 2, which is subsequently used for averaging cluster-aware Feed Forward. The cross-attention is disabled to fix the prototype embeddings in zero-shot forecasting. It is worth noting that zero-shot forecasting is applicable to both univariate and multivariate scenarios. We refer to Appendix B for detailed discussion.
4.3 Complexity Analysis
CCM effectively strikes a balance between the CI and CD strategies. On originally CI models, CCM introduces strategic clustering on channels, which not only reduces the model complexity but also enhances their generalizability. Simultaneously, CCM increases the model complexity on originally CD models with negligible overhead for higher capacities. We refer to Figure 5 for empirical analysis. Theoretically, the computational complexity of clustering probability computation (Eq. 2) and the cross-attention (Eq. 3) are O(C)d), where K, C are the number of clusters and channels, respectively, and d is the hidden dimension. One may also use other attention mechanisms [68, 69, 70] for efficiency. The complexity of cluster-aware Feed Forward scales linearly in C, K, and the forecasting horizon H.
5

---

5
Experiments
CCM consistently improves performance based on CI or CD models by significant margins across multiple benchmarks and settings, including long-term forecasting on public multivariate datasets (Sec. 5.2); short-term forecasting on 2 univariate datasets (Sec. 5.3); and zero-shot forecasting in cross-domain and cross-grauularity scenarios 5.4).
5.1 Experimental Setup
Datasets. For long-term forecasting, we experiment on 9 popular benchmarking datasets across diverse domains [11, 12, 71], including weather, traffic and electricity. M4 dataset [72] is used in short-term forecasting, which is univariate dataset that covers time series across diverse domains and various sampling frequencies from hourly to yearly. We further provide new stock time series dataset with 1390 univariate time series. Each time series records the price history of an individual stock spanning 10 years. Due to the potential significant fluctuations in stock performance across different companies, this dataset poses challenges for capturing diverse and evol ving stock patterns in financial markets. The statistics of long- and short-term datasets are shown in Table and Table 3.
Table 2: The statistics of datasets in long-term forecasting. Horizon is (96, 192, 336, 720).
Table 3: Dataset details of M4 and Stock in short- term forecasting.
Dataset
Channels
Length
Frequency
Dataset
Length
Horizon
ETTH
7
17420
hour
M4 Yearly
23000
6
ETTmI&ETTTm
7
69680
15 min
M4 Quarterly
24000
8
ILI
966
I week
M4 Monthly
48000
18
Exchange
7588
I day
M4 Weekly
359
Weather
21
52696
10 min
M4 Daily
4227
Electricity
321
26304
hour
M4 Hourly
414
48
Traffic
862
17544
hour
Stock (New)
10000
7/24
We follow standard protocols [11, 12, 13] for data splitting on public benchmarking datasets. As for the stock dataset, we divide the set of stocks into train/validationntest sets with a ratio of 7:2:1. Therefore, validation/test sets present unseen samples (Le., stocks) for model evaluation. This evaluation setting emphasizes the data efficiency aspect of time series models for scenarios where historical data is limited or insufficient for retraining from scratch given unseen instances. More details on datasets are provided in Appendix C.I.
Base Models and Experimental Details. CCM is a model-agnostir channel strategy that can be applied to arbitrary time series forecasting models for improved performance. We meticulously select four recent tf time series models as base models: TSMixer [7], DLinear [8], PatchTST [21] and TimesNet [13]. which mainly cover three mainstream paradigms, including linear models, transformer-baat based and convolutional models. For fair evaluation, we use the optimal experi- ment configuration as provided in the official code to implement both base models and the enhanced version with CCM. All the experiments are implemented with Py PoTcch on single NVIDIA RTX A6000 48GB GPU. Experiment configurations and implementations are detailed in Appendix C.3. Experimental results in the following sections are averaged on five runs with different random seeds. Refer to Appendix C.6 for standard deviation results.
5.2 Long-term Forecasting Results
We report the mean squared error (MSE) and mean absolute error (MAE) on nine real-world datasets for long-term forecasting evaluation in Table 2. The forecasting horizon is {96, 192, 336, 720). From the table, we observe that the model enhanced with CCM outperforms the base model in general. Specifically, CCM improves long-term forecasting performance in 90.27% cases in MSE and 84.03% cases in MAE across 144 different experiment settings. Remarkably, CCM achieves a substantial boost on DLinear, with a significant reduction on MSE by 5.12% and MAE by 3.04%. The last column the table quantifies the average percentage improvement in terms of MSE/MAE, which underscores the consistent enhancement brought by CCM across all forecasting horizons and datasets. Intuitively, the CCM method is more useful in scenarios where channel interactions are complex and significant, which is usually the case in real-world data. See more analysis in Appendix C.5.

---

Table 4: Long-term term forecasting results on real-world datasets in terms of MSE and MAE, the lower the better. The forecasting horizons are {96, 192, 336, 720]. The better performance in each setting shown in bold. The best results for each row are underlined. The last column shows the average percentage of MSE/MAF improvement of CCM over four base models.
Model Mettcc
MSE
TSMixer
MAE
MSE
MSE
Diiiear
MAB
MAE
PachIST MAB
MSE
MAB
MSB
TimesNet
MAE
MSE
.CCM
MAE
IMP/
Traftle Hecetiiit Weather 11.1 Kochange ETT12 ETTM ET1ml KTTTI
0.361
9.404
0.415
0.392
0.402
0.575
0.425
0.399
0.416
0.313
0.15
0.398
0.425
0.371
0.414
0.39%
0.384
0.466
0.431
0.380
0,400
0.425
0.442
0.534
0.422
0.645
0.420
0.433
0.640
0.317
0.491
0.485
0.461
0.908
0.463
0.472
0.463
0.689
0.488
0.497
0.468
0.337
0.521
0.520
0.493
0.333
0.285
0.283
0.343
0.345
0.289
0.338
0.335
0.371
1.123
0.335
0.365
0.365
0,333
0.376
0.373
0.383
0.482
0.361
0.370
0.400
0.412
0415
0.716
0.423
0.47
0.419
0.478
1.852
0.346
0.276
0.540
6.274
0.40
6.336
0.371
0.319
0.418
0,13
0,39
0,402
0.410
0.361
0.406
0.44
0.449
0.445
0.464
0.549
0.496
0,462
0.461
4,570
0.171
0.260
0.158
0.168
0.35%
0,185
0.189
0.270
0.660
0.221
0.330
0.384
0,353
0.333
0,251
1230
0.250
0.910
3.459
0.276
0.369
0.427
0.358
6351
6.315
0.347
0.420
B.36
0300
0.554
0.522
0.45%
0,374
0.394
7.139
96
11200
0,215
0.314
0
0,107
6.106
1.251
2,680
0.103
0.315
1300
0.185
0.256
0.234
0.340
3
0.343
1.421
0311
0.462
0.342
0337
0.361
5.875
724
0.898
0.710
0.966
0.455
0.813
0.739
5,570
24
1.914
0.579
1958
1.081
1935
0.935
1,561
0.759
2317
2.139
0.956
4,483
36
1.805
11356
1300
2.142
0.977
0.943
1.706
0,920
1.968
0.914
2.561
48
1.797
11973
1.7%
0867
2.335
1.221
1.366
1.79
0.906
1.514
0381
1250
0.440
1.229
0.957
1,602
1.859
0.395
1310
0.7
2.479
1.088
2.361
1.086
1,50
0.943
1.755
0.36
1002
0,928
2.041
0.930
2.691
96
0.149
0.245
0.147
0.17
6.169
0.315
1.729
0.201
0.245
0.193
0141
0.306
0.297
0.340
0.285
0.198
0.341
0.11
0.33
0210
0.261
6.315
0.357
2.539
0.264
0.291
0.244
0281
0.294
0,335
0.274
0.324
0211
0.382
0.345
0.36
0330
0.306
6.274
0.291
2.924
0.330
0.336
0316
0.339
0,574
0.320
0.357
0,300
0.314
0.316
0,33
0348
0,389
0.366
0.362
1,476
96
0.142
0.237
0.139
0.135
0,153
0.239
0.142
0.247
0.138
0.233
0.136
01331
0.168
0,272
0.158
0.259
2.480
192
0.154
0.265
0.47
0.266
0.158
0,251
0.151
0.248
0,153
0.247
0.153
0.26
0.184
0.259
6,172
0.362
3.226
336
0.163
0.264
0161
0.261
0.170
0.268
0.168
0.267
0,170
0.263
0.168
0.26
0.198
0,300
6.181
0.284
2.423
0.208
0.300
0.204
0.299
0.233
0.342
0.230
0.338
0,306
0.2%6
0,210
10
0.20
0,300
0.205
0.309
1.417
96
0.376
112664
0.375
0.262
0.411
0.254
0.411
0.282
0.360
0.24
0.357
1.246
0.593
0.321
0.554
I.316
1,488
192
0.397
0,277
0.340
0.270
0.423
0.287
0.421
0.286
0.399
0.256
0.39
0.3
0617
0.316
0,562
0.331
3.175
316
0.413
0.390
0.411
0.289
0.418
0.299
0.436
0.297
0.401
0.270
0.88
0.69
0.336
0.579
0.341
2,120
720
0.444
0.306
0.441
0.302
0.467
0.316
0.471
0.318
0.443
0.204
0.430
0.28
0.640
0.350
0.587
0.365
1.445
Comparison with Regularization Solution. PRReg [17] recently displays to improve the perfor- mance of CD models, by predicting residuals with a regularization term in the training objective. We evaluate the effectiveness of PRReg and our proposed CCM on long-term forecasting performance enhancement based on CI and CD models. Following the previous training setting [17], we develop CI and CD versions for Linear [8] and Transformer [41] and report MSE loss. Table 5 shows that CCM surpasses PRReg in most cases, highlighting its efficacy compared with regularization solutions. See full results in Appendix C.4.
5.3 Short-term Forecasting Results
In both M4 and stock datasets, we follow the univariate forecasting setting. For M4 bench- marks, we adopt the evaluation setting in prior works [42] and report the symmetric mean ab- solute percentage error (SMAPE), mean abso- lute scaled error (MASE), and overall weighted average (OWA). As for the stock dataset, we implement MAE and MSE metrics in Table 6. See more details on metrics in Appendix C.2. Remarkably, the efficacy of CCM is consistent across all M4 sub-datasets with different sam- pling frequencies. Specifically, CCM outper- forms the state-of of-te-of-the-art linear model (DLinear) by a significant margin of 11. 11.62%, and outper- forms the best convolutional method TimesNet by 8.88% In Table 6, we also observe a sig- nificant performance improvement on the stock dataset, achieved by applying CCM. In the stock dataset, we test on new samples (i.e., univariate stock time series) that the model has not seen during
Table 5: Comparison between CCM and existing regularization method for improved performance on CI/CD strategies. The best results are high- lighted in bold. The forecasting horizon is 24 for ILI dataset and 48 for other datasets. * denotes our implementation. Other results collect from [17]
CD
C1
+PRReg
+CCM*
Linear
0.402
0.345
0.342
0.342
ETThl
Transformer
0.861
0.655
0.539
0.518
ETTml
Linear
0.404
0.354
0.311
0,310
Transformer
0.458
0.379
0.349
0.300
Weather
Linear
0.142
0.169
0.131
0.130
Transformer
0,251
0,168
0.180
0.164
ILI
Linear
2.343
2.847
2.299
2.279
Transformer
5,309
4,307
3.254
3.206
Ellctricity
Linear
0.195
0.16
0.196
0.195
Transformer
0.250
185
0.185
0,183
7

---

Table 6: Short-term forecasting results on M4 dataset in terms of SMAPE, MASE, and OWA, and stock dataset in terms of MSE and MAE. The lower the better. The forecasting horizon is {7, 24} for the stock dataset. The better performance in each setting is shown in bold.
Modd
TSMiscr
CCM
DLinerr
CCM
PuchTST
-CMM
TimesNe:
+CCM
[MP(S]
SMATE:
14-702
14,76
16.965
14.337
13:477
13,384
15.378
14.426
7.286
M4 (Hially
MASE
3,343
3.570
4.283
3.144
3,099
2,997
3.554
3,446
9.580
OWN
0,715
0,875
1,005
0.854
0.291
0.781
0.918
0.802
11:346
11.187
10.513
10.121
M4 (Quanerlty
MASE
1336
1227
0.921
0.897
SMAPE
13,435
13,407
13,30
12.99
12.790
M4 (Manildy)
MASE
1022
1.019
1,887
0.970
0.941
1.09
0.942
4.358
OWN
0.96
0.944
0
0.96
0,505
0.895
0.957
0,891
3/067
SMATE
7J867
7.176
6.709
6.160
4.53
4,643
5.913
5,218
10.377
M4 (Uttens)
MASE
5.587
5.M03
4,553
4733
3.347
5.128
4.507
3
7,364
OWN
1662
1.536
1,487
1,389
1
0.997
1,438
1.217
9.472
12.867
11,914
M4 (Avgg)
1.740
0.569
0.894
Stock (Huizo 7)
MAE
MSE
0.939
0,307
0,938
0.806
0.831
0.992
0.83
0.774
0.396
0.771
0.892
0.771
0.330
0.802
0.915
0.745
2/026
3.388
Stuck (Hunioo 24)
MSR
MAE
0.829
1807
0.817
0.991
0,532
0.996
0.917
0.781
0.990
0.789
0.765
0.880
0.938
0.830
0.907
0.789
3,489
5.282
training to evaluate the model's generalization and robustness. By memorizing cluster-specific knowl- edge from analogous samples, the model potentially captures various market trends and behaviors and thereby makes more accurate and informed forecasting.
Table 7: Zero-shot forecasting results on ETT datasets. The forecasting horizon is (96, 720). The best value in each row is underlined.
Ouneraliiaaion Toak
Model
MIS
TSM MAR
MSF
CCM
MAR
DLinear MSR
MAR
MSF
CCM
MAF
PuchTST MSF
MAR
MSF
CCM
MAR
TimesNee MSR
MAR
MSF
CCM
MAF
IMP(S)
OUTIM-
730
96
0.258
0..54
0.357
0.414
0.283
0.370
0.353
0.413
0.305
0.569
0.571
0.549
0.253
0.330
0.349
0.517
1.313
0.414
0.362
0.442
0.292
0.586
0.346
0.423
0.391
0.540
0.412
0.506
01516
0,388
0.410
0.491
4.336
3,651
2 ETTTI -ETIml
96
730
0.763
1.252
0.315
0.677
0.710
1.315
0.652
0.726
1.981
0.948
0.658
0.681
1.188
0.634
0.309
0.729
1.49
0.845
0.567
0.698
1.249
0.798
0.647
0.887
1.835
0.718
0.981
015277
1.401
0,700
0.964
10.249
4.628
3 ETTbI <ETTm2
730
2,001
0.704
1,061
0.896
1.611
0.677
0.954
0.18
1925
0.694
1,014
0.67
2.204
0.704
1.123
1,874
1.012
4.457
7.924
C ETT12 ETTTbl
0.456
0.450
0.422
0.502
0.563
0,590
15,243
8.016
S BTi
96
730
0.945
1.472
0.725
0.872
0,576
1.464
0.697
0.866
0.736
1.513
0.656
0,938
0.700
1253
0.544
0.642
0.840
1882
0.708
1.052
0.771
1,532
0.688
0.863
1,250
1.561
0.850
1.016
1.611
1.864
0.967
0.793
11.43
6.344
96
1.284
0.771
1,073
6.714
1.347
0.746
0.894
0.669
0.907
0.721
0.789
0,699
1.049
0.791
0.804
0.687
18.016
720
2.255
1.137
1.754
1.665
1,992
1.001
1.740
0.970
2.651
1.149
1,65
0.771
2.185
1.108
1.742
0,983
15,952
5.4 Zero-shot Forecasting Results
Existing time series models tend to be rigidly tailored to a specific dataset, leading to poor gener- alization on unseen data. In contrast, CCM leverages learned prototypes to capture cluster-specific knowledge. This enables meaningful comparisons between unseen time series and pre-trained knowledge, facilitating accurate zero-shot forecasting. Following prior work [73], we adopt ETT collections [11], where ETThl and ETTh2 are hourly recorded, while E and ETTm2 are minutely recorded. "I" and "2" indicate two different regions where the datasets originated. Table 7 shows MSE and MAE results on test datasets. CCM consistently improves the zero-shot forecasting capacity of base time series models in 48 scenarios, including generalization to different regions and different granularities. Specifically, based on the results, we make the following observations. (1) CCM exhibits more significant performance improvement with longer forecasting horizons, high- lighting the efficacy of memorizing and leveraging pre-trained knowledge in zero-shot forecasting scenarios. (2) CCM demonstrates a better effect on originally CI base models. For instance, the averaged improvement rates on two CI models, i.e., DLinear and PStchTST, are 10.48% and 11.13% respectively, while the improvement rates on TSMixer and TimesNet are 5.14% and 9.63%. Overall, the experimental results verify the superiority and efficacy of CCM in enhancing zero-shot forecasting and the practical value of generalization within elosely related domains under varying conditions.
5.5
Qualitative Visualization
Channel Clustering Visualization. Figure presents the t-SNE visualization of channel and prototype embeddings within ETTh1 and ETTh2 datasets with DLinear the base model. Each point represents a channel within a sample, with varying colors indicating different channels. In ETThl dataset, we discern pronounced clustering of channels 0, 2, and 4, suggesting that they may
8

---

be capturing related or redundant information within the dataset. Concurrently, channels 1, 3, 5, and coalesce into another cluster.
The similarity matrix the lower left further corroborates these findings. Clustering is also observable in ETTh2 dataset, par- ticularly among channels O, 4, and 5, as well channels 2, 3, and 6. Comparatively, channel shows dispersion among clus- ters, partly due to its capturing of unique or diverse aspects of the data that do not closely align with the features represented by any clusters. The clustering re- sults demonstrate that CCM not only elucidates the intricate rela- tionships and potential redundan- cies among the channels but also offers critical insights for feature analysis and enhancing the interpretability of time series models.
Channe Channe (hanne
channel O
Charnel Charnol & channol
channel
ISME Faaure
NOUT Sestuee (4) ETTh
ESRE Fouture (b) ETT12
Figure 2: 1-SNE visualization of channel and prototype embedding by DLinear with CCM on (a) ETTh1 and (b) ETTh2 dataset. The lower left corner shows the similarity matrix between channels.
Weight Visualization of Cluster-aware Projection. Figure 3 depicts the weights visualization for the cluster-aware Feed Forward on ETThl and ETTml datasets, revealing distinct patterns that are indicative of the model's learned features [15, 51].
For instance, in the ETTml dataset, Cluster shows bright diagonal striping patterns, which may suggest that it is primarily responsible for capturing the most dominant periodic signals in the corresponding cluster. In contrast, Clus- ter exhibits denser stripes, indicating its role in refining the representation by capturing more subtle or complex periodicities that the first layer does not. The visualization implies the model's ability to identify and represent periodicity in diverse patterns, which is crucial for time-series forecasting tasks that characterized by intri- cate cyclic behaviors.
Dbiker
(a) ETTh1 Dataset
(b) ETTml Dataset
Figure 3: Weights visualization of cluster-wise lin- ear layers on (a) ETThl and (b) ETTml datasets. The input and output lengths are 336 and 96, re- spectively. We observe the different periodicities captured by different clusters.
5.6
Ablation Studies
Figure 4 shows ablation study cluster ratios, which is defined as the ratio of the number of clusters the number of channels. 0.0 means all channels are in single cluster. We observe that the MSE loss slightly decreases and then increases as the cluster ratio increases, especially for DLincar, PatchTST, and TimesNet. Time series models with CCM the best performance when the clus- ter ratio is in the range of [0.2, 0.6]. is worth noticing that DLinear and PST two CI models among four base models, benefit consistently from channel clustering with any number of clusters. Additional ablation studies on the look- back window length and clustering step provided in Appendix D.
030
039
3.00
0300
130
(6) MBE loss Wil.. cluster ratiou ETTia ditasoe
(b1 MSE loss W... diuster ratioa on cultaet
achieve
Figure 4: Ablation Study on Cluster Ratios in terms of MSE loss with four base models. The forecasting horizon is 96. (left: ETThl dataset; right: ETTml dataset)

---

5.7 Efficiency Analysis
We evaluate the model size and runtime efficiency the proposed CCM with various numbers of clusters on ETThl dataset, as shown in Figure 5. The batch size is 32. and the hidden dimension is 64, We keep all other hyperparameters consistent to ensure fair evaluation. is worth noting that CCM reduces the model complexity based on Channel-Independen models (e.g., PatchTST, DI Diinear), since CCM essentially uses cluster identity to replace channel identity. The generalizability of CI models is thereby enhanced as well. When it comes to Thaanel-Dependent models, CCM increases the model complexity with negligible overhead, considering the improved forecasting performance.
Model Size Analyss
Runn ng Time Emficencc Andlysis
154 not (SPM)
EL (001)
10
0.30
TSM
CLivarr (021)
Theshet (C04)
RLnc Votrattn wew CORNOO
DI OLiver thoeet
Tireste: bas
(baac)
10"
MacTTS 1009)
T5N Tzanst
NOTTTT Itaast
(a) Model Size Analysis on ETThl dataset
(b) Runtime Efficiency Analysis on ETThl dataset
Figure 5: Efficiency analysis in model size and running time ETThl dataset.
Conclusion
This work introduces novel Channel Clustering Module (CCM) address the challenge of effective channel management in time series forecasting. CCM strikes a balance between individual channel treatment and capturing cross-channel dependencies by clustering channels based their intrinsic similarity. Extensive experiments demonstrate the efficacy of CCM multiple benchmarks, including long-term, short-term, and zero-shot forecasting scenarios. Refinement of the CCM clustering and domain-specific similarity measurement could potentially improve the model performance further. Moreover, it would be valuable to investigate the applicability of CCM in other domains beyond time series forecasting in future works.
Acknowledgments and Disclosure of Funding
This research was supported in part by the National Science Foundation (NSF) CNS Division Of Computer and Network Systems (2431504), NSF-AoF FAIN (2132573), ARO and AWS Research Awards. We would like to thank the anonymous reviewers for their constructive feedback. Their contributions have been invaluable in facilitating our work.
10

---

References
[1] Francisco Martinez Alvarez, Alicia Troncoso, Jose Riquelme, and Jesus S Aguilar Ruiz. Energy time series forecasting based pattern sequence similarity. IEEE Transactions Knowledge and Data Engineering, 23(8): 230-1243, 2010.
[2] Irena Koprinska, Dengsong Wu, and Zheng Wang. Convolutional neural networks for energy time series forecasting. In 2018 international joint conference on neural networks (IJCNN), pages 1-8. IEEE, 2018.
[3] Rafal A Angryk, Petrus Martens, Berkay Aydin, Dustin Kempton, Sushant Mahajan, Sunitha Basodi, Azim Ahmadzadeh, Xumin Cai, Soukaina Filali Boubrahimi, Shah Muhammad Hamdi, et Multivariate time series dataset for space weather data analytics. Scientific data, 7(1):1-13, 2020.
[4] Remi Lam, Alvaro Sanzaez- Matthew Willson, Peter Wirnsherger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416-1444
2023.
[5] Li Li, Xiaonan Su, Yi Zhang, Yuetong Lin, and Zhiheng Li. Trend modeling for traffic time series analysis: An integrated study. IEEE Transactions on Intelligent Transportation Systems, 1666):340-3349, 2015.
[6] Yi Yin and Pengjian Shang. Forecasting traffic time series with multivariate predicting method. Applied Mathematics and Computation, 291:266-2773 278, 2016.
[7] Si-An Chen, Chun-Liang Li, Nate Yoder, Serean O Arik, and Tomas Pfister. Tsmixer: An all-mlp architecture for time series forecasting. arXiv preprint arXiv:2303306053 2023.
[8] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205. 13504, 2022.
[9] Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma. Junchi Yan, and Liang Sun. Transformers in time series: survey. arXiv preprint arXiv:2202.07122. 2022.
[10] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. arXiv preprini arXiv:2201. /2740, 2022.
[11] Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11106-11115, 2021.
[12] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 114:2219-222432 2021.
[13] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, Times- net: Temporal 2d-variation modeling for general time series analysis. arXiv preprint 10.0122200.02186, 2022.
[14] Minhao Liu, Ailing Zeng. Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma. and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816-5828 18288 2022.
[15] Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint UrXiv:230S.1072. 2023.
[16] Yuan Peiwen and Zhu Changsheng. Is channel independent strategy optimal for time series forecasting? arXiv preprint arXiv:2310. 17658, 2023.
[17] Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. arXiv preprint arXiv:2304.05206 2023.

---

[18] Jian Ni and Yue Xu. Forecasting the dynamic correlation of stock indices based on deep learning method. Computational Economics, (1(1):55-55, 2023.
[19] Xingkun Yin, Da Yan, Abdullateef Almudaifer, Sibo Yan, and Yang Zhou. Forecasting stock prices using stock correlation graph: A graph convolutional network approach. In 202/ International Joint Conference on Neural Networks (LJCNN), pages 1-8. IEEE, 2021.
[20] Phillip Jang and David S Matteson. Spatial correlation in weather forecast accuracy: a functional time series approach. Computational Statistics, pages 1-15, 2023.
[21] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.147300 2022.
[22] Sean J Taylor and Benjamin Letham. Forecasting at scale. The American Statistician, 72(1):37- 45, 2018.
[23] Oskar Triebe, Hansika Hewamalage, Polina Pilyugina, Nikolay Laptev, Christoph Bergmeir, and Ram Rajagopal. Neuralprophet: Explainable forecasting at scale, 2021.
[24] G Peter Zhang. Time series forecasting using hybrid arima and neural network model. Neurocomputing, 50:159-175, 2003.
[25] Nesreen K Ahmed, Amir F Atiya, Neamat El Gayar, and Hisham EI- El-Shishiny. An empirical comparison of machine learning models for time series forecasting. Econometric reviews, 29(5-6):594-62 621, 2010.
[26] José F Torres, Dalil Hadjout, Abderrazak Sebaa, Francisco Martinez-Alvare, and Alicia Troncoso. Deep learning for time series forecasting: survey, Big Data, -1):3-21,, 2022.
[27] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: survey. Philosoph- ical Transactions of the Royal Society A, $79(2194):2000010)) 2021.
[28] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv: /803.0/27,, 2018.
[29] Renzhuo Wan, Shuping Mei, Jun Wang, Min Liu, and Fan Yang. Multivariate temporal convolutional network: deep neural networks approach for multivariate time series forecasting. Electronics, 878):876, 2019.
[30] Rajat Sen, Hsiang-Fu Fu Yu, and Inderjit S Dhillon. Think globally, act locally: A deep neural network approach to nigh-dimensional time series forecasting. Advances neural information processing systems, 32, 2019.
[31] Hansika Hewamalage, Christoph Bergmeir, and Kasun Bandara. Recurrent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37(1):388 427, 2021.
[32] Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. Advances in neural information processing systems, 31, 2018.
[33] Slawek Smyl. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting. International Journal of Forecasting, 36(1))75-85, 2020.
[34] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 16(3):1181-119, 2020.
[35] Yong Liu, Haixu Wa, Jianmin Wang, and Mingsheng Long- Non-stationary transformers: Rethinking the stationarity in time series forecasting, arXiv preprint arXiv:2205.14411, 2022.
[36] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations, 2022.
12

---

[37] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:23 2023.
[38] Binh Tang and David Matteson. Probabilistic transformer for time series analysis. Advances in Neural Information Processing Systems, 34:23592-23608, 2021.
[39] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations, 2021.
[40] Aosong Feng, Jialin Chen, Juan Garza, Brooklyn Berry, Francisco Salazar, Yifeng Gao, Rex Ying, and Leandros Tassiulas. Efficient high-resolution time series classification via attention kronecker decomposition. arXiv preprint (tti:240030088822 2024.
[41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[42] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv: 10437, 2019.
[43] Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented oriented mlp structures. arXiv preprint arXiv:2207.01186, 2022.
[44] Abhimanyu Das, Weihao Kong, Andrew Leach, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.. 08424, 2023.
[45] Robert Cleveland, William S Cleveland, Jean McRae, and Irma Terpenning. Stl: seasonal-trend decomposition. J. Off. Stat, 6(1):3-73, 1990.
[46] Qingsong Wen, Zhe Zhang, Yan Li, and Liang Sun. Fast robuststl: Efficient and robust seasonal- trend decomposition for time series with complex patterns. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2203-2213, 2020.
[47] Djamel Benaouda, Fionn Murtagh, J-L Starck, and Olivier Renaud. Wavelet-based nonlinear multiscale decomposition model for electricity load forecasting. Neurocomputing, 70(1-3):139- 154, 2006.
[48] Donald B Percival and Andrew T Walden. Wavelet methods for time series analysis, volume Cambridge university press, 2000.
[49] Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. Mien: Multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations, 2022.
[50] Yifan Zhang, Rui Wu, Sergiu M Dascalu, and Frederick C Harris. Multi-scale transformer pyramid networks for multivariate time series forecasting, IEEE Access, 2024.
[51] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He. Defu Lian, Ning An, Longbing Cao, and Zhendong Niu. Frequency- mips are more effective learners in time series forecasting, arXiv preprint arXiv:2311.06788 2023.
[52] Tian Zhou, Ziqing Ma. Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems, 35: 15:77777-12600 2022.
[53] Pablo Montero-Manso and Rob J Hyndman. Principles and algorithms for forecasting groups of time series: Locality and globality. International Journal of Forecusting, 3334)163
2021.
13

---

[54] Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. Card: Channel aligned robust blend transformer for time series forecasting. In The Twelfth International Conference on Learning Representations, 2023.
[55] Jung-Y Yi Jiang, Ren-Jia Liou, and Shie-Jue Lee. A fuzzy self-constructing feature clustering algorithm for text classification. IEEE transactions on knowledge and data engineering, 23(3):335-344 349, 2010.
[56] Dmitrii Marin, Jen-Hao Rick Chang, Anurag Ranjan, Anish Prabhu, Mohammad Rastegari, and Oncel Tuzel. Token pooling in vision transformers for image classification. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 12-21, 2023,
[57] Lijimol George and P Sumathy, An integrated clustering and bert framework for improved topic modeling. International Journal of Information Technology, pages 1-9, 2023.
[58] Hao Li, Alin Achim, and D Bull. Unsupervised video anomaly detection using feature clustering. IET signal processing. 6(5):521-533, 2012.
[59] Iwan Syarif, Adam Prugel-Bennett, and Gary Wills. Unsupervised clustering approach for network anomaly detection. In Networked Digital Technologies: 4th International Conference, NDT 2012, Dubai, UAE, April 24-26, 2012. Proceedings, Part I 4, pages 135-145. Springer, 2012.
[60] Rajesh Kumar Gunupudi, Mangathayaru Nimmala, Narsimha Gugulothu, and Suresh Reddy Gali. Clapp: A self constructing feature clustering approach for anomaly detection. Future Generation Computer Systems, 74:417 09:417-449,.. 2017.
[61] Jiahao Ji, Jingyuan Wang, Chao Huang, Junjie Wu, Boren Xu, Zhenhe Wu, Junbo Zhang, and Yu Zheng. Spatio-tempone superviie learning for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 4356-4364, 2023.
[62] Gang Liu, Silu He, Xing Han, Qinyao Luo, Ronghua Du, Xinsha Fu, and Ling Zhao. Self- supervised spatiotemporal masking strategy- based models for traffic flow forecasting. Symmetry, 15(11): 2023.
[63] Zhe Li, Zhongwen Rao, Lujia Pan, and Zenglin Xu. Mis-mixers: Multivariate time series forecasting via factorized temporal and channel mixing. arXiv preprint arXiv:2302.05501, 2023.
[64] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
[65] Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang. Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. In sevenhh Conference on Neural Information Processing Systems, 2023.
[66] Hao Helen Zhang. Mark G Genton, and Peng Liu. Compactly supported radial basis function kernels. Technical report, North Carolina State University. Dept. of Statisties, 2004.
[67] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax softmax. arXiv preprint arXiv: 16/ 144, 2016.
[68] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint arXiv:2001.0445), 2020.
[69] Sinong Wang, Belinda Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04777 2020.
[70] Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Efficient attention: Attention with linear complexities. In Proceedings of the IEEF/CVE winter conference on applications of computer vision, pages 3531-3539, 2021.
14

---

[71] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The slst international ACM SIGIR conference on research & development in information retrieval, pages 95-104, 2018.
[72] Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34(4):802-
808, 2018.
[73] Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.0172S 2023.
[74] Lei Chen and Raymond Ng. On the marriage Ip-norms and edit distance. In Proceedings of the Thirtieth international conference on Very large data bases- Volume 30, pages 792-80 203, 2004.
[75] Hui Ding, Goce Trajcevski, Peter Scheuermann, Xiaoyue Wang, and Eamonn Keogh. Querying and mining of time series data: experimental comparison of representations and distance measures. Proceedings of the VLDB Endowment, 1(2):1542-1552, 2008.
[76] Omer Gold and Micha Sharir. Dynamic time warping and geometric edit distance: Breaking the quadratic barrier. ACM Transactions Algorithms (TALG), 14(4):1-17, 2018.
[77] Maša Kljun and M Ters ekk A review and comparison of time series similarity measures. In 29th International Electrotechnical and Computer Science Conference (ERK 2020). Portoroz. pages 21-22, 2020.
[78] Wei Li, Xiangxu Meng, Chuhao Chen, and Jianing Chen. Mlinear: Rethink the linear model for time-series forecasting. arXiv preprint arXiv:2305.0488 2023.
[79] Artur Trindade. EleetricttyLoaDDiigaammm303 12014. UCI Machine Learning Repository, 2015. DOI: https://bl..gg/11.243322/633
[80] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. corr abs/1703.07015 (2017). arXiv preprint 1rXiv:1703.07015, 2017.
[81] M4 Team et al. M4 competitor's guide: prizes and rules. See https://www. m4. unic. ac. pdf. 2018.
[82] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 141,,690, 2014.
[83] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
15

---

A Definitions
A.1 Channel Similarity
Essentially, the similarity between two time series X and X, is defined as SI,(X,, =
exp( -x,,x,,))) where -(:,-) can be any distance metric [74, 75], such as Euclidean Distance (Ip), Editing Distance (ED) and Dynamic Time Warping (DTW) [76]. One may also use other similarity definitions, such Longest Common Subsequence (L.CS)) and Cross-correlation (CCor).
Firstly, the choice Euclidean distance in this work motivated by its efficiency and low computa- tional complexity, especially in the case of large datasets or real-time applications. Let H denote the length of the time series. The complexity of the above similarity computation is shown in Table 8,
Secondly, it's worth noting that while there are vari- ous similarity computation approaches, studies have demonstrated strong correlation between Euclidean distance and other distance metrics [77]. This high cor- relation suggests that, despite different mathematical formulations, these metrics often yield similar results when assessing the similarity between time series.
Table 8: Complexity of similarity computa- tion
Euclidean O(H) O(H))
Edit Distance
LCSS
CCor
O(H²)
O(H))
O(H?)
This empirical evidence supports the choice of Euclidean distance as a reasonable approximation of similarity for practical purposes. In our implementation, we select = 5 in Eq. to compute the similarities based on Euclidean distance.
A.2 Channel Dependent and Channel Independent Strategy
The definitions of Channel Dependent (CD) and Channel Independent (CI) settings are pivotal to this work. The fundamental difference lies in whether a model captures cross-channel information. There are slightly varied interpretations of Channel Independent (CI) in previous works and we summarize as follows.
1. In some works [15, 54]. CI is broadly defined as forecasting each channel independently. where cross-channel dependencies are completely ignored. For linear models [78, 16), CI is specifically defined individual linear layers for channels in previous works. Each linear layer is dedicated to modeling a univariate sequence, with the possibility of differing linear weights across channels.
2. In previous work [17], CI also means all channels being modeled independently yet under a unified model.
All the above works acknowledge that CI strategies often outperform CD approaches, though this comparison is not the focal point of our work. It's also recognized that the specific CI strategy employed in Dninear and PStchTST contributes significantly to their performance. The CI setting in [17] represents specific instance within the broader CI setting in other works [15, 54]. To avoid ambiguity, we use (+) to represent the model for the i-th channel specifically, aligning with previous definitions without conflict.
B Multivariate and Univariate Adaptation
We provide pseudocodes for training time series models enhanced with CCM in Algorithm 1. Algorithm 2 displays pseudocodes for the inference phase, where both the training and test sets have the same number of channels. The components the pretrained model F, highlighted in blue, remain fixed during the inference phase. It's important to note that shot forecasting in Algorithm 2 is adaptable to various scenarios. Let's discuss these scenarios:
Training on a univariate dataset and inferring on either univariate or multivariate samples: In this case, the model learns prototypes from a vast collection of univariate time series in the training set. As a result, the model can effortlessly adapt to forecasting unseen univariate time series a zero-shot manner. To forecast unseen multivariate time series, we decompose each multivariate sample into multiple univariate samples, where each univariate sample can be processed by the pretrained model. The future multivariate time series can be obtained by stacking multiple future univariate time series.
16

---

Training on multivariate dataset and inferring on either univariate multivariate samples: For Btannel-Dependee models, test samples should have the same number of channels as the training samples, as seen in sub-datasets within ETT collections [11]. In contrast, for Channel- Independent models, zero-shot forecasting can be performed on either univariate or multivariate samples, even when they have different numbers of channels.
Algorithm 1 Forward function of time series models with channel clustering module. C' is the number of channels in the dataset. K is the number of clusters. is the length of historical data. H the forecasting horizon.
Input: Historical time series X RTx
Output: Future time series Y E RH Prototype embedding C Rxxd Initialize the weights of linear layer 0g for = 1, Initialize K cluster embedding C₁ E Rd for k 1, K.
Cluster Embedding C
X < Normalize(X)
S. < exp(
IIX, -XXX,xxI1"""
Compute Similarity Matrix Channel Embedding H via MLP Encoder in the Cluster
20²
h; +- MLP(X,) for each channel i.
Assigner
Pi,k + Normalize(
he
[0,
Compute Clustering Probability Matrix P b. Sample Clustering Membership Matrix
M Bernoulli (P).
C + Normalize cxp(
H))
④ (TM WyH.
Update Cluster Embedding via
Vd
Cross Attention
H Temporal Module(H).
b. Update via Temporal Modules
for channel in 2 ,C) do
Y, t hgs (H,) where 0 Ex
b Weight Averaging and Projection
end for
Algorithm Zero-shot forecasting of time series models with channel clustering module. C is the number of channels in both the training and datasets. K is the number of clusters. T is the length of historical data. H is the forecasting horizon.
Input: Historical time series X RTxC. Pretrained Models J Output: Future time series Y E RH
Load the weights of K linear layer 0g for k 1,... K from F Load K cluster embedding C& Rd for 1,... K from F.
Cluster Embedding C
X + Normalize(X))
St <- exp( -x11
Compute Similarity Matrix S > Channel Embedding H via MLP Encoder in the Cluster
h; + MLP(X,) for each channel i.
Assigner
Pi,k < Normalize( Tou E [0, 1].
Compute Clustering Probability Matrix P Sample Clustering Membership Matrix M Update via Temporal Modules
M Bernoulli
H Temporal Module(H).
for channel i in 21,2, ... C' do
Y, +- hg: (H;) where 0² Σx Pi
Weight Averaging and Projection
end for
C Experiments
C.1 Datasets
We utilize nine commonly used datasets for long-term forecasting evaluation. Firstly, ETT collection [11], which documents the oil temperature and load features of electricity transformers over the period spanning July 2016 to July 2018. This dataset further subdivided into four sub-datasets ETThs and ETTms, with hourly and 15-minute sampling frequencies, respectively.
Public Datasets.
17

---

s can be 1 or 2. indicating two different regions. Electricity dataset [79] encompasses electricity consumption data from 321 clients from July 2016 to July 2019. Exchange [80] compiles daily exchange rate information from 1990 to 2016. Traffic dataset contains hourly traffic load data from 862 sensors in San Francisco areas from 2015 to 2016. Weather dataset offers a valuable resource with 21 distinct weather indicators, including air temperature and humidity, collected every 10 minutes throughout the year 2021. ILI documents the weekly ratio of influenza-like illness patients relative to the total number of patients, spanning from 2002 to 2021. Dataset statistics can be found in Table 9.
We adopt M4 dataset for short-term forecasting evaluation, which involves 100,000 univariate time series collected from different domains, including finance, industry, etc. The M4 dataset is further divided into sub-datasets, according to the sampling frequency.
Table 9: The statisties of dataset in long-term term and short-term forecasting tasks
Tasks
Dataset
Channels
Forecast Horizon
Length
Frequency
Domain
ETTh!
[96. 332, 720)
17420
hour
Temperature
ETTh2
7
[96. 192, 336, 720
17420
1 hour
Temperature
ETTml
[96, 192, 336, 720
69680
15 min
Temperature
FTTm2
[96. 336, 720)
69680
15 min
Long-term
Illness
7
[96. 192, 336, 720)
966
week
IlIness Ratio
Exchange
[96, 192, 336, 720)
7588
day
Exchange Rates
Weather
21
[96. 192, 336, 720)
52696
10 min
Weather
Electricity
321
[96. 336, 720
26304
hour
Electricity
Traflic
862
[96, 192, 336, 720}
17544
hour
Traffic Load
M4- Yearly
6
23000
yearly
Demographic
M4 Quarterly
24000
quarterly
Finance
M4 Monthly
18
48000
monthly
Industry
Short-term
M4 Weekly
13
359
weekly
Macro
M4 Daily
14
4227
daily
Micro
M4 Hourly
48
414
hourly
Other
Stock
(7,24)
10000
min
Stock
Stock Dataset. We design a new time series benchmark dataset constructed from publicly available stock-market data. We deploy commercial stock market API to probe the market statisties for 1390 stocks spanning 10 years from Nov.25, 2013 to Sep.1, 2023. We collect stock price data from 9:30 a.m. to 4:00 p.m. every stock open day except early closure days. The sampling granularity is set to be 5 minutes. Missing record is recovered by interpolation from nearby timestamps and all stock time series processed to have aligned timestep sequences. We implement short-term forecasting on the stock dataset, which is more focused on market sentiment, and short-term events that can cause stock prices to fluctuate over days, weeks, or months. Thereby, we set the forecasting horizon as 7 and 24.
C.2 Metries
Following standard evaluation protocols [13], utilize the Mean Absolute Error (MAE) and Mean Square Error(MSE) for long-term and stock price forecasting. The Symmetric Mean Absolute Percentage Error (SMAPE), Mean Absolute Scaled Error (MASE), and Overall Weighted Average (OWA) are used as metrics for M4 dataset [72, 42]. The formulations are shown in Eq. 5. Let y: and y, denote the ground-truth and the forecast at the f-th timestep, respectively. H is the forecasting horizon. In M4 dataset, MASE is a standard scale-fre free metric, where s is the periodicity of the data (e.g., 12 for monthly recorded sub-dataset) [72]. MASE measures the scaled error w.Et. the naive predictor that simply copies the historical records of periods in the past. Instead, SMAPE scales the error by the average between the forecast and ground truth. Particularly, OWA is an M4-specific metric [81] that assigns different weights to each metric.
H lye-
MAE 1
Σ t=1
Rel,
MSE- 1 H
(ye
8t)
H
i=1
SMAPE 200
ly, Del
MASE H
Σ
ly,
Yr
(5)
H
Y
+ F
2j=s+1 |y; -yj-sl
H
SMAPE
MASE
OWA-
SMAPE Narve2
MASENave2
18

---

C.3 Experiment Details
To verify the superiority of CCM in enhancing the performance mainstream time series models, select four popular and state-of-the-art models, including linear models such as TSMixer [7], DLinear [8], transformer-basee model Patch'STT [21] and convolution-based model TimesNet [13].
We build time series models using their official codes and optimal model configuration
In the data preprocessing stage, we apply reversible instance normalization [64] to ensure zero mean and unit standard deviation, avoiding the time series distribution shift. Forecasting loss is MSE for long-term forecasting datasets and the stock dataset. Instead, we use SMAPE loss for M4 dataset. We select Adam [82] with the default hyperparameter configuration for ($1. 32) (0.9, 0.999). An early-stopping strategy is used to mitigate overfitting. The experiments are conducted on a single NVIDIA RTX A6000 48GB GPU, with PyTorch [83] framework. We use the official codes and follow the best model configuration to implement the hase models. Then we apply CCM to the base models, keeping the hyperparameters unchanged for model backbones. Experiment configurations, including the number of MLP layers the cluster assigner, the layer number in the temporal module, hidden dimension, the best cluster number, and regularization parameter ,3 on nine real-world datasets are shown in Table 10.
Table 10: Experiment configuration.
clusters
# lincar layers in MLP hidden dimensionn # layers (TSMixer) layers (PuchTST) # layess (TimesNet)
ETTh1
0.3
128
3
ETTml
1
0.3
1
BTTh2
2.
0.3
1
64
ETTm2
Exchange
0.9
1
III1
2.
0.9
1
36
Weather
[2.5]
Electricity
[3.10]
0.5
2
128
Trafiic
[3.10]
0.5
128
C.4
Comparison between CCM and Other Approach
Table 11: Full Results on Comparison between CCM and existing regularization method for enhanced
performance on CI/CD strategies in terms of MSE metric. The best results are highlighted bold.
CD
+PRReg
+CCM
((Th)(48)
Linear
0.402
0.345
0.342
0.342
Transformer
0,861
0.655
0.539
0.518
ETTh2(48)
Linear
0.711
0.226
0.239
0.237
Transfomer
1,031
0.274
0,273
0,284
Linear
0.404
0.354
0.311
0.310
FTTmi(48)
Transformer
0.458
0.379
0.349
0.300
ETTm2(48)
Linear
0.161
0.147
0.136
0.146
Transformer
0,281
0.148
0.144
0.143
Exchange(48)
Linear
0.119
0.051
0.042
0.042
Transformer
0.511
0.101
0.044
0.048
Weather(48)
Linear
0.142
0.166
0.131
0.130
Transformer
0.251
0.168
0.180
0.164
ILI(24)
Linear
2.343
2.847
2.299
2.279
Transfomer
5.309
4.307
3.254
3.206
Flectricity(48)
Linear
0.195
0.196
0.196
0.195
Transformer
0.250
0.185
0.185
0.183
Predict Residuals with Regularization. Prior work [17] demonstrates that the main drawback of CD models is their inclination to generate sharp and non-robust forecasts, deviating from the actual trend. Thereby, Predict Residuals with Regularization (PRReg for simplicity), specifically designed
19

---

regularization objective, is proposed to improve the robustness of CD methods as follows.
L Cr
1 N E Lr
- Al1) (1(1))-A() +
AN(f),
(6)
where the superscript j indicates the sample index. LF MSE loss. A X2. represents the last values of each channel in X Therefore, the objective encourages the model to generate predictions that are close to the nearby historical data and keep the forecasts smooth and robust. The regularization term S? (L2 norm in practice) further restricts the complexity of the model and ensures smoothness in the predictions. It was demonstrated that PRReg can achieve better performance than original CD and CI strategies [17]. We conduct extensive experiments on long-term forecasting benchmarks to compare PRReg and CCM. The full results are shown in Table 11. The numbers in
parentheses next to the method represent the forecasting horizon. We set the length of the look-back window to 36 for ILI and 96 for other datasets for consistency with previous works [17]. The base models are linear model [8] and basic transformer [41]. The values in the PRReg column are the best results with any A, collected from [17]. We observe from Table 11 that CCM successfully improves forecasting performance original CI/CD strategies (or reached comparable results) in 13 out of 16 settings, compared with PRReg method.
C.5 Results Analysis
We report the degree of multivariate correlation across multiple channels (measured by the average Pearson correlation coefficient) in Table 12 and Table 13. r denotes the degree of multivariate correlation. Then the Pearson correlation coefficient between r' and the performance improvement rate is 0.258 in long-term forecasting, indicating weak correlation. It demonstrates that CCM tends achieve greater boost on datasets that intrinsically correlated within channels. Compared with datasets used in long-term benchmarks, M4 demonstrates more significant correlations between time series. Therefore, capturing the correlation within the dataset in short-term cases potentially leads to greater improvement in the forecasting performance than in long-term cases.
Table 12: Multivariate intrinsic similarity for long-term forecasting datasets
Dataset
ETTh1
ETTm1
ETTh2 0.3224
ETTm2
Exchange
ILI
Weather
Electricity
Traffic 0.6325
Correlation
r 0.1876
0.1717
0.328
0,3198
0.508
0.1169
0.5311
Table 13: Intrinsic similarity for short-term forecasting datasets
Dataset
M4 Monthly
M4 Daily
M4 Yearly
M4 Hourly
M4 Quarterly
M4 Weekly 0.653
Correlation
0.62
0.646
0.712
0.55
0.671
C.6 Error Bar
Experimental results this paper are averaged from five runs with different random seeds. We report the standard deviation for base models and CCM-enhanced versions on long-term forecasting datasets Table 14, M4 dataset in Table 15 and stock dataset Table 16.
D Ablation Study
D.1 Influence of Cluster Ratio
The number of clusters is an important hyperparameter in the CCM method. To verify the effectiveness of our design, we conduct an ablation study for all base models on four long-term forecasting datasets. The full results are shown in Table 17. We tune different cluster ratios, defined as the ratio of the number of clusters to the number of channels. Original means the original base model without any channel clustering mechanism. 0.0 indicates grouping all channels into the same cluster. We make the following observations. (1) For most cases, the channel clustering module (CCM) with any number of clusters greater than 1 consistently improves the forecasting performance upon base models. (2) For
20

---

Table 14: Standard deviation of Table 2 on long-term forecasting benchmarks. The forecasting
horizon is 96.
ETTal
ETTml
ETTh2
ETTma2
Exchange
Weather
Ellettiiity
Trathic
TSMixer
MAB
MSE 6.360.0.001
0.392=0.000
0:285
0.399:10001
0.343
0.2841..001
0.071
0.260=00001
0.09910..04
0.209100009
0.879500009
1.1410.031
0.149_00008
0.198=0000
0.142100002
0.297:50004
0.264
0.37600.005
+00M
MAR
MSE
0.305=000001
0.395-00.001
0220
0.937+00002
0.278
0.338+0.000
0.0
0.260-0.001
0005
0
1998
0.874-0.012
0.177=0007
0. 0.194-0.009
0.139
0.235+00008
0.075
0.262-00001
DLinear
MAB
MSE
0.375=0.002
0.3991000001
0.290110001
0.343:10001
0.209:0.001
0.35320.001
0.067:0.001
0.260
0,215100110
0000
2.215:0.031
1081200009
0.250=00008
0.102-0011
0.153100004
0.229100005
0.254200000
20.000
+0CM
MSE 0.371=0.001
0:298:00000 001
0.285+00001
0:166-0.002
0085+000066
1.955-0.034
0118700015
0.142:00.00
0
MAB
0.303.0.000
0.3
0.24810.02
0.25
0.21110013
0,915
0.245.00020
0.247100006
0,2
PuchTST
MSE
0.29220002 0.5
0.278
0.174
0009
1.92200011
0.169-00000
0.13810004
0.30050.00
MAR
0.308-0.004
0,351+0.004
0,340+0.004
0.261-00003
0316+0012
0.757+0.015
0.196-0012
0:233+0005
0.249-0.005
+0UM
MAR
MSE
0.99K=00.015
0.338+00005
0.274
0.336+0.006
0:16
0.256-000013
000
0.28+00009
0,750+0:300
1.61100021
0.187-0008
0.197=0013
0.126100002
0:21++00006
0.35720.007
0.246+0.006
TimesNet
MAB
MSF
0.884-0.005
0,402.0.005
0 006
0.375100.066
0.394
0.540+0.005
0,267.10,003
0.187-00055
0.234100013
007
0,931000000
2.817+0:074
0.220_.0013
01122-0011
0.27210.005
0.168+0.002
0.321
0.503-0.010
+OCM
MAB
MSE
0.380=0.004
0,400
0.000
0.3
0,300
0.336+00003
0.270.00.005
0.189-0.003
0,21110:110
000++00006
0.936.10.018
2.139=0.058
01160-0015
0.215_00024
0.25910.006
0.158+00003
0.354=0.009
0,316.0.0088
Table 15: Standard deviation of Table 6 on M4 dataset
Model
TSMixer
CCM
DLincar
CCM
PatehTST
CCM
TimesNet
CCM
SMAPE
0.122
0.130
0.087
0.089
0.135
0.134
0.168
0.162
Yearly
MASE
0.022
0.022
0.017
0.018
0.021
0.017
OWA
0.002
0.002
0.002
0.002
0.007
0.009
0,010
0.011
SMAPE
0.100
0.079
0.073
0.106
0.105
Quaterly
MASE
0.016
0.016
0.015
0.015
0.008
0.009
0.013
0.011
OWA
0.008
0.007
0.006
0.008
0.013
0.016
0.009
0.009
SMAPE
0.113
0.113
0.110
0.111
0.122
0. 0.120
0.120
0.134
Monthly
MASE
0.013
0.015
0.009
0.013
0.017
0.019
0.011
0.012
OWA
0.001
0.001
0.002
0.001
0.003
0.002
0.004
0.004
SMAPE
0,113
0,110
0,126
0.128
0.137
0.130
0.129
0.125
Others
MASE
0.024
0.026
0.036
0.031
0.023
0.025
0.028
0,025
OWA
0.011
0.013
0.009
0.008
0.023
0.019
0.024
0.020
SMAPE
0.113
0.115
0.111
0.103
0.136
0.134
0.148
0.153
Avg.
MASE
0.027
0.025
0.021
0.017
0.026
0.021
0.027
0.042
OWA
0.006
0.004
0.004
0.004
0.013
0.016
0.021
0.036
Channel-Indepee Independen base models, such as DLinear and PatchTST, grouping all channels into one cluster results in channel fusion at the last layer, leading to degradation in forecasting performance compared to the original CI models. (3) For most cases, the cluster ratio in the range of [0.2, 0.6) typically results in the best performance.
D.2 Inffuence of Look-back Window Length
In this section, we conduct additional ablation experiments to investigate the effect of different look- back window lengths in the newly collected stock dataset, which determines how much historical information the time series model uses to make short-term forecasts. Specifically, the ablation study helps identify the risk of overfitting or underfitting based on the chosen look-back window length. An overly long window may lead to overfitting, while short window may cause underfitting. Table 18 display the forecasting performance on forecasting horizon and 24. The length of the look-back window ranges from two to four times the forecasting horizon. From Table 18, we make the following observations. (1) CCM effectively improves the base models' forecasting performance in 24 cases across different base models, forecasting horizons, and look-back window lengths consistently. (2) Especially, CCM achieves better enhancement when the look-back window is shorter.
D.3 Influence of Different Clustering Steps
We conducted an ablation study on different clustering steps to investigate its effect on downstream performance, reported in Table 19. The ablation study follows the setting in Table 5. Random means we randomly assign each channel to a cluster, leading to worse clustering quality. K-Means means using the k-means algorithm to replace our clustering step, leading to suboptimal prototype embedding. The proposed CCM is essentially an advanced variant of K-Means with learnable prototype embedding and cooss-attention mechanism. Results show that both clustering quality and
21

---

Table 16: Standard deviation of Table 6 on stock dataset
Horizon
Metric
TSMixer
+CCM
DLinear
CCM
ParchTST
CCM
TimesNet
CCM
MSF
0.001
0.001
0.001
0.001
0.003
0.003
0.004
0.004
0.001
0.001
0.001
0.003
0.003
MSE
0.002
0.002
0.001
0.002
0.005
0.004
0.005
0.005
24
MAB
0.001
0.001
0.001
0.001
0.003
0.002
0.003
0.004
Table 17: Sensitivity of cluster ratio terms MSE metric. The forecasting horizon is 96. 0.0 means grouping all channels into the same cluster. original means the base model without the CCM mechanism.
Cluster Ratio
Original
0.0
0.6
0.8
TSMixer
0.361
0.361
0.362
0.365
0.363
0.364
0.366
ETThI
DLincar
0.375
0.378
0.371
0.371
0.372
0.372
0.371
PatchTST
0.375
0.380
0.372
0.371
0.373
0.376
0.375
TimesNet
0.384
0.384
0.380
0.383
0.385
0.385
0.388
TSMixer
0.285
0.285
0.283
0.283
0.284
0.285
0,286
ETTml
DLinear
0.299
0.303
0.298
0.298
0.299
0,300
0,300
PatchTST
0.294
0.298
0.292
0.289
0.289
0.293
0.293
TimesNet
0.338
0.338
0.337
0.335
0.336
0.335
0.337
Exchange
TSMixer
0.089
0.089
0.086
0.087
0.088
0.090
0.092
DLinear
0.088
0.093
0.088
0.087
0.085
0.089
0.089
PatchTST
0.094
0.095
0.089
0,088
0,088
0,091
0.093
TimesNet
0,107
0,107
0,105
0.105
0.107
0.107
0,107
Electricity
TSMixer
0.142
0.142
0.139
0.139
0.140
0.143
0.143
DLiinear
0.153
0.160
0.143
0.142
0.143
0.147
0.150
PatchTST
0.138
0.142
0.136
0.138
0.140
TimesNet
0.168
0.168
0.160
0.159
0.167
0.168
0.169
prototype embedding will affect the downstream performance. Instead, CCM generates high-quality channel clustering results, compared with random assignment and K-Means clustering.
E Visualization Results
121
63
62
Chane
(a) Channel-wise MSE Loss Comparison on ETTh1 dataset with DLinear
(b) Channel similarity visualization of ETTh1 dataset
Figure 6: (a) Channel-wise forecasting performance and (b) Channel similarity on ETThl dataset illustrate the correlation between model performance and intrinsic similarity
E.1 Channel-wise Performance and Channel Similarity
Figure 6 illustrates the channel-wise forecasting performance in terms of MSE metric and channel similarity on ETThl dataset with DLinear. We use the model's performance difference on the original dataset and the randomly shuffled dataset as the model's fitting ability that MSE loss computed on channels that have been standardized, which means that any scaling differences between them have been accounted for. Figure 6 highlights noteworthy pattern: when two channels exhibit higher degree of similarity, there tends to corresponding similarity the performance on these channels. This observation suggests that channels with similar characteristics tend benefit similarly from the optimization. implies a certain level of consistency in the improvement of MSE loss when dealing with similar channels.
specific channel. Note
22

---

Table 18: Short-term forecasting on stock dataset with different look-back window length in {14, 21, 28}. The forecasting length is 7. The best results with the same base model are under- lined. Bold means CCM successfully enhances forecasting performance over the base model.
Forecast
Input Length
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
TSMixer
0.947
0.806
0.974
0.816
0.939
0.807
1.007
0.829
1.016
0.834
1.100
0.856
CCM
0.896
0.778
0.954
0.808
0.938
0,806
0.991
0.817
1.006
0.824
1.078
0.851
DLineur
1.003
0.834
0.995
0.833
0.992
0.831
0.998
0.832
0.996
0.832
0.998
0.832
CCM
0.778
0.782
0.774
0.786
0.798
PTtchTST
0.933
0.804
0.896
0.771
0.926
0.794
0,976
0.793
0,951
0.790
0.930
0.789
CCM
0.931
0.758
0.892
0.771
0.924
0.790
0.873
0.767
0.860
0.759
0.880
0.765
TimesNet
0.943
0.816
0.934
0.803
0.930
0.802
0.998
0.830
1.003
0.818
1.013
0.821
CCM
0.926
0.796
0.911
0.789
0.915
0.793
0.937
0.789
0.974
0.803
0.979
0.804
Imp. (%)
4.492
4.590
3.527
2,211
3,230
2.152
6.492
3.798
5.344
3.271
3.409
2.445
Table 19: Ablation on different clustering steps on ETTh1 and ETTml based on Linear and Trans- former architecture.
CD
C1
Random
K-Means
CCM
Linear
0.402
0.345
0.389
0.357
0.342
ETTh1
Transformer
0.861
0.655
0.746
0.542
0.518
Linear
0.404
0.354
0.371
0.326
0.310
ETTml
Transformer
0.458
0.379
0.428
0,311
0.300
F
Discussion
This paper presents the Channel Clustering Module (CCM) for enhanced performance of time series forecasting models, aiming to balance the treatment of individual channels while capturing essential cross-channel interactions. Despite its promising contributions, there still exist limitations and directions for future work that warrant consideration.
Limitation. While CCM shows improvements in forecasting, its scalability to extremely large datasets remains be tested. Moreover, the clustering and embedding processes in CCM introduce additional computational overhead. The efficiency of CCM in real-time forecasting scenarios, where computational resources are limited, requires further optimization.
Future Works. Future research can focus adapting CCM to specific domains, such as biomedical signal processing or geospatial data analysis, where external domain-specific knowledge can be involved in the similarity computation. Furthermore, exploring alternative approaches to develop dynamical clustering mechanism within CCM could potentially enhance forecasting efficacy. is also worth examining the effectiveness of CCM in contexts beyond time series forecasting.
Social Impact. The Channel Clustering Module (CCM) presented in this paper holds significant potential for positive social impact. By improving the accuracy and efficiency of forecasting, CCM can benefit a wide range of applications critical to society. For instance, in healthcare, CCM could enhance the prediction of patient outcomes, leading to better treatment planning and resource allocation. Additionally, in financial markets, CCM could aid in predicting market trends, supporting informed cecision-making and potentially reducing economic risks for individuals and organizations. Overall, the development and refinement of CCM could potentially enhance the quality of life and promote societal well-being.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's s contributions and scope?
Answer: [Yes]
Justification:
Clearly state the contribution in the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer this question will be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected generalize other settings.
It fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Discussed the limitations in conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specificatio, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Question:
Answer: [NA]

---

Justification:
the paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data provided or not)?
Question:
Answer: [Yes]
Justification: Provided implementation and hyperparameters in Appendix C.3.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments. No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily
algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. large language model), then there should either way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for repro- ducibility. In the case of closed-source models, it may be that access to model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
Open access data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
25

---

Answer: [Yes]
Justification:
The code is available at Graph-ande and- haa.retrrri-eaaaannnanneeeeeeeee
https://githubbccm com/
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. ec/ mnnides/CodeSubmt
Policy) for more details.
While we encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. cttps.cccppppo wwtppccccc/aceecccccccccccctttce iigide//ooddeiiii wwt.sumiisisiiiiii. sni aassenoolicyy Pollcy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information as possible in supplemental material (appended to the paper) is recommended, including URLs to data and code is permitted.
Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Provide training and test details in Appendix C.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Provided error bars in Appendix C.6.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed emrrr)).
26

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer:
[Yes]
Justification:
Provided details in Appendix C.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethies
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code Ethics https://neurips http./cpcc//.etciceeeittttto
Question:
Answer:
[Yes]
Justification:
well conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Discussed the social impacts in Appendix F.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied to particular upplications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: the paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Cited the original paper that produced the dataset used in this work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi theode. com comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Will document the newly released dataset and well specify the license after paper acceptance.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "xat0OVopoL", "reviewer": "Reviewer_9puR", "summary": "The paper proposes a new Channel Clustering Module and a corresponding Cluster Loss to group similar channels using a cross-attention mechanism. This creates a hybrid between channel independent and channel dependent approaches. Experiments in the paper compare both with and without the proposed module on a variety of existing state of the art methods and are run on time-series datasets in different domains on both short and long-term time horizons and demonstrate that this approach is generally applicable regardless of the domain. These experiments show an improvement in performance over prior work while also enabling zero-shot forecasting. The clustering produced by this module surfaces relationships between channels which are useful for feature analysis and interpretability.", "questions": "- In section 4.1, why shuffle channels in each batch? Channel independent implies each channel is handled individually while the motivating toy experiment here changes the channel every batch. What happens when not shuffling the channels?\n- How does the prediction error change when the prediction time horizon is changed between training and inference? Is it possible to modify the time horizon after training?\n- In section D.3, how do you use random and k-means for cluster assignment? Additional details about these experiments would help clarify quality improvement described in the table.\n- How do you expect the performance of the CCM to change when the channel count is on the order of millions?\n- What does a cluster ratio of 1.0 mean?", "limitations": "The authors list that the work needs to be tested on large datasets and that there is an additional performance overhead of the proposed CCM module.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and easy to comprehend and has good mathematical background. The CCM and the cluster loss is well motivated and utilizes the Gumbel-softmax to represent cluster assignment in a differentiable manner. The technique is evaluated on a variety of time series datasets across diverse domains in both a short and long-term setting. This demonstrates that the method is useful across different models and generally reduces error. The computational complexity of the CCM in the inference setting is also included.", "weaknesses": "The paper is evaluated on a variety of real-world datasets but evaluation on some synthetic data to highlight scenarios where existing methods produce a higher error but adding the clustering module reduces the error would help solidify the claims of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable feedback and positive comments.  We address the potential concerns as follows.\n\n>Adding evaluation on synthetic dataset will help solidify the claims.\n\nWhile synthetic datasets serve their purpose in controlled experiments, our study's emphasis on real-world datasets is crucial and more reasonable for validating the practical applicability and robustness of the proposed CCM. Unlike synthetic datasets generated in previous studies, which may not encompass the full spectrum of real-world complexities, we leverage diverse real-world benchmarks. These datasets inherently capture intricacies such as varied data distributions, noise levels, and nuanced interdependencies between channels in real-time forecasting scenarios. This focus allows us to demonstrate CCM's effectiveness in addressing practical challenges across different domains, highlighting its superiority over synthetic benchmarks that often fail to replicate such complex interactions.\n\n>Why shuffle channels in each batch? What happens when not shuffling the channels?\n\nShuffling channels aim to remove channel identity information. In the toy experiment in Sec 4.1, we train the model in two patterns: A) the model is trained on the original dataset and B) The model is trained on the randomly shuffled dataset.  Let $W_i$ represent the weights for the $i$-th channel. In Pattern A, $W_i$ is optimized upon the $i$-th channel only, while in Pattern B, the optimization of $W_i$ will lose channel identity information, which causes performance degradation on both CI and CD models (from Table 1). Therefore, this toy experiment justifies that channel identity information benefits model performance in general.\n\n>How does the prediction error change when the prediction time horizon is changed between training and inference? Is it possible to modify the time horizon after training?\n\nYes, it is possible to adjust the forecasting horizon after training, although it's not typically done in practice. When shortening the forecasting horizon post-training, the prediction error remains relatively stable because the training loss averages errors over each future step. However, if extending the horizon (e.g., from H1 to H2 where H2 > H1), the conventional approach involves forecasting H1 steps first, then using that prediction as input to forecast the subsequent H2 - H1 steps. This sequential process tends to increase prediction error due to accumulated inaccuracies over the extended horizon.\n\n>How do you use random and k-means for cluster assignment? \n\nIn the Random method, each channel is assigned to clusters with uniform probability. We use *sklearn.cluster.KMeans* to implement the k-means algorithm with a default maximum number of iterations (*max_iter*=300). The input channel embeddings remain the same as that of our proposed Cluster Assigner. We keep the number of clusters consistent for three clustering methods in the ablation study. We will add details in our revised version.\n\n>How do you expect the performance of the CCM to change when the channel count is on the order of millions?\n\nOur clustering technique, which scales linearly with respect to channel counts, has been tested to efficiently handle large datasets. Numerous experiments across diverse datasets, including large-scale datasets such as traffic (15,122,928 observations) and stock (13,900,000 observations), provide strong evidence for CCM's promising scalability on large-scale datasets: 1) CCM consistently reduces model complexity for CI models (e.g. DLinear and PatchTST), regardless of dataset scale or size (Fig. 5); 2) The linear scaling w.r.t channel count enables efficient handling of high-dimensional data, crucial for many practical applications; and 3) our experiments reveal that the performance gains achieved by CCM are maintained across different dataset sizes, suggesting that the method's benefits are robust and not limited to specific data scales. Therefore, CCM is supposed to effectively group channels based on similarities while maintaining optimal computational efficiency.\n\n>What does a cluster ratio of 1.0 mean?\n\nA cluster ratio of 1.0 indicates that the number of clusters equals the number of channels, but it does not imply channel independence. In our experiments, even with a cluster ratio of 1.0, we observed clustering phenomena in channel embeddings, albeit with some empty clusters. This ratio is maintained for experimental rigor and to validate the robustness of the model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Mlywi8wVj", "reviewer": "Reviewer_yXcm", "summary": "The paper presents a new Channel Clustering Module (CCM) for time series forecasting, which dynamically groups channels based on intrinsic similarities to balance the strengths of Channel-Independent (CI) and Channel-Dependent (CD) strategies. CCM improves forecasting accuracy by enhancing model generalization and capturing essential cross-channel interactions, achieving significant performance gains in both long-term and short-term forecasting. The module also supports zero-shot forecasting and improves the interpretability of complex time series models. Extensive experiments demonstrate CCM's effectiveness and adaptability across various mainstream time series models.", "questions": "1. How can the similarity metric be further refined to enhance the clustering quality and address performance variations across different domains?\n\n2. What strategies can be employed to test and ensure the scalability of CCM for extremely large datasets?\n\n3. How does CCM perform in real-world applications with diverse and dynamic datasets, and what adjustments are necessary to optimize its performance in such scenarios?\n\n4. What techniques can be implemented to mitigate the increased computational overhead introduced by CCM, especially for models with high channel and cluster counts?", "limitations": "1. CCM does not outperform CI/CD strategies in a few cases, possibly due to the underlying channel relationships in certain real-world domains not aligning well with the similarity metric used by CCM.\n\n2. The increased model complexity introduced by CCM may lead to higher computational costs, particularly for models with numerous channels and clusters.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a new Channel Clustering Module (CCM) that balances individual channel treatment and cross-channel dependencies, combining the strengths of Channel-Independent (CI) and Channel-Dependent (CD) strategies.\n\n2. CCM enables zero-shot forecasting, leveraging learned prototypes to handle unseen samples effectively.", "weaknesses": "1. The introduction of CCM increases the model's complexity, particularly for original CD models, which may result in higher computational overhead.\n\n2. The paper acknowledges that the scalability of CCM to extremely large datasets remains untested, which could be a limitation for practical applications requiring the processing of large-scale data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable feedback, which significantly improves the quality of this work. We also address below the potential concerns. \n\n>The introduction of CCM increases the model's complexity. The scalability of CCM to extremely large datasets remains untested. What strategies to ensure the scalability of CCM? What techniques to mitigate the increased computational overhead of CCM?\n\nAs discussed in Sec.4.3, the computational complexity of CCM scales linearly with the number of channels $C$, which is computationally efficient for real-world deployment. Our extensive experiments, including large-scale datasets such as traffic (15,122,928 observations) and stock (13,900,000 observations), provide strong evidence for CCM's promising scalability on extremely large-scale datasets: \n- CCM consistently **reduces model complexity for CI models** (e.g. DLinear and PatchTST), regardless of dataset scale or size (Fig. 5).\n- The **linear scaling w.r.t channel count** enables efficient handling of high-dimensional data, crucial for many practical applications.\n- Our experiments reveal that the **performance gains achieved by CCM are maintained across different dataset sizes**, suggesting that the method's benefits are robust and not limited to specific data scales. \n\nThis demonstrates the efficiency and scalability of CCM, making it promising for extremely large-scale real-world deployments. \n\nFurthermore, we've identified several strategies to optimize the scalability of CCM, including leveraging parallel and distributed frameworks for cluster assigner training and applying sampling on time series similarity computation to optimize computational resources. Algorithmic optimizations, such as efficient and fast attention with linear time complexity further support CCM's scalability. However, we would like to emphasize that **these techniques are compatible and orthogonal to the contribution of this manuscript** and deserve substantial additional research that would expand beyond our current scope. We appreciate your suggestion and will expand our discussion on computational efficiency in the revised version.\n\n>How can the similarity metric be further refined to enhance the clustering quality and address performance variations across different domains?\n\nThe similarity metrics and alternatives are discussed thoroughly in Appendix A.1. We select the Euclidean-based similarity metric due to its efficiency and generalization. Our extensive experiments also justify the effectiveness and robustness of this metric in evaluating the cross-channel similarity (see evidence in Appendix E) and consistently enhancing model performance.\n\nThe similarity metric can be refined by incorporating domain-specific feature selection and engineering or dynamically adjusting parameters based on data distribution characteristics. These techniques are compatible and orthogonal to the proposed method, so we consider them as future work. Again, we would like to emphasize that the current metric already demonstrates the efficiency across different domains and serves as a generalizable tool to evaluate channel similarity.\n\n>How does CCM perform in real-world applications with diverse and dynamic datasets, and what adjustments are necessary to optimize its performance in such scenarios?\n\nFirstly, we clarify that **our experiments already showcase CCM’s robust performance in real-world applications with diverse datasets** (such as electricity, traffic, etc.). As mentioned in Line 246-247, CCM improves long-term forecasting performance in 90.27% of cases in MSE and 84.03% of 247 cases in MAE across 144 different experiment settings. Moreover, **CCM's versatility extends beyond scenarios where cross-channel relations remain static over time**. It supports dynamic clustering across different minibatches and time steps, which is crucial for practical applications. Due to the space limit, we discussed possible improvements such as incorporating dynamic similarity metrics or using domain-specific similarity metrics in Appendix F.\n\n>Explanations of a few cases that CCM does not outperform CI/CD strategies.\n\nAs we clearly discussed in Appendix C.5, CCM method is more useful in scenarios where channel interactions are complex and significant, which is usually the case in real-world data. Therefore, in datasets where channels are nearly independent or where the inter-channel relationships are not as pronounced, the benefits of CCM's clustering mechanism may be less significant. \n\nWe would like to emphasize that the average improvement rate across all cases (with different base models / datasets / forecasting lengths) is 2.443% in the long-term and 6.456% in short-term benchmarks. Specifically, CCM improves the performance in 90.27% of cases in MSE and 84.03% of cases in MAE in long-term benchmarks, given the context that the base models are already optimized for high performance. Therefore, any additional gains through CCM are noteworthy and indicative of its effectiveness in refining forecasting performance. \n\n\nWe hope that the above clarification improves your confidence in our work. Let us know if you have any further questions/concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P7WabcpgmD", "reviewer": "Reviewer_rxdY", "summary": "Time series forecasting has been a topic of interest, with previous studies exploring different strategies. The Channel-Independent (CI) strategy treats channels individually, improving forecasting performance but lacking generalization and ignoring channel interactions. On the other hand, the Channel-Dependent (CD) strategy combines channels indiscriminately, leading to oversmoothing and reduced accuracy. A channel strategy is needed that balances individual treatment and essential interactions. Based on the correlation between performance and channel mixing, a novel Channel Clustering Module (CCM) was developed. CCM groups channels with intrinsic similarities and utilizes cluster information, combining the advantages of CD and CI. Experimental results show that CCM enhances the performance of CI and CD models, enables zero-shot forecasting, and improves interpretability of complex models by uncovering intrinsic patterns among channels.", "questions": "1. Can the author provide the percentage increase in runtime and memory usage after integrating CCM for each method on the Traffic dataset?\n2. Can a certain hyperparameter analysis be conducted for the parameter β (i.e., β ∈ {0, 0.25, 0.5, 0.75, 1.0}) to demonstrate the impact of the Cluster Loss on model performance?\n3.  Can the author provide a significance test (i.e., p-value test) for the experimental results, especially for cases where the values in Table 4 are close? As in some cases, the improvements are minimal, and the observed performance gains might be due to randomness rather than the effectiveness of CCM.", "limitations": "The author elaborates on the limitations of their work. The main limitations are as follows: CCM does not outperform CI/CD in a few cases in the long-term forecasting benchmark; CCM’s scalability to extremely large datasets remains to be tested.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed model-agnostic method CCM achieves optimal performance between single-channel and cross-channel modeling, and it can be integrated into existing time series prediction models to enhance their performance.\n2. By learning prototypes from clusters, CCM facilitates zero-shot forecasting on unseen samples, whether in univariate or multivariate scenarios.\n3. The author integrated CCM into four mainstream time series prediction models on multiple different datasets. The experimental results demonstrate that in most cases, CCM can bring about significant performance improvements.", "weaknesses": "1. The experimental section involves a limited number of baseline methods, for example, SOTA LLM-based time series prediction models [1, 2] were not selected.\n2. I noticed that CCM introduces additional model complexity, as it has an independent Feed Forward layer for each cluster. When the value of K is large, this may result in an excessive number of Feed Forward layers, leading to a significant increase in space complexity. On the other hand, the time complexity of CCM is linearly related to K and C. For certain datasets with a higher number of channels (e.g., Traffic), CCM may noticeably increase the time complexity of the base model. Some of the results in Figure 5 of the Appendix hint at this issue.\n3. According to Table 14, the improvements brought by CCM to the base model are sometimes overshadowed by the perturbation caused by randomness. This may indicate that CCM has certain limitations.\n\n[1] Zhou, Tian, et al. \"One fits all: Power general time series analysis by pretrained lm.\" Advances in neural information processing systems 36 (2023): 43322-43355.\n[2] Jin, Ming, et al. \"Time-llm: Time series forecasting by reprogramming large language models.\" arXiv preprint arXiv:2310.01728 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable feedback and positive comments.  We address the potential concerns as follows.\n\n>Some LLM-based models were not selected as baselines.\n\nAfter careful consideration, we've decided not to include LLM-based time series models in this manuscript for several reasons. Firstly, many LLM-based models treat multivariate time series as independent univariate series, which doesn't align with our core assumption about the importance of cross-channel interactions [1,2]. Secondly, some LLM-based methods that concatenate variates into a single long series [3] are incompatible with CCM's clustering approach, as the clustering will destroy the attention calculation along the single univariate time series. While we acknowledge the growing importance of LLM-based methods in time series forecasting, investigating clustering effects on these models requires substantial additional research that would expand beyond our current scope. We believe this warrants a separate, dedicated study. Our focus on traditional deep learning models allows for a thorough analysis of CCM's effectiveness within an established framework. We are more than happy to add discussions in our revised version.\n\n[1] One fits all: Power general time series analysis by pretrained lm\\\n[2] Time-llm: Time series forecasting by reprogramming large language models\\\n[3] Unified Training of Universal Time Series Forecasting Transformers\n\n>CCM introduces additional space and time complexity. CCM’s scalability remains to be tested.\n\nAs demonstrated in Sec 4.3, CCM's computational complexity scales linearly with the number of channels $C$, ensuring efficient real-world applications. Our extensive experiments provide strong evidence for CCM's promising scalability on extremely large-scale datasets:\n- CCM consistently reduces model complexity for CI models (e.g. DLinear and PatchTST), regardless of dataset scale or size (Fig. 5).\n- The linear scaling w.r.t channel count enables efficient handling of high-dimensional data, crucial for many practical applications.\n- CCM's performance gains persist across various dataset sizes, indicating robust benefits independent of data scale. \n\nThese findings collectively underscore CCM's efficiency and scalability, making it a highly promising approach for large-scale real-world deployments.\n\n>The improvements brought by CCM may sometimes be caused by randomness. Can the author provide a significance test?\n\nWe conducted a significance test using p-values for the ETTh1 dataset in Table 4, which represents the scenario where CCM shows a minimal performance boost among our experiments. The p-value test results are as follows (values lower than 0.05 are in bold):\n|ETTh1|96|192|336|720|\n|:-:|:-:|:-:|:-:|:-:|\n|TSMixer |-|  **0.0046**  |**0.007**|**0.011** |\n|  DLinear |**0.034** |0.093|**0.041** | **0.009** |\n| PatchTST |-|**1.12E-05** |**0.0004** |**0.002** |\n| TimesNet |0.089|**0.0019**|**0.043**|0.064|\n\nCCM's improvement effect is significant in general. Even in this \"worst-case\" scenario for CCM (i.e., ETTh1 dataset), we observe significant improvements across multiple forecast horizons and base models. Specifically, the majority of the p-values (11 out of 14) are below the conventional significance threshold of 0.05, indicating statistical significance. Despite a few cases where the p-values are above 0.05, the overall trend strongly supports the effectiveness of CCM. We will include a more comprehensive significance test in our revised version.\n\n>The percentage increase in runtime and memory usage on the Traffic dataset?\n\nWe provide the percentage increase in model size and runtime on the Traffic dataset based on TimesNet and TSMixer (two CD models) as follows. \n\n$\\Delta$ Param (%): \n| BaseModel / #clusters |2|3|4|5|\n|-|:-:|:-:|:-:|:-:|\n|TimesNet| 0.052 |  0.088 |0.125 | 0.162 |\n|TSMixer| 9.845 |18.420|26.995 |35.570|\n\n$\\Delta$ iter. time (%):\n| BaseModel / #clusters |2|3|4|5|\n|-|:-:|:-:|:-:|:-:|\n|TimesNet|2.907 |3.632|5.291|5.777|\n|TSMixer| 25.746 | 41.350| 53.352 | 64.462|\n\nWe observe that CCM represents a minimal increase in memory usage and runtime on TimesNet, demonstrating its parameter efficiency when applied to TimesNet. For TSMixer, there is a greater increase compared to TimesNet. However, it's important to note that TSMixer is inherently a more compact model. The absolute increase in parameters remains relatively small and reasonable. It's worth noting that these increases should be considered in the context of the performance improvements that CCM brings, which justifies this modest increase in model size. Moreover,  when applied to CI models, (eg, DLinear and PatchTST), CCM consistently reduces the model complexity and runtime, further highlighting its versatility and efficiency.\n\n>A hyperparameter analysis for the parameter $\\beta$?\n\nWe conducted a hyperparameter analysis for $\\beta$, ranging from 0.1 to 2.0 on ETTh2 dataset (H=720) as follows. \n|$\\beta$ |w/o CCM|0.1|0.3|0.5|1.0|2.0|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|\n|TSMixer | 0.445±0.006 | 0.442±0.006 |**0.438±0.003**|0.442±0.009|0.439±0.005 |0.440±0.007|\n|DLinear | 0.601±0.008 | 0.537±0.013 |**0.499±0.012**|0.501±0.010|0.510±0.008|0.524±0.013|\n|PatchTST | 0.381±0.005 | 0.381±0.004 | 0.379±0.004 |**0.378±0.007**| 0.379±0.004 |0.381±0.009|\n|TimesNet | 0.462±0.009 | 0.460±0.006 | **0.457±0.003** |**0.457±0.003**| 0.460±0.004 |0.468±0.008|\n\nWe observe that the introduction of CCM (with appropriate $\\beta$) consistently improves performance compared to the baseline without CCM. The optimal $\\beta$ value varies slightly across different architectures, but generally falls in the range of 0.3 to 0.5. Moreover, we observe that TSMixer, PatchTST, and TimesNet show more stable performance than DLinear across different $\\beta$ values. We will include the sensitivity analysis on $\\beta$ in our revised version.\n\nWe hope that the clarification improves your confidence in our work. Let us know if you have any further concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jVmFVKjsuR", "reviewer": "Reviewer_SRyH", "summary": "The paper introduces the Channel Clustering Module (CCM), a novel approach to enhance time series forecasting models. CCM addresses the limitations of traditional Channel-Independent (CI) and Channel-Dependent (CD) strategies by dynamically clustering channels based on their intrinsic similarities. This approach allows the model to balance individual channel treatment with capturing essential cross-channel dependencies, leading to improved forecasting performance. CCM is adaptable to various time series models and demonstrates its effectiveness through extensive experiments on multiple real-world datasets.", "questions": "- Where is the official code?\n- In terms of thinking, the principal component analysis method is actually similar to the proposed method. Please provide a detailed explanation.", "limitations": "The limitation of the Channel Clustering Module (CCM) outlined in the paper includes its scalability to extremely large datasets and the computational overhead introduced by the clustering and embedding processes. While CCM shows improvements in forecasting, its efficiency in real-time forecasting scenarios with limited computational resources remains to be optimized. Additionally, the clustering and embedding processes in CCM introduce additional computational overhead, which could be a concern in scenarios where computational efficiency is critical.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The originality of CCM lies in its novel approach to channel clustering for time series forecasting, addressing the limitations of existing strategies.\n- The quality of the work is evident in the well-designed experiments and the clear presentation of results, showcasing the effectiveness of CCM across different datasets and models.\n- The paper’s clarity in explaining the concept, methodology, and results enhances its readability and understanding.", "weaknesses": "- The paper could benefit from more detailed discussions on the selection of similarity metrics and the impact of hyperparameters on performance.\n- The computational efficiency of CCM, especially in large-scale applications, is not extensively discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable feedback and positive comments.  We address the potential concerns as follows.\n\n>The computational efficiency of CCM, especially in large-scale applications, is not extensively discussed.\n\nAs discussed in Sec.4.3, the computational complexity of CCM scales linearly with the number of channels $C$, which is computationally efficient for real-world deployment. Our extensive experiments, including large-scale datasets such as traffic (15,122,928 observations) and stock (13,900,000 observations), provide strong evidence for CCM's promising scalability on extremely large-scale datasets: 1) CCM consistently **reduces model complexity for CI models** (e.g. DLinear and PatchTST), regardless of dataset scale or size (Fig. 5); 2) The **linear scaling w.r.t channel count** enables efficient handling of high-dimensional data, crucial for many practical applications; and 3) our experiments reveal that **the performance gains achieved by CCM are maintained across different dataset sizes**, suggesting that the method's benefits are robust and not limited to specific data scales. This demonstrates the efficiency and scalability of CCM, making it promising for extremely large-scale real-world deployments. \n\nFurthermore, we've identified several strategies to optimize the scalability of CCM, including leveraging parallel and distributed frameworks for cluster assigner training and applying sampling on time series similarity computation to optimize computational resources. Algorithmic optimizations, such as efficient and fast attention with linear time complexity further support CCM's scalability. However, we would like to emphasize that these techniques are compatible and orthogonal to the contribution of this manuscript and require substantial additional research that would expand beyond our current scope. We appreciate your suggestion and will expand our discussion on computational efficiency in the revised version.\n\n>Where is the official code?\n\nAs we mentioned in Line677 (Appendix C.3), the code is available at the following anonymous link: https://anonymous.4open.science/r/TimeSeriesCCM-4E83. We will have an official GitHub repository once the paper is accepted.\n\n>The principal component analysis method is actually similar to the proposed method. Please provide a detailed explanation.\n\nPCA and the proposed CCM serve fundamentally **different purposes and operate on distinct principles**.\n\nPCA is primarily a dimensionality reduction technique that transforms high-dimensional data into a lower-dimensional space, preserving maximum variance. It does not inherently cluster data but rather restructures it into orthogonal components. In contrast, CCM is specifically designed for time series forecasting, focusing on grouping channels based on their intrinsic similarities to enhance prediction accuracy and interpretability. CCM dynamically clusters channels into cohesive groups, leveraging these similarities to capture complex inter-channel dependencies. It enhances forecasting by allowing models to treat clusters as coherent entities, thus improving both individual channel fit and cross-channel interactions.\n\nEssentially, while PCA emphasizes variance capture and dimensionality reduction, CCM prioritizes similarity-based clustering to optimize time series analysis and forecasting.\n\n>Detailed discussions on the selection of similarity metrics and the impact of hyperparameters on performance would benefit the paper.\n\nThe selection of similarity metrics and other alternatives are discussed thoroughly in Appendix A.1. We also conducted ablation studies on cluster ratio and look-back window length to investigate their impact on model performance. Please refer to Appendix D for detailed results and analysis.\n\nWe hope that the above clarification improves your confidence in our work. Let us know if you have any further questions/concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposed to model the correlation between channels to improve  time series forecasting via an attention-based channel clustering method. Experimental results have demonstrated the effectiveness of the proposed channel clustering approach. Although the channel clustering approach is new, the channel dependency modeling  is not the first time explored in literature, e.g., [1-2]. A detailed discussion is necessary.

[1] Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators

[2] Enhancing Multivariate Time Series Forecasting with Mutual Information-driven Cross-Variable and Temporal Modeling

Some reviewers have concerns on the computation complexity and scaling ability. Authors have provided responses to these questions.
Decision: Accept (poster)