CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecasting
Authors: Jiyuan Xu
Wenyu Zhang
Xin Jing
Jiahao Nie
Shuai Chen
Shuai Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Multivariate Time Series Forecasting
Channel Permutation Invariance
Spatio-temporal Decoupling
Meta-Learning
Foundation Models
Primary area: learning on time series and dynamical systems
TL;DR: CPiRi enables channel-permutation-invariant MTSF by combining frozen temporal encoding with lightweight spatial attention trained via channel shuffling, achieving SOTA accuracy with zero performance drop under dynamic sensor changes.

Abstract: Current methods for multivariate time series forecasting can be classified into channel-dependent and channel-independent models. Channel-dependent models learn cross-channel features but often overfit the channel ordering, which hampers adaptation when channels are added or reordered. Channel-independent models treat each channel in isolation to increase flexibility, yet this neglects inter-channel dependencies and limits performance. To address these limitations, we propose CPiRi, a channel permutation invariant (CPI) framework that infers cross-channel structure from data rather than memorizing a fixed ordering, enabling deployment in settings with structural and distributional co-drift without retraining. CPiRi couples spatio-temporal decoupling architecture with permutation-invariant regularization training strategy: a frozen pretrained temporal encoder extracts high-quality temporal features, a lightweight spatial module learns content-driven inter-channel relations, while a channel shuffling strategy enforces CPI during training. We further ground CPiRi in theory by analyzing permutation equivariance in multivariate time series forecasting. Experiments on multiple benchmarks show state-of-the-art results. CPiRi remains stable when channel orders are shuffled and exhibits strong inductive generalization to unseen channels even when trained on only half of the channels, while maintaining practical efficiency on large-scale datasets. The source code is released at https://github.com/JasonStraka/CPiRi.

Published as a conference paper at ICLR 2026
CPIRI: CHANNEL PERMUTATION-INVARIAN RELATIONAL INTERACTION FOR MULTIVARIATE TIME SERIES FORECASTING
Jiyuan Xu, Wenyu Zhang, Xin Jing, Jiahao Nie, Shuai Chen, Shuai Zhang' School of Information Technology and Artificial Intelligence Zhejiang University of Finance and Economics Hangzhou, China
(strakal, jingxin, mozo, zhangshual l @zufe. www.fe.edu.com wyzhangée
sg, jhnie@hdu edu. en
ABSTRACT
Current methods for multivariate time series forecasting can be classified into channcl-dependent and channel-inee -innneeeiddeeendentt models. Channel-dependent models learn cross-channel features but often overfit the channel ordering, which hampers adaptation when channels treat each channel in isolation to increase flexibility, yet this neglects inter-channel dependencies and limits performance. To address these limitations, propose CPiRi, channel permutation invariant (CPI) framework that infers cross- channel structure from data rather than memorizing fixed ordering, enabling deployment in settings with structural and distributional co-drift without retrain- ing. CPiRi couples spatio-temporal decoupling architecture with permutation- invariant regularization training strategy: a frozen pretrained temporal en- coder extracts high-quality temporal features, lightweight spatial module learns content-driven inter-channel relations, while channel shuffling strategy enforces CPI during training. further ground CPiRi theory by analyzing permuta- tion equivariance in multivariate time series forecasting. Experiments multiple benchmarks show state-of of the-art art results. CPiRi remains stable when channel or- ders are shuffled and exhibits strong inductive generalization to unseen channels even when trained on only half the channels, while maintaining practical effi- ciency on large-scale datasets. The source code is released Strasnntaaaa/CPPiiii
added reordered. Channel-independen models
INTRODUCTION
Multivariate time series forecasting (MTSF) is critical in domains like finance and transportation, where modeling inter-channel relationships is essential (Zhang et al.] (0025). Research in this area has largely bifurcated into two paradigms with paradoxical trade-off: chamnel-independent (CI) and channel-dependent (CD) models (Shao et al. 2025b).
CD models, spanning architectures from graph neural networks (GNNs) Transformers, explicitly model relational interactions across channels. Despite their sophisticated designs, many exhibit critical limitation: they overfit the static positional configurations of training data rather than learning semantic relationships. As illustrated in Fig. []] these models memorize channel order instead of content-driven dependencies. Consequently, they suffer catastrophic degradation when encountering channel permutations or new channels during inference, which are common scenarios real-world dynamic systems and typical of structural co-drift real deployments (e.g., sensor networks or evolving financial metrics). This positional rigidity fundamentally undermines their robustness and adaptability. Conversely, CI models (c.g.. DLincar (Zeng et al. 2023), PatchTST Nie et al. 2023) process channels independently, ensuring robustness against noise and channel heterogeneiry (Shao et al.) 2025b). However, they ignore cross-channel interactions, sacrificing the core advantage of multivariate analysis and limiting forecasting performance.
"Corresponding author

---

Published as a conference paper at ICLR 2026
Channel Dependent
Channel discriminability Chroose interaction
Ovain
Permutate ABC
+BAC
CPI
model
1
model
GPL
CRI
Channel Independent
J
Channel discriminability XCross interaction
train
Permutate
BAC
model
model
CPI.
model
model
CPI
model
model
CP
C CiiRi (ours)
CiChannel discrininability chrose Interaction
F rezen Encuder
Temperal inpeesntatoo
Spatial 'Moduee
Spatfall--wwam rrynsesbybbo
F rozen Decoder
Charme Shuffling
Permutate ABC
BAC
CPL
Channe-AA
Frozen Encoder
Spatial Modide
F Fruzen
CPI
Decader
Chasnel-3
CPL
CaannelCC
Conceptual overview of the CPiRi framework. Top: channel- channel-dependent models often overfit to channel order, leading performance collapse in channel permutation invariance (CPI) tests. Middle: channel-independent models lack cross-channel interaction. Bottom: CPiRi resolves this by (1) using a frozen encoder for robust temporal feature extraction, and (2) training chan- nel permutation--qquivaiiii
Figure 1:
spatial module with channel shuffling strategy to learn generalizable, content- based relationships, ensuring both high accuracy and invarintt robustness under chan- nel shuffling.
This dichotomy presents fundamental dilemma: CD models capture essential interactions but lack adaptability, while CI models ensure robustness at the cost of relational reasoning. To address this, we posit that models should achieve channel permutation invariance (CPI), i.e. maintaining per- formance regardless of channel ordering or additions, without compromising interaction modeling. To validate this need, we introduce a CPI diagnostic: models genuinely understanding channel re- lationships should exhibit stability under channel shuffling. Our tests reveal alarming fragility in state-of-the-as (SOTA) CD models. For instance, on PEMS-08 (Sec. 4.2). Informer's error in- creases by >400% under channel shuffling, confirming reliance positional memorization rather than relational reasoning. The ultimate goal of MTSF is not merely to pass permutation test but develop models that are both accurate and generalizable. The CPI diagnostic serves as a crucial litmus test that exposes index- and -egelogg-ddeendeneee a failure mode that blocks deployment under co-drift.
In this paper, we propose CPiRi (Channel Permutation-Invann Relational Interaction), novel framework that models cross-channel dependencies in CPI manner while synergizing CI and CD strengths. CPiRi integrates two core innovations: tecture and permutation -nnvaaiiiinnnniiantt regularization based training strategy. The architecture consists of frozen pre-trained temporal encoder and lightweight, trainable spatial module. The en- coder extracts channel-specific features independently, preserving CI advantages (robustness, noise immunity). The spatial module then models relational interactions across channels using only content-bass based cues, climinating positional bias. This decoupling isolates temporal learning from relational reasoning, enabling the model to inherit CI stability while capturing CD interactions.
decoupling based model archi-
The permutation-invarian regularization based training strategy employs dynamic channel shuffling during training, which removes positional shorteuts and forces the spatial module leam content- driven relational reasoning. By exposing the model to diverse channel configurations, acquires a meta-skill for generalizable interaction modeling, ensuring CPI without sacrificing accuracy.
Our contributions are as follows:
2

---

Published as a conference paper at ICLR 2026
The CPiRi framework, resolving the CI-CD trade-off via channel permutation invarrian relational interaction.
A spatio-temporal decoupled architecture combining a frozen temporal encoder (CI) and content-aware spatial module (CD).
A permutation-invariant regularization strategy using channel shuffling enforce content- driven relational reasoning.
SOTA results on benchmarks with: negligible degradation under channel shuffling; strong inductive generalization to unseen channels (e-g.. trained on only half the channels) without retraining; and improved robustness in low-data regimes, where the regularization strategy is most beneficial.
2 RELATED WORK
Channel-dependent models. CD approaches represent the mainstream of MTSF research. How- ever, early methods like GNNs (e.g.. MTGNN (Wu al.] (0200) learn rigid, static graphs, while subsequent Transformer-based models (e.g.. Crossformer (Zhang & Yan 2023) often overfit to channel order by treating channels as a fixed sequence. While many recent studies propose sophis- ticated architectures to improve robustness generalization, such as through adaptive hypergraphs
(Chen et al. 2025), channel-aligned transformers (Xue et al., 2024). dual unmixing structures (Zhu al.; 2024 context-aware transformers (Hong et al.) 2025). by focusing explicitly latent space generalization (Deng et al.) 2024), they tend to add model complexity without resolving the fundamental positional dependency. Furthermore, recent large models like Timer-XI (Liu et al., 2024b) are ill-suited for typical MTSF tasks. They demand extensive pre-training and are computa- tionally expensive, leading to poor performance on smaller datasets and practical scaling issues due to memory constraints. The framework of CPiRi, while primarily designed for CPI, offers a critical side-benefit: it resolves this dilemma by avoiding the need for massive data and complex spatial pre-training, making it a more practical and effective solution.
Channel-independent models. CI approaches, exemplified by DLinear ZZnng et al. 2023) and the SOTA PatchTST (Nie et al., 2023). prioritize robustly modeling each channel's 's temporal patterns independently. While efficient and inherently is the neglect of cross-channel dynamics. A new frontier involves large foundation models like Chronos-Bolt (Ansari et al., 2024) and Sundial (Liu et al.) 2025). These models are often discussed in the broader context of applying large model concepts to time series analysis (Jin et al. (024) and are trained on diverse datasets (Shao et al. 2025a), with some proposing novel objectives like pre- dicting curve shapes (Feng al. 2024). However, work is one of the first to effectively integrate a powerful, pre-trained univariate foundation model (Sundial) into a multivariate task. By using Sundial frozen feature extractor, CPiRi isolates the challenge of relational learning, allowing smaller, specialized module to focus entirely on learning generalizable spatial interactions.
iettuuttttionnvvviaann their fundamental limitation
METHODOLOGY
3.1 PROBLEM DEFINITION
The standard MTSF problem can be formalized follows. For multivariate time series containing C' channels (also referred to as variables or nodes), given historical data X' {X1,.... X²) ) e R4xC, where X, E RC is the vector of values for all channels at time step i, and L is the look-back window size. The time series forecasting task is to predict the values y {Xr.+1, XI+T} e RTxC for the next T time steps.
3.2 FRAMEWORK OVERVIEW
As illustrated in Fig. 2 the CPiRi framework processes multivariate time series through three dis- tinct, sequential stages, strategically isolating temporal and spatial learning. This decoupled design allows us to leverage powerful, pre-trained temporal priors while focusing the training entirely on learning& a generalizable relational reasoning skill. Then the permutation-invariint regularization strategy forces the model to learn generalizable, content- content-based relationships, moving beyond mem- orizing channel order.
3

---

Published as a conference paper at ICLR 2026
Time Stypt
Feregh
Eatheldiut
Emmhe dading
Triar Stops
Patch Numbbe
Perch
Chorrs
rozen
Spotial Module
Frozen Decader
Enceder
Dnpot
Stage
Stage 2
Stage 3
Output
Figure 2: Architectural overview of the CPiRi framework. CPiRi operates in three stages: (1) A frozen univariate foundation model (Sundial Encoder) independently extracts temporal features for each channel. (2) A lightweight, trainable spatial module processes the set of channel representa- tions to model permutation invaarian interactions. (3) The frozen Sundial Decoder independently generates the final forecast for each channel from its updated representation.
3.3 RADICAL SPATIO-TEMPORAA DECOUPLING MODEL.
The three-stage stage architecture embodies philosophy of decoupling, strategically leveraging pre- trained model for temporal features and a lightweight module for spatial relationships.
Stage 1: universal temporal feature extraction. For given input X' E R each of the C' channels processed isolation by the frozen encoder of the Sundial foundation model (Liu al. 2025). From the encoder's output, extract the final patch representation for each channel, yielding a set of temporal feature vectors (hi....,hy hc), where each h, E TR a D-dimensional feature vector.
Stage 2: prrmutation-equivariin spatial interaction. This stage is the core trainable component of CPiRi. The of temporal representations {hi..... he) is treated as an unordered set and fed into a lightweight spatial module, which consists of standard Transformer encoder block. The multi-head self-attention mechanism within this block is inherently permutation- -euuattiinnniiiiiiiiinnaat making the architecture structurally robust to the order of its inputs. The sole purpose of this module is to learn a generalizable function that models inter-channel dynamics based on the content of the feature vectors. The output is a set of spatially- representations (h',... h'c,, where each vector has been enriched with context from other relevant channels.
Stage 3: independent prediction generation. In the final stage, each enriched representation h; is independently passed to the frozen Sundial (Liu et al. 2025) decoder to generate the forecast result. This final independent step reinforces the model 's robustness by preventing structural entanglement at the generation stage, thereby completing our decoupled design.
This decoupling and frozen design overcomes two fundamental limitations. First, it transfers robust temporal priors learned from large-scale external datasets, effectively mitigating data scareity in dy- namic MTSF tasks. Second, it establishes modular paradigm where the spatial module focuses exclusively on learning cross-channel dynamics. This specialization avoids the prohibitive compu- tational cost of retraining monolithic architectures while simultaneously addressing inefficiency in capturing sparse relational patterns.
3.4 PERMUTATION-INVARAAT REGULARIZATIO STRATEGY
The key to unlocking CPiRi's generalization capability is its training strategy, which we frame as form of meta-learning. By exposing the model to distribution of different channel orderings, compel it to learn transferable "meta-skill" for relational reasoning that is agnostic to the arbitrary indexing of channels. The procedure is detailed in Algorithm I During each training step, we apply a random permutation to the channels of both the input batch X and the target batch Y. The processes channels independently. The spatial module,
frozen temporal encoder is unaffected, however, receives randomly ordered set of channel representations. To consistently minimize the loss, the module cannot rely on positional cues (e.g., "the 3rd channel is always noisy"). Its only recourse is to learn function that identifies relationships based on the intrinsic content of the temporal feature vectors themselves. This forces the model to transition from memorizing static correlations to learning generalizable relational reasoning capability. While our training strategy shares the high-level goal of improving generalization (similar to meta-leamning), it operates through saariance-focusee data augmentation rather than task adaptation.
4

---

Published as a conference paper at ICLR 2026
Algorithm 1 Permutation inwooionnnivaiiinnnn regularization strategy via channel shuffling
Require:
Training dataset D Drain with C' channels
1: for cach batch (X, Y) Diran do
Generate random permutation
IIc
Apply the same permutation: X,, Yr
4: Compute prediction: Yprea CPiRi(Xa))
5: Compute loss: C Loss(Y prea: Y²)
Back-propager C update spatial module
7: end for
3.5 THEORETICAL FOUNDATION: GENERALIZABLE REASONING VIA PERMUTATION INVARIANCE
The empirical success of channel shuffling is grounded in the mathematical principle of permutation invariance. This section provides the theoretical argument for how our methodology guarantees that CPiRi learns generalizable, content-based based relationships.
hc] be the set of temporal features The principle of ietpuuttttioneuvviiiiintt Let (RDj that maps H. to contextualized from Stage 1. The spatial module is function / representations H'. A function f is permutation-equivvaaaat cnmmmuttationequvvaiiinn equivariant if for any permutation n, it holds that (f(H)=())..... (P)) In essence, permuting the inputs only permutes the outputs in the same way, as illustrated in Fig. 3 Without explicit constraints, a standard CD) model might learn non-equivariant function by overfitting positional artifacts (e.g., learning specific mapping for the 3rd channel), leading to catastrophic failure in our channel shuffling test. Since the encoder/decoder of CPiRi independently channels (invariant) and the spatial module is equivariant, the full pipeline of CPiRi is equivariant by closure.
(h (c))
Enforcing CPI through permutation-invarian regularization. Our training objective is mini- mize the expected loss over the distribution of all possible permutations II
min (xx x~lle (X+)) V+)]
6
where represents the trainable parameters the spatial module f. Any non-equivariant component within fo that relies specific ordering will incur high loss for most permutations, failing to minimize the expected loss. The only stable solution is one that inherently equivariant. Thus, the optimization process naturally drives the learned function Se be permutation-equivarian.,
From invariance to relational learning. Foundational work like Deep Sets (Zaheer et al. 2017) function
must be decomposable into the form (h,) p(h, $ d(hy)), where ø is an element-wise transformation, ① a symmetric aggre- gation function (c.g., sum, mean), and p is a combination function. The self-attention mechanism is canonical implementation of this structure, as it computes the output for an element h, as weighted sum of transformations of all elements in the set, with weights determined by content- based similarity.
has shown that any sermutation-equivariin
In conclusion, direct logical chain links our methodology to its outcome: the channel shuffling strategy (the algorithm) necessitates pernutation-equuvaaiiiie function (the mathematical property), which turn requires an architecture based on symmetric aggregation (the structural form), for which self-attention is the ideal implementation. This guarantees, by design, that CPiRi is forced to leamn inter-channel relationships based on semantic content, instead positional artifacts.
2
Figure 3: Channel-level permutation equivariance. Rotating or reflecting the same directed graph permutes node indices while preserving edges, yiclding equivalent adjacency matrix. This illus- trates that MTSF permutation-equivane at the channel level, and that CPI models can adapt to dynamic multivariate time series settings with reordered, added, removed channels.
5

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. Our evaluation is conducted on five widely-used public benchmark datasets: METR-LA
(Li al. 2018). PEMS- BAY (Li et al. 2023), PEMS- 04, PEMS-08 (Chen et al. 200T), SD (a subset
of LTrgeST (Liu et al. 2023), Electricity (Lai et al. 2018 These datasets exhibit strong channel heterogeneity, where sensors capture diverse spatial relationships and dynamic patterns, making them idcal for evaluating MTSF modeling capabilities. For scalability analysis, we additionally include the larger LargeST subsets GBA (2,352 channels), GLA (3,834 channels), and CA (8,600 channels). We prioritize traffic datasets because they exhibit strong cross-channel dependencies and dynamic sensor networks in practice, while datasets with weaker channel heterogeneity can be more efficiently handled by CI models (Shao et al., 2025b). A complete summary of dataset scales and split protocols follows the BasicTS+ (Shao et al. 2025b) standard (see Appendix Table 7.
Evaluation metries. We employ two standard metrics to assess forecasting performance: Mean Absolute Error (MAE) and Weighted Absolute Percentage Error (WAPE). MAF provides direct measure of prediction error in the original scale of the data, while WAPE normalizes across datasets with distributional shifts and scale disparities, enabling fair cross-dataset comparison (Liang et al,
2023
Baselines. We compare CPiRi against a comprehensive suite CI and CD models. CI models:
DLinear Zeng et al. 2023). PatchTST (Nie et al. 2023 Chronos-Bolt (Ansari et al.) 2024), and
Sundial (Lu et al. 2025). > CD models: Informer (Zhou et al., 2021), STID (Shao et al. 2022
Crossformer (Zhang & Yan 2023), iTTranformer (Liu al. 2024a). CrossGNN (Huang al.
2023
TimeXer (Wang et al. 2024), ,, and Timer-XL (Liu ct al. (204bb).
Implementation details. All models are evaluated on the benchmark BasicTS+ (Shao al. 2025b)
with five separate training runs on a server equipped with NVIDIA A800 GPU, and adopt the optimal configurations. CPiRi's dropout rate is set 0.3 for better constructing sparse spatial relation. Both L and T are 336 for all experiments. More implementation details can be referred to appendix.
4.2 COMPARISON WITH SOTA METHODS
Forecasting accuracy. As shown in Table [ CPiRi achieves SOTA performance on four of five benchmark datasets under standard training protocols and significantly outperforms existing CI and CD models. The sole exception is METR-LA, where STID and Crossformer leverages exogenous holiday features unavailable to our sequence-only model. Crucially, CPiRi surpasses all large pre- trained baselines by substantial margins (> 12% WAPE on SD), validating that our decoupled design better handles limited data and weak spatial signals than monolithic architectures.
Table Main forecasting performance comparison. All models are trained with fixed channel order (except where noted), using official pre-trained weights (*) when full training was infeasible. CPiRi achieves SOTA performance while maintaining CPI, demonstrating the advantages of our hybrid approach. Best results are in bold; second-best underlined.
METR-LA
PEMS-BAY
PEMS-04
PEMS-08
SD
Electricity
Paradigm
Model
WAPE, MAE,
WAPE. MAE,
WAPEL MAE,
WAPE] MAE
WAPE] MAE
WAPE] MAE
Chronos-Bolt"
24.19%
11.76
8.52%
5.11
34.48%
73.97
32.83%
71.76
19.71%
41.11
10.22% 241.44
Sundial*
16.51%
8.14
5.79%
3.52
18.77%
36.44
22.69%
30.05
24.40%
53.94
11.81% 276.72
CI
Dlinear
14.93%
7.67
5.22%
3.17
17.77%
36.92
16.50%
33.42
19.46%
43.93
11
11.80%
244.86
PatchTST
10.51%
5.33
4.87%
2.95
15.54%
32.32
12.37%
23.83
13.41%
29.06
10.68%
241.32
Timer-LL* 23.64%
12.27
8.22%
4.95
36.33%
77.05
31.52%
68.07
46.07 %6
110.47
17.23%
405.63
Informer
10.36%
5.25
4.47%
2.70
13.57%
27.88
13.02%
27.36
19.55%
43.10
17.61%
347.94
CrossGiNN
12.82%
6.52
5.19%
3.15
17.90%
37.16
16.83%
33.81
19.51%
44.00
11.63%
258.94
CD)
TimeXer
11.18%
5.69
4.61%
2.79
16.43%
34.16
16.02%
31.66
14.43%
31.37
18.70%
357.05
[Transformer
11.28%
5.71
4.21%
2.55
12.99%
26.79
10.70%
20.17
12.45%
27.28
10.67%
237.59
STID
8.48%
4.21
3.91%
2.36
12.43%
25.65
10.90%
20.60
12.51%
26.64
10.65%
245.24
Crossformer
8.84%
4.42
4.07%
2.47
13.28%
27.30
11.43%
22.03
12.50%
27.21
13.57%
257.69
CI+CD
CPiRi (ours)
9.14%
4.62
3.90%
2.36
11.67%
23,96
9.43%
17.46
12.25%
26.85
9.90%
235.33

---

Published as a conference paper at ICLR 2026
Table 2: Channel shuffling robustness analysis. For each model, we show: (1) performance when trained normally but tested with shuffled channels (Test Shuffle), and (2) performance when trained with channel shuffling (Train Shuffle). The severe degradation of most CD models under test-time shuffling reveals their dependence on fixed channel order, while CPiRi maintains stable performance all conditions.
METR-LA
PEMS-BAY
PEMS-04
PEMS-08
SD
Model
Shuffle
WAPEL
MAE]
WAPEJ
MAE]
WAPE,
MAE]
WAPEL
MAE]
WAPE,
MAE,
Informer
Test Train
20.19%
10.38
9.99%
6.03
83.53%
150.02
118.19%
145.58
19.55%
43.10
16.48
8.11
7.30%
4.38
49.39%
90.51
74.00%
81.12
17.38%
38.15
Test 12.85%
6.53
5.20%
3.15
17.95%
37.23
17.01%
33.89
19.51
44,00
CrosSGNN
Train
12.78%
6.48
5.19%
3.15
17.92%
37.18
16.79%
33.78
19.49%
43.93
Test 13.79%
7.08
5.92%
3.57
17.22%
35.80
14.74%
33.27
18.46%
40.27
TimeXer
Train
11.84%
6.09
4.86%
2.95
16.72%
34.94
15.96%
31.88
14.77%
32.38
Test 11.27%
5.70
4.21%
2.55
12.99%
26.79
10.70%
20.17
12.45%
27.28
irranformme
Train
11.50%
5.86
4.21%
2.55
13.02%
26.84
10.58%
19.98
12.40%
27.21
Test 18.07%
9.23
7.20%
4.35
52.31
86.25
65.18%
69.20
12.51%
26.64
STID
Train
10.11%
5.15
4.30%
2.60
13.75%
28.17
11.82%
21.93
12.98%
28.15 18.18
Crossformer
Test 18.06%
9.12
6.66%
4.03
43.83%
78.36
39.85%
54.72
12.50%
27.21
Train
9.87%
4.90
4.47%
2.69
14.75%
30.38
12.82%
23.57
12.85%
27.65
Test 9.23%
4.67
4.02%
2.45
11.93%
24.57
10.08%
18.20
13.46%
29.21
CPiRi (ours)
Train
9.14%
4.62
3.90%
2.36
11.67%
23.96
9.43%
17.46
12.25%
26.85
Table 3: Performance degradation under partial channel shuffling on the PEMS-08 dataset. The performance of non-invariant CD models collapses progressively as the percentage of permuted channels increases at test time, while CPiRi remains perfectly robust across all conditions. The best results are highlighted in bold, while the second- best results are underlined.
100% % shuffle
75% shuffle
50% shufile
25% shuffle
0 shufile
Model
WAPE
MAEL
WAPE
MAET
WAPEJ
MAEL
WAPE
MAEE
WAPEL
MAEJ
Informer
118.19%
145.58
76.21%
113.26
52.73%
83.18
41.25%
59.07
13.02%
27.36
STID
65.18%
69.20
42.66%
56.13
30.17%
41.31
25.61%
32.19
10.90%
20.60
Crossformer
39.85%
54.72
28.33%
46.86
22,29%
38.41
19.90%
33.06
11.43%
22.03
Timer-XI.
31.52%
68.11
31.52%
68.07
31.54%
68.11
31.53%
68.11
31.52%
68.07
CrossGNN
17.01%
33.89
16.91%
33.86
16.86%
33.84
16.89%
33.85
16.83%
33.81
TimeXer
16.74%
33.27
16.57%
32.97
16.40%
32.63
16.19%
32.06
16.02%
31.66
iTransformer
10.70%
20.17
10.70%
20.17
10.70%
20.17
10.70%
20.17
10.70%
20.17
CPiRi (ours)
9,43%
17.46
9.43%
17.46
9.43%
17.46
9.43%
17.46
9.43%
17.46
Channel shuffling vulnerability analysis. Table exxoses critical limitations in mainstream CD models when subjected to channel shuffling at inference time. Models trained without permutation augmentation exhibit catastrophic performance degradation under channel-shuffled testing exempli- fied by Informer and STID, representing етror increases exceeding 400% and 235% respectively. This fragility stems from architectural dependencies on channel order, such as fixed positional en- codings that incentivize index memorization over content-basse based reasoning.
CPiRi's robustness advantage. CPiRi sustains stable performance under both training and test- ing permutations (Table 2 with minimal deviation between standard and shuffled evaluations (AWAPE < 0.25% across datasets). Unlike iTrrnsformer, which achieves CPI by performing spatio-temporal multi-head attention inside each layer and thus couples temporal and channel di- mensions with an approximate O((T 2)²) cost, CPiRi fully decouples them: a frozen tempo- ral encoder summarizes each channel into attention then operates on these summaries. This design delivers stronger robustness and higher efficiency. Full results with standard deviations are provided in appendix.
token representation, and single O(C2) spatial
Progressive permutation robustness. Table quantifies model degradation under increasing per- mutation intensities on PEMS-OS where 25% shuffle indicates sampling 25% channels uniformly at random and applying random permutation within this subset, while keeping the remaining chan- nels in their original order. CD models (e.g.. Informer, STID) exhibit progressive performance collapse shuffling rates increase, with WAPE deteriorating by > 100% under full permutation.

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study of the CPiRi framework about the effectiveness of its core principles and architectural components.
METR-LA
PEMS-BAY
PEMS-04
PEMS-08
SD
Variant
WAPE,
MAEL
WAPEL
MAE.]
WAPE,
MAEE
WAPEL
MAF.
WAPE..
MAE.
CPiRi
9.14%
4.62
3,90%
2.36
11.67%
23.96
9.43%
17.46
12.25%
26.85
w/o spatial-temporal decouple
9.21%
4.65
3,93%
2.38
11.91%
24.46
10.80%
18.99
13.37%
28.97
w/o regularization stralegy
9.23%
4.67
4.02%
2.45
11.93%
24.57
10.08%
18.20
13.46%
29.21
w/o pretrained weights
16.96%
8.91
7.54%
4.54
74.86%
90.45
52.29%
101.69
64.84%
120.22
w/ layer encoder from scratch
10.71%
5.41
4.28%
2.56
12.61%
26.29
11.17%
21.96
14.34%
30.39
w/ frozen Chronos-2 encoder
10.90%
5.47
4,32%
2.66
16.54%
34.16
13.16%
25.66
18.50%
40.39
w/ fine- tuninnnng in last 10 epochs
8.81%
4.46
4.00%
2.42
11.86%
24.35
9.73%
17.82
12.00%
26.47
w/o spatial module
16.51%
8.14
5.79%
3.52
18.77%
36.44
22.69%
30.05
24.40%
53,94
w/ mean pooling
13.72%
7.07
4.24%
2.56
12.81%
26.21
12.42%
21.89
13.33%
29.03
In stark contrast, CPiRi maintains invariant prediction quality regardless of permutation intensity (9.43% WAPE at all shuffling levels). This establishes channel shuffling tests as essential diag- nosties for dynamic real-world deployments where sensor configurations frequently change. Across datasets with different channel counts and under progressive shuffling, the degradation pattern aligns more with the strength of inter-channel correlations than with channel counts itself, indicating that brittleness under reordering is driven primarily by channel heterogeneity rather than the number of channels.
4.3 ABLATION STUDY
Framework components. We first validate our two primary contributions, with results summarized in Table 4 In the w/o apatial-temporal decouple variant, we fine-tune the encoder alongside the spatial module instead of keeping it frozen. This leads to performance degradation, confirming our hypothesis that decoupling is crucial. It prevents the model from overfitting dataset-speciee artifacts and preserves the robust priors from the foundation model. The w/o regularization strategy variant, which disables the channel shuffling strategy, shows consistent performance drop. This demonstrates that our regularization strategy is a potent regularizer, compelling the model to learn truly generalizable, content-driven relationships rather than relying on positional shortcuts.
Encoder variants and fine-tuning, Beyond the core ablations, Table 4 further clarifies how the temporal encoder choice and fine-tuning strategy interact with our spatio-temporal decoupling and CPI objective. First, w/o pretrained weights simply removes Sundial pretraining while keeping the same architecture; this variant collapses, indicating that high-quality temporal priors are indispens- able for our content-driven relational reasoning in the spatial module. Second, w/ 3 layer encoder from scratch replaces the original 12-layer Sundial encoder with a lightweight 3-layer counterpart trained from random initialization. Although this variant converges more readily than w/o pretrained weights, it still lags behind CPiRi, underscoring that a strong frozen encoder is key to our decou- pled design. Third, w/ frozen Chronos-2 encoder uses Chronos-2 (Ansari al. 2025) with 120M parameters as encoder to replace Sundial with 128M parameters. Since Chronos series models are designed around short forecasting horizons of 64, their representations transfer poorly to our long- horizon setting, and the performance is even inferior to the from-srcco variant. Finally, w/ fine-tuning in last 10 epochs unfreezes the encoder in last 10 epochs. While this brings slight gains on a few datasets, it raises training memory consumption by about five times and reduces representa- tion separability in UMAP analyses (Fig. 6). suggesting overfitting to a single dataset and weakening the CPI-oriented decoupling. Overall, these findings support our design choice: coupling a frozen equivariant spatial block trained under channel shuffling best balances accuracy, robustness under co-drift, and practical efficiency.
pretrained temporal encoder with permutation--qqiivrinnt
Model designs. We also ablate the key modules of our model architecture (Table Removing the spatial module (w/o spatial module) causes catastrophic drop in performance, reducing the model to a simple CI forecaster. This irrefutably proves that explicit cross-channel relational modeling is essential for achieving SOTA accuracy. Specifically, relying the final token for next-token prediction is well-suited forecasting; replacing this with an aggregated average all tokens results in notable performance drop w/ mean pooling, further validating design's 's effectiveness.
8

---

Published as a conference paper at ICLR 2026
surigg PRMS BAY
wollagg
S4CTB U.A.
FIMS
71345
SD
1042
2.30
15
1,65
14.5
125
WATEL
12.00
22.55
22
2 14.0
13.2
MAD
13
X
23-45
12.5
70
03
12
112
24
25
50
75
30
75
Clookc
Choolss
S.Madag
Figure 4: Analysis of inductive generalization to unseen channels and the impact of permutation- invariant regularization strategy. Performance is evaluated with and without the strategy as the percentage of available training channels varies. The results show that the strategy consistently im- proves performance, with the benefits becoming more pronounced in low-data regimes. For clarity, WAPE (%) values are annotated directly on the corresponding data points.
Table 5: Forecasting performance large-scale datasets.
Timer-XL
Sundial
CPiRi
Dataset
Channels
WAPE!
MAE,
WAPE]
MAE,
WAPE,
MAE,
SD
716
46.07%
110.47
24.40%
$3.94
12.25%
26.85
GBA
2,352
42.99%
100.10
21.02%
48.08
11.48%
25.50
GLA
3,834
40.83%
106.15
23,21%
54.46
11.00%
26.40
CA
8,600
42.75%
96.53
23.60%
49.56
12.68%
25.94
Table 6: Efficiency and scalability analysis the large-scale CA dataset (C=8600, 7=336). The comparison highlights the practical advantages of CPiRi's decoupled architecture over monolithic models like Timer-XL, which are often constrained by prohibitive memory requirements. "0OM" denotes an out memoory error.
Maximum Inference time Arg, time
GPU memory (GB) Avg. memory (GB)
Paradigm
Model
batch size Base Compiled
Complexity
[Base Compiled Base
Compiled
Base
Compiled
Sundial
2.55
1.61
0.64
0.40
54.44
20.68
13.61
5.17
O179)
CI+CD
CPiRi (ours)
4
2.66
1.62
0.67
0.41
54,56
32.00
13.64
8.00
0 C2)
CD
Timer-XL
00M
1.07
1.07
>80
75.68
75.68
O(T x C))
iTransformer
0.52
0.40
0.26
0.20
45.18
35.37
22.59
17.69
O([Tx x CP))
Generalization under data scarcity. To further probe the limits of CPiRi, we test its performance on the full dataset when trained on only a fraction of the available channels (25% 50%, 75%). As illustrated in Fig. 4 CPiRi demonstrates remarkable inductive generalization to unseen chan- nels. Notably, training with only 25% of channels reduces training time by 70% while incurring merely 2% drop in accuracy, further highlighting the efficiency of our framework in resource- constrained scenarios. CPiRi maintains competitive performance when trained on 50% of channels, demonstrating its capability for handling unseen channels based temporal patterns. The widening performance gap in low-data regimes confirms that our regularization strategy is critical for robust and generalizable relational reasoning under challenging, data-scarce conditions.
4.4 EXPLORATORY STUDY
Scalability and efficiency analysis. CPiRi delivers consistent gains large-scale benchmarks (up to 8,600 channels; Table 5). confirming real-world scalability while keeping runtime and memory modest. The decoupled design has complexity (O(C² + T2)): temporal encoding and decoding are handled by the frozen CI backbone, and the spatial module attends only across the C last tokens per channel rather than over all tokens, which far more tractable than Timer-XI's O(TT x C)²). On the CA with 8,600 channels, Table repports compiled per-instance inference of 0.41s for CPiRi versus Sundial (0.40s), with average GPU memory 8.00GB versus 5.IGGB. Timer-XL demands 75.68GB memory under comparable settings. CPiRi requires substantially less memory compared with iTransformer, which performs multi-layer spatiotemporal aggregation inside each attention block and raises computational cost and makes optimization harder. This confirms CPiRi strikes an effective balance between modeling capacity and computational cost, making it practical solu- tion for large-scale tasks.

---

Published as a conference paper at ICLR 2026
Sundial
CPIRI Permutation
CPIRI Permutitioon
CPIRI Pormutation
45.0
45.0
45.0
45.0
40.0
40.0
40.9
40.0
35.0
35.0
30.0
30.0
-10.0
.50
0.5
-10.0
-5.0
0.0
-10.0
-5.0
0.0
-10.0
-5.0
0.0
Figure 5: UMAP visualization of channel representations on METR-LA. Left: Sundial. Right:
CPiRi under three random channel permutations, near-identical geometry with clearer separation.
Sundial
CPIRI Permotation
CPIRI Permuttation
CPIRI Permitation
14.0
14.0
14,0
12.0
12.0
12.0
10.0
10.0
10.0
8.0
8.0
6.0
6.0
-2.00
22.5
20.0
17.5
24.0
22.0
20.0
24.0
22.0
-20.0
23.0
22.5
20.0
17.5
Figure 6: UMAP visualization of channel representations on METR-LA after fine-tuning CPiRi, embeddings become more compact and less separable.
Qualitative analysis. We use UMAP Ghojoghh et al., 2023 to visualize channel representations before and after the spatial module. Fig. 5 contains four panels: the left panel shows raw Sun- dial embeddings, and the three right panels show CPiRi outputs under three independently shuf- fled channel orders, all projected into the same space. The three CPiRi clouds exhibit nearly identical geometry and clearly improved cluster separability relative to Sundial, illustrating that the permutation--qqiivrriat spatial attention plus channei-shufflng regularization learns content- driven, order-agnostic relationships consistent with CPI. Fig. 6 visualizes embeddings after late- stage unfreezing of the encoder. Compared with the frozen-encoder CPiRi, the points become overly compact and less separable across channels, suggesting partial representation collapse and sataset-specific overfitting. Together with Appendix Fig. / across datasets, the embeddings after CPiRi exhibit a more semantically meaningful structure with clearer separability across channels, consistent with the higher-fidelity forecasts in the appendix case studies (Fig. 10
CONCLUSION
This paper proposes CPiRi, framework that successfully synergizes the strengths of the CI and CD paradigms. Its design is founded on two principles: radical spatio-temporal decoupling and a permutation-invarinn
regularization strategy. The architecture leverages a frozen univariate foun- dation model to provide robust temporal features, which are then processed by a lightweight spatial module. The strategy guided by CPI forces spatial module to learn content-driven relational rea- soning meta-skill, which not only enhances data diversity but also improves the channel discrim- inability. This approach leads to SOTA predictive accuracy on multiple benchmarks, and overcomes the brittleness of complex CD models that overfit to positional artifacts, critical flaw revealed by the channel shuffling test. Critically, CPiRi exhibits:
(1) strong inductive generalization to unseen channels (e.g., 50% training channels), and (2) enhanced robustness low-data regimes, where its regularization strategy prevents overfitting to sparse relational patterns. Beyond accuracy and robust- ness, CPiRi is practical at scale. These properties make CPiRi a compelling choice for real-world environments with structural and distributional co-drift, where relational reasoning and operational efficiency are both required.
Limitation discussion. While CPiRi synergizes CI/CD strengths effectively, its static fusion mech- anism between temporal and spatial modules presents a limitation for scenarios involving abrupt trend shifts. Future work will pursue dynamic fusion mechanisms by developing adaptive interac- tion protocols, potentially improving responsiveness to volatile patterns. A more ambitious frontier is move beyond purely endogenous signals by integrating external, unstructured information (c.g., news events, policy changes) within a causal reasoning framework to move beyond endogenous modeling, enhancing real-world applicability.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work has been supported by Zhejiang Province Science and Technology Leading Talent Plan Project of China (No.2023R5213), Zhejiang Province "Lingyan" Key R&D Project of China (No.2026C0222002), Zhejiang Province "Jianbing" Key R&D Project of Chima 200.20250,,,0,, No.2024C01034, No. 202201 144), and Zhejiang Provincial Natural Science Foundation of China (No.LON26F030003),
REFERENCES
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Syndar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Michael W. Mahoney, Kari Torkkola, Andrew Gor- don Wilson, Michael Bohlke-Schneider, and Yuyang Wang- Chronos: Learning the language of time series. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL. httpelewwweeeeeeeeee net forum www.eeeeoorooooeeeeeeeeeeeeeeeee -eeguunndd-ee
https://gpenreview..
Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erick- son, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael
Chronos-2: 2: From univariate universal forecasting, arXiv preprint 1xXiv:2510.158211 2025, BRLhhtts://aart www.wawwwww................. 251.218221111
Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway perfor- mance measurement system: Mining loop detector data. Transportation Research Record, 1748 (1):96-102, 2001. doi: 10.3141/1748 12. URL BRLhttps:: 0tt.:/......//111111/111/13//.00
Jiawen Chen, Qi Shao, Duxin Chen, and Wenwu Yu. Decoupling spatio-temporal prediction: When lightweight large models meet adaptive hypergraphs. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '25, 2025.
Songgaojun Deng, Olivier Sprangers, Ming Li, Sebastian Schelter, and Maarten de Rijke. Domain generalization in time series forecasting. ACM Transactions on Knowledge Discovery from Data, 18(5), February 2024. ISSN 1556-4681. doi: 10.1 10.1145/3643035. URL https: htthttps::/ /doo:..///ooooooooo org 10.1145/36433 34415/3363335
Cheng Feng, Long Huang. and Denis Krompass. General time transformer: encoder-only founda- tion model for zero-shot multivariate time series forecasting. In Proceedings of the 33rd ACM In- ternational Conference on Information and Knowledge Management, CIKM '24, pp. 3757-3761, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704369, doi: 11114//3622777336799111 URL hollhtps::00 //oii. org/10..goo..ooo...ooooooo 1145/16176633 11115//36667773379993337
Benyamin Ghojogh, Mark Crowley, Fakhri Karray, and Ali Ghodsi. Uniform Manifold Approxi- mation and Projection (UMAP), pp. 479-497. Springer International Publishing, Cham, 2023. ISBN
doi: 10. 10.1100/978-3-000 010779978--00111100---111111
httthhttss://ec org/
10.1007/978 3 031 10602-66000000 6_
Xiaobin Hong, Jiawen Zhang, Wenzhong Li, Sanglu Lu, and Jia Li. Unify and anchor: A context- transformer for cross-domain time series forecasting. CoRR, abs/2503.01157, 2025. doi: 10.48550AARXI22503.001 01157
URI. https:///ooiiii orgg10..4450//axxtt...... 2503.
Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. CrossGNN: Confronting noisy multivariate time series via cross interaction refinement. In Proceedings of Thirty-seventh Conference Neural Information Processing Systems, NIPS '23, 2023. htRtssttt::/ //tp::////eeevvveeeeee wwt.eeew.eee.tteeeeeeeeeeeeeeeee d= 1w000o
Ming Jin, YiFan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan. and Qingsong Wen. Position: What large language models tell about time series analysis. In Proceedings of The Forty-first International Conference on Machine Learning, ICML 224, 2024. /

---

Published as a conference paper at ICLR 2026
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In The s International ACM SIGIR Conference Research & Development in Information Retrieval, SIGIR 18, pp. 15-104, New York, NY, USA, 2018. ACM Association for Computing Machinery. ISBN 978 450356577. doi: 10.1145/ 3209978.32 3.209006..0066.... URL htt.tssss:///o.. wtg.:ioorgg100 10. 0ww.11115/339991 02177788.211000 3210006
Fuxian Li, Jie Feng. Huan Yan, Guangyin Jin, Fan Yang, Funing Sun, Depeng Jin, and Yong Li. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution. ACM Transactions on Knowledge Discovery from Data, 17(1), February 2023. ISSN 1556-4681.
doi: 10.1145/3532611. URL htttt::: /oi. 0t1/10.1145/33326111
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In Proceedings of International Conference on Learning Representations, ICLR '18, 2018.
Yubo Liang, Zezhi Shao, Fei Wang, Zhao Zhang, Tao Sun, and Yongjun Xu. Basicts: An open source fair multivariate time series prediction benchmark. In Ana Gainaru, Ce Zhang, and Chunjie Luo (eds.), Proceedings of Benchmarking, Measuring, and Optimizing, pp. 87-101, Cham, 2023. Springer International Publishing. ISBN 978-3-031-31180-2.
Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang, Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: A benchmark dataset for large-scale traffic forecasting. In Advances in Neural Information Processing Systems, 2023.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long- itransformer: Inverted transformers effective for time series forecasting, In Proceedings The Twelfth International Conference on Learning Representations, ICLR '24 2024a. URL https: 8fah Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. TIMER-XL Long-context transformers for unified time series forecasting. In Proceedings of The Twelfth International Conference Learning Representations, ICLR '24, 2024b. URL https:// openreview,ned wtteet//ormmmmd115s2.200
//openzeview.neteetee wttp://wwwww.eeeeeooooeeeeeeeee.
Yong Liu, Guo Qin, Zhiyuan Shi, Zhi Chen, Caiyin Yang, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Sundial: A family of highly capable time series foundation models. In Pro- ceedings of the International Conference on Machine Learning, ICML '25, 2025.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In Proceedings of International Conference on Learning Representations, ICLR '23, 2023.
Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM '22, pp. 4454 4458, 2022.
Zezhi Shao, Yujie Li, Fei Wang, Chengqing Yu, Yisong Fu, Tangwen Qian, Bin Xu, Boyu Diao, Yongjun Xu, and Xueqi Cheng. BLAST: Balanced sampling time series corpus for universal fore- casting models. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, SIGKDD '25, 2025a. URL https:://on httt::://do..or/10... www.oo...g././....... 01411.11715119999 3736860
Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao. Gao Cong, Christian S. Jensen, and Xueqi Cheng. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 37(1):211 2025b. ISSN 1041- 4347. doi: 10. 10.1109/KDDE.22444448A88
URL
2024 3484454
Yuxuan Wang. Haixu Wu, Jiaxiang Dong, Yong Liu, Yunzhong Qiu, Haoran Zhang, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with ex- ogenous variables. Advances in Neural Information Processing Systems, 2024.
12

---

Published as a conference paper at ICLR 2026
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con- necting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
Wang Xue, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. CARD: Chan- nel aligned robust blend transformer for time series forecasting. In Proceedings of The Twelfth International Conference on Learning Representations, ICLR '24, 2024. URL https:/7 d=
Manzil Zaheer, Satwik Kottur, Siamak Ravanbhakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J Smola. Deep sets. In Proceedings of the 31st International Conference Neural Information Processing Systems, NIPS' 17, pp. 3394-3404, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 5781510860964.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI '23, 2023.
Shuai Zhang, Jiyuan Xu, Wenyu Zhang, Yixiang Zhang, and Chengjie Ni. You only adapt once: An adaptive transformer for dynamic multivariate time series forecasting across time-varying Information Sciences, 717:122350, 2025. ISSN 0020-0255.
topologies and
doi: com/ http://ccaeeaaarreee//////011120
122350. URL https: http:://wwwww... wwt.wwwcccecccdree......
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In Proceedings of the International Conference on Learning
Representations, ICLR '23, 2023.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang- Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of The Thirty-Fifh AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, volume 35 of AAAI '21 pp. 11106-11115. AAAI Press, 2021.
Xuanbing Zhu, Dunbin Shen, Zhongwen Rao, Huiyi Ma, Yingguang Hao, and Hongyu Wang- Mts-unmixers: Multivariate time series forecasting via channel-time dual unmixing. CoRR, abs/2411..17700, 2024.
/ /oo.. http://oooooo.....//////// 48550/ arXiv. 2411.1111111 17770
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 MORE EXPERIMENTAL SETUP
A.1.1 DATASET SELECTION AND PROTOCOLS
Rationale for dataset selection. A rigorous evaluation of MTSF models hinges on datasets with significant channel heterogeneity (Shao et al. 2025b), where individual time series exhibit diverse and complex dynamics. Such datasets are essential for assessing model's ability to learn meaning- ful inter-channel relationships. We specifically select traffic forecasting benchmarks as they provide an ideal testbed for two reasons. First, they are known for their strong and complex inter-channel dependencies. Second, they are derived from dynamic sensor networks where events like sensor fail- ures or expansions are common, making Channel Permutation Invariance (CPI) practical necessity for robust real-world deployment.
Table 7: Statistics of the datasets used in our experiments.
Dataset
Channels
Timesteps
Granularity
Time Period
Data Type
Source
METR-LA
207
34,272
5 mins
Mar-Jun 2012
Traffic Speed
Li
2018
PEMS-BAY
52,116
5 mins
Jan-May 2017
Traffic Speed
PEMS-04
307
16,992
5 mins
Jan-Feb 2018
Traffic Flow
PEMS-08
170
17,856
5 mins
Jul-Aug 2016
Traffic Flow
C
2001
SD
716
525,888
5 mins
2017 2021
Trafic Speed
Liu
2025
GBA
2,352
525,888
5 mins
2017 2021
Traffic Speed
Lru
2023
GLA
3,834
525,888
5 mins
2017
Traffic Speed
Liu
2025
CA
8,600
525,888
5 mins
2017-200
Traffic Speed
Liu
2023
Electricity
321
26,304
60 mins 2012-2014
Electricity Consumption
LaI
2018
Benchmark summary. Our evaluation is conducted on a hierarchy six standard benchmarks (METR-LA, PEMS-BAY, PEMS-04, PEMS-08, SD, Electricity) and is further extended three large-scale datasets from LargeST (GBA, GLA, CA) to assess scalability. Detailed statistics for all datasets are summarized in Table 7 To ensure fair and reproducible comparisons, all dataset processing, including normalization and data splits, strictly adheres to the standardized protocols established by the BS open-source benchmarking framework (Shao et al. 2025b).
A.1.2 MODEL CONFIGURATION AND BASELINES
Our model, CPiRi, built upon a frozen, pre-trained Sundial (Liu al. 2025) model which serves as the backbone for temporal feature extraction. A lightweight, trainable Transformer encoder block functions as the spatial module, tasked with learning content-driven relational interactions. This design allows CPiRi to leverage powerful temporal priors while focusing training exclusively on the relational learning skill. Key hyperparameters, such as a dropout rate 0.3, are kept consistent across all datasets to ensure robustness and minimize tuning overhead, as detailed in Table S and Fig. 7
10.29%
5.2
0.102
WAPE MAE
5.1
0.100
WAPE
0.098
5.0
9.54%
9.45%
4.9
9.32%
9.39%
9.33%
4.8
9.14%
4.7
0.092
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Dropout
Figure 7: Hyperparameter sensitivity analysis of the dropout rate on the METR-LA dataset. The results indicate that dropout rate of 0.3 achieves the optimal balance for both WAPE and MAE metrics. For clarity, WAPE values are annotated directly on the corresponding data points, and the best-performing setting is marked with green square.
14

---

Published as a conference paper at ICLR 2026
Table 8: Model Hyperparameters.
Hyperparameter
Value
Input sequence length
336
Ouiput sequence length
336
Number of layers
4
Learning rate (LR)
0.001
LR milestones
1, 10, 25,
Weight decay
1 10
Gradient clipping
3.0
Dropout rate
0.3
Hidden dimension
768
Attention heads
12
Normalization
LayerNorm GELU
Activation function
9%
CPiRi
iTransfommer
18%
TimeXer
WAPE
36%
CrossGNN
Timer-XL
72%
Crossformer
144%
STID
0%
25%
50%
75%
100%
Informer
Channel Permutation Ratio
Figure 8: Performance degradation of CD models under partial channel shuffling on the PEMS- dataset. Models were trained with a fixed channel order. The performance of non-invariant models collapses progressively as the percentage of permuted channels increases at test time. In contrast, CPiRi and iTrrnsffrmer remain perfectly robust across all conditions.
For a comprehensive comparison, all baseline models were evaluated within the BasicTSS+ frame- work (Shao et al.] (025b) using their officially reported optimal hyperparameters. Our baseline selection spans the current MTSF landscape: we include channel-independent (CI) models like DLinear (Zeng al. 2023) and PatchTST (Nie al. 2023), which achieve permutation invariance (CD) models, such Informer (Zhou et al.) (02T), STID (Shao al. 2022), and Crossformer (Zhang & Yan 2023), that explicitly model these relationships but, due to archifectural biases like fixed posi- tional encodings, prove brittle under permutation tests (Fig. Rp. As direct competitor, we include the CPI-aware ITraanformer (Liu et al.) 2024a), which achieves invariance through an innovative architectural shift. Finally, to benchmark data and parameter efficiency, we compare against large foundation models like Chronos-Bolt (Ansari al. 2024) and Timer-XL (Liu et al. 2024b) using their official pre-trained weights.
by forgoing cross channel interaction. These are contrasted with classic
A.1.3 SCOPE: DYNAMIC MTSF AND EVALUATION PROTOCOL
Our experimental design is driven by the goal of dynamic MTSF rather than generic TSF. deploy- ment, channel configurations (ordering, availability) and spatial relations may change over time, and datasets exhibit strong channel heterogeneity and inter-channel dependence. Consequently, we pri- oritize (i) benchmarks where cross-channel structure matters and (ii) diagnostics that directly probe relational robustness (e.g., CPI tests). To make "dynamic" precise, we use structural and distribu- tional co-drift to denote deployments where the channel set topology changes (structural) and the underlying data distributions shift over time (distributional). Such co-drift is common in sensor networks, finance, and other operational settings.
15

---

Published as a conference paper at ICLR 2026
Metric rationale (WAPE vs. MSE/RMSE). Because datasets differ in magnitude and units, adopt WAPE as a scale-independent primary metric and report MAE in parallel for interpretability. Formally,
E, Dcl
WAPE- E, +
which normalizes absolute error by the aggregate signal scale and yields unit-free percentage that is directly comparable across datasets with different units/ranges. In contrast, MSE/RMSE are scpeendep:ndent: their values grow with the variance and units of the target, making cross- dataset comparisons misleading and biasing summaries toward high-magnitude series. WAPE also avoids the divide-by-zero instability of MAPE when yt by using the dataset-level denominator; follow common practice with a small (e.g., 10- mean lye)) for numerical safety, This protocol emphasizes adaptability and comparability across heterogeneous settings, instead strictly mirroring generic TSF test beds that assume static channel layouts.
A.1.4 ON CHANNEL ADDITION/REMONE EVALUATION
We operationalize "unseen channels" by training on subsets of channels (25%, 50%, 75%) and eval- uating on the full set (Fig. 4.. This protocol stresses inductive generalization without retraining and agnostic to model families. In contrast, direct plug-and-play test of channel addition/removal is not uniformly applicable to many CD baselines: their architectures bind the channel dimension via learned adjacency/positional structures, so adding or removing channels typically requires graph re- specification and (partial) retraining. Forcing comparators into this setting via ad-hoc workarounds (e.g., zero-masking channels to emulate missing/new inputs, as done in prior work such as YOAO (Zhang et al., 2025)) changes their effective training distribution and creates fairness issues across methods. For these reasons, we (1) use bubsstt-raininggfull--tet to approximate addition and removal in model-agnostie manner, (2) permutation (shuffle) tests to isolate order sensitivity, and (3) leave a comprehensive, architecture-specffi add/remove benchmark, requiring consistent reparame- terization rules for CD models, future work. Notably, CPiRi's s decoupled, permuuation-equiiariaa design is inherently compatible with plug-and-pla -lay channel changes without retraining: the above constraints arise from ensuring fair cross-paradigm comparison rather than from limitations of CPiRi itself.
A.5 COMPUTING WORKSTATION
All experiments were conducted on a server with the following specifications to ensure reproducibil- ity:
GPU: NVIDIA A800 (80GB)
CPU: 128 Intel Xeon Platinum 8358 CPU 2.60GHz
Operating System: Ubuntu 22.04 LTS
Software: Python 3.10, PyTorch 2.3.1, CUDA 12.1
A.2 EXTENDED EXPERIMENTAL RESULTS
A.2.1 FORECASTING PERFORMANCE AND STABILITY
Table reports the full forecasting results, extending the main paper by providing both the mean and standard deviation (STD) over five independent runs. The STD serves as a critical indicator of model stability and reliability. The results show that CPiRi not only achieves state-of-the-an the-art (SOTA) accuracy but also consistently exhibits a smaller standard deviation across most datasets compared to complex baselines. This heightened stability direct consequence of its robust, regularized design, which combines frozen pre-trained backbone with the channel shuffling strategy to prevent overfitting and enhance reproducibility.
A.2.2 ON THE SOURCE OF CPIRI'S SUPERIORITY
The superiority of CPiRi stems from a powerful synergy between its permutation-equivarian archi- tecture and its unique training methodology. We substantiate this claim through two key analyses:
16

---

Published as a conference paper at ICLR 2026
Table 9: Full forecasting performance comparison with standard deviations. This table extends and integrates the results from Table and Table 2 of the main paper, reporting both mean and standard deviation (STD) over five independent runs. We evaluate all models under three distinct conditions: (I) standard training with fixed channel order, (2) training with channel shuffling, and (3) testing a fixed-order model with channel shuffling. CPiRi achieves SOTA performance while maintaining perfect Channel Permutation Invariance (CPI). Best results are in bold; second-best are underlined.
METR-LA
PEMS-BAY
PEMS-04
PEMS-08
Paradigm
Model
WAPE
STD
WAPE
STD
WAPE
STD
WAPE
STD
WAPE
STD
Chronos-Boll
24.19%
8,52%
34.48%
32.83%
19.71%
Sundial*
16.51%
5.79%
18.77%
22.69%
24.40%
Dlincar
14.93%
0.11
5.22%
0.07
17.77%
0.13
16.50%
0.12
19.46%
0.16
PatchTST
10.51%
0.09
4.87%
0.06
15.54%
0.11
12.37%
0.09
13.41%
0.12
Timer-XI.
23.64%
8.22%
36.33%
31.52%
46.07%
Informer
10.36%
0.09
4.47%
0.06
13.57%
0.11
13.02%
19.55%
0.17
trained with
CrussGNN
12.82%
0.11
5.19%
0.07
17.90%
0.14
16.83%
0.13
19.51%
0.17
fixed
TimeXer
11.18%
0.10
4.61%
0.06
16.43%
0.13
16.02%
0.12
14.43%
0.13
channel
Trransformrr
11.28%
0.10
4.21%
0.05
12, 99%
0.10
10.70%
0.08
12.45%
0.11
order
STID
8.48%
0.07
3.91%
12.43%
0.10
10.90%
12.51%
0.11
Crossformer
8.84%
0.08
4.07%
0.05
13.28%
0.11
11.43%
0.09
12.50%
0.11
Informer
16.48%
0.25
7.30%
0.15
49, 39%
0.45
74.00%
0.60
17.38%
0.28
CD
CrossGNN
12.78%
0.18
5.19%
0.10
17.92%
0.25
16.79%
0.24
19.49%
0.28
trained with
TimeXer
11.84%
0.17
4.86%
0.09
16.72%
0.23
15.96%
0,22
14.77%
0.21
chamnel
iTransformer
11.50%
0.16
4.21%
0.08
13.02%
0.18
10.58%
0,15
12.40%
0.17
shuffling
STID
10.11%
0.15
4.30%
0.08
13.75%
0.19
11.8%
0,17
12.98%
0.18
Crossformer
9.87%
0.14
4.47%
0.08
14.75%
0.20
12.82%
0.18
12.85%
0.18
Informer
20.19%
0.30
9.99%
0.20
83.53%
0.70
118.19%
0.90
19.55%
0.35
CD tested with
CrossGNN
12.85%
0.20
5.20%
0.12
17.95%
0.28
17.01%
0.26
19.51%
0.30
TimeXer
13.79%
0.22
5.92%
0.14
17.22%
0.27
16.74%
0.25
18.46%
0.29
channel
iTransformer
11.27%
0.18
4.21%
0.10
12.99%
0.20
10.70%
12.45%
0.19
shuffling
STID
18.07%
0.28
7.20%
0.16
52.31%
0.50
65.18%
0.55
12.51%
0.20
Crussformer
18.06%
0.28
6.66%
0.15
43.83%
0.45
39,85%
0.40
12.50%
0.20
CI+CD
CPiRi (ours)
9.14%
0.08
3.90%
0.05
11.67%
0.09
9.43%
0.07
12.25%
0.11
first, by evaluating its robustness and accuracy in permuted channel environments, and second, by demonstrating its generalization capability in data-scarce scenarios.
The channel permutation tests, detailed in Table Jablelam and visualized in Fig. 8 reveal a clear hierarchy of model capabilities. Architecturally biased models like Informer and STID fail catastrophically. and even when trained with channel shuffling, they cannot overcome their inherent limitations, leading to compromised performance. While iTransformer's 's purely architectural solution achieves perfect ro- bustness, CPiRi consistently surpasses it in accuracy (e.g., 9.43% vs. 10.70% WAPE on PEMS-08). This highlights that CPiRi's combination of a robust architecture and targeted training approach more effectivv.
This principle, that the training process unlocks a deeper, more generalizable relational reasoning. is further validated by the model's performance in low-data regimes, presented in Table I0 When trained on progressively smaller subsets of channels, the performance gap between models trained PEMS-08, with and without our channel shuffling strategy widens dramatically. For instance, the WAPE gap is minor when using all channels (9.43% vs. 10.08%), but expands significantly when trained on only 25% of channels (10.72% vs. 14.22%). This confirms that this regularization technique acts as potent regularizer, compelling the model to learn a "meta-skill" of content-driven reasoning. This learned generalization the primary source of CPiRi's superiority.
A.2.3 STATISTICAL SIGNIFICANCE TESTS
To rigorously validate our claims, we conducted pairwise Wilcoxon signed-rank tests (Table II The analysis confirms that CPiRi's performance improvement is statistically significant (p<0.05) over all baseline models under the challenging channel shuffling settings. While this performance gap narrows in static, fixed-order environments against highly specialized models like STID, CPiRi's unique advantage is achieving competitive SOTA accuracy without compromising the robustness crucial for real-world dynamic deployments.
17

---

Published as a conference paper at ICLR 2026
Table 10: Validation of the channel shuffling strategy and its impact on generalization. This table compares performance with and without the strategy under data-scarce conditions, where training is performed on reduced subsets of the available channels (e.g.. 'w/o 75% signifies training on a 25% subset). The widening performance gap in low-data regimes highlights the strategy's critical role as a regularizer for learning generalizable relationships.
METR-LA
PEMS-BAY
PEMS-04
PEMS-08
SD
Variant Channels
WAPE,
MAE
WAPEJ
MAE,
WAPE
MAE
WAPE,
MAE,
WAPE,
MAE.
w/o 75%
9.35%
4.73
4.10%
2.48
12.07%
24.72
10.72%
19.57
13.40%
28.96
w/o 50%
9.29%
4.71
3.99%
2.42
12.06%
24.70
10.70%
18.93
13.01%
28.01
w/ strategy
w/o 25%
9.20%
4.66
3.96%
2.40
12.01%
24.62
10.12%
18.19
12.96%
28.00
9.14%
4.62
3.90%
2.36
11.67%
23.96
9.43%
17.46
12.25%
26.85
w/o 75%
10.27%
5.20
4.52%
2.73
15.27%
30.54
14.22%
25.86
16.91%
35.37
w/o 50%
9.55%
4.82
4.38%
2.65
13.12%
26.65
11.76%
20.64
15.97%
34.16
w/o strategy
w/o 25%
9.33%
4.70
4.16%
2.51
12.59%
25.60
10.91%
19.51
14.32%
30.95
9.23%
4.67
4.02%
2.45
11.93%
24.57
10.08%
18.20
13.46%
29.21
Table 11:
Significance test results of WAPE values using the Wilcoxon signed-rante test. Our model, CPiRi, is compared against other models across the 5 datasets. The p-values are for a one-tailed test with the alternative hypothesis that CPiRi's 's WAPE is lower. We significance level of 0.05. Results where CPiRi is statistically superior (p <0.05) are marked in bold.
Comparison Model Test Statistic (W) p-value p <0.05
Paradigm: Cf
Dlincar
0.0
0.031
Yes
PatchTST
0.0
0.031
Yes
Paradigm: CD (trained with fixed channel onder)
Informer
0.0
0.031
Yes
CrossGNN
0.0
0.031
Yes
TimeXer
0.0
0.031
Yes
iTransformer
0.0
0.031
Yes
STID
3.0
0.156
No
Crossformer
3.0
0.156
No
Paradigm: CD (trained with channel shuffling)
Informer
0.0
0.031
Yes
CrossGNN
0.0
0.031
Yes
TimeXer
0.0
0.031
Yes
iTransformer
0.0
0.031
Yes
STID
0.0
0.031
Yes
Crossformer
0.0
0.031
Yes
Paradigm: CD (tested with channel shuffling)
Informer
0.0
0.031
Yes
CrossGNN
0.0
0.031
Yes
TimeXer
0.0
0.031
Yes
iTransformer
0.0
0.031
Yes
STID
0.0
0.031
Yes
Crossformer
0.0
0.031
Yes
A.3 ADDITIONAL QUALITATIVE ANALYSIS
A.3.1 VISUALIZATIONS OF CHANNEL REPRESENTATIONS
To provide intuitive evidence of what the spatial module learns, we use UMAP (Ghojogh et al. 2023) to visualize the channel representations before and after its application Fig. 9 for all benchmark datasets, extending the analysis from the main paper. The results are remarkably consistent: the initial temporal representations from the Sundial encoder (red) form a dense, undifferentiated cluster. In stark contrast, after being processed by the CPiRi spatial module, the representations (blue) organized into well-separated, structured manifolds. This clearly shows that the spatial module successfully learns reconfigure the representations into more semantically meaningful structure,
18

---

Published as a conference paper at ICLR 2026
255
210
315
90
154 161
(a) METR-LA
PEMS-BAY
PEMS-04
(d) PEMS-08
(e) SD
115
339
(1) METR-LA
(g) JEMS-BAY
(h) PEMS-04
i PEMS-08
(i) SD
Sanial CPAL
22.0
4.
22.5
4.5
-5.0
-25.5
-24.
215
-24.
-6.5
20
2.0
3.3
10.0
-1.5
68
X15
3600
213
1.5
20
25
30
11.5
11.0
10.5
110
11.3
12.0
125
13.0
1.0
(k) METR-LA
(I) PEMS BAY
(m) PEMS-04 04
-1.5
-0.5
2.0
-1.0
-1.5
3.0
-2.0
3.5
-2.5
-.00
4.0
...5
8.5
-3.5
8.5
9.0
9.5
9.5
6.0
-.0
10.0
10.5
7.0
-1.0
11.0
10.5
8.0
(n) PEMS-08
(o) SD
Figure UMAP visualization of channel representations. Temporal feature distributions from the Sundial encoder (red) versus CPiRi-processed representations (blue) across datasets. CPiRi in- duces semantically meaningful clustering evidenced by the structured separation of blue manifolds, demonstrating its capacity for content-driven relational reasoning guides by CPI. Sub-figures (a-i) show two-dimensional projections. Sub-figures (k-o) show three-dimensional projections, where vectors illustrate the transformation from initial (Sundial) final (CPiRi) representations.
19

---

Published as a conference paper at ICLR 2026
improving their distinguishabilit.. This consistent transformation provides powerful visual proof that CPiRi has learned a transferable "meta-skill" for identifying and encoding relational patterns based purely on content, transcending the specifics of any single dataset.
A.3.2 CASE STUDY OF FORECASTING PERFORMANCE
To provide further intuitive evidence of CPIRi's 's effectiveness, Fig. [0 visualizes 336-step forecasting results for sample cases from the benchmark datasets. The plots clearly show that CPiRi's predic- tions maintain high fidelity to the ground truth across diverse patterns and time scales, demonstrat- ing that the spatial interaction module successfully utilizes inter-channel dependencies to refine its forecasts. In contrast, the channel-inde -ineneeeindeeeneent Sundial baseline struggles to capture the unique dy- namics of each channel, resulting in noticeable deviations. This qualitative comparison underscores the critical role of relational learning in achieving accurate multivariate forecasting.
A.4 EXTENDED ANALYSIS
A.4.1 THE STRUCTURAL COUPLING OF CHANNEL STRATEGY AND DECODING PARADIGM
The ongoing discourse regarding CI versus CD should be re-evaluated not merely as a data pro- cessing choice, but structural compromise necessitated by Non-Autoregressive (NAR) or Prefix-based architectures. Prevalent CI-based models (e.g.. PatchTST) implicitly factorize the prediction target into independent distributions conditional solely on the lookback window (i.e., X) 2 II ()).. just mapping the history to prediction. While this simplification miti- gates the noise accumulation inherent direct mapping and offers robustness on datasets with weak inter-series correlations, it fundamentally neglects the joint probability of future trajectories and the intricate interplay of multi-channel modalities.
Consequently, we argue that for MTSF tasks exhibiting strong channel heterogeneity. the expres- sivity of NAR-based CI methods theoretically bounded. Although recursive Autoregressive (AR) methods may inherently risk accumulated errors, their superior model capacity dictates that expand- ing the prediction window can yield lower error rates under an equivalent computational budget. Furthermore, by integrating dedicated channel interaction modules, AR architectures effectively adapt to complex inter-channel relationship modeling, overcoming the limitations of independent distribution assumptions and rendering them indispensable for strongly coupled channels. CPiRi exemplifies this paradigm shift: by anchoring a chameel-inerraction spatial module on a robust AR foundation (Sundial), explicitly restores the modeling of joint spatiotemporal distributions that purely Cl-based approaches fundamentally discard.
A.5 REPRODUCIBILITY
A.5.1 SOURCE CODE AND USAGE INSTRUCTIONS
The complete source code for CPiRi, along with all scripts required to reproduce the experiments, is released at the Github Repository JasonStraka/CPPiR) The repository structured facilitate ease of use:
/tutorial/ and / /sccripsss Contains scripts for downloading and preprocessing all datasets according to the standardized formats used in our experiments.
/experiments/ Includes the main experiment runner script (train py) and config- uration files for training and evaluating CPiRi and all baseline models on each dataset.
/basel i nes/: Provides the source code for the CPiRi architecture well as the imple- mentations of all baseline models integrated within the framework.
README md: A detailed guide is provided with step-by-step instructions how to set up the required software environment, prepare the data, and execute the training and evaluation scripts reproduce the paper's results.
20

---

Published as a conference paper at ICLR 2026
Smale
Tael
192
20
190
35.
drgg
dags
(a) METR-LA
(b) PEMS-BAY
Baay
100
200
300
400
306
690
100
204
300
400
314
63
PEMS-04
PEMS-
200
Value
Sindal
Value
500
500
Vahee
250
300
Mibe
Valos
500
01
100
200
500
400
500
500
700
Time steps
(e) SD
Figure 10: Forecasting case studies on five benchmark datasets. Each subplot visualizes 336-step forecast for four selected channels, comparing CPiRi's predictions against the ground truth and the Sundial baseline. CPiRi's 's forecasts maintain high fidelity across various patterns, demonstrating its effective relational learning, whereas the chaneee-independent Sundial baseline shows clear devia- tions.
21

---

Published as a conference paper at ICLR 2026
A.5.2 PSEUDOCODE OF THE CPIRI TRAINING PROCEDURE
Algorithm in the main paper provides a high-level overview of the permutation-invariant regu- larization strategy, To complement this, Algorithm 2 below offers a more detailed, step-by-step description of the CPiRi model's forward pass during a single training iteration. This pseudocode serves formal and unambiguous specification of our method, clarifying the flow of data through the three architectural stages and the precise point at which channel shuffling is applied. This level of detail is intended to eliminate any ambiguity and facilitate replication by other researchers.
Algorithm CPiRi Training Procedure with Bermutation-Invarian Regularization
Require: Training dataset D ((x*)) ))))) where X' E RBBLxC
Frozen temporal encoder Er trainable spatial module My. frozen decoder Drozen
Channel permutation set Ie (symmetric group of order C)
Ensure: Trained spatial module parameters 0* with permutation-innariann property
Initialize 0
2: for epoch = 1 to K do for batch (X') D do
4:
Apply channel shuffling:
Sample random permutation л Uniform(TTc)
X PermuteChannels((K
Yn + PermuteChannels(()n,
8:
Stage 1: Temporal Feature Extraction
9:
for = 1 to do
10:
he < Errozen ((),))))
11:
end for
12:
H Stack((h, .... be), dim 2)
13:
Stage 2: Spatial Interaction
14:
H'' + (H)
15:
Stage 3: Prediction Generation for
16:
do
17:
Y. t Dinonen c)) end for
18:
19:
y + (tack((y 8cl, dim = 2)
20:
Compute loss: t- Loss(), (+)
21:
Update parameters: + OptimizerStep(O, VoC)
22:
end for
23:
end for
24:
return optimal parameters l* +- 0
22


Reviews:
[{"review_id": "PWwbwnPuON", "reviewer": "Reviewer_DSFJ", "summary": "This paper tackles multivariate time series forecasting and points out that many models inadvertently entangle cross-channel interactions with the specific channel order seen in training. The authors introduce a three-stage framework: a frozen, pretrained univariate backbone to extract temporal features, a lightweight spatial module that views these features as an unordered set and models their relations with self-attention, and a frozen per-channel decoder. To enforce order agnosticism, training is done with random channel shuffling, discouraging any dependence on positional cues. On traffic style benchmarks, the approach proves effective, and the paper further shows it can generalize even when trained on only a subset of channels.", "questions": "The questions are included in the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The problem framing is timely and interesting. The paper not only says “permutations matter” but builds an explicit diagnostic: train with fixed order, test with shuffled order, show catastrophic failure for several competitive models. \n2. The proposed framework is reasonable and coherent, with the per channel frozen temporal encoder feeding a permutation aware spatial block, and the permutation based training strategy reinforcing the intended behavior.\n3. The reported improvements indicate that the method actually delivers robustness rather than just matching accuracy in the standard setting.", "weaknesses": "1. My main concern is the reliance on a large pre-trained backbone. Table 4 shows that removing the pre-trained weights leads to a substantial drop in accuracy, which suggests that much of the gain comes from the foundation model rather than from the proposed permutation invariant interaction itself. However, in Table 1 the competing CD baselines are trained from scratch and do not benefit from comparable pre-training. This raises a fairness question: to what extent are the improvements due to the architectural idea, and to what extent are they due to access to stronger prior knowledge? It would help to include baselines equipped with similar pre-trained features, or to report results for CPiRi without pre-training in Table 1.\n2. All benchmarks are traffic datasets. It is unclear whether the same channel ordering effect appears in other multivariate settings (e.g., energy, industrial telemetry, climate) or in higher-dimensional public datasets such as [1]. Showing at least one non-traffic dataset would clarify how general the phenomenon is.\n3. The paper shows that changing channel order can hurt performance, but it is not clear how this effect scales as the number of channels grows. Do larger channel sets make models more brittle to reordering, or does the effect plateau? A controlled study where channel count is progressively increased would make the robustness claim stronger.\n\n\n[1] U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecasting. 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Yv6nZnbRfi", "reviewer": "Reviewer_AJUX", "summary": "The paper proposes CPiRi, a channel permutation-invariant framework for multivariate time series forecasting. It combines a frozen pretrained temporal encoder with a lightweight spatial interaction module trained under random channel shuffling. This design makes the model rely on content-based relations instead of channel order, achieving strong generalization and state-of-the-art performance on multiple benchmarks.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Interesting motivation.\n2. Clear and easy-to-follow writing.\n3. Comprehensive theoretical analysis provides support for the proposed method.", "weaknesses": "1. Since this paper only uses Sundial as the temporal feature extractor, it lacks an explanation of why Sundial was chosen over other foundation models. Can this framework generalize to other pretrained models such as Chronos [1] or Moment [2]?\n2. I appreciate that the paper uses high-dimensional datasets with channel heterogeneity. This is an interesting attempt for scalability analysis. However, could you also test the model on Time-HD [3]?\n3. The main concern lies in efficiency (which the authors did not discuss in the main text). The framework invokes two large pretrained models and employs multi-head attention for high-dimensional inputs, which could lead to significant computational overhead.\n4. iTransformer [4] also applies multi-head attention along the channel dimension and is permutation-equivariant. How does this work differ from iTransformer in terms of channel modeling?\n\n[1] \"Chronos: Learning the language of time series.\" arXiv preprint arXiv:2403.07815 (2024).\n[2] \"Moment: A family of open time-series foundation models.\" arXiv preprint arXiv:2402.03885 (2024).\n[3] \"U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecasting.\" arXiv preprint arXiv:2507.15119 (2025).\n[4] \"iTransformer: Inverted Transformers Are Effective for Time Series Forecasting.\" arXiv preprint arXiv:2310.06625 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FmivrzCSk0", "reviewer": "Reviewer_Vucs", "summary": "CPiRi is a new framework for multivariate time series forecasting that stays accurate even when input channels are reordered or changed. It combines a frozen pre-trained temporal encoder with a lightweight spatial module trained using random channel shuffling, forcing the model to learn true content-based relationships instead of memorizing positions. This design achieves state-of-the-art accuracy, strong generalization to unseen channels, and robust performance under channel permutations.", "questions": "1. Why frozen temporal encoder? Will fine-tuning help improve performance?\n2. Will applying different temporal encoders, e.g., Chronos and Moment [1], affect performance?\n3. Does the strategy work for even higher-dimensional time series [2]?\n4. Can the authors visualize the latent representations before and after spatial encoding across different permutations?\n\n[1] Goswami M, Szafer K, Choudhry A, Cai Y, Li S, Dubrawski A. Moment: A family of open time-series foundation models. arXiv preprint arXiv:2402.03885. 2024 Feb 6.\n[2] Ni J, Wang S, Liu Z, Shi X, Zhong X, Ye Z, Jin W. U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecasting. arXiv preprint arXiv:2507.15119. 2025 Jul 20.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear Problem Identification: The paper clearly diagnoses a critical flaw in existing CD models using a simple \"channel shuffling\" diagnostic test. This test reveals that many SOTA models rely on \"positional memorization,\" leading to catastrophic performance collapse (e.g., >400% error increase for Informer) when channel order is changed.\n2. Effective and Sound Design: The \"spatio-temporal decoupling\" is an elegant solution. It leverages the power of a robust, pre-trained temporal model (CI strength) while using a separate, lightweight module to explicitly learn cross-channel interactions (CD strength). The channel shuffling strategy directly enforces the desired permutation-invariant property.\n3. Comprehensive Experimental Validation: The experiments are thorough and directly support the claims. CPiRi achieves SOTA accuracy , remains perfectly stable under channel shuffling , and shows strong inductive generalization to unseen channels. The ablation studies (Table 4) clearly demonstrate the necessity of each component: the spatial module, the pre-trained weights, and the shuffling strategy.", "weaknesses": "1. The framework's success is entirely dependent on the frozen Sundial encoder. The ablation study \"w/o pretrained weights\" results in \"complete failure\". This makes it difficult to separate the contribution of the novel CPiRi training strategy from the powerful priors of the (very large) foundation model it relies on.\n2. The individual components are standard: a \"standard Transformer encoder block\" for the spatial module and a data augmentation technique for training. The novelty is in the clever system design and training methodology that combines these parts, rather than a fundamentally new architectural mechanism.\n3. By freezing the temporal encoder, the model can handle structural co-drift (changes in channel order) but cannot adapt its temporal feature extraction to new patterns or \"abrupt trend shifts\". This limitation is noted by the authors and means it may struggle in scenarios where the underlying temporal dynamics of the data change over time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z7IsTu4793", "reviewer": "Reviewer_Kq3U", "summary": "This paper addresses a central paradox in multivariate time series forecasting: channel-dependent (CD) models overfit to channel order rather than learning true inter-channel relationships, while channel-independent (CI) models achieve robustness by sacrificing performance, as they neglect these critical dependencies. To resolve this trade-off, the authors propose CPiRi, a novel framework that employs a spatio-temporal decoupling architecture. CPiRi leverages a frozen, pre-trained foundation model to independently extract robust temporal features, which are then fed into a lightweight, trainable spatial module. This module is specifically trained with a permutation-invariant regularization strategy (channel shuffling) to learn content-driven, permutation-equivariant relationships. This design allows CPiRi to achieve state-of-the-art forecasting accuracy while demonstrating exceptional robustness, maintaining its performance even when channel orders are permuted during inference.", "questions": "1. The paper convincingly shows that CPiRi is robust to channel shuffling, while standard CD models are not. However, it's unclear how much of CPiRi's gain comes from its permutation-equivariant *architecture* versus its \"permutation-invariant regularization\" *training strategy*.\n2. The \"channel shuffling\" in Figure 4 is a key experiment. The methodology is unclear. How is \"25% shuffle\" defined? Does it mean 25% of the channel indices are randomly permuted *among themselves*, or 25% are swapped with other random channels? This needs a precise definition.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The proposed spatio-temporal decoupling architecture, which integrates a *frozen* time series foundation model (for temporal features) with a *lightweight, trainable* spatial module (for relational learning), is a novel and efficient design.\n2. The introduction of the permutation-invariant regularization strategy (channel shuffling) is a simple yet highly effective training technique to enforce the desired invariance.\n3. The experiments are pointedly designed to test the central hypothesis. The 'channel shuffling robustness analysis' (Table 2) is particularly impactful, providing a stark and convincing contrast between CPiRi's stability and the fragility of other CD models.", "weaknesses": "1. The paper *does* apply the shuffling strategy to baselines (Table 2, \"Train Shuffle\"), which is a strong point. Could the authors elaborate on *why* this strategy fails to rescue models like Informer or STID?\n2. The \"w/o regularization strategy\" ablation (Table 4) shows only a *minor* performance drop (e.g., 9.21% vs 9.14% on METR-LA; 10.80% vs 9.43% on PEMS-08). This suggests that the permutation-equivariant architecture alone provides almost all of the robustness, and the \"potent\" and \"critical\" regularization strategy actually has a minimal impact.\n3. The calculation cost of CPiRi, which adds an $O(C^2)$ spatial module, is non-trivial. However, in some cases (e.g., METR-LA, Table 1), it does not achieve SOTA performance compared to lighter models like STID.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)