XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis
Authors: Israel Zexer
Omri Azencot
Venue: Accepted by TMLR ()

Abstract: Multivariate time-series analysis involves extracting informative representations from sequences of multiple interdependent variables, supporting tasks such as forecasting, imputation, and anomaly detection. In real-world scenarios, these variables are typically collected from a shared context or underlying phenomenon, suggesting the presence of latent dependencies across time and channels that can be leveraged to improve performance. However, recent findings show that channel-independent (CI) models, which assume no inter-variable dependencies, often outperform channel-dependent (CD) models that explicitly model such relationships. This surprising result indicates that current CD models may not fully exploit their potential due to limitations in how dependencies are captured. Recent studies have revisited channel dependence modeling with various approaches; however, these methods often employ indirect modeling strategies, which can lead to meaningful dependencies being overlooked. To address this issue, we introduce \textbf{XCTFormer}, a transformer-based channel-dependent (CD) model that explicitly captures cross-temporal and cross-channel dependencies via an enhanced attention mechanism. The model operates in a \emph{token-to-token} fashion, modeling pairwise dependencies between every pair of tokens across time and channels. The architecture comprises (i) a data processing module, (ii) a novel Cross-Relational Attention Block (CRAB) that increases capacity and expressiveness, and (iii) an optional Dependency Compression Plugin (DeCoP) that improves scalability. Through extensive experiments on three time-series benchmarks, we show that \textbf{XCTFormer} achieves strong results compared to widely recognized baselines; in particular, it attains state-of-the-art performance on the imputation task, outperforming the second-best method by an average of 
 20.8\% in MSE and 15.3\% in MAE. Our code is publicly available at \url{https://github.com/azencot-group/XCTFormer}.

Published in Transactions on Machine Learning Research (03/2026)
FoTmer: Leveraging Cross-Channel and Cross- Time Dependencies for Enhanced Time-Series Analysis
Israel Zexer
zezer@post.bgu..cc
The Stein Faculty of Computer and Information Science
Ben- Gurion University of the Negev
Omri Azencot
azencot@bgu.ac.t
The Stein Pacully of Computer and Information Science Ben- Gurion University of the Negev
Reviewed on OpenReview: https: // openreveew. net/ per//frrm?? id= TEfyR4:0Tw
Abstract
Multivariate timo-series analysis involves extracting informative representations from quences of nultiple interdependent variables, supporting lasks such as forecasting, imputation, and anomaly detection. In real-world scenarios, these variables are typically collected from shared context or underlying phenomenon, suggesting the presence of latent dependencies across time and channels that can be leveraged to improve performance. However, recent find- ings show that channel-independent (CT) models, which assume no inter-variable dependencies, often outperform channcl-dopendent (CD) models that explicitly model such relationships. This surprising result indicates that current CD models may not fully exploit their potential due to limitations in how dependencies captured. Recent studies have revisited channel dependence modeling with various approaches; however, these methods often employ indirect modeling stralegies, which can lead to meaningful dependencies being overlooked. To address this issue, we introduce XCTFormer, a transformer- channel-dependent (CD) model that explicitly captures cross cross-tempont temporal and cross-channel dependencies via an enhanced attention mechanism. The model operates a token-to-ton fashion, modeling pairwise dependencies ween every pair of tokens across time and channels. The architecture Attention Block (CRAB)
prises (i) data processing module, (ii) novel that increases capacity and expressiveness, and (iii) optional Dependency Compression Plugin (DeCoP) that improves scalability. Through extensive experiments on three time- series benchmarks, we show that XoTFrrer achieves strong results compared to widely recognized baselines; in particular, it attains state-of-the-art performance on the imputation task, outperforming the second-best method by an average of 20.8% in MSE and 15.3% in MAE. Our code is publicly available at htttp://ggttheei wttp:///eeeeeeeggrooooooo---/-
1 Introduction
Forecasting, anomaly detection, and imputation are critical tasks across wide range of real-world domains Jin al. 2024). For instance, forccasting is utilized in energy management, weather prediction, healthcare, and more Mystakidis al. 2024; Brunet et al. 2023 Duarte et al. 2021). Time-series analysis plays a vital role in extracting key information from sequential data to facilitate these tasks. The effectiveness of this information extraction is crucial, as it directly impacts the performance of subsequent time-scrios tasks (Trirat
et al. 2024 Accurate time-series analysis enables organizations to enhance decision-making and optimize resource allocation (Bi et al.) 2018 Wang et al. 2024b), highlighting the importance of the information extraction component as a key area of research.
Time-scries data can be modeled using two main approaches HHa al. 2024). Univariate approaches treat each channel independently, disregarding any potential relationships between them. In contrast,

---

Published in Transactions on Machine Learning Research (03/2026)
multivariate approaches take into account not only the temporal behavior within each channel but also potential dependencies across channels. In real-world scenarios, multivariate datasets are often derived from a common underlying process or phenomenon, which typically leads to dependencies among the features (Chen et al. 2024b. Incorporating relevant signals enhances representation quality and improves accuracy in downstream lasks (Isik el al., 2025: Domingos, 2012). As a result, multivariate models are generally expected to outperform univariate models by leveraging both cross-channel dependencies and temporal dynamics. Therefore, time-series analysis can benefit significantly from richer representations when cross-channel dependencies are utilized.
However, recent work in time-scries forccasting by Han et al. (2024) challenged this assumption by showing that channel-independent (CI) models, which treat multivariate time-series as separate univariate channels and ignore potential inter-channel correlations, outperform channel-dependent (CD) models that explicitly model such dependencies. They attribute this surprising outcome to trade-off between capacity, defined as a model's ability to fit complex patterns, and robustness, defined as its ability to remain accurate in the presence of noise, input variation, or distribution shifts. While CD models gain capacity by incorporating cross-channel information, this often comes at the expense of robustness, making them more sensitive to distribution shifts. In contrast, CI models sacrifice some capacity by ignoring cross-channel dependencics, thereby improving robustness and generalization accuracy. These findings challenge the common belief that adding relevant information typically improves representation quality and accuracy, revealing a gap between channel-dependdntt methods and their unrealized potential. Motivated by these findings, we seek in this paper to address the following question:
How should we model sequential cross-chanmel information to realize its potential?
Recent rescarch has revisited channel dependence with cross-channel modeling approaches that often outper- form channel-indeeendeer (CI) baselines. i'Tranformer (Liu et al. 2024) largets cross-channel dependencies by treating each channel as token and applying a Transformer on the token sequence. CrossFormer (Zhang & Yan 2023) and CARD (Wang et al. 2024c) address both cross-channel and cross-time relationships, by employing a two-stage pipeline for sequence modeling and channel processing. Despite recent advancements, most methods model dependencies across different channels and time indirectly, thereby potentially over- looking important interactions. Additionally, cross-channel dependencies are often unknown in advance, as the underlying generative process is typically unknown. These dependencies may also change over time (Zhao & Shen 2024), raising the need for simultaneous cross-channel cross-time modeling. To address these challenges, we propose a direct modcling strategy with a token-to-token approach that explicitly captures each token's pairwise dependencies across all channels and time-steps. This potentially minimizes essential information loss associated with existing indirect models. To accomplish this, we introduce XCTFormer, a Transformer-based framework that models all pairwise dependencies directly within a single attention block, token-to-token, effectively identifying the most relevant dependencies for downstream tasks.
The backbone of the XCTFormer consists of three novel components: (i) data processing unit (ii) the Cross-Relations Attention Block (CRAB), and (iii) the Dependency Compression Plugin (DeCoP). First, we independently patch each channel and tokenize the data. Next, we flatten the channel and time dimensions, which allows CRAB and DeCoP to capture all pairwise dependencies in a token-to-token manner. CRAB extends the standard attention block (Vaswani et al. 2017) with two key modifications to improve expressivity and robustnoss. First, introduces a lcarnable, non-boolean masking mechanism that supplements conventional binary masks by weighting dependencies according to their learned importance. This allows the model to focus on the most crucial dependencies for the downstream task. Second, CRAB replaces the standard softmax function with new normalization technique that retains the properties needed for attention activation (Saratchandran et al. 2025) while allowing negative weights. This extension increases the model's expressiveness by enabling it to capture a wider range of relationships, as suggested by Lv et al. (2024). Lastly, DeCoP is an optional module that partly modifies CRAB with the addition of a learnable compression mechanism, designed to enhance scalability for datasets with numerous channels. addresses the memory limitations imposed by the transformer's quadratic attention mechanism. DeCoP compresses the quadratic attention into lincar form while minimizing information loss through learnable compression transformation.
2

---

Published in Transactions on Machine Learning Research (03/2026)
We evaluated XCTFormer against various baseline models multiple downstream tasks, inchuding forecasting, anomaly detection, and imputation, demonstrating strong results. Our main contributions are:
We identify a key limitation in the current literature on time-series modeling: while analysis methods have advanced substantially, little emphasis has been placed on explicitly capturing both cross-channel and cross-time dependencies in unified manner. Most existing approaches either model temporal patterns inter-channel relations separately, which restricts their ability to exploit the full structure of multivariate timo-scrics data.
2. To address this gap, we propose XCTFormer, a general-purpose framework that models all pairwise cross-channel and cross-time dependencies directly through token-to-token mappings. XCTFormer integrates two complementary components: (i) CRAB, which enhances expressiveness by learning importance-aware attention masks and allowing signed attention activations, and (ii) DeCoP, which mitigates scalability bottlenecks on high-dimensional data through learnable compression while minimizing information loss.
3. We evaluate our approach on three core time series tasks, forecasting, anomaly detection, and imputation, obtaining consistent improvements or competitive performance against strong baselines. In particular, we achieve state-of-the-art (SoTA) performance in the imputation task, with average reductions in MSE and MAE of 20.8% and 15.3%, respectively. We also observe notable gains in forecasting accuracy and anomaly detection performance. Furthermore, we introduce synthetic dataset, evaluated the forecasting task, to evaluate our model on capturing cross-variate and cross-time relationships and to test its robustness to spurions correlations (Appendix Db.
2
Related Work
From Classical Methods to Deep Architectures. Multivariate time-series analysis has progressed from traditional statistical models like ARIMA (Box & Jenkins 1970), which often struggle to capture nonlinear dynamics, to deep neural approaches such as LSTM (Hochreiter & Schmidhuber 1997 and TCN (Franceschi
et al. 2019). While these deep models improve expressiveness, they may still fall short in modeling very long- range dependencies. More recently, time-series tasks have utilized both simple MLP-based architectures (Zng
iet al. 2023 Wang et al.
2024a
Nochumsohn et al. 2025b) and Transformer based models (Zhou et al. 2021
Nie et al. 2023 Liu al.
2024 Zhang & Yan 2023 Wang et al. 2024c). Broadly, these models adopt either
indamnel-ddeeendent (CI) strategy, treating each variable separately, or a channel-dependent (CD) approach that explicitly leverages cross-variable structure.
Early CD designs: temporal focus with implicit cross-channel modeling. Early CD models emphasized efficient temporal modeling and attention computation. These methods implicitly incorporated cross-channel information by generating tokens representing all channels at the same or nearby time-steps.
typically using 1D convolutions, before applying cross-time attention (Li et al. 2019b Zhou et al. 2021 Wu et al. 2021 Liu et al. 2022b Zhou et al. 2022). However, since inter-channel relationships were not explicitly embedded, these approaches failed to fully leverage cross-channed dependencies (Zhang & Yan 2023). Conscquently, the attention mechanism struggled to recover missing structure, leading to suboptimal representations (Liu al., 2024).
CI baselines and channel as token formulations. On the CI side, PatchTST partitions each channel into overlapping time patches, treating these patches as tokens. These channel tokens then passed to stacked transformer architecture that exclusively models cross-time dependencies (Nie et al. 2023 Linear models, when applied independently to cach channel, have also demonstrated competitive performance (Zeng et al. 2023 Das et al. 2023). MTLinear (Nochunsohu el al., 2025b) is CI: it first clusters channels and then trains a predictor for each cluster to mitigate conflicts the multi-task objective, but cross-channel dependencics are not explicitly modeled. To reintroduce cross-channel interactions, iTransformer represents each entire channel as a single token, enabling self-attention to operate across variables [Liu et al.) 20244).
LEDDAM YYu et al. 2024) takes different approach by decomposing each series into trend and seasonal
3

---

Published in Transactions on Machine Learning Research (03/2026)
components, processing the seasonal part via parallel cross-channel and cross-time pathways before combining them. However, it still lacks unified mechanism that jointly models both dimensions within its attention module.
Two-stage explicit cross-time and cross-channel modeling. CrossFormer addresses the limitations of earlier models by dividing each channel into equal-length segments and embedding these segments individually to better preserve semantic information (Zhang & Yan 2023). This approach, along with CARD (Wang et al. 2024ct, utilizes Iwo-stage atlention scheme: first atlending along the temporal dimension, then explicitly across channels. While this sequential treatment is effective, it captures cross-channel temporal dependencies only indirectly, which may result in limited expressiveness.
Time-series foundation models. Time-series foundation models (TSFMs) have attracted growing interest unified architectures for zero-shot and few-shot forecasting across multiple datasets. They pretrained diverse time-series corpora to learn general-purpose temporal patterns that transfer across domains. Most
TSFMs follow channel-independers designs (Das et al., 2024 Ansari et al. 2024 Shi et. al. 2025 Auer el al. 2025 Nochumsohn et al., 2025a), handling multivariate inputs by processing each variable independently
as a univariate scries. This choice improves scalability and helps pretraining remain broadly applicable across datasets with varying numbers and types of variables. But it may fail to fully leverage cross- variable dependencies that are crucial in many real-world multivariate systems. Recent efforts have begun to address this cross-channel challenge. For example, Chronos 2 (Ansari et al., 2025) introduces group attention to share information within sets of related series, while Moirai-1 (Woo et al. 2024 proposes an any-variate architecture that flattens multivariate time series into single token sequence, allowing it to handle an arbitrary number of variables and jointly model cross-channel structure. Overall, these works highlight that effectively modeling cross-channel structure remains a key challenge, also within the time-series foundation model paradigm.
3 Vanilla Transformer Attention
To facilitate clear understanding of our proposed modifications, first outline the standard transformer attention mechanism (Vaswani et al. 2017). Consider an input sequence X E NxD. where N denotes the sequence length and D: the per-token input feature dimension. In our case, the same sequence serves to form queries, keys, and values. The atlention block projects X with learnable matrices (with Dm being the per-head attention dimension):
W. WR. W. E RDixD..,
XW,, K XW V XW. R.R.V..
Scaled dot-product scores quantify pairwise query-key affinity:
= QKT
NxN
VDm
An optional mask M E NN encodes disallowed positions (e.g,, padding or future time-steps) via
0, allowed
Mij
-0, blocked
We then convert scores into attention weights row-wise and aggregate values accordingly:
Softmax(A + M) E NNN
D=WV NDD...
Before normalization, the optional mask M is added to the score matrix A. Applying row-wise softmax to A + M effectively assigns zero weight to blocked entries; hence, the mask serves as a selection mechanism that suppresses specific relationships (e.g., to prevent information leakage from future time steps). The resulting attention matrix W nonnegative with each row summing to one, yielding a probability-Ii distribution over keys for each query. Consequently, the output O a row-wise weighted combination of the value vectors, governed by these attention weights, representing the attention block's output.
4

---

Published in Transactions on Machine Learning Research (03/2026)
4 Former
A.1 Data Processing
Modifed Transformer
A.2 Data Processing
C. Downstream task heads
Nx bm
Flatten
NarchNorm
Horzzn
Linear
xx Dm
Permute
Dropout
Positional Encoding
0xCxTx00
Feed-Forward
8xCxTxDm 8xT'xCxOm Om
Flatten
Reshape
8xCxTxL
Patching
8xCxT
Premute
BatchNorm
8.1 Plain CRAB
8.2 CRAB DeCop
Nx
Dot Prod
Dot-Prod
AbsAct
Learnable
Multi-Head Crose Relational Attention Block (CRAB)
Compression
Chanciel
Element Wize Mui
Dot-Prod
Matrix
Pos
BxTxC
Batch
Learnable
Dat-Prad
Dot Prod
Mask
Figure 1: XCTFormer model overview. Multivariate inputs are divided into patches channel, tokenized, and then passed as a flattened time-and-channel sequence through stacked Transformers with CRAB attention. CRAB utilizes learnable mask and a signed, non-softmax normalization to model direct token-to-token dependencies. The optional DeCoP module enhances scalability by compressing pairwise attention into a compact representation, reducing memory and compute requirements; since the compressed attention loses the original pairwise structure, the learnable mask is not applied when DeCoP is used. In the diagram, panel B.1 illustrates the CRAB flow and panel B.2 illustrates the CRAB with DeCoP flow.
To model all pairwise dependencies through direct loken-lo-loken modeling, present XCTFormer, Transformer-bases general-purpose, encoder-only time-series model. XCTFormer comprises a universal backbone and a task-specific head. The backbone has three components: (i) a data-proccssing unit, (ii) a Attention Block (CRAB), and (iii) a Dependency Compression Plugin (DeCoP). Figure summarizes the pipeline: panel A lokenizes the input and flattens across channels and patches to form a unified sequence that exposes all pairwise dependencies to the Transformer; Panel applies stack of our modified Transformer equipped with CRAB (Sec. 4.2 and the optional DeCoP module (Sec. 4.3) Panel C maps the resulting representations to predictions via. a task-specific head.
4.1 Data Processing
To effectively capture the diverse and unknown dependency structures present in multivariate time-series, XCTFormer is designed to explicitly model all pairwise cross-channel and cross-temporal relationships. For each token, we define potential pairwise dependencies across channels and time points throughout the entire time-serics. These dependencies can take one of four potential forms: (i) self-lag relationships, where past values of the same channel may influence future states; (ii) cross-channel synchronous tenships, where channels identical time points may influence one another; (iii) cross- channel lagged relationships, where other channels may exert temporal influence through their historical values; and (iv) forward-in-time relationships, where current values may influence subsequent values within the same or different channels.
For visual representation of these dependencies, see Figure 2
Modeling dependencies at the level of individual measurements is both computationall expensive and impractical, as single measurements lack meaning without temporal context (Zeng et al. 2023). To address this, we adopt patching strategy (Nic et al.] 2023),, segmenting each channel independently into short temporal patches that capture local patterns. We project each patch through learnable linear layer and add learnable positional encoding along the time axis for each channel, generating tokens. Finally,

---

Published in Transactions on Machine Learning Research (03/2026)
t-2
t-
t+1
t+2
C4
Figure Potential cross-channel and temporal dependencies for token at channel time
permute the data dimensions that the patch sequence comes first, then flatten the tokens across the patch and channel dimensions to create unified sequence. This enables the Transformer to model all pairwise dependencies (see Panel A, Figure D]. We apply this permutation to simplify the structure of the attention mask, making it easier to analyze further (see App. C.1
4.2
Attention Block (CRAB)
CRAB modifies the standard attention (Vaswani et al. 2017) block with two complementary components designed to increase models' expressivity: (i) a learnable, non-Boolean relational mask designed to learn important dependencies, and (ii) a signed, absolute-sum normalization activation function that replaces the original softmax, designed increase expressiveness by allowing negative values, inspired by Lv al. 2024
findings. For visual representation of the model, refer Panel B.1, Fig. I
Our learnable non-Boolean mask is designed to learn the most dominant dependencies between tokens. We apply this mask to posilive-transformee attention score matrix. Starting from the score matrix A E NxN, we remove sign information via global shift,
A+ A min(A)
Then, we initialize learnable real-valued mask M E NxN with zero mean and standard deviation 22/N following He initialization (He et al. 2015). We apply in an clement-wise fashion:
A = M MoA.
Thus, this shift removes sign while preserving the relotive ranking and pairwise differences of the original scores, while the learnable mask M sets their signs and reweighs magnitudes. The produced output then passed the activation function.
Our modified attention activation function replaces softmax with row-wise normalization that yields signed allention weights. Allowing negalive weights can increase the model's expressive power (Lv el al. 2024). To ensure stable training, our proposed activation must preserve the stability property that underlies softmax's success [Saratchandran et al. 2025): maintaining a bounded Frobenius norm of VN for the produced activation matrix,
|| Activation(A)]]= S VN.
For an attention-score matrix A E R N we define our activation function as a normalization of values by the absolute sum of the corresponding row. The AbsAct function is defined as:

---

Published in Transactions on Machine Learning Research (03/2026)
Aiy + Σx= Ait +
AbsAct(Ais)
vi,j fI.....NN
(1)
where E = x 1 and 6 = 1 x 10 are numerical stabilizers. The parameter shifts each attention score before normalization, while adds a positive margin ensure the denominator remains non-zero. Our activation function satisfies the bounded-nore constraint, ensuring stable training (see Appendix A.1 for formal proof). Additionally, allowing negative attention weights enables the model to capture wider range of dependencies, thereby increasing its expressiveness (see Appendix C.4 for additional analysis). Note that since AcsAct divides each entry by its row's absolute-value sum, the standard 1/vd,, scaling of attention scores cancels out and is therefore omitted (see Appendix A.2 for a formal derivation).
4.3 Dependency Compression Plugin (DeCoP)
Since we model all pairwise cross-channel and cross-time dependencies (Figure 2 using the attention mechanism, memory and compute scale as O(N2), where N is the total number of modeled relations. Applying such attention to datasets with many channels can exceed hardware capacity, potentially leading to running failures. To address this limitation, we introduce DeCoP, a plugin which compresses each row of the attention matrix A E N with a learnable transformation, yielding a compressed matrix A E Nx with & < N. This reduces the dominant cost from quadratic to linear in N for fixed k, improving scalability regardless of dataset size. For visual plot of the model, refer to Panel B.2, Fig. DeCoP is defined follows:
DeCoP introduces a learnable compression transform whose parameters are initialized with He initialization He
et al. 2015) and fine-tuned during training. Our compressor is a matrix, C E R such that k < N. Let.
Q =x K = XW with Q, K DW.. We utilize in the attention computation as follows:
A. = Q (K CC R
Due to the associative property of matrix multiplication, computing Q(KT C) is equivalent to computing (QKT)C, enabling us to obtain a compressed version the full token-to-token attention without materializing the quadratic x N attention matrix. While vanilla attention incurs O(N' cost for computing QK" DeCoP's reordered computation achieves O(NDmk) complexity, scaling linearly in N when < N while preserving essential attention relationships through the compressed representation. For full complexity analysis, see Appendix A.3
Finally, we also need to modify V calculations as the attention dimension is reduced to k. The modification is defined as follows:
W E K
= V VER
The new V represents the values corresponding to the compressed attention dependencies. The final output is calculated as follows:
W. = Ab) Nxk = W.V E NxD..
Note that when DeCoP is applied, the learnable mask M from CRAB (Sec. 4.2 is not used. Since the compressed attention A, E Nx does not preserve the direct pairwise structure of the full x matrix, element-wise masking of individual token-to-token relationships is no longer applicable; instead, AbsAct is applied directly to the compressed scores A.
5 Experiments
We evaluate the proposed XCTFormer across three fundamental timo-scrics tasks: long-term forecasting, imputation, and anomaly detection. Our experiments use well-cstablished benchmark datasets commonly used in prior work to ensure fair comparison with existing approaches. Across all experiments, apply
7

---

Published in Transactions on Machine Learning Research (03/2026)
DeCoP (Sec. 4.3) to datasets with more than 60 channels; datasets with 60 or fewer channels use the plain CRAB module without. DeCoP (Sec. 4.2) For each lask, we present the experimental setup and datasets and report comparative results against strong baselines. The following subsections detail our experiments for each timo-series task. App. B provides formal task formulations, extended training and evaluation details, including hyperparameter search protocol and values.
5.1 Long- Term Forecasting
Time-series forecasting aims to predict future values from historical observations. We evaluate our model on seven widely used multivariate datasots, comprising four ETT subsets (ETTml, ETTm2, ETTh1, ETTh2),
Weather, Electricity (ECL), and Traffic, following Autoformer (Wu et al.) 2021 We adopt the TemesNet
setup (Wu et al. 2023) with a lookback window of 96 time-steps and forecasting horizons {96, 192, 336, 720).
We compare against twelve widely recognized forecasting models: (i) Trranformmr--aaee
Autoformer (Wul
ct al. 2021). FEDformer (Zhou et al. 2022), Crossformer (Zhang & Yan 2023). PStchTST (Nic ct al.
2023
Transformer (Liu et al.
2024); (ii)
DLinear (Zeng et al. 2023), TiDE (Das et al.
2023
TimeMixer (Wang et al.
2024ah, MTLinear (Nochumsohn et al.
2025b); (ii) Hybrid Transformer and Linear:
LeDDAM (Yu al. (004)); (iv) TCN-basod: SCINet (Liu et al.
2022a TimosNot (Wu et al. 2023). We also
include synthetic dataset whose target channel is constructed from lagged tross-variate signals. The dataset
also includes distractor channels (random walks) to test robustnesss to spurious correlations. Models are (MS) setting, which allows to specifically test each model's capacity to
evaluated in the aultivariate--o-singe capture cross-channel and cross-ting toss-timee dependencies. To ensure a fair comparison, all models are trained using the same hyperparameters as ETTml ( (Appendix D
Table 1: Average long-tern forecasting results comparison. Synthetic is evaluated under multivariate-to-singe (MS) setting while all other datascts use multivariate (M). We compare extensive competitive models under different prediction lengths. Aog is averaged from all four prediction lengths, that (96, 192, 336, 7203.
Modela
XCTFurmee
T
MTTlow
Loidan
Tharskmes PaddIST Creabeuer 2004) (CLB
TIDE Timmser
DLinarr
SCINNe
PEDooner
(Ouna)
(ICLB
(AISTATS
OCSIL
2125) (CCR
(JCLR 2023)
(NeutPS
20021
(NealPS
Metric
MAR
MaR
MAR
MAR
von Mor vsn MAR Mar
MAR Mow
Mse
MAR
MAF
KAR
MAR
MAR
0.3se
p.ass
0:87
10307 0,37 (1007 0.30 0.000 Ja1r o 000 (0.00 oitre ta ase 0. ja39 0.001
0.407
p.009
0.897
0.433
0.00
0.439
Weather
|0.237
0.207
10.206 0.202
Josas
0.00
(1:00 o Juaso poss wars jusss 0.285 (o 0.260 0.300 joo 0.300 just9 0,786
(0:05 0:s1
0.033
10:000
0.300
0.007
ECL" 00.100
0.205/0.105
0.253
Ja.os
0,283
Jo.os 0.2sa J1.822
|225|o.125 0.270 juato 0,518 (0.444 0.504 (025 0.944 juise 0.304
(0:255
0.s19|p.20
0.965
(0211
0.327
p.227
0.961
Toatue
Ja.435
0.357
(n.016
0.261
Juon
0.373
957
0.301 (a. 0.297 jo.ras 0.292//.500 0.311 (0.007 0.r3d Ja 750 0.075 ju.00 0.306
0.383
1.901
0.000
|0.009
0.376
0.379
Symthetse
J0,00
0.125|
|1.00
0.311
Juuzo 0.19 Jouss 0:390 jaarz
o.124/1
0.995 juits
|.009/p.00
0.50
0.330
12.076
p-119
1" Coun
4
0
6
0
Aug PLOP|
8.380 105
7,000/06
5.0EE00 |
10948100
|
8.288) (o5
2.31 (00
1.818 100
|
1:288 (11
3.00E
14418100
2000 (00
5,00
Ang P
5:66-406
2.01
2.88405
1134446
2,505405
|
770004055
8308406
5.200-00
6,001404
1/408-10
2.154405
Fonlisated molticoriaateeeen toleeigge (MS) witting: typerparemeterrs idhiticall BTTmi for modelee Appiniii
dacaentt detalls.
Reported ATLlincor results rofloct the per-dataect bost MUNNiineo MIDLinncrr DeCCa mabled Tormeet this datawet.
Results. shown in Table XCTFormer delivers strong lorecasting results compared to widely recognized baselines across diverse benchmarks, achieving second-best performance on 8 out of 10 evaluation metrics. These results highlight the model's effectiveness in capturing cross-channel dependencies in the long-term forecasting task. On the synthetic dataset, where the target is generated from lagged cross-variate signals, XCTFormer ranks second (Avg MSF= while using 55x fewer parameters and 364x fewer FL.OPs than the top-ranked model. These results suggest that XCTFormer models cross-channel and temporal dependencies effectively in this set ling, while being less affected by distractor channels, indicating robustness to spurious correlations. Full results and additional details are provided in Appendix D
5.2 Imputation
Time-series imputation reconstructs missing values from observed data. We evaluate our model on six widely used multivariate datasots: four ETT subsets (ETTml, ETTm2, ETTh1, ETTh2) (Zhou et al., (021)) Electricity (ECL), and Weather. We adopt the + setup, using a lookback window of 1024 time steps and applying random missing-mask rates of {12.5%, 25%, 37.5%, 50%]. We compare against eleven
8

---

Published in Transactions on Machine Learning Research (03/2026)
Table 2:
To evaluate our model performance on imputation, we randomly mask (12.5%, 25%, 37.5%, 50% of the time points in a time scries of length 1024. The final results are averaged across these different masking ratios. Red indicates best performance (lowest error), blue indicates second best.
Models XCTFarmer TimeMisars--
TinmMaar (Trasafommr PuidITT Crnedfommn FEDfarmer
TIDE
DLinner
TimasNNt
MICN
Autafarmer
(Ours) [ICLR 2025) (ICLR 2024) (ICLR 2024) (ICLR 2020) (JCLR 2020) (ICOL, 2022) (TMLR 2023)
(AAA) 2028) (JOLR 2023) (ICLR 2022)
(NouttPS 2021)
Metric
MSE MAE
MSE
MAE [MSE MAE [MSE MAE IMSE MAE (MSE MAE |ASE
[MSE
MAE
[MSE
MAE
[ASE MAE
|ASE MAE
JASE
MAE
ETT(A02)/0000 0.141
(0.055
0.154
[0.097 0.220
|0.006 0.205 (0.120 0.225 0.150
0,258
|0.14
0,200
(0.314
0.306
[0.115
0.229
(0.079
0,182
[0.129 0.254
[0.104
0.215
Wodter j0.022 0.049 (0.049 0.078 |0.001 0.114 jo.096 0.302 (0,082 0.149 1.160
0.111
Joost
0.189
(0.063
0.131
|0.07]
0.307
[0.06]
0.088
|0.056
0.126
|0.066
0.107
BCL? |0.046 0.140 (0.100) 0.197 Ju.122 0,261 jo.141 0.223 (0.129 0.198 1o.266 0:204
joast
0.114
10.182
0.202
|o.0s0o
0.200
(o.2at
0,256
J0.188
0.246
[0.141
0.294
DeCoP was enabled for XCTFormer on this dataset.
Table 3: Fl scores (%) for anomaly detection across five benchmark datasets. Red indicates best (highest), blue indicates second best.
Modkk.
NCTPirrre Thwllbere- Thadderer
THE
TiraNfe PREllrow:: (CSS)
ETTS
Disee LAME DETE
LOSS
Prodenet
Ancuayy
Dellewen
TIN
Lagttow (Neetressn
Transheeer
LITM
(Durs)
(CLB 3000
DCUR 201
TVER DOS
(CEE SOU
OUNE, WNE
WEL DM22 OTED SOT
IAXAI
84.21
80.50
71.16
05.01
85.81
85.08
K2.63
53.18
77.10
84,62
71.31
85.11
63.04
65.49
81.66
76.82
[1.49]
76.21
70.56
71.41
79.05
85.82
72.51
70.15
86.15
78.57
78.95
85.03
81.88
77.50
89.53
79.66
64.50
85.31
86.00
81.10
[7.00]
70.57
78:48
$1.95
[MAP
66.08
73.10
06.87
61.00
11.32
70.76
19.21
501.0
09266
71.08
06.90
71.32
71:00
71.18
04.92
70.00
[0.45]
60:97
09.70
F0.48
12.60
76.75
91.78
95.99
91.78
[5.00]
$1.30
1991
00.30
07.00
90.17
02.50
97-47
97.23
07.15
91.70
95.35
97.20
77.00
90.20
89.08
79
40
77.00
73.61
p0.37]
76.70
76.07
61.07
Arg
87.57
87.47
71.45
86.34
86.97
84.25
50.57
82.46
52.08
76.74
64.20
82.37
80.50
78.53
77.91
(7.24)
76.00
70.88
77.97
"Aweeay
widely recognized models: (i) Transformer- based: Autoformer (Wu et al. 2021), FEDformer (Zhou et al.
2022), Crossformer (Zhang & Yan 2023). PatchTST (Nic et al. 2023) iTransformer (Liu et al. 2024); (1i
MILP-based DLinear (Zeng et al.) 2023). TiDE (Das et al. (2023). TimeMixer (Wang et al. 2024a): (iii)
Convolutional-baseel SCINet. (Liu et al. 2022a), TimesNet (Wu et al. 2023). MICN (Wang et al. 2023
Results. shown in Table 2 XCTFormer delivers state-of-the-art (SoTA) imputation results in comparison to competing baselines across diverse benchmarks. With the best performance all 6 evaluation metrics, particularly, our approach outperforms the second-best baseline by an average of 20.8% on MSE and 15.3% on MAE across all datasets, highlighting the model's offectiveness in capturing cross-channel dependencics.
5.3 Anomaly Detection
Anomaly detection seeks to identify unusual or abnormal patterns in time-series, often corresponding to faults, attacks, or rare operational modes. We evaluate on five widely used benchmarks: SMD (Server Machine Dataset, (Su et al. 2019)), ), SWaT (Secure Water Treatment, (Mathur & Tippenhauer 2016 ) PSM (Pooled Server Metrics, (Abdulaal et al., 2021)), and NASA telemetry datasets MSL and SMAP (Hundman et al. 2018). We compare against nineteen widely used models: (i) RNN/ICN: LSTM (Hochreiter & &Schmidhuberr Transformer (Vaswani al.
1997), TCN (Franceschi et al., 2019]: : (ii) Transformer-
2017). Log Trans (Luu et al. 2022b),
Li et al. 2019a), Reformer (Kitaey et al. 2020), Informer (Zhou et al. 2021) Pyraformer
Autoformer (Wu et al. 2021). ,, FEDformer (Zhou et al.] 2022). ETSformer (Woo et al.
2022) 0, Stationary
(Non-stationary Transformer) [Liu et al. 2022c), Anomaly Transformer (Xu et al. 2022
Light (Zhang)
ct al. 2022), iTransformer
(Liu et al. 2024): (ifi) Statc-space: LSSL (Gu et al. 2022):
(iv) Linear MLLP
DLinear (Zeng et al.) )202)), TiDE (Das el al. 2023); (v) Convolutional/Mivcs TimesNel (Wu et al. 2023
TimeMixer+ (Wang et al. 2025).
Results. As shown in Table 3 XCTFormer performs competitively against the considered strong baselines
(for detailed comparison table refer to Appendix [.6). Our model achieves high F1 score, suggesting it captures cross-channel dependencics effectively for anomaly detection.
9

---

Published in Transactions on Machine Learning Research (03/2026)
6 Analysis
Ablation Study. To evaluate cach component's contribution, we conducted an ablation study across three fundamental time-series tasks: long-term forecasting, imputation, and anomaly detection. Our methodology involved two categories of experiments: (i) component wise analysis, where we systematically removed or altered individual architectural modifications introduced to the vanilla Transformer to isolate each component's impact. Modifications include: removing the learnable mask, reverting the activation function from our proposed approach back to the standard softmax and both; (ii) dependency modeling analysis, where we examined variants that model only cross-channel dependencies (inspired by illrnsformer (Liu et al., 2024 or only temporal relationships (similar to PStchTST (Nic et al. 2023)) to validate the necossity of our integrated cross-channel and cross-time modeling strategy. Results presented in Table 3 report the average performance metrics across all datasets and experimental configurations specific to each task for every model variation. The full XCTFormer consistently outperforms all variants across all three tasks and evaluation metrics. These findings validate our architectural design choices and provide empirical evidence that each proposed component contributes meaningfully to the model's overall performance across diverse time-series applications. For detailed experimental configurations and full results, refer to Appendix D.2
Table 4: Ablation study results across different tasks, evaluated with different XCTFormer variations.
XCTFormer
Long-ten term Forecasting
Imputation
Anomaly Detection
vs Others
MSE
MAE
MSE
MAE
Precision
Recall
F-Score
(%)
XCTFormer (Original)
0.328
0.337
0.044
0.124
92.1
83.7
87.6
W/o mask"
0.351
0.369
0.051
0.131
90.6
74.5
81.1
8.0%
Original softmax activation
0.359
0.364
0.053
0.143
90.3
75.4
81.5
9.7%
Vanilla transformer
0.361
0.364
0.060
0.149
90.9
76.1
82.0
11.2%
Sequence modeling
0.341
0.343
0.052
0.131
91.2
78.8
83.9
5.6%
Channel modeling
0.341
0.348
0.081
0.174
91.2
76.1
82.3
14.2%
W/o mask averages exclude DeCoP datasets (ECL, Traffic) where the learnable mask is not applicable.
Robustness Across Random Seeds. To evaluate the stability and reliability of XCTFormer, we assessed its performance across different random initializations. Neural models are often sensitive to parameter initialization randomness and the order of training samples, leading to variability in results. To address this, we trained FCTFrrmer using the optimal hyperparameters selected by validation on five distinct random seeds (2021 to 2025). For each the three primary time-series tasks: long-term forecasting, anomaly detection, and imputation, we report both the mean and standard deviation of the relevant performance metric, providing more robust estimate of model effectiveness, We further quantify robustness using a confidence score, calculated from the coefficient of variation (Reed et al.) 2002), which reflects the model's precision and repeatability. In this context, lower standard deviation indicates greater consistency and, therefore, higher reliability. Summarized seed robustness results are presented in Table 5 For more information on confidence score calculation and the complete analysis tables, refer to Appendix E.3
Table 5: Averaged metrics of trained models, evaluated on five different seeds (2021 2025) across all datasets, are reported for each metric and lime-series task, along with the corresponding confidence interval.
Task
Metric
Mean + Avg. Std
Confidence Score (%)
Long-Term Forecasting
MSE
0.330 + 0.003
99.05%
Long-Term Forecasting
MAE
0.339 + 0.002
99.28%
Imputation
MSE
0.047 + 0.005
89.14%
Imputation
MAE
0.128 1 0.009
93.10%
Anomaly Detection
Precision
91.382 0.868
99.05%
Anomaly Detection
Recall
79.660 + 4.298
94.60%
Anomaly Detection
84.600 3.100
96.34%
10

---

Published in Transactions on Machine Learning Research (03/2026)
7 Limitations
XCTFormer explicitly models all pairwise channel-time dependencies via a unified attention block, improving expressiveness and delivering strong performance relative to well-established baselines. However, this design also introduces practical limitations. Flattening time channel tokens makes attention quadratic in the number of tokens, increasing memory and runtime as the lookback length and channel count grow. DeCoP mitigates this cost by compressing attention to linear form, but it still scales with sequence length and dimensionality and adds a decent parameter overhead. Finally, the gains are not uniform across datasets and tasks, with some settings showing smaller improvements modeling strategy is sensitive to the underlying dependency structure and may offer limited benefits when cross- cross-channel relations are weak or difficult to capture.
higher variance, suggesting that the presented pairwise
8 Conclusion
In this paper, we address fundamental paradox in multivariate time-series analysis: although leveraging cross-channel structure should improve performance, recent findings show that channel-indepenndet models often outperform channcl-dependens models. This counterintuitive result suggests that existing channel- dependent methods do not. fully exploit cross-channel information. We argue that this limitation arises from current approaches that model cross-channel and cross-time dependencies indirectly, thereby overlooking interactions. To bridge this gap, we introduce XCTFormer, which revisits channel dependence through direct, token-by-token modeling. Instead of treating channels and time steps as separate entities processed through multi-stage pipelines, XCTFormer treats each channel-time data point as an individual token and models all pairwise dependencies within a unified attention mechanism, which is important for capturing time-evolving dependencies. Through the Cross- Relational Attention Block (CRAB) with learnable masking (when DeCoP is not applied) and an enhanced attention activation function, XCTFormer improves expressivity while maintaining robustness, and the optional Dependency Compression Plugin (DeCoP) supports scalability on high-dinensional datasets. Extensive evaluation across forecasting, anomaly detection, and imputation highlights XCTFormer's effectiveness: it delivers state-of-the-art imputation accuracy, with average error reductions of 20.8% in MSE and 15.3% in MAE, while also achieving strong performance gains in forecasting and anomaly detection. Additional evaluation on a synthetic dataset, where the target depends on lagged cross-variate signals with distractor channels, suggests that XCTFormer effectively captures cross-channel and cross- cross-time dependencies while showing robustness to spurious correlations. At the same time, this direct modeling strategy introduces practical limitations: Unified loken-to-token attention scales quadratically with the number of time-channel tokens, and while DeCoP reduces this cost, the parameter count still grows lincarly and remains non-negligible. In addition, gains are not uniform across datasets and tasks, with some settings showing smaller improvements higher variance, suggesting sensitivity to the underlying dependency structure. This motivates further research into more robust channel-time modeling strategies that balance expressiveness, efficiency, and consistency across diverse datasets and tasks. Despite the limitations presented, our proposed direct modeling approach represents a substantial step toward a more comprchensive capture of dependencies and toward realizing the full modeling potential of multivariate time-series data.
Acknowledgments
This research was partially supported by the Lynn and William Frankel Center The Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, ISF grants 668/21 and 1299/25, an ISF equipment grant, and by the Israeli Council for Higher Education (CHE) via the Data Science Research Center, Gen-Gn University of the Negev, Israel.
11

---

Published in Transactions on Machine Learning Research (03/2026)
References
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical Approach to Asynchronous Multivariate Time Scries Anomaly Detection and Localization. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Dala Mining, 2021.
Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Türkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda-Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Sohlke-Schneider, and Beruie Wang. Chronos: Learning the language of time series. Truns. Mach. Learn. Res., 2024.
Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sun- dar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateck Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael Bohlke-Schneider. Chronos-2: From univariate to universal forecasting. CoRR, 2025.
Andreas Auer, Patrick Podest, Daniel Klotz, Sebastian Böck, Günter Klambauer, and Sepp Hochreiter. Tirex: Zero-shot forecasting across long and short horizons with enhanced in-context learning. CoRR, 2025.
George E. P. Box and Gwilym M. Jenkins. Time Series Analysis: Forecasting and Control. Holden-Daa first edition, 1970.
Gilbert Brunet, David B Parsons, Dimitar Ivanov, Boram Lee, Peter Bauer, Natacha B Bernier, Veronique Bouchet, Andy Brown, Antonio Busalacchi, Georgina Campbell Flatter, et al. Advancing weather and climate forecasting for our changing world. Bulletin of the American Metcorological Society, 2023.
C Bui, N Pham, A Vo, A Tran, A Nguyen, and Le. Time series forecasting for healthcare diagnosis and prognostics with the focus on cardiovascular discascs. In 6th International Conference on the Development of Biomedical Engineering in Vielnam (BME6). Springer, 2018
Xiaodan Chen, Xiucheng Li, Xinyang Chen, and Zhijun Li. Structured Matrix Basis for Multivariate Time Series Forecasting with Interpretable Dynamics. In Annual Conference on Neural Information Processing Systems, NeurIPS, 2024.
Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat, Sen, and Rose Yu. Long-term Forccasting with TiDE: Tine-series Dense Encoder. Transactions Machine Learning Research, 2023.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for timo-scrics forecasting. In Forty-forst International Conference on Machine Learning, ICML, 2024, 2024.
Pedro Domingos. A Few Useful Things to Know about Machine Learning. Communiccations of the ACM, 2012.
Diego Duarte, Chris Walshaw, and Nadarajah Ramesh. A comparison of time-series predictions for healthcare emergency department indicators and the impact of COVID-19. Applied Sciences, 2021.
Gal Fadlon, Idan Arbiv, Nimrod Berman, and Omri Azencot. A diffusion model for regular time scries generation from irregular data with completion and masking, In Advances in Neurol Information Processing Systems (NeurIPS) 89, 2025.
Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised Scalable Representation Learning for Multivariate Time Series. In Annual Conference Neural Information Processing Systems, NewrIPS, 2019.
Tal Gonen, Itai Pemper, Ilan Naiman, Nimrod Berman, and Omri Azencot. Time series generation under data scarcity: A unified generative modeling approach. In Advances in Neural Information Processing Systems (NeurIPS) 39, 2025.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Albert Gu, Karan Goel, and Christopher Ré, Efficiently Modeling Long Sequences with Structured State Spaces. In International Conference. on Learning Representations (ICLR), 2022.
Lu Han, Han-Jia Ye, and De-Chuan Zhan. The Capacity and Robustness Trade-Off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting. IEEE Trans. Knowl. Data Eng., 36(11), 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving Deep into Rectifiers: Surpassing Human- Level Performance on ImageNet Classification. In IEEE International Conference on Computer Vision, ICCV, 2015.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Newral compulation, 1997.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Söderström. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2018. Datasets: SMAP and MSL.
Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, and Sanmi Koyejo. Scaling Laws for Downstream Task Performance in Machine Translation. In The Thirleenth International Conference on Learning Representations, ICLR, 2025.
Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey I. Webb, Irwin King, and Shirui Pan. A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection. IEEE TPAMI, 2024.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Ho Choi, and Jaegul Choo. Reversible Instance Normalization for Accurate Time-Series Forecasting against. Distribution Shift. In The Tenth International Conference on Learning Representations, ICLR, 2022.
Diederik P. Kingma and Jimmy Ba. Adam: Method for Stochastic Optimization. In 3rd International Conference on Learning Represenlations, ICLR, 2015.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The Efficient Transformer. In International Conference on Learning Representations (ICLR), 2020.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting, In Advances in Neural Information Processing Systems (NeurIPS), 2019a.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting. In Annual Conference on Newral Information Processing Systems, NewrIPS, 2019b.
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction. In Annual Conference on Neural Information Processing Systems, NewrIPS, 2022a.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Low-Complent Pyramidal Attention for Long- Range Time Series Modeling and Forecasting. In The Tenth International Conference on Learning Represenlations, ICLR, 2022b.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting, In Advances in Neural Information Processing Systems (NewrIPS), 2022c.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. Transformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, ICLR, 2024.
13

---

Published in Transactions on Machine Learning Research (03/2026)
Ang Lv, Ruobing Xie, Shuaipeng Li, Jiayi Liao, Xingwu Sun, Zhanhui Kang, Di Wang, and Rui Yan. More Expressive Attention with Negative Weights. arxie preprinl arXiv:2411.07116 2024.
Aditya P. Mathur and Nils Ole Tippenhauer. SWaT: A Water Treatment Testbed for Research and Training TCS Security. In Proceedings of the International Workshop on Cyber-Physsiee Systems for Smart Water Networks. IEEE, 2016.
Aristeidis Mystakidis, Paraskevas Koukaras, Nikolaos Tsalikidis, Dimosthenis Ioannidis, and Christos Tjortjis. Energy Forecasting: A Comprehensive Review of Techniques and Technologies. Energies, 2024.
Ilan Naiman, Nimrod Berman, Itai Pemper, Idan Arbiv, Gal Fadlon, and Omri Azencot. Utilizing image transforms and diffusion models for generative modeling of short and long time series. Advances in Neural Information Processing Systems, 37:121699-1211 121730, 2024a.
Ilan Naiman, N. Benjamin Erichson, Pu Ren, Michael W. Mahoney, and Omri Azencot. Generative modeling regular and irregular time series data via Koopman VAEs. In The Twelfth International Conference on Learning Representations, 2024b.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A Time Series is Worth 64 Words: Long-term Forccasting with Transformers. In The Eleventh International Conference on Learning Representations, ICLR, 2023.
Liran Nochumsohn and Omri Azencot. Data augmentation policy search for long-term forecasting. Trans. Mach. Learn. Res., 2025, 2025.
Liran Nochumsohn, Michal Moshkovitz, Orly Avner, Dotan Di Castro, and Omri Azencot. Beyond data 11. scarcity: A frequency-driven framework for zero-shot forecasting. arXiv preprint 11.:2441.157/44
2024.
Liran Nochumsohn, Raz Marshanski, Hedi Zisling, and Omri Azencot. Super-linear: lightweight pretrained mixture of linear experts for time series forecasting. arXiv preprint arXiv:2509.15005 2025a.
Liran Nochumsohn, Hedi Zisling, and Omri Azencot. A Multi-Task Learning Approach to Linear Multivariate Forccasting. In International Conference on Artificial Intelligence and Statistics, ATSTATS, 2025b.
Boris N Oreshkin, Mayank Jauhari, Ravi Kiran Selvam, Malcolm Wolff, Wenhao Pan, Shankar Ramasub- ramanian, Kin G Olivares, Tatiana Konstantinova, Andres Potapezynski, Mengfei Cao, et al. Zero-shot forecasting by simulation alone. arXiv preprinl Xiv:22601 00970, 2026.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, Higgh-Peffrrmance Deep Learning Library. In Annual Conference on Neural Information Processing Systems, NewrIPS, 2019.
Gerald F. Reed, Frances Lynn, and Brian D. Meade. Use of the coefficient of variation in assessing variability in quantitative assays. Clinical and Diagnostic Laboratory Immunology, 2002.
Facebook Research. fvcore: FAIR computer vision library. http::///tthhn.ccc//ccceeeeerrrr 2021.
Hemanth Saratchandran, Jianqiao Zheng, Yiping Ji, Wenbo Zhang, and Simon Lucey. Rethinking Attention: Polynomial Alternatives to Softmax in Transformers, 2025.
Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-MoE: Billion-scale time series foundation models with mixture of experts. In The Thirteenth International Conference on Learning Representations, ICLR 2025, 2025.
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Moving, 2019.
14

---

Published in Transactions on Machine Learning Research (03/2026)
Patara Trirat, Yooju Shin, Junhyeok Kang, Youngeun Nam, Jihye Na, Minyoung Bae, Joeun Kim, Byunghyun Kim, and Jae-Gil Lee. Universal Time-Series Representation Learning: A Survey, arXin, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Annual Conference on Neural Information Processing Systems, NeurIPS, 2017.
Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. MICN: Multi-scale Local and Global Context Modeling for Long-tern Series Forecasting. In The Eleventh International Conference on Learning Representations, ICLR, 2023.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and Jun Zhou. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. Tn The Twelfth International Conference Learning Representations, ICLR, 2024a.
Shiyu Wang, Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. +: A General Time Series Pattern Machine for Universal Predictive Analysis. In The Thirteenth International Conference on Learning Representations, ICLR, 2025.
Xinlin Wang, Hao Wang, Binayak Bhandari, and Leming Cheng. Al-Empowered Methods for Smart Energy Consumption: A Review of Load Forecasting, Anomaly Detection and Demand Response. International Journal of Precision Engineering and Monufacturing Green Technology, 2024b.
Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. CARD: Channel Aligned Robust Blend Transformer for Time Scries Forccasting. In The Twelfth International Conference on Learning Represenlations, ICLR, 2024c.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. ETSformer: Exponential Smoothing Transformers for Timo-scries Forecasting. arXiv preprint ar arXiv:2202.0133 2022.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Forty-first International Conference on Machine Learning, 1CML 2024, 2024.
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, In Annual Conference on Neural Information Processing Systems 2021, NewrIPS, 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. TimesNet: Temporal 2D- 2D-Variation Modeling for General Time Scries Analysis. In The Eleventh International Conference on Learning Representations, ICLR, 2023.
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In International Conference Learning Representations (ICLR), 2022.
Guoqi Yu, Jing Zou, Xiaowei Hu, Angelica I. Avilés-Rivero, Jing Qin, and Shujun Wang. Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling. In Forty-first International Conference on Machine Learning, 1CML, 2024.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Thirty-Seventh Conference on Artificial Intelligence. AAAT, 2023.
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less Is More: Fast Multivariate Time Series Forecasting with Light campling-orienter MLP Structures. arXiv preprint 0001022207.01186, 2022.
15

---

Published in Transactions on Machine Learning Research (03/2026)
Yunhao Zhang and Junchi Yan. Crossformer: Transformer Utilizing Cross- Dissessonn Dependency for Multi- variate Time Series Forecasting. In The Fleventh International Conference on Learning Represenlations, ICLR, 2023.
Lifan Zhao and Yanyan Shen. Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators. In The Twelfth International Conference Learning Representations, ICLR, 2024.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In Chirly-Fifhh Conference Artificial Intelligence. AAAI, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency Enhancod Decomposed Transformer for Long-term Scries Forccasting. In International Conference on Machine Learning, ICML, 2022.
16

---

Published in Transactions on Machine Learning Research (03/2026)
A Appendix: Extended Notes on XCTFormer
A.1 Theoretical validity of the proposed activation function
Proof. We show that the proposed AbsAct activation function satisfies the sufficient stability criterion of Saratchandran et al. (2025), namely that the Frobenius norm the produced matrix is bounded by the square root of the number of rows:
ActivattonnA)]le & VN
Given a matrix A = (aiz) RNNN and using the activation defined in Section
we have:
a1
iin
a1
ain
Dj=i/a13 a1yl
2j=1 j=1 ajj
AbsAct
(2)
Oml
Onn
an1
E==1(an) [nn
2==1 [nny
Note. In practice, add small positive offsct and denominator stabilizer to prevent division by zero. We set dej- dij 10-4 elesent-wise, then normalize each row by |0,1++ i.e, use dz:/(Zn ///t//illle | + 10--)..
These constants (10-4 and 10- are included only for numerical stability and are omitted from the proof for simplicity.
By definition of the Frobenius Norm:
N
AbsAct(A)][
Oij
(3)
i=1
/=1
k=1
Aik
N
(4)
aask
k=
N
E |0is]
(since
5 (zy1)?)
(5)
|ask
N Σ
(6)
Hence, ||AbsActAA]lp S VN.
Note. The same bound holds when DeCoP is applied: the matrix still has N rows, and after rowwise & normalization each row's Frobenius norm strictly less than one. Consequently, the sum of the Frobenius norm across all rows does not exceed N.
A.2 Scaling invariance of AbsAct
In standard attention, the score matrix is scaled by 1/Vdz before applying softmax keep dot-product magnitudes coltrolled and avoid overly sharp attention distributions. Under AbsAct, this scaling is unnecessary. Let a > 0 be any positive scalar (e.g. 1/vdk). For any row i with ΣN AAal > O, we have for every entry (i, j):
aAi Am |aAikl
aAi
Aij AbsAct(AA);
AbsAct(aA)ig
(7)
a Ei
A
>
A
Akml
17

---

Published in Transactions on Machine Learning Research (03/2026)
Hence, AbsAct is homogencous of degree zero for positive scalars:
AbsAct(aAA) = AbsAct(A), Va 0,
whenever the row-wise denominator nonzero. Because the 1/vd; factor is positive scalar applied uniformly to every entry, it cancels exactly and can be omitted without affecting the output.
A.3 DeCoP: Complexity Analysis
We show that DeCoP reduces the attention cost from quadratic in the sequence length N linear (for fixed k < N), thereby lowering both compute and memory costs.
Setup.
Let X E NND. and Wg. Wa E RDIXD= Define
Q XW, R NxD.. == XW R.N
DeCoP introduces learnable compression matrix CERNXA with & & N.
DeCoP computation (no N x N intermediate).
S = K
A QS RN
When DeCoP is applicd, the learnable non-boolean mask from CRAB (Sec. 4.2 is not used. Since the compressed altention A. E Ng* does not retain the direct pairwise structure of the full N x N attention matrix, element-wise masking of individual token-to-token relationships is no longer applicable.
Total complexity (after).
Total = O(
(form Q)
+.(ND,D..)
(form K)
+O(NDm.)
(S K KTC)
+O(ND
(A QS)
= -( + 22mkk) = O(2ND, + 2N
(D.=Dm)
O(N(D², Dmk))
Growth in N. For fixed D,, Dm, & with &o < N, the cost is linear in N.
Memory, Store K: O(NDm); A and A4: O(Nk); temporary S: O(m parameters C: O(Nk), M: O(Nk). No N N matrix materialized.
18

---

Published in Transactions on Machine Learning Research (03/2026)
B Appendix: Implementation Details
B.1
Time-Series Tasks Formulation and Metrics
This section presents the mathematical formulation and evaluation metries for three fundamental time-series tasks: long-tem term forecasting, anomaly detection, and imputation. Each task addresses distinct challenges in temporal data analysis while sharing common architectural foundations.
B.1.1 Long-Tere Forecasting
Long-term forocasting aims to predict future values multivariate time-scrics given historical observations. Given a historical sequence X E L' where is the lookback window length and C' is the number of channels, the objective is predict the future sequence Y R where H the prediction horizon.
Training Objective: The model minimizes the Mean Squared Error (MSE) loss:
N
L forecast
Y. Y,11
(8)
where N is the number of training samples, Y; is the ground truth, and Y, is the predicted sequence.
Evaluation Metrics: Performance is assessed using multiple regression metrics:
Mean Absolute Error (MAE): MAE NH ΣN E# Yi, Y.dl
H Mean Squared Error (MSE): MSE NH Σ EE (Y Y2,)²
B.1.2 Imputation
Time-series imputation reconstructs missing values in partially observed sequences. Let X E RLxC be the ground-truth sequence and M E 1) binary mask where Mr. 0 marks a missing entry; the observod input obtained via an dement-wise product X X M.
Problem Formulation. An imputation model with parameters và reconstruets the complete sequence:
Imputer, (X M).
Training Objective. During training, artificial masks are sampled with mask rate p. The reconstruction loss is computed only on masked positions:
Lmpute
Σ (t.c)em
(X,a &,.
m ((t,c) Mc. = 0}.
(10)
m
Evaluation Protocol.
1. Apply random mask (rate p) to test sequences to obtain Xobs and M.
2. Impute the missing entries: filed = xobs @ M X (1-
3. Compute metrics ezclusively on the masked set M.
Evaluation Metrics (masked-only). All metrics are calculated only on masked values (t, E m:
MlAEmask
1
Σ
X1,c
&r,cl
(11)
mm
(t.c)em
MSF Faakk
1
(Xe,
xe,02
(12)
m
(t.cem
19

---

Published in Transactions on Machine Learning Research (03/2026)
B.1.3 Anomaly Detection
Anomaly detection identifies unusual patterns or outliers in time-serics data using a reconstruction-based approach. The model learns to reconstruct normal patterns and flags samples with high reconstruction errors as anomalous.
Problem Formulation: Given time-series X E RR.x. the model Jø learns to reconstruct the imput:
x
(13)
The anomaly score computed as the reconstruction error: s = X 1
Training Objective: The model trained exclusively on normal data using reconstruction loss:
N X; Do (
1
Lrecon
(14)
N
Detection Mechanism: We use a percentile-based threshold over the pooled anomaly-score distribution. Let = [Strain] J [Stest]. For a target anomaly rate a, the threshold is
quantile(S, 1- l-a).
In our experiments we set 0.01 (1%) for all datasets, except SMD Suu al. 2019 where a 0.005 (0.5%). A sample with score is labeled as
[1, if > v (anomaly),
label
0, 8 S r (normal).
Evaluation Metrics: Performance is measured using binary classification metries:
Precision: P
TPFFP (proportion of correctly identified anomalics)
TP
Recall: TP+FN (proportion of actual anomalies detected)
F1-Score: Fi 2 PxR (harmonic mean of precision and recall) P4R
Accuracy: Acc = TP4TN (overall classification correctness)
TP4TN
where TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively.
B.2 Experiment Datasets And Evaluation Setups
We evaluate long-tere term forecasting on seven widely used multivariate datasets (Weather, Electricity, Traffic, ETTh1, ETTh2, ETTml, ETTm2).. For forecasting, we follow the TimesNet (Wu et al., 2023) setup with look-back window L 96 and horizons // (96, 192, 336, 720}; dataset specifications appear in Table For time-series imputation,
the same datasets forecasting, except for Traffic, and follow the + [Wang et al.) 2025) setup with L 1024 and masking ratios E {12.5% 25% 37.5%, 50% ) refer to Table 7 Anomaly detection focuses on identifying finc- aie-ggraindd patterns. To assess this, we selected the following datasels: SMD (Server Machine Dataset, (Su et al.) 2019)), SWaT (Secure Water Treatment, Mathur Tippenhauer 2016)), PSM (Pooled Server Metrics, (Abdulaal et al. 2021)), and NASA telemetry datascts MSL. and SMAP (Hundman et al., 2018). The details the datasets used for anomaly detection are provided in Table g For long-tere forecasting. we additionally evaluate on synthetic dataset where the target is derived from lagged cross-channel signals; full construction details appear in Appendix D
20

---

Published in Transactions on Machine Learning Research (03/2026)
Table 6: Benchmark datasets and evaluation settings for long-term forecasting.
Dataset
Dim
Look-back
Prediction Horizons
Dataset Size
Frequency
Information
ETTml
7
(96, 192, 336, 720)
(34465, 11521, 11521)
15 min
Temperature
ETTm2
7
96
(96, 192, 336, 720)
(34465, 11521, 11521)
15 min
Temperature
ETTh1
7
96
{96, 192, 336, 720)
(8545, 2881, 2881)
15 min
Temperature
ETTh2
7
96
{96, 192, 336, 720)
(8545, 2881, 2881)
15 min
Temperature
Weather
21
96
{96, 192, 336, 720)
(36792, 5271, 10540)
10 min Hourly
Weather
ECL
321
96
(96, 192, 336, 720)
(18317, 2633, 5261)
Electricity
Traffic
862
96
(96, 192, 336, 720)
(12185, 1757, 3509)
Hourly
Transportation
Table 7: Benchmark datasets and evaluation settings for time-series imputation.
Dataset
Dim
Look-back
Imputation Mask Ratios
Dataset Size
Frequency
Information
ETTml
7
1024
[1.5%% 25%, 37.5%, 50%]
(34465, 11521, 11521)
15 min
Temperature
ETTm2
7
1024
[1..5% 25%, 37.5%, 50%]
(34465, 11521, 11521)
15 min
Temperature
ETTh1
7
1024
[12.5%, 25%, 37.5%, 50%]
(8545, 2881, 2881)
15 min
Temperature
ETTh2
7
1024
11.55% 25%, 37.5%, 50%]
(8545, 2881, 2881)
15 min
Temperature
Weather
21
1024
[12.5%, 25%, 37.5%, 50%]
(36792, 5271, 10540)
10 min
Weather Electricity
ECL
321
1024
[12.5%, 25%, 37.5%, 50%]
(18317, 2633, 5261)
Hourly
Table 8: Dataset detailed descriptions for anomaly detection. Validation, Test).
The dataset size is organized in (Train,
Dataset
Dim
Series Length
Dataset Size
Information
SMD
38
100
(566724, 141681, 708420)
Server machines
MSL
55
100
(44653, 11664, 73729)
Spacecraft telemetry (Mars)
SMAP
25
100
(108146, 27037, 427617)
Spacecraft telemetry
SWaT
51
100
(396000, 99000, 449919)
Water treatment ICS
PSM
25
100
(105984, 26497, 87841)
Server metrics
21

---

Published in Transactions on Machine Learning Research (03/2026)
B.3 Training Details.
implemented in PyTorch (Paszke et. al. 2019 and NVIDIA RTX 3090 GPUs. We fix the random seed to 2021. We applied RevIN transformation Kim et al. 2022 to mitigate distributional shifts in the data. Models are trained with the Adam optimizer [Kingma & Ba 2015 using mean squared error (MSE) loss, logether with a OneCycleLR scheduler. At high level, OneCycleLR first. increases the learning rate from its initial value to peak (while inversely adjusting momentum), and then gradually anneals it to small value for the remainder training; this promotes fast early progress and stable late-stage convergence. We set pct_start 0.4, allocating 40% of the total training steps to the warm-up//neress phase and 60% to the annealing phase. For each run, we select the checkpoint with the lowest validation MSE and report the corresponding test performance in the tables.
All experiments
Hyper-Paramcee Search. We tuned hyperparamcters with Optuna and chose the configuration yiclding the lowest validation MSE. The selected configuration is tuned per dataset and task, and then kept fixed across all forecasting horizons imputation mask ratios, reported in Table
Table 9: Hyperparameter settings for XCTFormer per dataset per time-scrics task
Processing
Transformer
Training
Dataset
patch
lom
stride
layers
heads
d model d If dropont To dropout
altn dropout
batch size
learning rate
opochs
Long-term time-series Forecasting
KTThi
16
16
0.2
0.3
0.6
32
0.001
10
ETT12
16
0.1
0.2
0.8
52
0.01
FFTTml
52
ETTm2
16
1
224
438
0.1
0.05
0.8
32
0.005
Weather
16
8
3
2
248
496
01.3
0.05
0.8
382
0U00S
Thaffic
192
0.001
ECL
16
0.1
0.05
64
32
0.066
Imputation
ETTTh2
0.05
32
0.006
ETTmi
16
96
192
0.1
0.05
0..
322
0.005
10
ETT
0.05
0.001
Weather
0.05
32
0.001
ECL
64
92
2
2
192
384
0.1
0.05
0.7
128
32
0.008
10
Anomally Detection
MSSL
16
256
512
0.1
0.005
0.7
123
0.01
10
PSM
16
256
612
(1.)
0.05
01.8
128
0.001
SMAP
0.05
128
0.006
0.05
128
SWAT
16
216
452
0.]
0.05
0.4
128
0.0066
10
Hyper-Parameter Search Space. We searched over bounded hyperparameter space per dataset and task, while fixing few coupled settings to reduce degrees of freedom (we set stride patch len/2 and d ff = 2 x d model). For ETTh1 /ETTT2 in forecasting, where observed stronger overfitting, narrowed the d model range and explicitly tuned dropout to improve generalization. The search area is represented in Table 10
Table 10: Hyperparameter Search Space
dropout
beads
Jayers
model
patch Jen dropout
dropout
ETTb1, ETT22
6d
0.110.33a.
[.05, 10.05, 0.3)
Long-teru termm Forecast
Others
(5e-4, lo4. Se-3, 1e
(01.0.8a
(1224)
[..]
8. 258|s
664 250
Impotation
322 256 (16,61,125)
256
Anomaly Detection
(5,66
/alggr//llotttt Notation: (a, ble denotes integer/ /flat range from with step (...) denotes categorical choicos; indicates parameter not used. ik was searched only for ECL and Traffic datasets.
B.4 Baseline Implementation Details
For baselines evaluated under the same experimental sotting as our main study, we directly used the reported results from the + Wang et al. (2025) paper when available, and otherwise reported from the corresponding original papers.
22

---

Published in Transactions on Machine Learning Research (03/2026)
For the synthetic dataset, all compared models were trained using the ETTml hyperparameters. Each model was run using the code and hyperparameters from its official repository where available; otherwise, the implementation was taken from https://githu: httpaa/.ooootttt//eeeeeeeeeeeear
B.5 Technical Evaluation Note
We compute final metrics for imputation and forecasting weighted averages across batches to account for varying batch sizes during evaluation. This adjustment is necessary because the last batch in an epoch may contain fewer samples than the standard batch size. When computing performance metrics by simply averaging across batches without considering batch sizes, smaller batches receive disproportionate weight in the final metric calculation, leading to biased performance estimates that do not accurately reflect true model performance across the entire dataset. In many older works, researchers addressed this problem by setting the drop_last last=TTrue parameter in PyTorch's DataLoader, which discards the final incomplete batch to ensure identical batch sizes. However, this approach wastes data and can be particularly problematic for smaller datasets, where discarding samples reduces available training or evaluation data. In recent works, it is more common to solve this problem by setting drop_ last_aaataeeeee and computing weighted averages, where each batch's metric contribution is weighted by its actual size, cnsuring that the final averaged metric accurately represents performance across samples the datasel without discarding any data.
B.6 Computational Profiling
To assess the computational cost each model, we measure both floating-pon point operations (FLOPs) and trainable parameter counts across all datasets. Full per-dataset results are reported in Tables 27 and 28
FLOPs. For all real-world datasets, FLOPs are computed using fvcore's FlopCountAnalysis Research 2021),, which performs symbolic forward pass to trace the model's computation graph and counts multiply- accumulate operations (MACs) across all layers, including custom modules.
Trainable parameters. Parameter counts are obtained via PyTorch's native enumeration: sum (p. numel () for in model. parameters() if pequirres_rad]) ensuring all learnable weights are captured regard- less of layer lype.
Excluded models. TimeMixer+ is excluded because its source code is not publicly available. Crossformer values on ETT and ET are unavailable (N/A) due to configuration incompatibility with those datasets.
23

---

Published in Transactions on Machine Learning Research (03/2026)
C
Appendix: Extended Analysis
C.1
Interpretable Learned Masks Analysis.
This analysis applics to configurations without DeCoP, where the learnable mask is active (see Soc. 4.3
CRAB (Sec. 4.2 introduces a learnable non-boolean mask that learns the most dominant cross-channel and temporal dependencies. The mask learns dominant dependencies by directly modulating the strength of attention valuos during training. Specifically, the mask multiplies attention weights clement-wisc, with higher absolute values amplifying the corresponding altention relationships and values near zero suppressing them. Through mradient-based optimization, the mask automatically discovers which cross-channel and temporal interactions are most informative for the downstream task, effectively learning a data-driven weighting scheme that prioritizes the most predictive dependencies.
Examining the learned mask can therefore provide data-specific insights about these dependency structures, as the mask values directly reflect relationship dominance, with higher absolute values indicating stronger learned dependencies. In this section, we explain how to interpret the attention mask as a foundation for further analysis so can be leveraged for different analytical needs.
Following data processing (Sec. 4.1 the input data is first permuted so that the patch-sequence dimension is placed before the channel dimension, and then the sequence and channel dimensions are flattened. This creates attention mask structure that can be visualized as grid of squares where each square represents cross-channel relationships between pairs of time steps (refer to Figure 3 for visual representation), with the main diagonal squares capluring cross-channel interactions within the same time step and off-diagonal squares revealing temporal cross-channel dependencies.
T * C x
x
T *
t2
43
t4
L1 C0
C4
C1 C2 C3
02 C3
C1
c3
C4
t
C3
CxT
TxC
C4
61
C2
t2
03
Permute
Flatten
Final Attention
C4
01
t3
C2
C4
C1
t4
03
C4
Figure 3: Interpretable Learned Mask Structure: The data permutation step places the patch sequence dimension first, creating an attention mask that can be visualized as grid of squares where cach square represents cross-channel relationships between pairs of time-steps. Note: batch and data dimensions are excluded from this diagram for clarity.
Analysis of Learnable Masks on ETTml Dataset Figure analyzes learnable attention masks trained on the ETTml dataset across two forecasting scenarios: 96-996 (top row) and 96->192 (bottom row). Each displays three visualizations: initial random masks initiated from normal distribution (left), learned
24

---

Published in Transactions on Machine Learning Research (03/2026)
patterns after training (middle), and corresponding heatmaps quantifying cross-channel dependency strength (right).
Data Processing and Architecture. The ETTml dataset was processed using patches of length 16 with a stride of 8, generating 12 patches across ETTml's channels. This configuration produces an 84 x 84 attention matrix (12 84 dimensions) that captures both temporal and cross-channel relationships.
Heatmap Interpretation. The dependency strength heatmaps are derived from the trained masks by averaging the absolute values within each cross-channel grid. Since masks are applied to attention weights, higher absolute mask values correspond to more dominant dependencies, with darker red regions in the heatmap indicating stronger cross-channel relationships between specific time steps.
Key Findings. The trained masks exhibit several notable patterns. First, they develop structured grid formations that align precisely with the 12-patch architecture, suggesting the model learns systematic cross- channel dependencics. By examining the heatmaps from both configurations, we observe a high density of dominant dependencies along the main diagonal. This diagonal concentration indicates that the model learns strong self-attention patterns, in which each time step primarily attends to itself and its immediate temporal neighbors. Such patterns suggest that the most informative relationships for forccasting are local temporal dependencies, in which recent observations carry the greatest predictive power for future values. This finding aligns with the intuitive understanding that in time-series analysis, temporally proximate data points are typically more relevant than distant historical information.
ETTm1 After Train
Cross-Channel Average
Before Train
Figure 4: Analysis of learnable attention masks on ETTml dataset. Top row: 96-996 forecasting; bottom
row: 96-+192 forecasting. Left colum:: initial random masks; middle colum:: learned structured patterns after training; right column: heatmaps of cross-channel dependency strength derived from trained masks. The heatmaps visualize the strength of cross-channel dependencics between time points, with darker red regions indicating stronger relationships.
25

---

Published in Transactions on Machine Learning Research (03/2026)
long term forecasting traffic
l <erm_forecastinn traffic
0.4500
Test Validation
0.296
0.400
0.4475
0.294
0.398
Validation MSE
0.4450
Test MSE
Test MAE
0.292
0.4425
0.396
0.4400
0.394
0.290
0.4375
0.392
0.288
0.4350
0.390
0.286
100
200
k
Figure 5: DeCoP k scnsitivity on Trafic (forccasting). {96, 192, 336, 720} for different compressed representation sizes k.
Average MAE/MSE across horizons
C.2 DeCoP compression size analysis
As recalled, DeCoP compresses loken-to-loken interactions into a low-dimensional representation; the choice of & directly controls the expressive capacity this bottleneck and thus can affect both accuracy and efficiency. In particular, larger & increases the dimensionality of the compressed attention embedding, which summarizes each token's 's pairwise interactions with all other tokens, enabling richer dependency modeling. However, this comes higher compute and memory cost. Therefore, analyzing sensitivity to k is important for providing practical guidance on choosing & that accounts for resource constraints.
Concretely, we tested & E [2, 32] (step 2) and & E [64, 256] (step 32). For each k, we report the average MAE/MSE, across forecasting horizons 96, 192, 336, 720 and the average MAF/MSE across imputation mask ratios O. 1252, 0.25, 0.375, 0.5. Overall, we observe dataset-dependent behavior. On Traffic (forecasting analysis Table 5), performance tends to improve as increases, indicating that a higher-capacity compressod representation better captures the complex multivariate dependencies in this dataset. In contrast, on ECL (forecasting analysis Table imputation analysis Table 7. smaller or mid-range k values often competitive and occasionally slightly better, suggesting that stronger compression can provide useful regularizing effect and that further increasing & yields limited additional benefit. Based on these results, we recommend treating k as a speccfi hyperparameter and tuning it to balance accuracy with compute and memory costs, as larger & values often yield only marginal gains.
C.3 Patch length and stride analysis.
Patching determines how the input sequence divided into fixed-length windows, where patch len sets the window size, and each window is projected into token embedding. The stride controls the overlap between consecutive windows and thus the density with which the sequence is covered. Together, they affect which information is represented in each token and how many tokens the transformer processes. With larger patch_ len stride, the model uses fewer tokens, but each token must represent longer window, which can make harder to preserve meaningful temporal information. With a smaller patch_less _en or stride, tokens represent more local information and overlap increases, but the longer token sequence raises the computational cost of attention. Therefore, patch len and stride define fundamental accuracy efficiency tradc-off. This sensitivity analysis evaluates how robust our method to this configuration and if dataset-specific tuning may be required.
26

---

Published in Transactions on Machine Learning Research (03/2026)
Long_term term_forecassingg Electricity
long_ term tttptteemfffffaee forecasting Electricity
Test Validdttion
0.22
0.20
0.30
Validation MSE
Test MSE
0.20
0.18
Test MAE
0.29
0.19
00.17
0.18
0.16
0.27
0.17
0.15
0.26
0.16
0.14
50
100
150
200
250
k
k
Figure 6: DeCoP k sensitivity on ECL (forecasting). Average MAE/ MSE across horizons
[96, 192, 336, 720} for diffcrent compressed representation sizes
imputation Electricity
imputation Electricity
Test
0.085
0.080
0.20
+0,080
0.075
Validation MSE
Test MSE
Test MAE
0.18
0.070
0.065
0.060
0.16
0.055
0.050
+0.050
0.14
0.045
0.045
100
200
k
Figure 7: DeCoP sensitivity on ECL (imputation). Average MAE/MSI across mask ratios {0. 125, 0.25, 0.375, 0.5) for different compressed representation sizes k.
27

---

Published in Transactions on Machine Learning Research (03/2026)
long term forecasting ETTm1
long long_term_ forecasting ETTm1
0.382
Test Validition
-0.620
0.380
0.615
Validation MSE
0.610
Test MSE
Test MAE
0.376
0.605
0.600
0.392
0.372
0.595
0.390
0.368
Patch Length
Patch Length
Figure 8: Patch length sensitivity on ETTm1 (forecasting.. Average validation and test losses across horizons {96, 192, 336, 720} for different patch_las len values with stride = patch_len/2
Following PatchTST's (Nie et al. 2023) best practice, we kept the patching configuration fixed in most of our main experiments. For forecasting and anomaly detection, we used patch_len- _ln___nnn___ 16 and stride= 8. For imputation, where the lookback was substantially larger (L=1024 versus 96 in forecasting and 100 in anomaly detection), we tested small set of patch len values 16, 64, 128 with the corresponding stride= patch_len) Based on validation error during hyperparameter search, we selected patch_ len= 64 and stride= 32 for some imputation datasets.
In this subsection, we conducted patching sensitivity experiment for forecasting by fixing stride = patch_lend _en/2 and testing different values of patch_ len from to 96 in increments of The plots report the average validation and lest. losses the ETTml (Figure 8) and Weather (Figure 9 datasets, where the average is computed across horizons {96, 192, 336, 720}. Overall, performance varies only slightly across different sctups, indicating that our method is robust to this hyperparameter. While the effect is small on ETTml, it is more noticeable on Weather, yet the differences remain limited to roughly 2-4%, suggesting that the preferred patching configuration can be dataset-dependent without requiring highly precise tuning.
C.4 digned-attention mask analysis.
Distribution of negative weights. As recalled from the Method (Sec. 4.2) we do not use the vanilla attention scores directly. Instead, we first apply a positive transformation and then adjust the scores with a learnable, non-Boolean allention mask, which controls both the sign and the magnilude of the resulting attention weights. Concretely, given an attention score matrix NNN, we remove sign information via global shift A4 A min(A), which ensures A, 2 clement-wise and then form the signed attention as M o A+, where M N is a learnable real-valued mask applied element- wise. Since A4 is non-negaiive, the sign of each entry in A is determined entirely by the corresponding mask value: My 0 produces negative attention weight, while Mij > 0 produces a positive one. We initialize M from a zero-mean Gaussian distribution, so roughly half of its entries are negative at initialization, and consequently, about half of the resulting attention weights are negative as well. Although M fine-tuned during training, its distribution remains close to normal, so negative weights persist and can contribute throughout optimization. Take a look at Figure 10 which shows the distributions of the mask values and the activated attention weights the end of the first and last training epochs.
Ablation: clipping negative weights. We further isolate the role of negative weights with targeted ablation. We keep our signed-attention activation unchanged, and only add final ReLU that clips all negative
28

---

Published in Transactions on Machine Learning Research (03/2026)
long term forecasting weather
long term forecasting weather
Test Validation
0271
Validation MSB
Test MSE
Test MAE
0.239
0.508
0.269
0.238
0.50G
-0.504
0.267
0.237
0266
Patch Length
Patch Length
Figure 9: Patch length sensitivity Weather (forecasting). Average validation and test losses across horizons {96, 192, 336, }20} for different patch_las values with stride = aatch_len//e
XCTFormer ETTm1_90 96 192 Epoch Attention Distributions
XCTFormer ETTm1 96 192 Epoch 10 Mask & Attention Distributions
Mask: Layer 1
Mask: Layer 2 [-0,708, 0.696)
Mask: Layer 1 [-1 1.583)
Mask: Layer 2 [-1.574. 1.448)
[-0.616, 0.636)
Hean-0.021 0.0335
Heann
1.50
0.75
Donisoo
0.5
0.2
Attention: Layer [-0 062, 0.080)
Attention: Layer 2 [00101], 0,069]
Attention: Layer
Attention: Layer 2 [-0.075, 0,102]
[-0.074, 0.100)
0,0300
Mcar-50002
Maan-000000
Mcan-0
Dursity
20
bxrrity
10
Value
Value
Velue
Villuw
Figure 10: Distributions of mask and signed-attention weights. Histograms of the learnable mask values M and the resulting activated attention weights. Results are shown for the forecasting task upon the ETTml dataset with lookback T.-96 and horizon #=192. The left panel shows the distributions after the first training epoch, and the right panel after the final (10th) epoch. The distributions remain approximately Gaussian over training, indicating that negative weights persist.
values to This variant is trained with the exact same hyperparameters and experimental settings, covering all forecasting horizons (96, 192, 336, 720} and all imputation mask ratios (0.125, 0.25, 0.375, 0.5). We then report performance averaged across these settings (see Table [1.. Overall, the original model that permits negative weights improves performance by about 1.3% over the clipped variant, suggesting that negative weights provide a consistent but modest performance gain.
C.5 Empirical Scalability Analysis with Respect to Input Dimension
As noted in the Limitations, XCTFormer sensitive to input size because explicitly models all pairwise time and channel relationships, which induces quadratic scaling. We design DeCoP to mitigate this cost. by
29

---

Published in Transactions on Machine Learning Research (03/2026)
Table 11: Ablation study results across different tasks, evaluated with different XCTFormer with clipped values.
XCTFormer
Long-term Forecasting
Imputation
Anomaly Detection
vs Others
MSE
MAE
MSE
MAE
Precision
Recall
F-Score
(%)
XCTFormer (Original)
0.328
0.337
0.044
0.124
92.1
83.7
87.6
AbsAct ReLU
0.331
0.339
0.044
0.124
92.1
79.5
85.2
1.3%
Parameters Data Dimension Size (Linear Scale)
12M
XCTFormer
XCTFormer (with k=64)
10M
Parameters
8M
6M
4M
2M
100
200
300
400
500
600
700
800
Data Dimension Size (Number of Features)
Figure 11: Scalability with respect to channel dimensionnlity (n fealures). XCTFormer's parameter count. grows quadratically, while the DeCoP variant scales approximately linearly.
compressing per-token dependencies and thereby achieving approximately linear scaling. To verify these theoretical expectations, we conduct empirical scalability analysis. Because the effective input equals the product of the sequence tokens (after patching) and the mumber of channels, vary each factor independently and measure how the paramcter amount changes. Table 12 represents the hyperparamcter used.
Table 12: Hyperparameter settings for XCTFormer scalability analysis
Trasoornce
XCTForeer
Variable
Experiment
parch stoide
layers
dropout dropout
dropost
Ronge
Dimumiio Staling Amalais
XCTFomet
16
128
266
0.1
0.85
0.
Satuuos 1 (sta
XC Toomee (DeCaP)
16
128
206
0.65
no
Seatures 100 istep 21, 100 800 (step 16)
Sequemee Length Scaling Analysis
Xc Teeeus
16
120
206
ILL
110
Jus DE 1028 R2)
XC XCTBemee (TaCaP)
16
756
a.6
00
lec 61-192: (step 1091 1066 198)
Channel scaling. We fix the lookback sequence length to 96 and vary the number of channels to match the dataset's 's dimensionality. Specificallyy, we tested n__eetures from 1 to 100 in steps of 2, and further extended from 100 to 800 in steps of 16 for the DeCoP variant. Because utilizing XCTFormer without DeCoP on larger datasets is impractical, the original XCTFormer evaluated only over the smaller range. As shown in Fig. 11 XCTFormer exhibits quadratic parameter growth with respect to the number of channels, whereas adding DeCoP yields approximately linear growth.
Sequence scaling. We fix the data dimensionality to 10 and vary the sequence length, which determines the number of tokens after patching. We evaluate sequences from 64 1024 in steps of 32, and extend from 1024 to 4096 in steps of 128 for the DeCoP variant. Again, XCTFrrmer is evaluated only the shorter range.
30

---

Published in Transactions on Machine Learning Research (03/2026)
Parameters vs Sequence Length (Linear Scale)
16M
XCTFormer
XCTFormer (with DeCoP, km64)
12M
Parameters
10M
8M
4M
2M
500
1000
1500
2000
2500
3000
3500
4000
Sequence Length (Lookback Window)
Figure 12: Scalability with respect to sequence length len). XCTFormer exhibits quadratic parameter growth, while the DeCoP variant scales approximately linearly.
Fig. 12 shows the same trend: XCTFrrmer grows quadratically with sequence length, whereas XCTFormer with DeCoP scales approximately linearly.
Appendix: Synthetic Dataset
D.1
Overview & Motivation
While existing syuthetic datasets are available (Nochumsohn et al. 2024 Nochumsohn & Azencot 2025
Oreshkin et al. 2026) and generative approaches enable sampling from empirical data (Naiman al. 2024b a
Gonen al. 2025 Fadlon al. 2025), they do not explicitly target the interplay between temporal and cross-variate dependencies. We propose new synthetic dataset, evaluated on the forecasting task, to emphasize settings in which forecastability depends jointly cross-variate and lagged cross-tine relationships. The dataset consists of seven channels and is evaluated under the multivariate- multivariatc-to-single (MS) setting: all channels serve as imput, but training and evaluation loss is computed only on the 7th channel (the synthetic target). The MS setting is adopted to specifically evaluate each model's ability to forecast data that depends solely on cross-channel and cross-time relationships. This target channel constructed entirely from lagged signals in the first four channels, making it difficult predict from own history alone. Channels and (var 5, var 6) are independent random walks that distractors, meant probe whether model latches onto spurious correlations, i.e., learns to rely channels that share no causal relationship with the target, which would degrade generalization.
D.2 Dataset Construction
The dataset comprises 7 channels and 10,000 time steps. At high level, the channels fall into three groups:
Informative sources (channels 1-4): four sine waves with distinct amplitudes and frequencies that carry the information needed to predict the target.
Distractors (channels 5 6): two random walks that share no causal relationship with the target. They test whether a model can ignore spurious correlations.
Target (channel 7): constructed entirely from lagged patches of the four source channels. At each point in time the target blends patches from different sources at different look-back distances, and the blend weight itself changes over time. Because the target is derived solely from other channels'
31

---

Published in Transactions on Machine Learning Research (03/2026)
past values, it is difficult to predict from its own history alone. Small Gaussian noise is added for realism.
The remainder of this section specifies each construction step in detail.
D.2.1 Step 1: Source Signals
Table 13 lists all seven channels. The four informative sources are sine waves with varying amplitudes (1.0 5.0), frequencies (0.002-0.03)) and phase offsets, ensuring the signals diverse and not trivially correlated. The two distractor channels random walks, which are non-stationary and can exhibit spurious short-range correlations with any signal. A model that naively attends to all variates equally would be misled by these distractors.
Table 13: Synthetic dataset variate specifications.
Index
Name
Type
Role
var
Sine wave (A=1.0, f=0.02)
Informative source
var
Sine wave (A=3.0, /=0.03)
Informative source
3
var 3
Sine wave (4=2.0, /=0.01)
Informative source
var
Sine wave (A=5.0, f=0.002)
Informative source Distractor
var 5
Random walk (a=0.1)
var 6
Random walk (a=0.15)
Distractor
target
Patch-dependent blend
Target (predicted)
D.2.2
Step Patching
The source signals are divided into non-overlapping patches of length 16 with stride of 8 (50% overlap). All subscquent operations are defined at the patch level.
D.2.3
Step 3: Target Construction
The target channel is built. by blending patches from two pairs of source variates, each read at specific lag. The two pairs are:
Pair 1: var 1 (lag=1) = 1) and var 2 (lag= 2), contributing weight 0.5.
Pair 2: var 3 (lag=2) and var 4 (lag = 3), contributing weight 0.5.
The different lags ensure that the model cannot simply aligu patches at single fixed offset; it must learn look back different distances for different variates.
Within each pair, the relative contribution of the two sources is controlled by blend weight wokk) that varies over time. Specifically, wi() follows triangle wave cycling between 0 and with period of 20 patches (160 time steps), creating non-stationary dependency structure:
[ (k mod 20)/10, if (k mod 20) 10 2 (1 mod 20)/10, otherwise
w(k)
(15)
That is, wok) cycles through the values 0.1, ...
as I increases.
........
Putting it all together, the target patch at index k is:
target[k) = 0.5- [w(k) varrvvva__aa []]]]] ((1-ww())))) var 2 2 +0.5- (u(k)-var_3i - 21- var var_ 4[k-3]
(16)
where var par_ilk- denotes the 16-clement patch extracted from variate patch position k-f. Overlapping patch regions averaged.
32

---

Published in Transactions on Machine Learning Research (03/2026)
D.2.4
Step 4: Noise
Finally, Gaussian noise (a=0.02) is added to the target channel to prevent trivial exact recovery and simulate real-world measurement noise.
D.3
Dataset Splits & Preprocessing
Table summarizes the dataset splits and evaluation configuration.
Table 14: Synthetic dataset splits and evaluation configuration.
Property
Parameter
Value
Train / Val Test split
Proportion
70%
10% 20%
Total points
?
10,000
Normalization
StandardScaler
Fitted on train only 96
Lookback window
seq_len label_les
Decoder label length
48
Prediction horizon
pred_les
(96, 192, 336, 720)
Training epochs
train @oii__epocss
10
D.4 Results
Table 15 reports per-horizon forecasting results on the synthetic dataset, logether with the trainable parameter count and FLOPs for each model on this dataset.
Table 15:
Forecasting results on the synthetic dataset (MS setting) across prediction horizons, with computa- tional cost on this dataset. All models use ETTml hyperparameters without dataset-specific tuning.
192
336
720
Avg
Model
Params
FLOPs
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Crossformer
0.018
0.106
0.026
0.126
0.027
0.129
0.031
0.137
0.025
0.124
1.1E+07
1.2E-49
XCTForrmer (Ours)
0,038
U.151
0.043
0.161
0.045
0.165
0.035
0.144
0.040
0.155
2.0E+U5
3.3E+06
TimeMixer
0.025
0.123
0.070
0.204
0.053
0.178
0.133
0.281
0.070
0.197
1.2E+05
1.6E+07
Leddam
0.043
0.159
0.094
0.235
0.095
0.234
0.082
0.216
0.079
0.211
2.9E+066
3.5E+00
SCINet
0.130
0.277
0.092
0.235
0.085
0.225
0.079
0.218
0.097
0.239
2.8E+05
3.4E++06
TTansformmer
0.217
0.362
0.236
0.375
0.282
0.411
0.282
0.413
0.254
0.390
2.6E+055
2.9E++06
PatcITST
0.222
0.365
0.319
0.442
0.441
0.249
0.384
0.277
9.2E++0
3.8E+07
TimesNet
0.554
0.597
0.471
0.538
0.404
0.499
0.350
0.459
0.445
0.523
2.7E+066
3.8E+09
DLinear
0.516
0.377
0.496
0.572
0.482
0.565
0.428
0.323
0.480
0.539
6.5E+04
4.5E+05
MTLinear
0,670
0.640
1.189
0,881
1,388
0.975
1.149
0.845
1.099
0.835
2.0E 05
3,4E 05
TiDE
0.836
0.746
1.290
0.934
1.476
1.016
1.182
0.875
1.196
0.893
9.6E+006
9.2E
Autoformer
0.573
0.605
1.753
1.094
1.743
1.079
1.606
1.019
1.419
0.949
1.2F+00
2.4R+007
FRDformer
1.537
1.006
2.422
1.282
4.065
1.663
1.879
1.110
2.476
1.265
1.6F+07
1.88+099
The two models that explicitly capture both cross-variate and cross-time patch interactions, namely Cross- former and XCTFormer, achieve the lowest errors, consistent with the dataset's construction in which the target depends on lagged patches from other channels. Crossformer ranks first (Avg MSF 0.025) and XCTFormer ranks second (Avg MSE 0.040). However, Crossformer uses approximately 55x more trainable parameters (11M vs. 200K) and 364x more FLOPs (1.2E+09 vs. 3.3E+06) than XCTFormer this dataset, highlighting substantial efficiency gap.
We note that no dataset-specific hyperparameter tuning was performed: all models were trained with ETTml hyperparameters. Consequently, the reported results may not reflect each model's optimal performance on this dataset.
Chamnel-independent models (e.g., PatchTST, DLincar) tend show higher error, which is expected given that the target is difficult. to predict from its own history alone. The distractor channels (random walks) do
33

---

Published in Transactions on Machine Learning Research (03/2026)
not appear noticeably affect the performance of cross-variate models, suggesting these architectures can, to some extent, distinguish informative from non-informative channels.
A visualization of the synthetic dataset is shown in Figure 13 and qualitative forecasting examples on this dataset are provided in Figure 14
Lagut
100
200
300
400
500
Time step
Figure 13: Synthetic dataset visualization: source signals (var through var 6) and the constructed target channel.
34

---

Published in Transactions on Machine Learning Research (03/2026)
Forecast Examploc: patch_no_ wwwioongnnnnnnneeeeoeeooee oscillating (prad_len-19)) XCTromerr #1794) 0,015150)
10
M
10
65
05-
60
-45
-10
50
100
251
150
250
Laddam (MSE: 0.023822)
TimeMizer (MSE: 0.052539)
Londact
10-
-15
-15-
-20
-20
50
200
150
200
232
300
50
100
:30
220
230
330
PatchTST (MSE: 0.098802)
SCINet (MSE: 0.133987)
100
Oraand
10
(MSE: 0.290194)
Londact
10
Grand
6.
-65
-10
1.
-20
20
-25
350
100
150
233
230
DLlinar (MSE: 0.505761)
Autoformer (MSE: 0.570745)
Lendack
-20-
:30
230
0.739445)
0.846645)
1.0
Grand Trm
65
0-
-1-
50
150
250
Figure 14: Prediction examples on the synthetic dataset: ground-truth target vs. model forecasts for selected models.
35

---

Published in Transactions on Machine Learning Research (03/2026)
E Appendix: Full Results
E.1 Statistical significance tests
To assess whether the observed performance differences are robust to random initialization, we conducted statistical significance tests for the forecasting task against two competitive baselines, LeDDAM (Yu et al., 2024] and iTransformer Liu et al. (2024). under the same evaluation protocol used throughout the paper.
The results are reported in Tables 16 and 17 respectively.
Experimental setup. For each dataset, we evaluated all models on the four standard prediction horizons {96, 192, 336, 7203. For every method (ours and each baseline), we trained the model five times using different random sceds {2021, (2021,2022, 2023, 2024, 2025]. For our modol, we used the hyperparamoters reported in Appendix [.3 For the baselines, we used the best hyperparametens provided in their official GitHub repositories. All runs followed the same training procedure as the main experiments, including model selection based on validation loss and reporting test-set errors. We applied two-sided paired t-test with 0.05. We mark a result as significant when the mean difference favors our model and p S 0.05; otherwise, we treat it as inconclusive.
Table 16: Statistical comparison of XCTFormer vs LeDDAM on forecasting datasets. Results averaged over prediction lengths (96, 192, 336, 720} across five seeds (2021-2025). Confidence level derived from Welch's
I-test (99.9%: p < 0.001, 99%: p < 0.01, 95%: p < 0.05). Bold indicates statistically significant. better
performance.
XCTFormer (Ours)
LeDDAM
Confidence
Dataset
MSE
MAE
MSE
MAE
Level
ETTh1
0.449 1: 0.002
0.436 1 0.001
0.436 1 0.0073
0.432 : 0.0034
95%
ETTh2
0.374 + 0.007
0.399 1 0.004
0.374 + 0.0019
0.398 1 0.0007
n.s.
ETTml
0.371 + 0.003
0.393 + 0.002
0.388 + 0.0034
0.398 + 0.0023
99%
ETTm2
0.271 + 0.001
0.319 0.001
0.282 + 0.0019
0.326 + 0.0009
99.9%
ECL"
0.176 = 0.007
0.270 0.007
0.171 = 0.0042
0.264 0.0027
n.S.
Traffic"
0.435 0.001
0.287 = 0.001
0.468 = 0.0075
0.294 = 0.0052
95%
Weather
0.237 + 0.001 0.267 + 0.001
0.244 + 0.0018
0.273 + 0.0012
99.9%
Not statistically significant (p 2 0.10).
DeCoP was enabled for XCTFormr on this dataset.
Table 17: Statistical comparison of XCTFormer vs iTransformer on forccasting datasots. Results averaged over prediction lengths (96, 192, 336, 720} across five seeds (2021-2025). Confidence level derived from Welch's t-test (99.9%: p 0.001, 99%: p 0.01, 95%: p 0.05). Bold indicates statistically significant. better performance.
XCTFormer (Ours)
iTransformer
Confidence
Dataset
MSE
MAE
MSE
MAE
Level
ETTh1
0.449 + 0.002
0.436 + 0.001
0.457 + 0.0014
0.449 + 0.0013
99.9%
ETTh2
0.374 0.007
0.399 + 0.004
0.383 + 0.0022
0.407 + 0.0011
95%
ETTml
0.371 + 0.003
0.393 + 0.002
0.408 = 0.0022
0.412 0.0013
99.9%
ETTm2
0.271 + 0.001
0.319 + 0.001
0.291 + 0.0010
0.335 + 0.0011
99.9%
ECL"
0.176 + 0.007
0.270 + 0.007
0.176 + 0.0037
0.267 + 0.0026
n.S.
Traffic"
0.435 + 0.001
0.287 1 0.001
0.430 1 0.0010
0.283 1 0.0010
99.9%
Weather
0.237 + 0.001
0.267 + 0.001
0.260 + 0.0014
0.281 + 0.0016
99.9%
D.A. Not statistically significant (p 2 0.10).
DeCoP was enabled for XCTFormer on this dataset.
36

---

Published in Transactions on Machine Learning Research (03/2026)
Results. Overall, our model achieves strong performance and shows statistically significant improvements across most datasets relative to both baselines. Compared to LeDDAM, the results are inconclusive on ECL and ETTh2, while ETThl favors LeDDAM. Compared to iTrensformer, the result is inconclusive on ECL, and Traffic favors iTransformer. These outcomes suggest that gains are often consistent across soeds, but in some datasets, the differences are not statistically significant.
E.2 Ablation Study: Complete Analysis
We conduct a systematic ablation study with six configurations to isolate the contribution of each architectural component in XCTFormer. All variants maintain identical data processing (patch mnnth/stridds training procedures, and model parameters, except for the specific component being modified.
Configuration Details
Full XCTFormer (Baseline): Complete architecture including CRAB module with learnable non-boolean mask (for dalasets with <60 channels) or DeCoP (for datasets with >60 channels), cross-time and cross-channel attention, and our proposed attention activation function.
2. W/o Learnable Mask: Removes the learnable mask component. Attention masks are not converted to positive values and no element-wise multiplication is applied. The CRAB module remains unchanged otherwise.
3. Standard Softmax Activation: Replaces our proposed activation function with standard Trans- former softmax while preserving CRAB and the learnable mask. Note that our attention dropout rate paramcter is replaced with the standard dropout argument commonly used in related work for fair comparison.
Vanilla Transformer: Substitutes CRAB (and DeCoP) with standard attention blocks following Vaswani et al. (0177)
5. Sequence Modeling Only: Retains only temporal self-attention within each channel, disabling cross-channel modeling Gthannel-independenn processing). This configuration tests the necessity of modeling cross-channel relationships, mirroring approaches like PatchTST (Nie al. 2023).
6. Channel Modeling Only: Preserves only cross-channel attention at each time step while removing temporal self-attention. This configuration tests the necessity of modeling temporal relationships, similar to designs that emphasize cross-variable mixing like iTransformer (Liu et al. 2024
Full Ablation Study Results Complete ablation study results for each time-series task are presented in Tables [8
19 and 20 For long-term forecasting and imputation tasks, the results shown for each dataset represent averages across all prediction horizons and mask ratios, respectively.
Table 18: Ablation study results for Long-tern term Forecasting across different datasets, evaluated with different XCTFormer variations.
XCTFomee Otlirs
ETTb1
ETT12
ETTml
ETTm2
ECLP
Traffie"
Weather
MSE
MAE
MSE
MSE
(%)
XCTFomeer (Origina)
0.250
0.136
0.389
0.396
0.369
0.392
0.270
0.319
0.166
0.263
0.135
0.287
0.237
0.267
W/o mask?
0.453
0.440
0.383
0.405
0.379
0.334
0.282
0.326
0.256
0.280
+2.7%
Original softmax activation
0.443
0.435
0.390
0.009
0.110
0.413
0.280
0.326
0.224
0.317
0.520
0.368
0.2c7
0.278
+8.0%
Vanilla transformer
0.4522
0.429
0.397
0.412
0.385
0.338
0.285
0.330
0.224
0.317
0.520
0.308
0.263
0.284
+8.3%
Sequence modeling
0.440
0.204
0.325
0.333
0.195
0.258
Channel modeling
0.461
0.450
0.388
0.409
0.378
0.333
0.278
0.321
0.166
0.263
0.476
0.529
0.229
0.269
+3.4%
DeCaP was enabled for XCTFormer on dataset.
W/o mask results exclude DeCoP datasets where the learnable mask
applicable; % is recomputed over remaining datasets.
37

---

Published in Transactions on Machine Learning Research (03/2026)
Table 19: Ablation study results for Imputation across different datasets, evaluated with different XCTFormer variations.
XCTFFrrme Others
ETTml
ETTm2
ETTh!
ETT12
Weather
ECT
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAF
MSE
MAE
MSE
MAE
(%)
XCTForeer (Original)
0.087
0.201
0.050
Woo mask!
0,041
0.132
0.129
0.100
0,082
0.200
0.064
0.169
0.031
0.047
+1399%
Original softmax activation
0,032
0.117
0.027
0.099
0.078
0.192
0.065
0.178
0.040
0.075
0.077
0.196
+11.7%
Vanilla transformer
0.043
0.137
0.033
0.113
0.208
0.193
0.048
0.077
0.196
+119.9%
Sequiee modeling
0.040
0.130
0.029
0.101
0.089
0.201
0.081
0.166
0.031
0.047
0.046
0.139
+8.7%
Chamnel modeling
0.065
0.174
0.044
0.136
0.203
0.305
0.060
0.167
0.041
0.072
0.075
0.191
+-34.6%
DeCoP was enabled for XCTFormer on this dataset.
W/o mask results exclude DeCoP datasets where the learnable mask is
not applicable; % is recomputed over remaining datasets.
Table 20: Ablation study results for Anomaly Detection across different datascts, evaluated with different
XCTFormer variations.
XCTFormer
PSM
SWaT
MSL
SMAP
SMD
Others
F- F-Score
F-Score
F- F-Score
F-Score
F-Score
(%)
XCTFormer (Original)
95.3
92.6
79.0
86.7
84.2
W/o mask
95.3
88.2
72.3
66.7
83.0
+8.0%
Original softmax activation
95.3
90.2
69.0
68.6
84.3
+7.5%
Vanilla transformer
95.3
93.1
71.8
66.9
83.2
+6.7%
Sequence modeling
95.3
93.5
79.4
67.5
83.9
+4.4%
Channel modeling
92.9
92.5
76.9
66.5
82.8
+6.4%
E.3 Robustness Across Random Seeds: Complete Analysis
Coefficient of Variation (CV) for Single Metric We quantify run-to-run stability using the coelficient. of variation, unitless meastre of dispersion relative to the mean (Reed et al. 2002). For metric with mean and standard deviation across five seeds (2021 to 2025), we compute:
CV(%) = 100
[
The coefficient of variation tells us how much results vary around their mean relative to the mean itself. Since CV is unitless, it enables comparison across datasets and metrics: smaller values indicate greater stability, while larger values indicate greater variability.
Confidence score mapping For intuitive interpretation, we report complementary confidence score:
Conf(%) 100 CV(%).
This confidence score inverts the scale that lower variability corresponds to higher confidence. For example, if CV = 3.2%, then Conf = 96.8%, indicating that repeated runs with identical setups produce very similar results.
Full Seed Analysis Results To enhance readability, we include only the averaged analysis table for all time-series tasks in the main paper, while the complete results are provided in Tables 21 22 and 23 The confidence score presented for each dataset represents the average confidence across all of its metrics.
38

---

Published in Transactions on Machine Learning Research (03/2026)
Table 21: Standard deviation for XCTFormer on forecasting datasets, evaluated across five seeds (2021-2025).
Results averaged over the four prediction lengths {96, 192, 720).
Model
XCTFormer (Ours)
Confidence Score
Dataset
MSE
MAE
Score %
ETTh1
0.449 + 0.002
0.436 = 0.001
99.7%
ETThh
0.374 + 0.007
0.399 = 0.004
98.5%
ETTml
0.371 = 0.003
0.393 = 0.002
99.3%
ETTm2
0.271 +: 0.001
0.319 + 0.001
99.6%
ECL"
0.176 1 0.007
0.270 + 0.007
96.6%
Traffie
0.435 0.001
0.287 = 0.001
99.7%
Weather
0.237 + 0.001
0.267 = 9.81e - 04
99.5%
P DeCoP was enabled for XCTFormer on this dataset.
Table 22: Results of the imputation task across datasets, evaluated across five seeds (2021-2025). We randomly mask (12.5%, 25%, 37.5%, 50%} of the time points; the final results are averaged across these four masking ratios.
Model
XCTFormer (Ours)
Confidence Score
Dalaset
MSE
MAE
Score %
ETTh1
0.090 + 0.002
0.204 + 0.003
98.0 %
ETTh2
0.052 + 0.013
0.153 0.020
80.9
ETTml
0.031 + 0.004
0.116 + 0.007
90.2 %
ETTm2
0.026 + 0.003
0.097 = 0.007
90.8 %
(Avg))
0.049 + 0.006
0.143 0.009
91.1 %
ECL"
0.051 + 0.008
0.149 + 0.014
87.6 %
Weather
0.031 5.26e
0.049 0.002
96.9 %
DeCoP was enabled for XCTFormer on this dataset.
Table 23: Results for the anomaly detection task (P, R, and F1 are precision, recall, and Fi-score in %), evaluated across five seeds (2021-2025).
Model
XCTFormer (Ours)
Confidence Score
Dataset
Precision
Recall
F1
Score %
MSL
87.84 1.73
66.66 3.92
75.77 3.18
96.0%
PSM
98.31 + 0.09
93.05 + 0.57
95.61 = 0.34
99.6%
SMAP
91.81 + 1.78
64.59 14.53
75.22 + 10.43
87.2%
SMD
87.01 + 0.25
81.99 + 1.15
84.42 - 0.69
99.2%
SWaT
91.94 + 0.49
92.01 + 1.32
91.98 + 0.86
99.0%
E.4
Long- Term Forecasting Results
To improve readability, present only the averaged table for long-term forecasting in the main paper and provide the full results here.
E.5
Imputation Full Results
To improve readability, we present only the averaged table for imputation in the main paper and provide the full per-mask-ratio results for XCTFormer in Table 25
39

---

Published in Transactions on Machine Learning Research (03/2026)
Table 24: Long-term forecasting results comparison across multiple datasets and horizons. Synthetict is evaluated under multivariate-to-single (MS) setting while all other datasets use multivariate (M). We compare extensive competitive models under different prediction lengths. Aug is averaged from all four prediction lengths, that {96, 192, 336, 720}.
Models
Lofiin
Cineforwe
TinweNe
SCttive
Autoforme
(Ours)
(CCR
2035)
(AISINTS 2056)
(IMML 3004)
(CCR 2084)
(ICLR 3r04)
(KCLR
2025;
(ICLR 5008)
(TMLR
3008)
(ICLR
[AXA]
2085)
(NoulPP
STG2)
[CCML 2022;
(XoutIPS
2001)
Metric
SUA5
MAB
3800
MAA
MAR
MAR
MAK
MAR
M0A
126
MAX
MDE
MAA
MAAS
MAR
MAG
06
b.302
0.355
0.310
0.334
[0,377
0.363
0.319
0:350
0,200
03377
0.354
0.300
0.333
0.501
0.404
0.136
b.364
0.387
0.338
0.375
0.595
0.374
0:415
0.036
0.379
0.019
(..05
0:125
192
0.354
0.350
0.345
0.362
0:279
0.387
0.300
0.383
0.303
0.381
0.300
0.574
0.387
0:450
0.451
0.376
0.857
0.391
0:430
0.406
0.053
0.096
836
0.885
0.434
0,370
0.891
R1412
0.400
0.804
0.402
0.000
0.414
0.420
0.420
0.421
0.414
0.832
0.515
0.438
0.425
0.410
0.411
0.415
0.450
0.445
(002)
0.507
720
0-48G
0.485 0.440
0.423
R8:055
0.443
0.400
0.442
0:454
(1:41 0.401
0.420
0.602
0.449
0.600
0.550
0.467
0.361
0.478
0,450
0.473
0.451
0.005
0.050
0.548
0.430
(0.07)
0.003
Jos
10.300 0.302 0.008
0.976
10.205
0.401
00.35
0.307
[0:883
01236
0,430
0.430
0.002
0.406
0.518
0.496
0.430
0.419
0.400
0.406
(c.
0:308
0.455
0.551
|0448
0.450
00.58
0.5L7
b.308 0.282
00.17
0.245
[1:155
0.254
0.387
0.125
0.238 0.000
0.254
3.183
0.200
0.287
0.366
0.305
0.185
0.267
0.203
0.285
0.267
0.255
0.331
192
0.2012 0.205
0.220
0.291
[8:200
0.25M
0.244
0.307
1.237
112210
0.254
0.300
0.255
0.514
0.414
0.412
0.200
0.364
0.240
0.300
0.254
0.361
0.000
0.445
0.326
(..361
0.540
306
0.280
0.332
0.303
0:943
00:001
0.335
0.303
0.341
(1,290
01340
0.311
0.318
0:305
0.517
0:597
0.512
0.391
0.351
0.507
0-470
0/637
0.591
0:366
0.:31
0:372
720
0.001 0.005 0.073
0.209
00402
0.303
0.400
0.300
0.801
0.407
0:412
0.401
1.730
1.040
0.400
0.525
0.060
0.421
0.415
0:032
Ans
(0:77
0.319/00200
00.200
0.320
[0,279
0.320
10:20
0.325
0.223
14325
0.286
0.382
0.200
0.504
0.757
0.011
0.300
0.404
0.291
0.353
6.504
0.002
0.373
0.37
J.304
(4.527
0.322
10,300
0.400
00.001
0.003
10,380
0.308
0.337
0.304/0.575
0.400
3,000
0.447
0.428
0.448
0.471
0.400
0.307
0:412
0.654
0.000
0.424
0 0.:49
0.630
192
0.400
0.420
0.410
0.641
0.421
0.424
0.422
0.429
0.421
0.441
0.512
0.477
0.42)
0.471
0.474
0.422
0.450
0.429
0.411
0.71
0.011
0.400
0.420
00.000
0.082
306
0.479
0.547
0.430
0.454
(2:
0.441
0.45M
0.802
1:981
0.187
0.158
0.506
0.00
0.577
0.546
0.565
0.515
0.451
0.000
0.000
0.467
0.776
0.039
0.550
0.000
0.006
720
0.400
0.458
0.407
0.451
1:472
0.400
0.403
0,500
0.401
0.561
0.517
0.053
0.558
0.501
0.500
0.510
0:816
0.000
0.541
0.001
0.51
Jos
(0.400
0.436
00.418
0.602
[0:443
0.429
0.431
0.420
0:647
00440
0.454
0.407
|.500
0.472
0.520
0.022
0.541
0.507
0.458
0.450
(0.001
0:458
0.347
0.047
J0.000
0.084
(0
0.087
06
0.312
00.376
0.325
10,358
0.306
0.2000
0.343
0.009
0:341 0.297
0.340
5,305
0.003
0.745
0.554
5.000
0.410
0.340
0.37.
(,300
0.354
0.707
0.021
0.297
0.334
0.355
0.370
0.338
0.342
0.979
0.388
0.307
0,380
0.972
0.400
0,903
0,877
0.656
0.414
0.960
0.420
0.452
336
0.402
0.417
00.308 0.300
(.413
0.423
0:412
0.424
(0.306
0.414
0.028
0.452
0.027
0.436
1.048
0.231
0.001
0.452
0.452
0.501
0.541
1.000
0.244
0.00
0.387
(1.
0.206
730
p.111 0.433, 0.302
0.415
[1:
0.400
0.110
0.138
00:11 6 0:131
0.127
0.415
p.030 0.400
1.304
0.703
0.574
0.079
0 0.000
0.409
0.509
0.061
1.510
0.836
0.003
0.174
0.515
0.51
|0.30 0.330
0,350
[0,033
0.307
00,372
0.306
0.407
0.668
0.427
0.954
10.400
0.450
86
b.150
0.180 0..855
[8139
0.311
0.300
1:3
0.000
0.374
0.310
100
0.217
0.195 0:371
0.300
0.361
01127 0.000
(.105
0.205
0.931
0.306
0.296
(0.006
0:336
Westher
192
0.100
0.243 0.201
0.945
[02002
0.200
0.207
0:200
0.208
0:350
0.251
0.204
0,234
0.265
0.250
0.277
0.342
0.258
0.230
0.201
0.597
0.255
0.263
0.340
6:270
0.330
0.307
0.907
0.257
0.286
0:331
0.900
0.350
0.905
730
(0:359
0.3140 00.112
0.334
(1,3322
0.316
0.333
0.343
(1330
0.91
0.356
0.317
0.000
0.54
0.377
0.000
0.351
0.346
0.315
0.350
0.515
0.383
0:377
0.017
0.40
0.178
(
Aox
J0.287
0.207 (0.220
0.262
[0.235
0.270
0.,32
0.272
[:240
0.202
0,258
0.27M
p.205
0.285
0.254
0.320
(.200
0.320
0.250
0,250
|..05
0.815
0.202
0.353
10,300
0.000
00.815
0.982
10.338
0.237
[1183
00::41
1:333
0.310
0.2M
0.219
0.314
0:372
0,302
0.947
10.00
0.306
0.317
192
0.16)
0.251
0.147
0.335
[1183
0.208
0.160
0.252
0.106
0.356
0.302
0.258
0.109
0.301
0.251
0.332
0.396
0.330
0.184
0.352
0.010
0.305
0:357
0.355
0.201
0.315
0.222
0.334
836
0.170
0.200
0.104
0.245
[1196
0.283
0.177
0:208
[.186
0.277
0.178
0.200
3.217
0.819
0.26
0.357
0.240
0.344
0.108
0.200
6:223
0.319
0.269
0.389
0:214
0.029
0.211
D.445
p.330
0,212
0.310
[0:201
0.317
0.201
0.317
0.365
0.320
0,300
0:259
Jos
Jo. 10000
0.205
0.105
0.368
[a10s
0.283
00.000
0.260
0:180
0:273
0.378
0.270
0.318
0.244
0.334
0.202
0.344
0.108
0.200
0.310
0.265
0.806
J0.31a 0.227
10:227
0.364
10.402
0.259
10.302
0.253
0.383
1.003
012855
0.3M5 0.208
3.525
0.547
0.844
0.420
0.505
0.450
0.350
0.321
0.000
0.306
0.205
0.009
10387
0.966
(1613
Traffee
192
0.434
0.051
00:022
0.309
0.080
0.605
0.431
0.336
0.370
0.799
0/004
0.375
0.389
836
0.464
0.291
0.428
0.363
[0001
0.302
0.480
0.297
0.438
0.296
0.
23517
0.884
0.674
0.420
0.702
0.477
0.029
0,350
0.005
0.878
0.797
0.908
0.621
0.385
(.022
0.337
720
0.492
0.307
(0.441
0.2352
0.382
0.408
0,313
(1:06
0..1.
0. 0.467
0.302
2.00
0.382
0.883
0.424
0.440
0.640
0.350
0.655
0.304
0.41
0.325
0.362
(.950
11:008
Aog
[0.435 0.357
00.410
0.264
[1002]
0.373
00:467
0.004
0.0
2.282j..00
0:341
0.667
0.436
(.700
0.336
(..05
0.353
0.804
10.c00
0.376
[0.025
0.370
Senbortte
192
Jusos 0.151
b.042 0.161
Jantoo
1.189
0.540
0.551
0.001
0.100
0:935
0:000
0.901
0.123
0.217 0.302
0.236
0.375
b.222
0.519
0.305
0.077
0.026
0.0180.1000
b.500
1,200
0:240
0.554 0.357
0:538
0.09
0.516
0.507
0.579
0.022
0.130
0.213
2.499
1:359
1.006
1.733
(1.25
1.051
01.905
236
0.045 0.105
1.385
0.975
0.000
0.884
0.033
0.352
0.411
0,919
0.441
0.027
0.120
0.404
0,400
0.565
0.055
4.0G
1.053
1.743
1.070
720
p p.oas 0.144
1.140
0.545
0.082
0:310
[1133
(1261
0.282
0.410
0.200
0.584
0.051
0.137
..182
0.538
0.350
0.450
6.425
0.525
0079
0.2.8
L.70
1.110
1.006
1.019
p.000 0.155
1000
0.533
|occmu 0:211
1.000
0.107
0.254 0.300
Ja
077
008
0.12
196
0.853
0.385
(o 180
0.520 0.057
22.476
1.955
1
0.010
1d Count]
7
20
21
4
0
Ang F10%|]
3338-000
7:500-00
8/545-00
20316000
1,313:000
1/200-31
1.018-00
1,415000
200000000
Jug Parans
7:21E-06
2:50E-000
1.91440
2.308-000
X.004405
Ta0B-00
8.58E-00
4.744400
0.308-04
T.44-407
1.0B447
213B406
Evaauutedd under mulhivarizz tooilll (315) setting: lyperparamatees diasen iduatical to BTTul for all models. Reported MTTLinon resulta seflect the pet-dulaset beat of MTNLineu and MIDDLnntr. P DeCoP was coabled for XCTForerr on this dateset.
40

---

Published in Transactions on Machine Learning Research (03/2026)
Table 25: Full imputation results for XCTFormer across all datasets and mask ratios. We randomly mask
(12.5%, 25%, 37.5%, 50%} of the time points in time series of length 1024.
Mask 12.5%
Mask 25%
Mask 37.5%
Mask 50%
Avg
Dataset
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.069
0.179
0.080
0.192
0.093
0.208
0.105
0.221
0.087
0.200
ETTh2
0.045
0.143
0.065
0.168
0.068
0.181
0.060
0.165
0.059
0.164
ETTml
0.019
0.092
0.026
0.107
0.034
0.122
0.035
0.124
0.028
0.111
ETTm2
0.020
0.083
0.022
0.089
0.025
0.095
0.028
0.101
0.024
0.092
(Avg)
0.038
0.124
0.048
0.139
0.055
0.152
0.057
0.153
0.050
0.141
ECLP
0.037
0.124
0.043
0.135
0.049
0.146
0.056
0.157
0.046
0.140
Weather
0.029
0.047
0.031
0.048
0.033
0.049
0.036
0.052
0.032
0.049
DeCoP was enabled for XCTFFormr on this dataset.
Table 26: Full results for the anomaly detection task. The P, R, and F1 represent the precision, rocall, and Fl-score, (%) respectively, Fl-score is the harmonic mean of precision and recall. A higher value of P, R and F1 indicates a better performance. Red indicates highest. F1 score, blue indicates second highest F1 score.
Datasets
SMD
MSL
SMAP
SWaT
PSM
Avg
Metrics
R
F1
P
F1
P
F1
P
(%)
LSTM
1997
78.52
65.47
71.41
78.04
86.22
11.93
91.06
57.49
70.48
78.06
91.72
84.34
09.24
99.53
81.67
77.97
Transformer
2017
83.58
76
79.56
71.57
87.37
78.68
89.37
57 12
69.70
68
84
96.53
80.37
62.75
96.56
76.07
76.88
LogTrans
2019a
83.46
70.13
76.21
73.05
87.37
79.57
89.15
57.
69.97
68.65
97
80.52
63.06
98.00
76.74
76.60
TCN
2019
84.06
79.07 81.49
75.11
82.44
78.60
80.90
59.23 70.45
76.59
95.71
85.09
54.59
99.77 70.57
77.24
Reformer
2020
82.58
69.24 75.32
85.51
83.31
81.40
90.91
57.44
70.40
72.50
96.55
82.80
59.93
95.38 73.61
77.31
Informer
2021
86.60
77.23 81 66666555
81.77
86.48
84.06
90.11
57.13
69.922
70.29
96.77 81 .433
64.27
96.33 77.10
78.83
Anomaly"
88.91
82.23 85.49
79.61
87.37
83.31
91.85
58.11 71.18
72.51
97.32 10
68.35
94.72 79.40
80.50
Pyraformer
2122
85.61
80.61 83.04
833.81
85.93
84.86
92.55
57.71 71.09
87.99/
96.00 91.78
71.67
96.02 82.08
82.57
Autoformer
2021
88.06
82.35 85. 11
77.27
80.92 79.05
90.40
58.62 71.12
89.
95.81 81 92, 7-
99.08
88.15 93.20
84.26
LSSL
2022
78.51
65.32 71.31
77.55
88.18 82.53
80.44
533.43
66.90
79.05
93..2 85.
66.02
92.93 77.20
76.77
Stationary
2022c
88.33
81.21 84.62
68.55
89.14 77.50
89.37
59.02
71.09
68.03
96.75 79.88
97..82
96.76 97.29
82.08
DLinear
2023
83.62
71.52 77. 1
844.34
85.42 84.88
92.33
55.44 69.26
80.91
95.30 87.52
98.28
89.26 93.55
82.46
ETSformer
2022
87.44
79.23
13
85.
84.98 85.03
9/2.25 55.75
69.50
910.02
SU.36 84.91
999.31
85.28
91.76
82.887
LightTS
2022
87.10
78.42
82.55 53
822.40
75.78 78.95
912.58
55.27
69.27
91 .98
94.72 913.333
988.37
95.97
97.15
84,22
FFDDormmr
2022
87.95
82.39 85.08
77.14
80.07 78.57
90.47
56 70.76
90.17
96.42 99
97.31
97.16
97.28
84.97
TimesNet
2023
88.66
83.14 85. 81
83.92
86.42
92.52
58.29 71.52
86.76
97.32 91.74
98.19
96.776 76.76.11 97.47
86.31
TiDE
2023
76.00
63:00
91
8-1.00
60.00
1..18810
88.00
50.00 64.00
96. 155.00
63.00
76.73
93.00
92.00
92.50
74.46
(Transformer
2024
78.45 60 71.15
86.15
62.65
72.54
90.67
52.96 66.87
99.96
65.55 19.18
95.65
94..69 95.17
76.98
+imesMixer++
2025
88.59 84.50 86.50
89.73
82.23 85.82
93.47 60.02 73.
92.96
94..33 94.64
98.33
96.90 97.60
87.47
XCTFormer
(Ours)
86.94 81.64 84.21
89.46
70.81 79.05
933.79 80.37 86.68
912.25 92.96 92.66
96. 92..2 95.30
87.57
* The original paper of Anomaly Transformer (Xu et al.]
)022) adopts the temporal association and
reconstruction error joint. anomaly criterion. For fair comparisons, we only use reconstruction error
here.
E.6
Anomaly Detection Full Results
To improve readability, present only the averaged plot for anomaly detection in the main paper and provide the full results in Table 26
E.7 Computational Complexity
We report computational complexity in terms of floating-pots point operations (FLOPs) and trainable parameter
counts. For detailed description of the measurement methodology, see Section B.6 Tables 27 and 28 present. the full per-dataset results.
41

---

Published in Transactions on Machine Learning Research (03/2026)
Table 27: FLOPs (floating-point operations) per datasel, averaged over prediction lengths {96, 192, 336, 7203.
Model and dataset ordering follows Table 24
See Section B.6 for measurement methodology.
Model
ETTml
ETTm2
ETThl
ETTh2
Weather
ECL
Traffic
Synth.
Avg
MTLinear
0.4E+05
0.4E+005
3.4E+05
3.4E+05
1.0E++06
1.6E++07
4.2E+007
3.4E 1:4E+05
05
7.59E+ +
DLinear
0.5E+05
4.5E+05
4.5E+05
4.5E+05
1.4E+066
2.1E+07
6,6E-+07
4.5E 1.5E+05 05
1.01E 101E++07
Autoformer
2.4E+077
2.4E+07
2.4E+07
2.4E+07
2.7E+07
8.4E+07
1.9E-08
2.4E+07
5.26E
PatchTST
3.8E+07
3.8E+07
1.9E+06
1.9E406
1.1E+08
1.7E++09
4.7E+09
3.
8.28E+08
Leddam
3.5E+07
4.7E+07
4.7E+07
1.2E+07
3,5E+07
1.6E++09
5,1E+409
3.5E-07
8.64E
TimeMixer
1.6++07
3.2E+07
1.6E+07
1.6E+07
6,0E+07
9.2E++08
5,9E+009
1.6E+07
8.72E+-08
TIDE
9.2E++07
7.7E+007
2.5E+07
1.1E+08
1.2E+08
6.0E+09
3.9E-00
9.2E-077
1.31E+099
iTransformer
2.9E+066
2.9E+006
1.0E+07
2.9E+06
1.3E+08
1.9E+09
8.7E++09
2.9E-06
1.34E+099
SCINet
3.4E+066
3.4E+06
7.3E+05
7.3E+05
4.8E+066
4.7E+09
6,6E-009
3.4E+06
1.41E+09
FEDformer
188++09
L8E+09
1.84+09
1.84+09
1.8E+09
2.2E+09
3.1E+09
2.01E+009 -.8
Crossformer
1.2E+09
N/A
9.7E+08
N/A
2.5E+09
2.6E+09
5.4E-09
1.2E-09
2.31E
XCTFormer (Ours)
3.3E+006
4.4E+07
4.0E+05
4.1E+06
4.9E+08
6.1E+09
2.0E+100
36
3.33E+099
TimesNet
3.8E+09
1.9E+09
1.3E+09
2.5E+09
2.6E+09
3.2E+11
6.5E+11
3.8E+09
1,23E+11
Syuthetic uses the multivariate-to--ingle (MS) setting with ETTml hyperparameters.
Crossformer values ETT12 and ETTm2 N/A due configuration incompatibility. TimeMixer+++ excluded its code publicly available.
Table 28: Trainable parameter counts per dataset, averaged over prediction lengths (96, 192, 336, 720]. For MTLinear, values are the average of MTDLinear and MTNLinear. See Section B.6 for measurement methodology.
Model
ETTml
ETTm²
ETTHl
ETTH2
Weather
ECL
Traffic
Syuth.
Avg
DLinear
6.5E+04
6.5E+04
6.5E+04
6.5E+04
6.5E+01
6.5E+00
6.5E+04
6.5E+04
6.50E+004
TimeMixer
1.2E+05
1.2E+05
1,2E+05
1.2E+05
1.5E+05
1.5E+40
1.7E+05
1.2E+05
1.34E+05
Autoformer
1.2E+05
1.2E+05
1.2E+05
1.2E+05
1.3E+05
3.2E+005
6.7E+00
1.2E+05
2,15E+05
PateiTSS
9.2E+05
9.2E+05
8.2E+04
8.2E+04
9.2E+05
9.2E+05
9.2E++05
9.2E
+.10E+05
MTLincar
2.0E+05
2.4E+05
2.0E+05
2.4++05
5.9E+05
2.2E+066
2.3E-+06
2.0E
7.71E+055
iTrausformer
2.6E+05
2.6E+05
9.0E+05
2.6E+05
5.0E+06
5.0E++06
6,5E+006
2.6E 05
2.30E
Leddam
2.9E+006
36
3.8E+06
1.3E+06
1.3E+06
3.4
3.4E+066
2.9E 06
2.85E 06
XCTFormer (Ours)
2.0E+05
4.1E+04
1.7E+05
2.9E+06
368E
3.2E++07
2.0E
5. 5.08E
Crossformer
L.1++07
N/A
1.1E+07
N/A
1.1E+07
1.3E+06
1.9E+06
1.1E 07
7.87E 06
TiDE
9.6E++0
0.
2.4E+06
1.4E+07
4.3E+06
1.6E+07
3.4E+066
9.6E 06
8,53E 06
FEDformer
1.6E+077
L.6E+07
1.6E+07
1.6E+07
1.6E+07
1.8E+07
2.1E+077
1.6E
1.69E
TimesNet
2.7E+006
1.1E+06
6.3E+05
1.2E406
1.2E+06
1.5E+08
3.0E
2.7E 06
5. 5.74E
SCINet
0.8E+00
+.8E+05
7.2E+04
7.2E404
3.1E+05
1.2
4.5E+08
2.8E 05
7.14E
Syuthetic uses the mullávarratee-to-ingge (MS) setting with ETTal hyperparaneters.
Crussformer values ETTT2 and ETTm2 N/A due configuration incompatibilliy. TimeMixrr+ excluded its code is nut
publicly available.
42


Reviews:
[{"review_id": "aPFPYW6Ej9", "reviewer": "Reviewer_F3Af", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\nThe authors propose a new model based on transformers that takes into account multivariate dependencies with token to token attention. To do this, the authors propose three modules: learnable masks, signed normalisation and a compression plugin for efficiency gains. The authors evaluate their model on a range of tasks and results suggest good performance, particularly on imputation.\n\n### Strengths\n\n1. It addresses a real gap in the literature. Alternatives to two stage approaches are useful and can potentially capture and retain more information through a principled solution\n2. Evaluation tests three different task types on well established benchmarks. It also compares against a wide range of baselines.\n3. Ablation study isolates the contribution of each module in the system.\n4. Interpretability analysis is very interesting, and maybe should be part of the main text.\n\n### Weaknesses\n\n1. The claim that CrossFormer and CARD can/may lead to information loss at the stage interfaces is not backed up. No citations for prior work that show this or any empirical demonstration. The authors don’t claim this as fact, but this is the basis for the motivation. Without this ablation/comparison study we do not know if the improved performance comes from the additional modules that are introduced.\n2. I am not convinced that the signed attention weights claim to “increase expressiveness” because softmax performs worse. It would be interesting to see the distribution of the weights (or how many are negative). Clipping the weights, rather than passing them through the softmax function, might give a better signal for the importance of negative weights, as the softmax brings its own benefits on top of clipping negatives.\n3. Does the learnable mask not grow quadratically with patches * channels? Even with the compression, the mask would be quadratic, no?\n4. Performance improvements on anomaly detection is marginal. SOTA performance isn’t a requirement, but the claims might need to be better reflect the actual performance.\n5. High variance across many tasks reduces confidence. Particularly as no statistical significance tests of difference are done to compare the different methods. Seed analysis is useful but it serves a different purpose.\n6. No sensitivity analysis for patch hparams. This seems like a very important hyper parameter but there is no indication of its effect on the performance or how this should be chosen for a given task. In the experiments some use length of 16 with stride 8 while others use 64 / 32. It’s not clear how or why these were chosen and what effect it has.\n7. The paper is completely lacking a discussion or limitations section. The conclusion is very positive and does not mention any of the caveats of the approach. I would expect an honest limitations section that overview the key limitations that the authors have identified, I have mentioned some of them above. A discussion on some of the results (e.g. high variance) would be an improvement as well.\n8. Not really a weakness but a mention or discussion of TS foundation models would be apt. Although they are a different paradigm, they are increasingly relevant in time series modelling.", "claims_and_evidence_yn": "No", "claims_explanation": "Some claims lack support.\n\n1. The claim to model “all possible cross-channel and cross-temporal relationships” is overstated as the attention mechanism is inherently pairwise. There is no modelling of higher order (>2) interactions. This should rather say “all pairwise …\"\n2. I feel the forecasting performance is overstated as well. It is hard to be sure of its superiority over other method since the margins are relatively small and no statistical significance test to actually test this.\n3. Although not a direct claim, much of the motivation hinges on two stage methods losing information. The limitations of this assumption and lack of evidence should be more clearly stated, as a simple read of the paper does leave the impression this is a matter of fact.\n4. The importance of negative weights (see above) is not fully explained with the softmax experiment. It lacks the proper experimental setup to demonstrate this effect because of the confounding.", "audience_yn": "Yes", "audience_explanation": "It is suitable for TMLR and many readers will find it interesting.", "requested_changes": "Critical:\n1. Validate information loss in two-stage methods (or adjust claims)\n2. Perform statistical significance tests to check for difference (or adjust claims)\n3. Check negative weights claims with more appropriate experimental set up\n4. Add a discussion of limitation\n\nLess critical:\n1. Add sensitivity analysis of patch and stride hyperparams\n2. Scalability analysis\n3. Adjust the “all possible relationships” claim\n4. Compare the computational cost and scalability of the methods", "broader_impact_concerns": "None identified.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KFcXCjhpnp", "reviewer": "Reviewer_92Lb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes new architectural components for transformer-based time series forecasting models. The goal is to enable modeling cross-channel and temporal dependencies simultaneously without impacting model performance. The key ideas in this paper are: a) learn a real-valued mask --- encoding the temporal/cross-channel dependencies of a token --- that can ultimately be used to scale the attention-logits; b) use signed attentions by replacing softmax with a different variety of row-normalization. The architecture is evaluated on 3 kinds of time-series tasks: (1) long-term forecasting, (2) imputation and (3) anomaly detection.", "claims_and_evidence_yn": "No", "claims_explanation": "Partially, no. The key claim in this paper is that the paper establishes state-of-the-art results in time-series related tasks --- namely, forecasting, imputation and anomaly detection. I disagree with the claim related to SOTA on forecasting because direct comparison of forecasting results in this paper with PatchTST results reveal that PatchTST has better or same results for almost all dataset/forecast pairs. I am open to being convinced otherwise and seek clarification from the authors on how to interpret this difference and how this method establishes a new state-of-art.", "audience_yn": "Yes", "audience_explanation": "The ideas in this paper are useful for machine learning practitioners working in the area of time-series-forecasting/classification.", "requested_changes": "1. I request the authors to clarify how this paper is positioned with respect to other results reported in the literature and update the paper if needed. \n2. DeCoP: While DeCoP is designed to address runtime complexity, it appears to me that it can also address overfitting due to spurious correlations. I probably missed this, but is there any analysis on how varying the k affects the results?\n3. Hyperparameter-search space: Could you please clarify the hyperparameter-search space for this study? For example, what are the ranges/values that optuna searched over for Table 9? And, it appears that the hyperparameters are fixed for all forecasting horizons of a specific dataset? \n4. Out of curiousity, how does this method perform on the Exchange dataset? PatchTST explicitly stated that this is a challenging problem for them but wondering if this method is robust?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pOHvDh3lw0", "reviewer": "Reviewer_H7Wf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors aim to address the paradox in time-series analysis where CI models often outperform CD models, even though CD models often have more information to leverage. To achieve this, the authors propose XCTFormer, which is able to explicitly leverage cross-channel dependencies. This leverages three key architectural pieces including CRAB (Cross-Relational Attention Block) which significantly enhances performance, a data processing module, and an optional Dependency Compression Plugin (DeCoP). The results are generally better than almost all existing models for long-term forecasting, imputation, and anomaly detection.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- The results are overall very strong, and the motivation is also very strong.\n- The ablations confirm the design choice regarding XCTFormer's changes to a standard baseline, with every changes helping performance\n- The authors go above and beyond with ensuring the experiments hold across seeds and datasets, making a strong arugment for the proposed XCTFormer.", "audience_yn": "Yes", "audience_explanation": "The proposed XCTFormer generally achieves very strong results for time-series analysis and thus would be of great interest for the time-series analsysis community.", "requested_changes": "- It's unclear to me how much of the performance boost comes from the XCTFormer just being potentially more expensive.\n    - Is it possible to compare the XCTFormer to the baselines, not just in terms of performance, but also in terms of parameters and FLOPs? Without this information, it is difficult to ensure a fair comparison is done.\n    - This change is relatively critical for my recommendation, as without this information it's not possible to know how fair the comparisons made were.\n- Is it possible to do an ablation regarding how much of the performance benefit was due to the data processing module?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)