Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series
Authors: Guoqi Yu
Juncheng Wang
Chen Yang
Jing Qin
Angelica I Aviles-Rivero
Shujun Wang
Venue: ICLR 2026 Oral (2026)
Keywords: EEG
ECG
Deep learning
Transformer
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose a centralized module to replace decentralized attention in Transformer for centralized medical time series like EEG and ECG.

Abstract: Accurate analysis of Medical time series (MedTS) data, such as Electroencephalography (EEG) and Electrocardiography (ECG), plays a pivotal role in healthcare applications, including the diagnosis of brain and heart diseases. MedTS data typically exhibits two critical patterns: **temporal dependencies** within individual channels and **channel dependencies** across multiple channels. While recent advances in deep learning have leveraged Transformer-based models to effectively capture temporal dependencies, they often struggle to model channel dependencies. This limitation stems from a structural mismatch: ***MedTS signals are inherently centralized, whereas the Transformer's attention is decentralized***, making it less effective at capturing global synchronization and unified waveform patterns. To bridge this gap, we propose **CoTAR** (Core Token Aggregation-Redistribution), a centralized MLP-based module tailored to replace the decentralized attention. Instead of allowing all tokens to interact directly, as in attention, CoTAR introduces a global core token that acts as a proxy to facilitate the inter-token interaction, thereby enforcing a centralized aggregation and redistribution strategy. This design not only better aligns with the centralized nature of MedTS signals but also reduces computational complexity from quadratic to linear. Experiments on five benchmarks validate the superiority of our method in both effectiveness and efficiency, achieving up to a **12.13%** improvement on the APAVA dataset, with merely 33% memory usage and 20% inference time compared to the previous state-of-the-art. Code and all training scripts are available in this [**Link**](https://github.com/Levi-Ackman/TeCh).

Published as a conference paper at ICLR 2026
DECENTRALIZED ATTENTION FAILS CENTRALIZED SIGNALS: RETHINKING TRANSFORMERS FOR MEDI- CAL TIME SERIES
Guoqi Yu', Juncheng Wang', Chen Yang', Jing Qin?, Angelica Aviles-Riveroo Shujun Wangf!
DDepartment of Biomedical Engineering, PolyU School of Nursing, PolyU 3 Yau Mathematical Sciences Center, Tsinghua University
ABSTRACT
Accurate analysis of Medical time series (MedTS) data, such Electroen- cephalography (EEG) and Electrocadiooraphy (ECG), plays a pivotal role healthcare applications, including the diagnosis of brain and heart diseases. MedTS data typically exhibits two critical patterns: temporal dependencies within individual channels and channel dependencies across multiple channels. While recent advances in deep learning have leveraged Transformer basooommer-bbeddddddd models effectively capture temporal dependencies. they often struggle with modeling channel dependencies. This limitation stems from a structural mismatch: MedTS signals are inherently centralized, whereas the Transformer's attention is decen- tralized, making it less effective at capturing global synchronization and unified waveform patterns. To address this mismatch, we propose CoTAR (Core Token -eeiatiionnReisssibuttinnn centralized MLP-based module tailored replace the decentralized attention. Instead of allowing all tokens to interact directly, as attention, CoTAR introduces a global core token that acts proxy facilitate the inter-token interaction, thereby enforcing centralized aggregation and redis- tribution strategy. This design not only better aligns with the centralized nature of MedTS signals but also reduces computational complexity from quadratic to linear. Experiments on five benchmarks validate the superiority of our method in both effectiveness and efficiency, achieving up to 12.13 % improvement on the APAVA dataset, with merely 33% memory usage and 20% inference time com- pared to the previous ofate-of-the-art Code and all training scripts are available in /ttpps://ithiii com Levi Ackman/TeCh
INTRODUCTION
Medical time series (MedTS) data are temporal sequences of physiological data used to monitor subject's health status (Badr et al. 2024), such as Eleetroencephallograhy (EEG) for neurological assessment (Arif et al. 2024] Jafari et al. 2023) and Electrocardiography (ECG) for cardiac diagno- sis (Xiao et al. 2023 Wang et al. 2023). Accurate classification MedTS facilitates early anomaly detection, timely diagnosis, and personalized treatment (Liu et al. 2021 Murat et al. 2020). This requires adequate modeling for two critical patterns: temporal dependencies within individual chan- nels and channel dependencies across multiple channels, illustrated in Figure (a). Temporal dependencies reflect the intrinsic signal dynamics over time within each channel, such as oscilla- tory rhythms and event-related potentials for EEG (Niedermeyer & da Silva 2005), and P&T wave for ECG (Goldberger et al. 2000). In contrast, channel dependencies capture the interactions and entanglements among multiple channels, such as functional connectivity for EEG (Stam) 2005) and the biophysical geometry of the heart for ECG (Macfarlane et al. 2005).
Previous deep-learning methods have achieved remarkable performance by focusing on model- ing temporal dependencies, using architectures such as recurrent neural networks (RNNs) (Roy
al. 2019), convolutional neural networks (CNNs) (Wang al. 2024a Lawhern al. 2018 or
Correspondence Shujun Wang (c-mail: shu-jun. www.:aagg@oolu..dd...

---

Published as a conference paper at ICLR 2026
Temporal Degendencies
Channel
Brain controls the Interaction between EEC channels
Heart controls the interaction between ECG chonnels
EEG
Channel 2
Channel 3
Channel Dependencies
(b) Channel dependencies in EEG/ECG
ECG
Takens ottention
directll interact
Token
Token
indireccly interact
Takles CoTAR
with other
Taken
using as pracy
Token
Token
Token
Token 3
(a) Temporal Dependencies Channel Dependencies (c) Interaction Attention
(d) Interaction CoTAR
Figure I: (a): Illustration of Temporal dependencies within each channel, and channel dependencies across channels. (b): Interaction between channels in EEG/ECG signals is centrally controlled by the brain/heart. (c): Attention module is a decentralized structure, where each token attends to all other tokens equally. (d): The proposed Core Token Aggregation-Redisribbbiii Redistribution (CoTAR) module operates in a centralized manner, with a core token as a proxy.
CNN-attention hybrids (Miltiadous et al., 2023a). However, each of these methods has limitations: RNNs suffer from sequential bottlenecks and difficulty capturing long-term dependencies, while CNNs are limited by local receptive fields and struggle with global temporal context. In contrast, Transformer /Vaswani al.) 2017) employs a decentralized attention mechanism, where each to- ken can directly interact with all other tokens, which enables global receptive fields, allowing it to capture long-range and complex temporal dependencies effectively (Wang et al.) [025] Qiu et al.] 2025 Liu et al. 2024a). This makes transformer-based models deliver state-of-the-e art MedTS clas- sification performance (Wang et al., 2024b; Mobin et al. (025). Despite their success in modeling temporal dependencies, Transformers face fundamental challenges when applied to modeling chan- nel dependencies in MedTS. As illustrated in Figure (b), MedTS signals typically originate from centralized biological source. For example, EEG rhythms emerge from mallamo-cortich circuits synchronizing cortical neurons into coherent scalp oscillations (Schaul 1998 Scherg et al.) 2019),
and ECG waveforms arise when impulses from the sinoatrial node propagate uniformly across the heart's conduction network (Rieta & Alcaraz 1999 AlGhatrif & Lindsay 2012). In contrast, Trans- former's attention operates a decentralized graph (Figure (c)): every token attends equally to every other token (Vaswani et al. [017 Gao et al. 2025). This uniform treatment of inter-channel interactions overlooks the inherent central coordination present in MedTS data. As result, the attention mechanism tends to dilute the principal, centrally driven patterns, such as the cardiac pace- maker rhythms, and thus fails to capture the global synchronization and unified waveform features that are essential for accurate modeling of channel dependencies in MedTS.
To address this mismatch between the centralized nature of MedTS and the decentralized structure of attention, we ask: can maintain the benefits of attention (flexible, dynamic cross-channel inter- action) while renovating it to reflect the centralized organization of MedTS? Inspired by star-shaped architectures in distributed systems, where a central server mediates all communication for improved efficiency and robustness (Roberts& & Wessler 1970 Guo et al., (211)). we propose CoTAR (Core a lightweight, MI.P-based module that seamlessly replaces the
Token Aggregation-Redistee conventional attention. Instead of pairwise token interactions, CoTAAR introduces a global core token that first aggregates information from all tokens and then redistributes it into each token, enabling centralized and flexible communication (Figure [ (d)). This architecture not only better mirrors the central coordination inherent in signals like EEG and ECG, but also reduces the computational complexity of token interaction from quadratic to linear. This shift enables significant gains in scalability and efficiency, particularly for long or high- sigh-dimensional sequences common in medical applications (Arif et al.) 2024 Jafari et al.) 2023).
With CoTAR, we propose TeCh, a unified CoTAR-based framework that adaptively captures Temporal dependencies, Channel dependencies, or both, by tuning the tokenization strategy (Tem- poral, Channel, or Dual). Such flexibility is particularly desirable in real-world medical time series, where not all datasets simultaneously exhibit strong temporal and inter-channel patterns. We con- duct extensive experiments across five MedTS datasets, including three EEG datasets and two ECG datasets. Results show that Tech not only achieves the best performance across all datasets, but also introduces significantly lower resource consumption, highlighting its superior effectiveness, efficiency, and potential for broader real-world applications.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Medical Time Series. Medical time series (MedTS) are time series data collected from the human body, used for disease diagnosis (Liu et al. 2021 Xiao et al. 2023), health monitoring (Badr et al.)
2024), and brain-computer interfaces (BCls) (Musk et al. 2019 Altaheri al.] (022)) MedTS in- clude BEG Tang et al., 2021). ECG (Xiao et al. 2023). EMG (Xiong et 2021). and EOG (Jiao al., 2020), each offering crucial information for medical applications. For example, EEG and ECG data are critical in assessing brain and heart health (Tang et al. 2021] Xiao et al. 2023). Such MedTS are characterized by temporal dependencies within each channel and channel dependencies between channels. Temporal dependencies include oscillatory rhythms and event-related potentials for EEG (Niedermeyer da Silva 2005), wave and wave for ECG (Goldberger et al. 2000). While the channel dependencies consist of functional connectivity for EEG (Stam (205). biophysi- cal geometry of the heart for ECG (Macfarlane et al. 2005). Accurate modeling these two patterns presents unique challenges. Recently, deep learning methods have significantly advanced the field of MedTS classification by providing precise temporal dependencies modeling using RNNs (Roy et al., 2019, Alhagry et al. 2017), CNNs (Lawhern et al. 2018), and Transformer (Wang et al. Mobin et al. (025). but the channel dependencies remain underexplored (Li et al. [024 Fan
2024b Mobin
al.
2025 Kim et 2025).
Transformers for Time Series. Transformer-base models have been extensively adopted for time series analysis, with growing attention to both temporal and channel dependencies. For example, Informer (Zhou et al. 2021) proposes the Temporal embedding that aggregates values across chan- nels as token model temporal dependencies. Autoformer (Wu et al. 202T) utilizes seasonal and trend decomposition to capture disentangled temporal information. PSt (Nie et al. 2023) splits the series from one channel into multiple patches, which improves the extraction of long- term temporal variations. i'rranformer (Liu et al. (024h) embeds the whole series of a channel into the Variate embedding, which maintains its complete context, thereby enhancing channel dependencies modeling. Finally, Leddam (Yu et al. 2024b) introduces a dual attention module to capture both temporal and channel dependencies.
Though the effective extraction of temporal dependencies has been addressed in MedTS using Tem- poral embedding and Transformer (Wang et al.) 2224bb] Mobin et al.) 2025). the mismatch between the current decentralized attention structure and the centrally organized MedTS fails the Trans- former in channel dependencies modeling. To address this, we propose a centralized MLP-based Core Token irrrgaation-Redistiibbtti (CoTAR) module, which delivers higher channel depen- dencies modeling ability while introducing only Linear complexity. By replacing attention using CoTAR, we propose framework that can adaptively model Temporal dependencies or Channel dependencies or both (denoted as TeCh) by tuning the tokenization strategy (Temporal, Channel, or Dual), whose effectiveness and efficiency are validated five benchmarks.
3 PRELIMINARIES
Inneeeee-nde Setting. Medical time series (MedTS) data exhibit hicrarchical structure, spanning subjects (individuals), sessions (recordings per visit), trials (repeated measurements), and samples (short segments used for diagnosis model training) (Wang al. 2024a). In clinical diag- nosis tasks, the goal is to predict disease status at the subject level using tools such as deep models trained on MedTTS samples. Independent" protocol (Wang al. 2024c b). which splits the dataset by subjects. Each subject, and all associated samples, appears exclusively in either the training, validation, or test set. This setting better reflects real-wol sul-world deployment, where models must generalize to unseen patients, therefore providing practical comparison.
ensure clinically meaningful evaluations, we adopt the 'Subject-
Problem Formulation. Consider an input MedTS sample X E TxC where I denotes the number of timestamps and C represents the number of channels. Our objective is to learn a function that can predict the corresponding label Y E R Here, K denotes the number of classes, such as various disease types different stages of one disease.
3

---

Published as a conference paper at ICLR 2026
4 METHOD
4.1 ATTENTION CORE TOKEN ACGREGAATOO-RLDUITTRIITTTO
(a) Attention
(b) CoTAR
Query
Ø(s*)
MLP
Softmax
O(S)
Muttivrriate Correlations Map
Matmul
Scale
Softmax
Key
MLP
Value
Matmul
Figure 2: Illustration of attention and Core Token Aggregation Aggregation-Rediistbtttioo (CoTAR). Attention organized in a decentralized way where each token directly interacts with all tokens, introducing a Quadratic complexity. CoTAR first aggregates core token and then redistributes it across channels to facilitate centralized channel interaction, bringing only Linear complexity.
The standard Attention. Transformer has demonstrated strong performance in many domains due to its ability to capture complex inter-token relationships, benefiting from the attention mecha- nism (Zhang et al. 2025aa 2024b, (025b).. Formally, for an input embedding O E RSxD (where S is the number of tokens and D the embedding dimension), as in Figure 2 (a), attention operates via:
Q = OWQ + by
K =WOWK + bbr ==OW, + bb
QKT
A A-Softmarl
VD VDV VV, Q.K.V.AERSXD
As mentioned before, such decentralized structure does not fit the centrally controlled MedTS data. Besides, its quadratic complexity stemmed from the matrix multiplications between Query and Key, making it inefficient for long and high-dimensional MedTS (Albuquerque al. 2019
Redistribuiion (CoTAR).. To better match the MedTS and break the scalability bottleneck of attention, we borrow insight from the star-shaped centralized system in software engineering. Traditional peer-to-pect structure lets the clients communicate directly with each other, which is time- and resource-consuming So more reliable and efficient way is server to aggregate and exchange the information between clients (Roberts & Wessler 1970 Guo al. 2019). Motivated by this, we propose the Core Token plug-in module that can seamlessly replace attention, as shown in Figure 2 (b). CoTAR first projects the token each channel, aggregates global context across channels into core vector, and redistributes it back to every token. Given input O E 5x5,, where S denotes the number of tokens and D the hidden dimension, CoTAR performs aggregation and redistribution as follows:
Core Token
Redistribution (CoTAR),
Ô GELU + W22 bR,
W1 DXD, by ERP, W2 E PxD: b2 RR.
O = Softmax(O,dim 0),
Ō RSD: Ow E RSDD.
C2 Sum(ō , Ow, dim = 0),
C, E RD:
C. Rep time- dim 0),
C, RSD.
Oco = Concat([O, C.]. dim = 1).
Oco E SX
A GEL,(Oc..Ws + by) W4 + ba:
W3 E ((D+D)) W. E DxD by E RR.
(2)
D. is the dimension of core token, C. is the obtained core token by aggregating information across all channels, and A E R S is the final output. CoTAR employs a centralized structure that first gets the global core token by aggregating information from all channels. Then the core token is redistributed into each token. This realizes an indirect interaction between channels using the core token proxy (like the brain/heart in EEG/ECG), And since each token only needs to interact with a single core token, it only brings Linear complexity. Thus, CoTAR delivers higher effectiveness with lower resource consumption.
4

---

Published as a conference paper at ICLR 2026
E E R
Temporal Embedding
M
O E R PxD
Average
Ōu. E R
LayerNorm
Feed Forward
LayerNorm
Projection
CoTAR
y Cc
H R
ch R
Average Oah E IR CxD
X RTxx
Channel Embedding
N
Figure 3: Overview of TeCh. MedTS signals X E R are embedded into Temporal embedding and Channel embedding. Then, each embedding is processed using Transformer encoders, with attention replaced by CoTAR. The final output representation from each branch is averaged across channels and added, then projected to the final predicted logits Y E RRK
4.2 OVERVIEW OF TECH
The proposed Tech framework is illustrated in Figure 3
The raw MedTS is embedded into Temporal and Channel embedding, each is processed using a set of Transformer Encoders (M for Temporal and N for Channel, M and N are tunable to match with data, and the Temporal or Channel branch will be removed if M = 0 or = 0); the learned representations are average across channels, fused and projected to the final output E K
Adaptive Dual Tokenization. Existing methods mainly rely Temporal embedding that treats single or multiple timestamps across channels as a token, favoring temporal dependencies modeling while hindering channel dependencies extraction (Yu al. 2024a Qiu et al. 2024). So we take balanced adaptive consideration of both patterns by using Adaptive Dual Tokenization.
Specifically, we form temporal token by aggregating
multiple timestamps across channels:
Ei: = vec(X((-1)L.AL,,) + be + W
i i= P, P= [T/L,, W, E LLODD E RD, Wiroos IR PxD.
where vec : mR + mR flattens a 2D tensor into a ID tensor, a predefined hyperparameter that decides the granularity, Witpos is the classical position embedding (Vaswani al. 2017). This will result in Temporal embedding E E RPxD Then, following iTransformer (Liu et al. 2024b Yu al. 2025), form token by aggregating the whole series across all timestamps of a channel:
h3:- = =X We + b W
j=1...,CC.
W. RxX b ER.. WOros R
This will result in Channel embedding H e RCDD By embedding the whole series of each channel as a token, the unique semantic information of each individual channel is well-retained. Such a channel-centric token is proven be effective in modeling multivariate correlations (Qiu et al, 2024 Wang et al. 2024d Han et al. 2024).
In the real world, not all signals simultaneously exhibit strong temporal and inter-channel patterns. Thereby, our Adaptive Dual Tokenization strategy can better match with them by tuning M and N.
Classification Paradigm. After Adaptive Dual Tokenization, the Temporal embedding F. and Chan- nel embedding H are processed using M and N standard Transformer Encoders with attention re- placed by CoTAR, respectively. Then the leamned Temporal representation Ote PxD from the Temporal embedding is averaged across channels into Ôr RRD, Similarly, the learned Channel representation Oah CCD from the Channel embedding is averaged into Och E RRP. Notably,
5

---

Published as a conference paper at ICLR 2026
if set M = or N O, this will remove the Temporal Channel branch, and Öze 0 or Och = 0. The final predicted logits obtained via:
= (Öte + Oct,) + by: Wa E DXK
ERRX
E
(5)
With the Adaptive Dual Tokenization strategy, Tech adaptively model temporal dependen- cies or channel dependencies or both, and CoTAR allows for more effective and efficient token correlation extraction. These innovations make Tech powerful, stable, and scalable framework for MedTS classification.
EXPERIMENTS
5.1 EXPERIMENT SETTING
We compare our Tech with 10 Transformer-based baselines across five MedTS datasets, including 3 EEG datasets, 2 ECG datasets. Our method is evaluated under the Subject-Indepennent setting, where training, validation, and test sets are split based on subjects. Additionally, we also conduct extensive experiments on two human activity recognition (HAR) datasets to the generalizability.
Table 1: The information of utilized datasets, including the number of subjects, samples, classes, sample channels, and timestamps (TS).
Dataset
# #-Subject
#-Sample
#-Class
#-Channel
#-TS
ADFTD
88
69,752
19
256
APAVA
23
5,967
16
256
TDBrain
6,240
256
PIB
64,356
300
PTB-XI.
17,596
191,400
12
250
HLAPP
13123
10
100
UCI-HAR
30
10,299
128
2006) is an EEG dataset where each sample is assigned binary label indicating whether the subject has Alzheimer's disease. (2) TDBrain (van Dijk et al.) 2022 is an EEG dataset with a binary label assigned each sample, indicating whether the sub- ject has Parkinson's ss disease. (3) ADFTD (Miltiadous et al.) (2233 a) is an EEG dataset with three-class label for each sample, categorizing the subject as Healthy, having Frontotemporal De- mentia, Alzheimer's disease. (4) PTB (PhysioBank 2000) is ECG dataset where each sample labeled with binary indicator of Myocardial Infarction. (5) PTB-XL (Wagner et al., 2020 is an ECG dataset with five-class label for each sample, representing various heart conditions.
Datasets. (1) APAVA (Escudero
FLAAP (Kumar& & Suresh, 2022) is a smartphone-based HAR dataset that records accelerometer and gyroscope data for activity pattern recognition. (7) UCI-HAR (Anguita al. 2013) com- prises accelcrometer and gyroscope data collected via waist-mounted smartphones, widely used for evaluating HAR models. Table provides critical information, such as subjects, channels, and timestamps. The data preprocessing and dataset split follow Medformer (Wang et al. 2024b).
Baselines. We compare with 10 cutting-edge time series Transformer-based methods: Auto- former Wu et al. 2021), FEDformer (Zhou et al. 2022). Informer (Zhou et al. 2021). iTTans-
former (Liu et al. 2024b). MTST (Zhang al. 2024a). Nonformer (Liu et al. (022). PatchTST (Nie) et al. 2023), Reformer (Kitaev et 2011,, vanilla Transformer (Vaswani et al. 2017), and Med-
former Wang al., 2024b) (state-of-the-art Transformer-based MedTSS classification model).
Implementation. We employ six evaluation metrics: accuracy, precision (macro-averaged), re- call (macro-averaged), Fl score (macro-averaged), AUROC (macro-averaged)) and AUPRC (macro- averaged). The training process is conducted with five random seeds (42-46) to compute the mean and standard deviation. All experiments are run on an NVIDIA RTX 4090 GPU. The results of all baselines the five MedTS datasets are directly taken from Medformer (Wang et al. 2024b). And the results on the two HAR datasets are reproduced using the official code from Medformer (Wang al. 2024b). We save the model with the best F1 score the validation set.

---

Published as a conference paper at ICLR 2026
Table 2: Results on five MedTS datasets. The training, validation, and test sets are distributed based on subject IDs. The best is Bolded and second is Underlined.
Datasets
Models
Accuracy
Precision
Recall
F1 score
AUROC
AUPRC
Avg
Autoformer
45.25=1148
43.67+1.4
42.96+2.8
42.59
61.02=1.2
43.10+230
16.60+1.87
FEDformer
46.30 46.30
46.0510.6
44.22+1.38
43.91
62.62=1.75
46.11
48.70 48.70
Informer
48.4511.66
161541
46.0611.84
45.74
65.87
47.6011.00
S0.21
(Transformer
52.60=11.9
46.79
47.28+1.20
46.79=1.13
67.26=1116
49.53+1.1
21.38+1..77
ADFTD (3-Clasees)
MIST
45.601200
44. 14.701133
45.0511.30
44.31
44.3
62.501o.1
45.161035
47.3911.19
Nonformer
49.95=1.05
47.71
47.46211.0
16.96=11335
66,23+1137
47.33+1.88
$0.61
PatchTST
44.37=0.95
42,40+1.13
42.06+1448
41.97+1.37
60.08+1.50
42.49+1.79
45.73++1.7
Reformer
50.78=1.17
49,64+149
49
49.89
47.94=.0.9
69.17
51.73+194
$1.69
Transformer
50.47=2.14
19.13+1133
48.01
48.09:1.59
67.93=1.59
48.93+2.00
50,26+1
Medformer
53.27=1.54
51.02+157
10.71
1..65
51.21+1332 70.93=1.19
1..80
TeCh
54.54.0..0
53.02..0.7
49.25
48.84
68.67
50.6211326
50.82
Autoformer
68.6411.82
68 68.4812.10
68.7 68.77
68 1.99
75.94
74, 38
70.724268
FEDformer
74.94=2.15
74.59+1.5
73.56+33.55
+3.51
83.72=1.97
82. 42.94
77.21 +:249
Informer
73.11::4.40
15.17
69.17
69 4:5006
70.46
75.75
7102
Transformer
74.5511.6
77..10
71.7611.72
12.30
85.5911555
84. 17
76.401177
APAVA
MIST
71.14=1.99
79.30
65.27
64.01
68.87=2.34
71,06+1.0
69.10
(2 Classes)
Nonformer
71.891381
71 8014.8
69 143.56
69. 19.74
70.55
70.781408
70.031330
PatchTST
67.03=1165
78.7611128
59.91
55.97
65.65=0018
67.9910.6
65,22+118
Reformer
78.70=2.0
15
16.00+1611
75.93+1.82
13.94=1440
16.04+1114
77.52+2.32
Transformer
76.3014472
77.641596
73.0915.01
73.7515.38
72.50:660
73.231780
74.421504
Medformer
78.74=064
81. +0.84
75, 40
23.20=0001
83.66-+0922
79.06+0
TeCh
86.86=1.09
16.85
16:10
86030=1006 1,06
94.02=0.52
93.79
18.
Autoformer
87.3313.99
88, 36.35.63
87.3313779
87.2613884
93.8
93,321242
89.02.13:28
FEDiormer
78.13=1.98
18.52+1.91
78.1 1
78.04+2001
86.56=196
18+49+1.99
80.8 80.81+1.79 11779
Informer
89.02=2.50
89,43+2.34
89.02+2.00
88.98+254
96,64=0.68
75.75
91.81+167
iTransformer
74.67
74.71
7..67
74.65
83.37
83. 83.7311277
1..1
TDBrain
MIST
76.96=3.76
77.24+33.9
76. 36
76. 3883
85.27=4446
82.81+364
+9.85+3306
(2-Classes)
Nonformer
87.88=244
88, 8.:11.4444
87, 88.844
87.78=2.56
97.05=008
96. 96.99
91.74+1662
PatchTST
79.25
79.2513.99
79, 79,6014.09
79 19.2513.79
79 201
87.951496
86. 86.3616677
81.601400
Reformer
87.92:2.01
88. 14
87.92+2.01
87.85
96.30:0054
90.02 46:401445
Transformer
87.17
87. 17.99
87.17
87 1.61
96.2810092
96. 3410.81
89,68 19,681148
Medformer
89.622081
89.6810.78
89.6 89.621081
89. 29.6221811
96.41
96. 51.31
90.81 10060
TeCh
93.21:0061
93.39
93.21
93.20=0611
98.68
98.72+
95.07
Autoformer
73.35
12.11
63.2443.17
63.6913.84
18.54
15335
69 69.003.3.33
FEDiormer
76.0512254
77.581361
1013.55
67. 14..3
85.93144M
82.59 12.542
36.401330
Informer
78.69=168
82. 1.00
69.19+
70.84=147
92.09=0.5
90.02+040
10.45
(Transformer
83.89=0.71
88. 88.25+1.J8
76.39+1.0
1..06=106
91.18=1.16
90 9340m
PTB (2-Classes)
MIST
76.59=1.00
79. 18:0
66.31
67 67.38:177
86.8622.75
85,75+284
76. 13
Nonformer
78.66=0.49
82.77+0.66
69.12+0.87
10.90=1.0
89.37+2.51
86.67+2.38
79.75+125
PatchTST
74.74=1.4
76.94+1.51
63.89+2.71
64.36+3.3M
88.79=00.1
83, 23.39
75.02+168
Reformer
77.9612.13
81.721161
68. 68.2013.35
69.65 69.6513.88
91.1310.74
88 88.4211130
79, 19.1812.17
Transformer
77.37=1.02
81.8440-666
67.14+1.80
68.4742119
90.08=1.76
87.22+168
15.52+135
Medformer
83.50-2.01
85. 15.194
77.11
79.1813.11
92.81
90.3211.54
84.02.11.4
TeCh
85.9612.52
89.9210.74
79.4314113
81.97
94.57
94,3610M
89.9412.37
Autoformer
61.68=2.72
51.60+164
49.10=1.52
48.85+227
82.04=1144
57.53+155 51.93+1.71
FEDiormer
57.2019.47
32.3816.09
49.0417.26
49,04
47 47.891844
82.1324.17
52.31
86.831708
Informer
71.4310.30
10.64
$9, 59.12.1047
60 14.44
88.65100.0
66.7610.17
67. 14.35
iTransformer
69.28=:0.2
SU, 59:4444
54.62+00.18
56 56.20
86.71=0.10
60.27+0021
64 4::423
PTB-XL (5-Classes)
MIST
72.1410.27
63.8410.72
60.0110..1
61.4310.3
18.97
65.831051
68. 70..00
Nonformer
70.56:0.55
61.57:0066
57.75+0.72
59.102006
88.32220.6
63.4010.79
66.781002
PatchTST
73.23=0025
65.70+004
60.82
62.61=00.4
89.74=0.19
67.32+0022
69. 49
Reformer
71.72=0.43
63.12+1.00
59, 59.20+a075
60.69=0..8
88, 88,80=0024
64.72+0.7
68.04+0552
Transformer
70.59=0044
61.57+0055
$7.62+0.35
59.05=0025
18.21
63., 36.3---.229
66.73+0036
Medformer
72.87=0.23
64.144042
60.60+0.46
62.02=0.7
89.66=0.13
66.39+0.2
69 20.28+0.30
TeCh
73.53
65.921..52
60.6110.5
62.441027
90.03.0.12
69.191025
69.95.0..0
5.2
MAIN RESULT
the results under the Subject-Independent setup. Our Tech consistently outperforms Medformer (the previous state-of-the-art) across all six metrics on four datasets, achieving up to 12.13% improvement in the average of all metrics on the APAVA dataset. Even in the challenging case of ADFTD, Tech remains comparable to Medformer (Avg: 51.80 vs. 50.82). Aggregated across all six metries on these five MedTS datasets, Tech achieves an overall 4.59% % performance gain over Medformer, which is remarkable. In Table Tech substantially outperforms Medformer across all metrics on both datasets, with an average improvement of 4.23% Since HAR tasks involve multi-sensor channels and fine-grained activity classes, these consistent and significant gains indicate that Tech generalizes better to noisy, high-variation, multi-channel time series inputs. In terms of robustness, Tech also outperforms Medformer, reflected in the lower average std across all datasets (0.86 vs. 0.96, a 10.42 % reduction). These results demonstrate that Tech is both more effective and more robust than Medformer.
7

---

Published as a conference paper at ICLR 2026
Table Results of two HAR datasets. To evaluate the performance of our method on general time series, we test it on two human activity recognition (HAR) datasets: FLAAP and UCI-HAR, which exhibit potential channel correlations inherently. The best Bolded and second is Underlined.
Datasets
Models
Accuracy
Precision
Recall
score
AUROC
AUPRC
Avg
Autoformer
38.93=1.01
38.22+1.31
37 17.40=111
33.5
74.12=035
35.77
42.90 22.99
FEDformer
19.5
59.84+8.0
58.57#897
57.73+9.9
89.75+537
60.88=961
64.3822852
Informer
72.87=0.99
73.20=0.7
72.76=n92
72.59+096
#029
77.57=121
77.48+0.36
i'rrnsformer
75.15=048
75.09=0.5
75.1442447
74.91
96.642214
80.81
79.62:0.6
FLAAP (10-Classes)
MIST
70.57=0.54
71.090033
70.97+073
70.61
94.56+0118
73.28+0.99
75.18+0.22
Nonformer
7..85
75.1921377
74.51
74.39
96.4322077
79. 19.2292180
19
PatchTST
56.3410.31
36.3610065
35.2910032
55.581045
89.24110.1
58.92.10.6
61.951.0.6
Reformer
71.13
71.20
70.57:1166
70.541179
95. 95.161042
73.8012200
15.40
Transformer
76.36
76.53:125
76.23-i095
76.05
96, 26.6510.11
80.70 80.70
80.42.:.00
Medformer
76.441004
76, 76.6
76,6311.3
76, 15065
95 44-10.2
81. 12.100
80.4 80.4
TeCh
80.60=0.30
80.29
80.36:003
80.23
97.67
86.18:99.1
84.22::0.55
Autoformer
41.86:246
49.62=115 48
44.30+255
32.69
83.72
58 -8.56624477
51.79
FEDformer
76.8919.9
75 66.66.19.4
77.561979
75.03
95. 95.161466
83.28
80.37
Informer
88.33
88. 28:22
28.47
88 88.20
98. 362001
94. 20:2033
89.8 10077
Trannformer
92.41
92.24 241061
92.331207
92.391064
99.07 10007
10,01 10.29
03.7440.477
UCI-HAR
MTST
90.99
90.96.10.9
90.921008
90.831.088
98.21 20.11
14
93.17.1051
(6-Classes)
Nonformer
91.0420.58
90.98.0.00
91.14:0366
91.01
99.02
96.07
93.3 93.37
PatchTST
87.67.0.19
88, 10.45
87.97
88.021038
98. 50.00
93.86.10,4
90.5611034
Reformer
88.70=1.14
88. 1.:
88,82+1.13
88. 88.59
28.6684026
94.60=1.7
91.53++.50
Transformer
89.36=1.74
89 1.33
89.49+119
89.33+1175
98. 98.87=023
95, 85.58+0088
91.83+1.13
Medformer
89.62=0051
89.70+0.18
89.80=014
89.62+081
00,80=0.7
91.61+045
TeCh
94.15:0.96
94.27=nm.
94.30=a97
94.262098
99.32:1005
96.74
95.02
5.3
ABLATION STUDY
Model Efficiency Analysis. Since CoTAR introduces only Linear complexity compared to the Quadratic complexity of attention, our Tech achieves higher performance with significantly lower resource consumption, as in (a) Compared to Medformer, Tech delivers 8% better accuracy while using just 33% of the memory usage and 20% of the inference time.
Channel CoTAR
Temporal CoTAR
89.0
900 MB
Channel Attention
Temporal Attention
(Gurs)
85.0
84.0
Medi
75.0
Accuracy
79.0
F1-Score
Transt
FEDR
65.0
74.0
MTST
Ink
69.0
55.0
PatchTST
Autor
64.0
15
115
215
315
415
45.0
0
4
8
12
16
20
Inference Time (ms)
Noise Level
(a) Efficiency analysis on APAVA
(b) Robust Test on PTB
Figure 4: (a): Efficiency and Effectiveness analysis of TeCh and other baselines on APAVA dataset 128. '#' stands for "former' to save space. (b): Robustness of attention
with batch size and CoTAR to noise when using Channel Temporal embedding. We consistently increase the intensity (the standard deviation) of Gaussian random noise from 0.0 to 20.0 on the last channel of the PTB dataset. F1-Score is used quantify the change.
Robustness Analysis. To test the robustness of attention and CoTAR, we introduce noise progres- sively during training by adding perturbations to the last channel of the PTB dataset. This formulated as X.c = X..c c + 3 noise, where X.,, X.C E RIxT is the last channel, noise E 1 is Gaus- sian noise with mean and standard deviation 1, B E RR. controls the noise intensity. Then, the processed sample X, c embedded into Channel embedding (Liu et al. 2024b) or Temporal em- bedding (Wang et al. 2024b). Figure (b) reveals that attention is highly sensitive to noise. This is because attention is a decentralized structure, which means each channel can be directly influenced by the corrupted, noisy channel. In contrast, CoTAR employed a centralized strategy, which pre- vents the noisy channel from directly interfering with others, therefore enhancing the robustness to noise. Meanwhile, compared to Temporal embedding, which is more common practice in previous
8

---

Published as a conference paper at ICLR 2026
Table 4: Ablation result of the proposed Dual Tokenization strategy, We include a general Human Activity dataset, UCI-HAR, to test its generalizability. (i) w/o: No tokenization is performed and directly uses the raw series as input-withood representation learning, single linear projection as classifier. (ii) Temporal: Only Temporal embedding is used. (iii) Channel: Only Channel embedding
used. (iv) Dual: Both Temporal and Channel embedding are used. The best is Bolded
ADFTD
APAVA
TDBrain
PTB
UCI-HAR
Accuracy
F1-Score Accuracy F1-Score
Accuracy F1-Score
Accuracy F1-Score Accuracy F F1-Score
w/o Temporal
33,79_004
32.67 37.67
50.68100
50.13
53,77a120 23.79.1
72.62
24 54.2220.7
51.72.
53.78-0.22
49.10=1m
53.71= 55.93 06
93.21 :0 93.20:001
74.74 85.96-255
62.90a114 91.562003 91 91.522062
47.06-135
32.92-030 75.68
73.54-249
07.58=1.04
67.5 67.54
81.97440 02.981041 23.00=044
Both
04.54-070
48.84
86.862109
86.30-100
89.79:006
89.77+097
84.15=206
79.11 94.15+036
94,26:4.90
Table 5: Ablation result of the proposed 'Core Token Aggregate-Reddisrrbbuu (CoTAR) module. (i)
w/o: No Token interaction is performed, which means directly removing the CoTAR module. (ii)
Attention: Replacing CoTAR with the Attention module. (iii) CoTAR: bascline with the CoTAR module. The best is Bolded.
ADFTD
APAVA
TDBrain
PTB
UCI-HAR Accuracy F1-Score
Accuracy FI-Score Accuracy
FI-Score
Accuracy FI-Score Accuracy F1-Score
w/o Attention 52.77 48.65+12:
53. 33.324007 47.262051
83.31 20.95
81 99
92. 22.69.075
92.67a066 85. 22 80.82 92.4010 92.55 92.55a001
83.42
82. +2.09
90.40-2.1 90.35+223
85.74 81.93122
03.13 93.21 =0,0
CoTAR 54.54:4m0 48.84
86.86-1.9
16.30+1006
93.21=041
93.20=as1
85,96=252 81.97=40 94.15=066 94,26=0.98
work (Mobin et al. 2025 Wang et al. 2024b), Channel embedding delivers higher robustness and
classification performance. This aligns with general time series analysis, where Channel embedding is more suitable for modeling channel dependencies, as it can better preserve the unique context of each channel, even when noise is entangled (Liu et al. 2024b Wang al. 2024d).
Ablation Study on 'Adaptive Dual Tokenization'. The results in Table 4 demonstrate the ef- fectiveness of the proposed Adaptive Dual Tokenization design. When skipping the representation leamingg phase (the w/o setting), the performance significantly deteriorates across all datasets, high- lighting the necessity of structured token embedding. Temporal tokenization excels on TDBrain, while Channel tokenization excels on PTB. And combining both yields an 11% improvement of Accuracy and 13% improvement of F1 F1-Score on APAVA. Moreover, Dual tokenization also excels on the UCI-HAR dataset, a well-known benchmark for Human Activity (HAR) tasks. Since HAR tasks involve multi-sensor channels and fine-grained activity classes, the significant gains of Dual Tokenization indicate that by simultaneously capturing both patterns, Tech can generalize to noisy, high-variaten variation,,,,, multi-channel time series. These findings confirm that the Adaptive Dual Tokeniza- tion strategy enables Tech to better align with the unique characteristics of each dataset, providing more versatile modeling of Temporal dependencies Channel dependencies, both.
Ablation Study on 'Core Token Eegregate-Redistrinng
Table 5 provides comprehensive (Co-
ablation study validating the effectiveness of the proposed Core Token
TAR) module, which yields consistent performance gains across all five datasets and both metrics. Moreover, CoTAR also demonstrates competitive or lower standard deviations, indicating higher robustness. These results suggest that CoTAR not only captures richer inter-token dependencies through core-token centric redistribution but also leads to more stable and generalizable representa- tions, thereby justifying its architectural necessity.
5.4 VISUALIZATION OF CORE TOKEN
In Figure 5 we visualized the core token generated by CoTAR and other embeddings across both temporal and channel spaces. Interestingly, in both embedding spaces, the core token consistently occupies central position, suggesting that it captures latent global physiological state integrating information across sensors (channel dimension) and across time (temporal dimension).
In the temporal space, this behavior reflects cross-temporal integration, which aggregates patterns over time into a stable representation of the system's evolving state. For EEG, such temporal ag- gregation resembles slow cortical dynamics, in which distributed neuronal populations maintain low-frequency coherence (e.g.. alpha or beta bands) to stabilize perception and working working-memory memory states (Niedermeyer & da Silva, 2005; Buzsáki, 2006; [Scherg et al. 2019). For ECG, it parallels the beat-to-ber coordination within the cardiac cycle: the sinus node's 's rhythmic discharge orchestrates each P-QRS-T sequence, and the consistent temporal integration of these cycles ensures stable and

---

Published as a conference paper at ICLR 2026
ADFTD
APAVA
PTB
Temporal
Channel
Core Token
Others
Figure 5: T-SNE visualization of the core token generated by CoTAR and other tokens. We visualize the embedding space of both temporal and channel.
regular cardiac pacing (AIGhatrif & Lindsay 2012 Goldberger et al. 2000). Thus, the core token can be interpreted latent summary of temporal coherence both neural and cardiac dynamics.
In the channel space, such centralization mirrors spatial integration across sensors. For EEG, this aligns with the global workspace and hub-based integration observed in frontoparietal networks that unify activity from distributed cortical regions (Dehaene & Changeux, 2011 Sporns 2010). For ECG, reflects pacemaker synchronization across myocardial conduction pathways, where central excitation orchestrates coherent activation throughout the heart (Rieta & Alcaraz, 1999. AlGhatrif
Lindsay 2012).
Together, these observations indicate that CoTAR's centralized proxy learns physiologically inter- pretable representations of both temporal and spatial coordination, effectively mirroring the central- ized integration mechanisms that underlic real biological systems. We posit that such centralized architecture could also inform the modeling of signals originating from centralized sources, e.g.,
fMRI (Xun et al., 2025).
CONCLUSION
Existing Transformer models suffer from the mismatch between the centralized nature of medical time series (MedTS) and the decentralized structure of the attention module. This work proposes the Core Token Aggregation- Redistibi (CoTAR) module, which models inter-token relationships in centralized way using a core token as a proxy, to replace attention seamlessly. Beyond being more effective in channel dependencies modeling. also reduces complexity from quadratic to linear. Based on CoTAR, our Tech framework can adaptively capture temporal dependencies or channel dependencies,
both, and achieves superior performance and efficiency on three EEG and two ECG datasets. This work demonstrates the effectiveness of introducing domain -secifi inductive biases into deep learning architectures MedTS analysis and paves way for more effective and scalable solutions.
7
ACKNOWLED
This work was partially supported by the Research Grants Council (RGC) of Hong Kong under the Collaborative Research Fund (CRF) (No. C5055-24G), the Start-up Fund of The Hong Kong Poly- technic University (No. P0045999). the Seed Fund of the Research Institute for Smart Ageing (No. P0050946), the Tsinghua-Poly Joint Research Initiative Fund (No. P0056509), and the University Grants Committee (UGC) funding of The Hong Kong Polytechnic University (No. P0053716).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Isabela A Albuquerque, João Monteiro, Olivier Rosanne, Abhishek Tiwari, Jean-Franngee Gagnon, and Tiago H Falk. Cross-subject statistical shift estimation for generalized electrencephalograply based mental workload assessment. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 3647-3653. IEEE, 2019.
Majd AlGhatrif and Joseph Lindsay. A brief review: history to understand fundamentals of electro- cardiography. Journal of community hospital internal medicine perspectives, 2(1):14383, 2012.
Salma Alhagry, Aly Aly Fahmy, and Reda A El-Khoribi. Emotion recognition based eeg using Istm recurrent neural network. International Journal of Advanced Computer Science and Appli- cations, 8(10), 2017.
Hamdi Altaheri, Ghulam Muhammad, Mansour Alsulaiman, Syed Umar Amin, Ghadir Ali Altuwai- jri, Wadood Abdul, Mohamed A Bencherif, and Mohammed Faisal. Deep learning techniques for classification of electroenceehaloggan (eeg) motor imagery (mi) signals: A review. Neural Com- puting and Applications, 35(20): 14681-1472222 2023.
Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, Jorge Luis Reyes- et al. A public domain dataset for human activity recognition using smartphones. Esann, volume pp. 3, 2013.
Aniqa Arif, Yihe Wang, Rui Yin, Xiang Zhang, and Ahmed Helmy. Ef-net: Mental state recognition by analyzing multimodal eeg-fnirs via cnn. Sensors, 24(6): 2024.
Yara Badr, Usman Tariq, Fares Al-Shargie, Fabio Babiloni, Fadwa Al Mughairbi, and Hasan Al- Nashash. A review on evaluating mental stress by deep learning using eeg signals. Neural Com- puting and Applications, pp. 1-26. 2024.
György Buzsáki. Rhythms of the Brain. Oxford University Press, 2006.
Stanislas Dehaene and Jean-Pierre Changeux. The global neuronal workspace model conscious access: from neuronal architectures to clinical applications. Neuron, 69(2):200-213, 2011.
J Escudero, Daniel Abásolo, Roberto Hornero, Pedro Espino, and Miguel López. Analysis of elec- troencephalograms in alzheimer's disease patients with multiscale entropy. Physiological mea- surement, 27(11):100) 2006.
Wei Fan, Jingru Fei, Dingyu Guo, Kun Yi, Xiaozhuang Song, Haolong Xiang, Hangting Ye, and Min Li. Towards multi-resolution spatiotemporal graph learning for medical time series classification. In Proceedings of the ACM on Web Conference 2025, pp. 5054-5064, 2025.
Xianqiang Gao, Pingrui Zhang, Delin Qu, Dong Wang, Zhigang Wang, Yan Ding, and Bin Zhao Learning 2d invariant affordance knowledge for 3d affordance grounding. In AAAI, volume 39, 2025.
Ary L. Goldberger, Luis AN Amaral, Leon Glass, Jeffrey Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 101(23)=e215-e22) 2000.
Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, and Zheng Zhang. Star- transformer. arXiv preprint arXiv: 1902.099000 09113, 2019.
Lu Han, Xu- Yang Chen, Han-Jia Ye, and De-Chuan Zhan. Softs: Efficient multivariate time series forecasting with series-core fusion. arXiv preprint arXiv:2404.14197, 2024. URL https://
StXiv..or3/bbs/4444.122
Mahboobeh Jafari, Afshin Shocibi, Marjane Khodatars, Sara Bagherzadeh, Ahmad Shalbaf, David López García, Juan M Gorriz, and Rajendra Acharya. Emotion recognition in eeg signals using deep learning methods: A review. Computers in Biology and Medicine, pp. 107450, 2023.

---

Published as a conference paper at ICLR 2026
Yingying Jiao, Yini Deng, Yun Luo, and Bao-Liang Lu. Driver sleepiness detection from eeg and eog signals using gan and Istm networks. Neurocomputing. 408: 11101111111111111111111111111111 2020.
lan T. Jolliffe and Jorge Cadima. Principal component analysis: a review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sci- ences, $74(2965):20150200) 2016.
Jongseon Kim, Hyungjoon Kim, HyunGi Kim, Dongjun Lee, and Sungroh Yoon. A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges. Artificial Intelligence Review, 5 1075, 2025.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In ICLR, 2019.
Prabhat Kumar and $ Suresh. Flaap: An open human activity recognition (har) dataset for learning and finding the associated activity patterns. Procedia Computer Science, 212:64-73, 2022.
Vernon J Lawhern, Amelia Solon, Nicholas R Waytowich, Stephen M Gordon, Chou Hung, and Brent J Lance. Eegnet: compact convolutional neural network for eeg-based brain-con computer interfaces. Journal of neural engineering, 1565)0056013, 2018.
Huayu Li, Ana S Carreon- Casreon-Rascon, Xiwen Chen, Geng Yuan, and Ao Li. Mts-lof: medical time- series representation learning via ocelusion-invariant features. IEEE Journal of Biomedical and Health Informatics, 2024.
Xinwen Liu, Huan Wang, Zongjin Li, and Lang Qin. Deep learning in ecg diagnosis: A review. Knowledge- Based Systems, 227:107187, 2021.
Yangcen Liu, Ziyi Liu, Yuanhao Zhai, Wen Li, David Doerman, and Junsong Yuan. Stat: Towards generalizable temporal action localization. arXiv preprint arXiv:2404.13311, 2024a.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. NewrIPS, 15:9881-9899 2022.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long- itransformer: Inverted transformers are effective for time series forecasting. ICLR, 2024b.
Jiecheng Lu, Xu Han, Yan Sun, and Shihao Yang. CATS: Enhancing multivariate time series fore- casting by constructing auxiliary time series as exogenous variables. In Forty-first International Conference on Machine Learning, 2024. URL httpss://openrevitt
net / forum? ?id
11DAGDeOUR
PW Macfarlane, B Devine, and E Clark. The university of glasgow (uni-g) ecg analysis program. In Computers in Cardiology, 2005, pp. 451 454.. IEEE, 2005.
Andreas Miltiadous, Emmanouil Gionanidis, Katerina Tzimourta, Nikolaos Giannakeas, and Alexandros T Tzallas. Dice-net: a novel convolution-traansforne architecture for alzheimer de- tection in eeg signals. IEEE Access, 2023a.
Andreas Miltiadous, Katerina D Tzimourta, Theodora Afrantou,, Panagiotis Ioannidis, Nikolaos Grigoriadis, Dimitrios G Tsalikakis, Pantelis Angelidis, Markos G Tsipouras, Euripidis Glavas, Nikolaos Giannakeas, et al. A dataset of scalp eeg recordings of alzheimer's 's disease, frontotem- poral dementia and healthy subjects from routine eeg. Data, 8(6):95, 2023b.
Md Kamrujjaman Mobin, Md Saiful Islam, Sadik Al Barid, and Md Masum. Cardioformer: Advancing ai in ecg analysis with multi-granularity patching and resnet. arXiv preprint arXiv:2505.055388 2025.
Fatma Murat, Ozal Yildirim, Muhammed Talo, Ulas Baran Baloglu, Yakup Demir, and U Rajendra Acharya. Application of deep learning techniques for heartbeats detection using ecg signals- analysis and review. Computers in biology and medicine, 120:103726 2020.
Elon Musk et al. An integrated brain-machine machine interface platform with thousands of channels. Jour- nal of medical Internet research, 2111))4161 2019.
12

---

Published as a conference paper at ICLR 2026
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.
Ernst Niedermeyer and FH Lopes da Silva. Electroencephaalggaapyy basic principles, clinical applications, and related fields. Lippincout Williams & Wilkins, 2005.
PhysioToollkit PhysioBank. Physionet: components of new research resource for complex physio- logic signals. Circulation, 101(23)e215-e2220 2000.
Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duct: Dual clustering enhanced multivariate time series forecasting, arXiv preprint arXiv:24/2. 10859, 2024.
Xiangfei Qiu, Hanyin Cheng, Xingjann Wu, Jilin Hu, Chenjuan Guo, and Bin Yang. A comprehen- sive survey of deep learning for multivariate time series forecasting: A channel strategy perspec- tive. arXiv preprint arXiv:2502.10721. 2025.
José Rieta and Raál Alcaraz. The genesis of the electrocardiogram (ecg). Wiley Encyclopedia of Electrical and Electronics Engineering, pp. 1-15, 1999.
Lawrence Roberts and Barry D Wessler. Computer network development to achieve resource sharing. In Proceedings of the May 5-7, 1970, spring joint computer conference, pp. 543-549, 1970.
Yannick Roy. Hubert Banville, Isabela Albuquerque, Alexandre Gramfort, Tiago H Falk, and Joce- lyn Faubert. Deep learning-basse electroenceepallgraphh analysis: a systematic review. Jour- nal of Neural Engineering, (655)051001, aug 2019. doi: 10.
URL
https: //tp//xxxxp/ www.xxoi.oooooooooooooooooo.oo.. owt. 108801008811111 21172411125222221 ab266c
Neil Schaul. The fundamental neural mechanisms of electreencephalography. Electroencephalog
raphy and clinical neurophysiology. 106(2):011-107, 1998.
Michael Scherg, Patrick Berg, Nobukazu Nakasato, and Sándor Beniczky. Taking the eeg back into the brain: the power of multiple discrete sources. Frontiers in neurology, 10:855, 2019.
Anil K. Seth, Adam B. Barrett, and Lionel Barnett. Granger causality analysis in neuroscience and neuroimaging. Journal of Neuroscience, 35(8):3293-3297 2015.
Olaf Sporns. Networks of the brain. MIT Press, 2010.
Comelis Stam. Nonlinear dynamical analysis of eeg and meg: review of an emerging field. Clinical neurophysiology. 116(10):226622))) 22016,221122200 2005.
Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang, Qianying Huang, Florian Dubost, Daniel Rubin, and Christopher Lee-Messer. Self-supervised graph neural networks for improved electreencephalographic seizure analysis. In ICLR, 2021.
Thomas W. Valente, Katie Coronges, Cynthia Lakon, and Elizabeth Costenbader. How correlated are network centrality measures? Connections, 28(1):16-26, 2008
Hanneke van Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns. The two decades brainclinics research archive for insights in neurophysiology (tdbrain) database. Scientific data, 9()):333, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention all you need. NewrlPS, 30, 2017.
Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima Lunze, Wojeiech Samek, and Tobias Schaeffter. Ptb-xl, large publicly available electrocardiography dataset. Scientific data 711):1-15, 2020.
Juncheng Wang, Chao Xu, Cheng Yu, Zhe Hu, Haoyu Xie, Guoqi Yu, Lei Shang, and Shujun Wang. Language model based text-to-audio generation: Anti-causally aligned collaborative residual transformers. In EMNLP, 2025. URL
net ffrrmmidd
zFfol r&Cmel
13

---

Published as a conference paper at ICLR 2026
Yihe Wang, Yu Han, Haishuai Wang, and Xiang Zhang. Contrast everything: A hierarchical con- trastive framework for medical time-series. NeurIPS, 36, 2024a.
Yihe Wang, Nan Huang, Taida Li, Yujun Yan, and Xiang Zhang. Medformer: A mulli-granularity patching transformer for medical time-series classification. In NeurIPS, 2024b. URL https:
//openreview.. net www.ooodddiiiiiiiieiiieeei
Yihe Wang, Taida Li, Yujun Yan, Wenzhan Song, and Xiang Zhang. How to evaluate your medical time series classification? arXiv preprint arXiv:2410.0305/ 2024c.
Yuxuan Wang, Haixu Wu, Jiaxiang Dong. Yong Liu, Mingsheng Long, and Jianmin Wang. Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.13278 2024d. URL
http://axxxvvv.g//aa... wwwpg//bbb///e 13278
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jian- min Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. In The Thirty-eighth Annual Conference Neural Information Pro-
cessing Systems, 2024e. URL http///pppe///ggeeeeeeeeeeeeeeee
Zekai Wang, Stavros Stavrakis, and Bing Yao. Hierarchical deep learning with generative adversarial network for automatic cardiac diagnosis from eeg signals. Computers in Biology and Medicine, 155:106641, 2023.
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. NeurlPS, 11:22419-224300 2021.
Jingyun Xiao, Ran Liu, and Eva L Dyer. GAFormer: Enhancing timeseries transformers through group-aware embeddings. In The Twelfth International Conference on Learning Representations,
Qiao Xiao, Khuan Lee, Siti Aisah Mokhtar, Iskasymar Ismail, Ahmad Luqman bin Md Pauzi, Qiuxia Zhang, and Poh Ying Lim. Deep learning-base based ecg arrhythmia classification: systematic review. Applied Sciences, 13(8):4964, 2023.
Dezhen Xiong. Daohui Zhang, Xingang Zhao, and Yiwen Zhao. Deep learning for emg-based human-machine interaction: A review. IEEE/CAA Journal of Automatica Sinica, 8(3):512-533, 2021.
Yue Xun, Jiaxing Xu, Wenbo Gao, Chen Yang, and Shujun Wang. Ada-fen: Adaptive frequeecy- coupled network for fmri-based brain disorder classification. In MICCAI, pp. 35-45. Springer, 2025.
Guoqi Yu, Yaoming Li, Xiaoyu Guo, Dayu Wang, Zirui Liu, Shujun Wang, and Tong Yang. Lino: Advancing recursive residual decomposition of linear and nonlinear patterns for robust time series forecasting. arXiv preprint arXiv:2410.17159, 2024a.
Guoqi Yu, Jing Zou, Xiaowei Hu, Angelica Aviles-Rivero, Jing Qin, and Shujun Wang. Revitaliz- ing multivariate time series forecasting: Learnable decomposition with inter-series dependencies and intra-series variations modeling. In ICML, 2024b. URL httph: htttps://eperreiiiiiii .et.teeeeeewwweeee//eeee.....
Guoqi Yu, Yaoming Li, Juncheng Wang, Xiaoyu Guo, Angelica I Aviles-Rivers Tong Yang, and Shujun Wang. Refocus: Reinforcing mid-frequenee and rey-frequency modeling for multivariate time series forecasting. arXiv preprint arXiv:2502. 0225600.668990 16890, 2025.
Pingrui Zhang, Yifei Su, Pengyuan Wu, Dong An, Li Zhang, Zhigang Wang, Dong Wang, Yan Ding, Bin Zhao, and Xuelong Li. Cross from left to right brain: Adaptive text dreamer for vision-and- language navigation. arXiv preprint arXiv:2505. 20897, 2025a.
Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised con- trastive pre-training for time series via time frequeeyyy consistency. NeurIPS, 35:3988-400 4003, 2022.
14

---

Published as a conference paper at ICLR 2026
Yitian Zhang. Liheng Ma, Soumyasundar Pal, Yingxue Zhang, and Mark Coates. Multi-resolution time-series transformer for long- Bang-term forecasting. In International Conference on Artificial Intel- ligence and Statistics, pp. 4222 4230. PMLR, 2024a.
Yu Zhang, Kehai Chen, Xuefeng Bai, Zhao Kang, Quanjiang Guo, and Min Zhang, Question-guided knowledge graph re-scoring and injection for knowledge graph question answering. In EMNLP, pp. 8972- 8985, 2024b.
Yu Zhang. Jinlong Ma, Yongshuai Hou, Xuefeng Bai, Kehai Chen, Yang Xiang, Jun Yu, and Min Zhang. Evaluating and steering modality preferences in multimodal large language model. arXiv preprint arXiv:2505. 20977 2025b.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, vol- ume 35, pp. 11106-11115, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. ICML, pp. PMLR, 2022.
15

---

Published as a conference paper at ICLR 2026
A DATA AUGMENTATION BANKS
In the embedding stage, we apply data augmentation to the input time series. We utilize bank of data augmentation techniques to enhance the model's robustness and generalization. During the forward pass in training, each time series will pick one augmentation from available augmentation options with equal probability. The data augmentation methods include temporal flipping, channel shuffling, temporal masking, frequency masking, matting.jittering. and dropout, and can be further expanded to include more choices. We provide a detailed description of each technique below.
Temporal Flippling We reverse the MedTS data along the temporal dimension. The probability of applying this augmentation is controlled by a parameter prob, with a default value of 0.5.
Channel Shuffling We randomly shuffle the order of MedTS channels. The probability of applying channel shuffling is controlled by the parameter prob, also set by default to 0.5.
temporal masking We randomly mask some timestamps across all channels. The proportion of timestamps masked is controlled by the parameter ratio, with a default value of 0.1.
Frequency Masking First introduced in (Zhang al., 2022) for contrastive learning, this method involves converting the MedTS data into the frequency domain, randomly masking some frequency bands, and then converting it back. The proportion of frequency bands masked controlled by the parameter ratio, with default value of 0.1.
Jittering Random noise, ranging from 0 to 1, is added to the raw data. The intensity of the noise is adjusted by the parameter scale, which is set by default to 0.1.
Dropout Similar the dropout layer in neural networks, this method randomly drops some values. The proportion of values dropped is controlled by the parameter ratio, with default setting of 0.1.
B DATA PREPROCESSING
We obtain all the well-preproceessed datasets from Medformer (Wang et al. 2024b) (https:// wwthub.cc./Diiii... wwwm:/D...mllaalttt/aaarrreeee Thanks for their brilliant work.
B. APAVA PREPROCESSING
The Alzheimer's 's Patients' Relatives Association of Valladolid (APAVA) dataset', referenced in the paper (Escudero et al.) 2006), is public EEG time series dataset with 2 classes and 23 subjects, including 12 Alzheimer's 's disease patients and 11 healthy control subjects. On average, each subject has 30.0 + 12.5 trials, with each trial being 5-second time sequence consisting of 1280 timestamps across 16 channels. Before further preprocessing, each trial is scaled using the standard scaler. Subsequently, we segment each trial into half-overlapping samples, where each sample is a 1- second time sequence comprising 256 timestamps. This process results in 5,967 samples. Each sample has a subject ID to indicate its originating subject. For the training, validation, and test set splits, we employ the (7,2,17,18] are assigned to the validation and test sets, respectively. The remaining samples are allocated to the training set.
setup. Samples with subject IDs {15,16,19,20) and
B.2 TDBRAIN PREPROCESSING
The TDBrain d referenced in the paper (van Dijk et al., 2022), is a large permission- accessible EEG time series dataset recording brain activities of 1274 subjects with 33 channels. Each subject has two trials: one under eye open and one under eye closed setup. The dataset includes a total of 60 labels, with each subject potentially having multiple labels indicating multiple diseases simultaneously. In this paper, we utilize subset of this dataset containing 25 subjects with Parkin- son's disease and 25 healthy controls, all under the eye-closed task condition. Each eye-closed trial is segmented into non overlllaag I-second samples with 256 timestamps, and any samples shorter
httas://osf.in//byyse https://brainc cOm/ wwwm///eeuree.
16

---

Published as a conference paper at ICLR 2026
than second are discarded. This process results in 6,240 samples. Each sample is assigned sub- ject ID to indicate its originating subject. For the training, validation, and test set splits, we employ the wubjject-Indepeneen setup. Samples with subject IDs (18.19.20.21,444477.8 are assigned to the
are assigned
to the validation set, while samples with subject IDs test set. The remaining samples are allocated to the training set.
B.3 ADFTD PREPROCESSING
The Alzheimer's Disease and Fron Fronmotempo Dementia (ADFTD) dataset', referenced in the pa- pers (Miltiadous et al. (023b a), public EEG time series dataset with 3 classes, including 36 Alzheimer's 's discase (AD) patients, 23 Frontotemporal Dementia (FTD) patients, and 29 healthy control (HC) subjects. The dataset has 19 channels, and the raw sampling rate is 500Hz. Each subject has trial, with trial durations of approximately 13.5 minutes for AD subjects (min=5.1, max=21.3), 12 minutes for FD subjects (min=7.9, 9, max=16.9), and 13.8 minutes for HC subjects (min=12.5, max=16.5). A bandpass filter between 0.5 is applied to each trial. We downsam- ple each trial to 256Hz and segment them into non-overlapping 1-second samples with 256 times- tamps, discarding any samples shorter than second. This process results in 69,752 samples. For the training, validation. and test set splits, we employ the 60%, 20%, and 20% of total subjects with their corresponding samples into the training, validation, and test sets, respectively.
setup by allocating
B.4 PTB PREPROCESSING
The PTB dataset', referenced in the paper (PhysioBank 2000), is public ECG time series recording from 290 subjects, with 15 channels and total of 8 labels representing 7 heart diseases and 1 health control. The raw sampling rate is 1000Hz. For this paper, we utilize a subset of 198 subjects, including patients with Myocardial infarction and healthy control subjects. We first downsample the sampling frequency to 250Hz and normalize the ECG signals using standard scalers. Subsequently, process the data into single heartbeats through several steps. We identify the R-Peak intervals across all channels and remove any outliers. Each heartbeat is then sampled from its R-Peak position, and we ensure all samples have the same length by applying zero padding shorter samples. with the maximum duration across all channels serving as the reference. This process results in 64,356 samples. For the training, validation, and test set splits, we employ the Subject- Indeeendenn setup. Specifically, we allocate 60%, 20%, and 20% of the total subjects, along with their corresponding samples, into the training, validation, and test sets, respectively.
B.5 PTB-XL PREPROCESSING
The PTB-XL dataset?, referenced in the paper (Wagner et al. 2020). is a large public ECG time series dataset recorded from 18,869 subjects, with 12 channels and 5 labels representing 4 heart diseases and I healthy control category. Each subject may have one or more trials. To ensure consis- tency, we discard subjects with varying diagnosis results across different trials, resulting in 17,596 subjects remaining. The raw trials consist of 10-second time intervals, with sampling frequencies of 100Hz and 500Hz versions. For our paper, we utilize the 500Hz version, then we downsample to 250Hz and normalize using standard sealers. Subsequently, each trial is segmented into non- overlapping I-second samples with 250 timestamps, discarding any samples shorter than I second. This process results in 191,400 samples. For the training, validation, and test set splits, we employ the
setup. Specifically, allocate 60%, 20%, and 20% of the total subjects, along with their corresponding samples, into the training, validation, and test sets, respectively.
17

---

Published as a conference paper at ICLR 2026
C IMPLEMENTATION DETAILS
C.I IMPLEMENTATION DETAILS OF ALL BASELINES
We implement all the baselines based on the Medformer (Wang et al. 2024b), which integrates all methods under the same framework and training techniques to ensure a comprehensive, strict fair comparison. The compared 10 baseline time series transformer methods are Autoformer (Wu al.][1221 (02I). FEDformer (Zhou et al. 2022). Informer (Zhou et al. (02T), iTransformer (Liu al.
2024b), MTST (Zhang et al. 2024a), Nonformer (Liu et al. 2022). PatchTST (Nie et al., 2023).
Reformer (Kitaev et al., 2019), Medformer (Wang et al. 2024b) (the previous state-of-the-art), and
the vanilla Transformer Vaswani et al. 2017).
For Medformer, we directly reproduced its result using their official implementations. For all other methods, we employ 6 layers for the encoder, with the self-attention dimension D set to 128 and the hidden dimension learning rate of 1e-4. The batch size is set to (32,32,128, 122, 128,128) for the datasets APAVA, TDBrain, ADFD, PTB, and PTB-XL, respectively. Training is conducted for 100 epochs, with early stopping triggered after 10 epochs without improvement in the F1-Score on the validation set. We save the model with the best Fl score on the validation set and evaluate it on the test set. We employ six evaluation metries: Accuracy, Precision (macro-averaged), Recall (macro-averaged), F1-Score (macro-averaged), AUROC (macro-averaged), and AUPRC (macro- (macro-averaged). Each experiment is run with 5 random seeds and fixed training, validation, and test sets to compute the average results and standard deviations.
the feed-forward networks set to 256. The optimizer used is Adam, with a
Autoformer Autoformer (Wu et al. 2021 employs an cotre-oorelatoon mechanism to replace self- attention for time series forecasting. Additionally, they use time series decomposition block to separate the time series into trend-cyclical and seasonal components for improved learning. The raw source code is available at htttt::t///ggt sewiithub.coom coomthumm hww.cchwwmmuoooo/./eaaaaeaa /tttttttorrrrooooo rmerr
FEDformer FEDformer (Zhou et al., 2022) leverages frequency domain information using the Fourier transform. They introduce frequency-enhance blocks and frequency-enhance attention, which are computed in the frequency domain. novel time series decomposition method replaces the layer norm module in the transformer architecture improve learning. The raw code is available at htttts:///gttutii com
Informer Informer (Zhou al. 2021 is the first paper to employ one- morwaor procedure instead of an autoregressive method in time series forecasting tasks. They introduce ProbSparse self-attention to reduce complexity and memory usage. The raw code is available at httpt::::s /tte:://iittt /ggtttb....u. com/
iTrrnsformer iTransformer (Liu al., 2024b) questions the conventional approach of embedding attention tokens in time series forecasting tasks and proposes an inverted approach by embedding the whole series of channels into token. They also invert the dimension of other transformer modules, such as the layer norm and feed-forward networks. The raw code is available at https: 7/github. com/ /
MTST MTST (Zhang et al. 2024a) uses the same token embedding method as Crossformer and PStchTST. highlights the importance of different patching lengths in forecasting tasks and designs method that can take different sizes of patch tokens as input simultaneously. The raw code is available at https::/github.
Nonformer Nonformer (Liu et al. 2022) analyzes the impact of non-stationarity in time series forecasting tasks and its significant effect on results. They design a de-stationary attention module and incorporate normalization and denormalization steps before and after training to alleviate the over-stationaarizaion problem. The raw code is available at /tttps:// com / Menstattlonnryy_Pansiie..e formers
PatchTST PSt (Nie et al. 2023) embeds a sequence of single-channel timestamps as a patch token to replace the attention token used in the vanilla transformer. This approach enlarges the receptive field and enhances forecasting ability. The raw code is available at httts:: http:::////hbb...
com/yug n P
18

---

Published as a conference paper at ICLR 2026
Reformer Reformer Kitaev et al. 2019) replaces dot-product attention with Docality-sensitivy hash- ing. They also use a reversible residual layer instead of standard residuals. The raw code is available at Pattps://giie hww.tttth.cooooo.eoooooooooooott com www./uuuuoo drainss wewaaainnnnneeeeeeeeee zeformer wat.ooommm--ttoorrirr.o.
Transformer Transformer (Vaswani et al. 2017). commonly known as the vanilla transformer, was introduced in the well-known paper "Attention is All You Need." It also be ap- plied to time series by embedding each timestamp of all channels as an attention token. The PyTorch version of the code available at https:: httpss///gttuab..ccm///reeeeee attention on is-alll-pee -olllyyuuuuo need pytorch
Medformer Medformer (Wang et al. 2024b) is multi-granularity patching transformer specifi- cally designed for medical time-series classification. It constructs patch tokens at multiple temporal resolutions capture both fine-grained local dependencies and long-range contextual patterns. This design improves the model's ability to handle heterogeneous temporal dynamics in physiological signals. The raw code available at httttps://giii
C.2 IMPLEMENTATION DETAILS OF OUR TECH
Our Tech is trained with a unified batch size (B = 128) and dimension core token D. = D across all datasets. The selection of other critical hyperparameters is listed in Table 6 We present the pseudo-code of the proposed CoTAR module in Algorithm
Table 6: Critical hyperparameters for TeCh by dataset. We listed the model dimension (D), patch length of Temporal embedding (L), number temporal encoders (M), number of channel encoders (N)), ) and learning rate (Ir).
Dataset
D
L
M
N
Ir
ADFTD
128
1
6
3e
APAVA
256
le-4
TDBRAIN
128
6
6
1e-4
PTB
256
1
0
3
le-4
PTB-XL
128
8
5
le-4
UCI-HAR
256
12
5
1e-4
FLAAP
512
1
6
1e-4
Algorithm Pseudo-Code of Core Token EnteregattionRRddisttbbittt (CoTAR).
Require: Input tensor: E Sx
Require: Parameters: Linear mapping layers Lin1, Lin2, Lin3, Lin4, dimension of core token D.. p
Require: Definition: Lin1 TR - R Lin2
DD - D.
Require: Definition: Lin3 D+D D Lin4
: RDD
1: Ô Lin2(GELU(Linn
Ô RSDD.
First MLP to obtain representation
2: On Softmax(O, dim dim
0), O ERSD.
Attention-like weights across channels
3: C," =E=
Öidd / @ Oidd, C R..
Weighted across channels to get token
4: C. Repeat(Con N times), C, E BSxD.
Repeat align the channel dimension of input Concatenate along last dimension b Fuse information through second MLP
5: Oco +(O: Oco (D++D.)
6: A Lin(((HLU(Limm(Oc..))) A A RSXD,
7: Return A E RSxD
C.3 FULL ABLATION RESULTS
To save space in the main text, we only present the ablation result of five representative datasets. We provide the full results all datasets in Table and Table
19

---

Published as a conference paper at ICLR 2026
Table 7: Full ablation result of the proposed Dual Tokenization strategy. (i) w/o: No tokenization is representation learning, a single linear
performed and directly uses the raw series as projection as classifier. (ii) Temporal: Only Temporal embedding. (ūi) Channel: Only Channel
embedding. (iv) Dual: Both Temporal and Channel. The best is Bolded
ADETO
TDBraln
PTB.NL
FLAAP
UCEHAR
Accurasy
F1.Soore Accuracy F1 Score Accuracy FT-Soore Accuracy F1 Seooe Accuracy F FlSScere Accuracy E1-Scere Accuracy 1 1'.Score
52.00 uad
50tdiinns 50. Banas 53. Mott 5377atan 2a 14 14.84aoo
2010
200 mnt 25.34-20
2xiiaaai 34. 24.25am 31 31.724807
Chainnll
50.92-0m
75 da 73 St 67 States 67 Stauus 65.062222
H1.97 to Bunks
54.70 26mir 77aBaam 22 22:86aa 92 dsasss 13 13
Both
$6.86-11m 36.30-11s Niraeass 77770 84.1 54.15
73. 73.15-mmm Pelka
Ba io 78.03 77.86 in 94.15aw 36-30
Table 8: Full ablation result of the proposed 'Core Token Aggregate-Reddstribuu Redistribut' (CoTAR) module.
(i) w/o: No Token interaction is performed, which means directly removing the CoTAR module. (äi)
Attention: Replacing CoTAR with the Attention module. (iii) CoTAR: baseline with the CoTAR module. The best is Bolded.
Accurasy F1-Scare Accuraty Storr TDdcain
ADFID
APAVA
FLScore Accuracy FL.Scure PTB-NL
PTB
FLAAP
UCI-HAR
FiScore Accuracy F1-Seare Accurncy F1.ccare
who Ta ii w lase a1 Mi 92.0 MiAlaam S5.Bte M0330are 22.2 50 Strt 14 74 D2 72 7235
Aftention 32-77A1 48 11 i 35.421m so 3204ica: 50.00.2as 50.15:135 55774-145 $1.93:132 72.01 uas 10: 77.6..m 7687.am 13uss 45.21 aa
C.4 COMPARISON WITH CUTTING-EDGE TEMPORAL MODELS
To position TeCh within the broader landscape beyond current MedTS classifiers and relative to general time-series backbones exhibiting partial similarity, we present a comparative analysis that maps overlaps and distinctions between recent backbones and TeCh.
(i) Methods employed a dual- deaa-deppenenccess modeling. We select two representative works: GAFormer (ICLR24) (Xiao al. 2024) and Leddam (ICML24) Yu et al. (2024b). GAFormer enhances token representations with group-aware embeddings for series clustering; Leddam intro- duces learnable decomposition into inter-series dependencies and intra-series variations; TeCh uti- Though all capture dual dependencies
lizes Adaptive Dual Tokenization Ciemporal/Channel//Pe (temporal and inter-channel), GAFormer and Leddam target forecasting and are Transformer-based thus decentralizing inter-channel interactions via attention, whereas TeCh uses a centralized CoTTA to better align with McdTS' biologically centralized sources (brain/heart). TeCh focuses on MedTS classification with physiological interpretability and linear complexity, while GeFormer/Leddam primarily focus on time series forecasting with quadratic attention costs. Consequently, GAFormer and Leddam are well-suited for broad forecasting scenarios; TeCh's centralized communication is more appropriate for MedTS channel dependencies. This is validated in our comparative result in T (Since there is no official implementation of GAFormer, and the information in the paper is not enough to reproduce, we take Leddam as baseline for its high reproducibility.)
(ii) Methods employed global or auxiliary tokens. We select two representative works: CATS (ICML24) (Lu et al.) 2024) and TimeXer (NIPS24) (Wang et al. (024c)) They both employ global/auxiliary tokens that are parameter-initializze and learned jointly with the model, remain- information (often tied to exogenous-
ing largely input-agnostic while aggregating/redisrrbbut variable modeling). In contrast, TeCh's 's core token is generated adaptively from each input (subject) via CoTAR, making it data-conditional and thus better suited to MedTS heterogeneity where the "central source" differs across individuals. Moreover, TimeXer and CATS still operate within de- centralized quadratic attention, while TeCh enforces centralized communication and achieves linear complexity. Additionally. TimeXer focuses on forecasting with exogenous variables and CATS constructs auxiliary time series to aid prediction, whereas TcCh targets MedTS classification with physiologically aligned central coordination. This dynamic, per-input core token mitigates the risk of poorer generalization from pre-defined global/aux tokens in clinical settings, as in Table (Since there is no official implementation of CATS, and the information in the paper is not enough to reproduce, we take TimeXer as baseline for its high reproducibility.)
Table 9: We compare our Tech with two representative models in general time series analysis that are similar to ours in certain respects. (i) Leddam (Yu et al. 2024b): like GAFormer (Xiao et al., 2024) and our Tech, all employ a dual-dependency modeling structure. (ii) TimeXer (Wang et al. 2024e): like CATS (Lu et al. (2244) and our Tech, all employ global or auxiliary tokens to aggregate
and redistribute information. The best is Bolded.
ADFTD Accuracy F1-Score Accuracy F1-Score Accuracy F1-Score Accuracy F1-Score
APAVA
TDBrain
PTB
PTB-XL
Accuracy F1-Score
Leddam
53, 53.142 46.64-103 75.92a124 74.08k-2. 71.27-0a 71.22;007 8384-161 78.761377 67.41=035 51.842a
TimeXer
52. 32.962050 43.4140a5 72.442041
70.09:00.
72.882157 72.56:145
83.32-022
78.43109
14-01 50.00ga3
Tech (Ours)
54.54.079 48.842172 86.86:199 86.30-1.
93.21 93.20 25.96=251 81.974417 73.53-907 62.44
20

---

Published as a conference paper at ICLR 2026
Table 10:
To further validate the generalizability, we further conduct a five-fold cross-validation based on the subject ID. The best is Bolded.
ADFTD
APAYA
TDBrain
PTB
PTB-XL
Accuracy Fl-Seore Accuracy F1-Score Accuracy F1-Score Accuracy F1-Score
Accuracy F FI-Score
Medformer 53,41=305 49.03-357 68.01=913
66.634971 82.92-903
81.1349.6 83, 83.30-546 72.46+5.17 71.76=066 61.
Tech (Ours) 55.05:243 49.82 80.662653 79.62 87.062621 86.00 89.48-3.18 84.59 73.65=041 62.79:a.2
Table 11: Quantitative comparison of the centralized property. We measure centralization using: (1) Spectral Centralization Index (SCI), the ratio of the largest eigenvalue to total variance, and (2) Dynamic Influence Centralization (DIC), the normalized out-strength imbalance of a first-order VAR model. Higher values indicate stronger centralized behavior.
EEG
ECG
Energy
Climate Weather
Metric/Dataset
ADFTD
APAVA
TDBrain
PTB
PTB-XL
ETTh2
ETTm2
SCI
0.918
0.520
0.616
0.622
0.652
0.397
0.296
0.381
DIC
0.668
0.731
0.747
0.825
0.777
0.241
0.119
0.342
Table 12: Further comparison with MedGNN (the latest MedT'S classifier in WWW 2025).
ADFTD
APAYA
TDBrain
PTB
PTB-XL
Accuracy F1-Score Accuracy Fi-Score Accuracy F1-Score Accuracy F1-Score Accuracy F1-Score
91.042009 91.0440.0 84.53=022 80,40 73.87=0.18 62.54=n.0 Tech (Ours) 54,54g030 48.84:1.72 1. N6.30 93.21 93.20:aa1 85.96-282 81.97 73.532007 62.44
MedGNN
56.12=a11 55,00:9.24 82.60 22.60:033 80.25
C.5 FIVE-FOLD CROSS- VALIDATION RESULT
To mitigate the bias of fixed Subject-Independeen split, we further performed a five-fold cross- validation based on subject IDs. As shown in Table IO TeCh consistently surpasses Medformer across all datasets. For example, APAVA, TeCh improves Accuracy and F1-Score by +12.6% and +13.0% while on PTB, the gains reach +6.2% % and +12.1%, respectively. TeCh also yields lower standard deviation (e.g.. 9.71 6.79 on APAVA F1-Score), indicating greater robustness. These results confirm that TeCh generalizes more effectively across subjects and remains robust to inter-subject noise, benefiting from CoTAR's centralized
mechanism.
C.6 CENTRALIZATION ANALYSIS
To formally quantify the degree of centralization in multivariate time series J RSXT, where S the number of channels, and T is the length, we introduce two complementary metries:
(1) Spectral Centralization Index (SCI):
Amax
T-1 X
1 X x)T)
1 X = T XII
SCI(X) Tr
X) -X)T)
(2) Dynamic Influence Centralization (DIC):
Let = [x1,X2, "r,,xx,,.,......... 11- y= [x2, Xa. Xr].
estimate A = Y where is the i-th column of X.
DIC(X) max; 8
S/
5 =
(6)
3
S
SCI measures spatial dominance as the energy concentration in the principal component of the co- variance matrix (Jolliffe & Cadimal 2016), while DIC captures temporal dominance as the nor- malized imbalance of out-strengths
first-order vector autoregressive model (Seth et al. 2015] Valente et al., (0088.. As shown in Table I EEG and ECG signals exhibit significantly higher centralization than signals generated from decentralized systems (Energy: ETTh2, ETTm2 (Zhou er al. 20 Climate: Weather (Wu et al.) 202T)). This confirms that MedTS possesses inherently centralized structures, where a few dominant channels or physiological processes govern the global dynamics. In contrast, energy and climate datasets are more decentralized. These findings explain why TeCh's 's centralized perrggaainngrrdistribtttii design is particularly effective for MedTS.
C.7 COMPARISON WITH THE LATEST ADVANCEMENT
We further compare against MedGNN, the latest MedTS classifier, in Table 12
21


Reviews:
[{"review_id": "mdAxbqrEK2", "reviewer": "Reviewer_3453", "summary": "This paper addresses a \"structural mismatch\" between Transformer models and Medical Time Series (MedTS) data like EEG and ECG. The authors argue that while MedTS signals originate from a centralized biological source (e.g., the heart or brain), the standard Transformer attention mechanism is decentralized, with all-to-all token interactions. This mismatch, they claim, makes it difficult for Transformers to model channel dependencies effectively.\n\nTo solve this, the paper proposes CoTAR, an MLP-based module designed to replace attention. CoTAR introduces a \"global core token\" that acts as a proxy. All tokens first aggregate information to this core token, which then redistributes the integrated information back to all tokens. This star-shaped architecture mimics the centralized nature of MedTS signals while reducing the computational complexity from quadratic to linear.\n\nThe full model, TeCh, uses CoTAR within a Dual Tokenization framework that processes the input in parallel: once with \"Temporal Embedding\" (patches of time) and once with \"Channel Embedding\" (whole channels as tokens).\n\nExperiments on five MedTS and two HAR datasets show that TeCh achieves state-of-the-art performance, significantly outperforming prior SOTA (Medformer) with large gains in efficiency (e.g., 33% memory and 20% inference time) and robustness to noise.", "questions": "1. Clarification of TeCh Architecture: The central framing of the paper is confusing. Table 6 shows that the SOTA results for TDBrain, PTB, PTB-XL, and FLAAP are achieved using a single-branch model ($M=0$ or $N=0$), not the \"Dual Tokenization\" model ($M>0$ and $N>0$) described in Section 4.2 and analyzed in Table 4.\n\n   - Could you confirm that the SOTA results in Tables 2 & 3 are achieved by tuning $M$ and $N$ and often setting one to 0?\n   - If so, why is the paper framed around \"Dual Tokenization\" as the primary architecture? This seems to misrepresent the final model and makes the \"Dual Tokenization\" ablation (Table 4) misleading. Wouldn't it be more accurate to present TeCh as a CoTAR-based model where the tokenization strategy (Temporal, Channel, or Both) is a key hyperparameter?\n\n2. Clarification of CoTAR Math (Equation 2): Could you please provide an unambiguous, step-by-step definition of the CoTAR module's computation? Specifically, what are the dimensions of all the weights and biases? A clear definition would resolve confusion about the module's precise mechanism.\n\n3. Mismatch in Ablation Results: There ablation studies are only performed on 5 of the 7 datasets and in a inconsistent manner. Is there a reason why the ablations were not performed on the other datasets? Including these would provide a more complete picture of the model's behavior across all evaluated datasets.\n\n4. Mismatch in Model Implementation: The paper provides the code link, but the imported names of the TeCh model seems to suggest that it uses the same Transformer encoder layer and not the CoTAR module. Since I cannot access the actual content of the files under the layers directory, could you clarify if the provided code implements the CoTAR module as described in the paper?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong, Intuitive Inductive Bias: The paper's primary strength is the clear and compelling motivation. The argument that decentralized attention is a poor structural match for centralized biological signals is an excellent insight and provides a strong foundation for the CoTAR module.\n\n2. Superior Performance: The proposed model achieves state-of-the-art results on a wide range of MedTS datasets, often by a significant margin over the previous SOTA, Medformer. This demonstrates the empirical effectiveness of the CoTAR module.\n\n3. Massive Efficiency Gains: The paper's most significant practical contribution is the efficiency of CoTAR. By reducing complexity from $O(S^2)$ to $O(S)$, the model achieves up to a 5x speedup in inference and a 3x reduction in memory usage compared to the prior SOTA. This is clearly visualized in Figure 4(a) and is critical for real-world medical applications.\n\n4. Improved Robustness: The experiments in Figure 4(b) provide strong evidence that CoTAR's centralized proxy design makes the model significantly more robust to noise in the input channels. This is a key practical advantage for noisy MedTS data.\n\n5. Thorough Evaluation: The experimental setup is strong, using 5 MedTS datasets (EEG and ECG) and 2 HAR datasets to show generalizability. The authors correctly use a subject-independent splitting protocol, which is crucial for clinically relevant MedTS evaluation.", "weaknesses": "1. Misleading \"Dual Tokenization\" Framework: The paper's biggest weakness is the framing of the \"TeCh\" model around \"Dual Tokenization\". The SOTA results in Tables 2 & 3 are NOT achieved by a consistent dual-branch model. As Table 6 reveals, 4 of the 7 datasets (TDBrain, PTB, PTB-XL, FLAAP) use a single-branch model ($M=0$ or $N=0$) to get the reported results. This makes the \"Ablation Study on 'Dual Tokenization'\" (Table 4) highly misleading. The ablation's conclusion that \"combining both yields overall superior performance\" is cherry-picked (it's only true for 2/5 datasets) and contradicted by the final model's own hyperparameters. The paper should be reframed to present \"TeCh\" as a family of CoTAR-based models where the tokenization (Temporal, Channel, or Dual) is a hyperparameter to be tuned, rather than presenting \"Dual\" as the definitive architecture.\n\n2. Confusing Mathematical Notation: The formal definition of CoTAR in Equation (2) is unclear. The notations are a bit confusing, particularly the use of symbols to represent the matrices (e.g., both $O$ and $Co$ represent a single matrix/vector). And the function names do not follow a consistent notation (e.g., upright \"GELU\" vs. italicized \"GELU\" in different parts of the equation). Notably, the equations do not clarify the shape of the matrices/vectors involved, making it hard to follow the operations. A clearer, more consistent notation with explicit shapes would improve clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hj3WJx0UZs", "reviewer": "Reviewer_KGK1", "summary": "This paper proposes the **CoTAR** (Core Token Aggregation-Redistribution) module to replace the attention module in medical time-series (MedTS) classification. This method is motivated by the assumption that MedTS signals typically originate from a centralized biological source. The design of the CoTAR module is inspired by client-server communication, which uses a core token to aggregate and exchange information between clients, rather than self-attention, where each token attends to all others equally. A new method called **TeCh** is proposed, aligned with the CoTAR module, similar to the Transformer architecture, but with attention replaced by CoTAR. Results are compared against 10 baselines across 5 MedTS datasets and 2 general time series datasets for classification, achieving SoTA performance.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method is motivated by MedTS's domain knowledge and inspired by server-client communication, which is a good approach. It is interesting to see that the linear complexity CoTAR module performs similarly to, and in some cases even better than, SOTA transformer methods. The comprehensive ablation study and direct comparison with the attention module are good and demonstrate the effectiveness of the CoTAR module.", "weaknesses": "It is better to provide more detail in equal (2), as Figure 2 lacks notation for the variables used. I can get the idea of the core token being redistributed to each token, but reading the equal (2) is still a little confusing about the details. The performance on the ADFTD dataset is limited to the F1 score. Sometimes, a fixed subject-independent split makes it hard to demonstrate the superiority of a method, as specific subjects in the training set may contain too much noise and make results across methods similar. You could provide a cross-validated (5-fold or Monte Carlo) subject-independent evaluation result on the dataset, demonstrating the effectiveness of your method, even when performance is limited on a fixed split. Besides, more advanced SOTA methods, such as MedGNN, should be compared with.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ESdK6YTc2j", "reviewer": "Reviewer_XcRP", "summary": "This paper proposes CoTAR (Core Token Aggregation–Redistribution), a centralized alternative to self-attention for modeling medical time series (MedTS) such as EEG and ECG. The authors argue that MedTS signals are centrally coordinated (e.g., brain or heart acting as a signal source), while Transformer attention is inherently decentralized, making it ill-suited to capture the global dependencies between channels.\n\nTo address this, CoTAR introduces a central “core token” that aggregates global information from all tokens (channels) and redistributes it back via a lightweight MLP mechanism, achieving linear computational complexity. Combined with a dual tokenization strategy that separately encodes temporal and channel embeddings, the resulting model (TeCh) jointly captures both temporal and inter-channel dependencies.\n\nExtensive experiments across five MedTS datasets (EEG/ECG) and two human activity recognition (HAR) datasets show that TeCh achieves state-of-the-art accuracy and efficiency", "questions": "-  Is there a formal way to distinguish between centralized and non-centralized MedTS?\n- Can the core token be interpreted or visualized to correspond to physiological latent processes?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper convincingly articulates the mismatch between decentralized attention and the centralized nature of many physiological signals. This conceptual framing is both intuitive and novel for the MedTS domain.\n\n- CoTAR is a well-engineered module that reduces the quadratic cost of self-attention to linear, while retaining flexibility in cross-token communication.\n\n- Experiments span seven datasets (five MedTS + two HAR) with six evaluation metrics. The method consistently outperforms ten Transformer-based baselines.\n\n- Code and training scripts are publicly released.\n\n- Provide robust test, i.e., standard deviation.", "weaknesses": "- The authors repeatedly assert that MedTS are “centralized” but provide no quantitative validation.\n- The paper omits direct comparisons with recent dual-dependency or TeCh-style models (e.g., GAFormer)\n- The proposed CoTAR conceptually resembles several prior Transformer modifications that employ global or auxiliary tokens to aggregate and redistribute information. The authors should discuss them. e,g,. CATS\n- Different datasets use different hyperparameters.\n\n\n\n\n\n\n\n[1] GAFormer: Enhancing time-series transformers through group-aware embeddings\n\n[2] CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)