Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification
Authors: Yihe Wang
Nan Huang
Taida Li
Yujun Yan
Xiang Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Transformer
Time Series
Healthcare
Primary area: machine_learning_for_healthcare
TL;DR: A novel transformer model designed for medical time series classification that utilizes cross-channel multi-granularity patching, and intra-inter granularity self-attention within and among granularities.

Abstract: Medical time series (MedTS) data, such as Electroencephalography (EEG) and Electrocardiography (ECG), play a crucial role in healthcare, such as diagnosing brain and heart diseases. Existing methods for MedTS classification primarily rely on handcrafted biomarkers extraction and CNN-based models, with limited exploration of transformer-based models. In this paper, we introduce Medformer, a multi-granularity patching transformer tailored specifically for MedTS classification. Our method incorporates three novel mechanisms to leverage the unique characteristics of MedTS: cross-channel patching to leverage inter-channel correlations, multi-granularity embedding for capturing features at different scales, and two-stage (intra- and inter-granularity) multi-granularity self-attention for learning features and correlations within and among granularities. We conduct extensive experiments on five public datasets under both subject-dependent and challenging subject-independent setups. Results demonstrate Medformer's superiority over 10 baselines, achieving top averaged ranking across five datasets on all six evaluation metrics. These findings underscore the significant impact of our method on healthcare applications, such as diagnosing Myocardial Infarction, Alzheimer's, and Parkinson's disease. We release the source code at https://github.com/DL4mHealth/Medformer.

Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification
Yihe Wang", Nan Huang", Taida Li* University of North Carolina Charlotte (ywang145, ahuangl, t1i14) @charlotte.edu
Yujun Yan Dartmouth College
Xiang Zhang
University of North Carolina Charlotte xiang. whange edu
edu
Abstract
Medical time series (MedTS) data, such as Electroencephalograahy (EEG) and Electrocardiography (ECG), play a crucial role in healthcare, such as diagnosing brain and heart diseases. Existing methods for MedT'S classification primarily rely on handcrafted biomarkers extraction and CNN-based models, with limited exploration of transformer-basee models. In this paper, we introduce Medformer, a -uultiggraullarity patching transformer tailored specifically for MedTS classifi- cation. Our method incorporates three novel mechanisms to leverage the unique characteristics of MedTS: cross-channel patching to leverage inter-channel cor- relations, multi-granularity embedding for capturing features different scales, and two-stage (intra- and inter sieer-rrnullaitt) multi granularity self-attention for learning features and correlations within and among granularities. We conduct extensive experiments on five public datasets under both subject-dependent and challenging indpeeeetn setups. Results demonstrate Medformer's periority over 10 baselines, achieving top averaged ranking across five datasets all evaluation metries. These findings underscore the significant impact of our method on healthcare applications, such as diagnosing Myocardial In- farction, Alzheimer's, s, and Parkinson's disease. We release the source code wwt.://ww...omeeAAeaaaeeaaa....e
Introduction
Medical time series refers sequences of health-related data points recorded successive times, tracking various physiological signals over time 2]. Effective classification of Med'TS data enables continuous monitoring and real-time analysis of subject's physiological state, supporting early ab- normality detection, accurate diagnosis, timely intervention, and personalized crcatmnnt-ulimate enhancing patient outcomes and healthcare efficiency [3, 4J. For instance, Electroencephallggrahy (EEG) provides insights into subject's neurological status [5, 6]. while Electrocardiography (ECG) aids in diagnosing heart conditions [7, 8, 9]. Most current MedTS classification approaches rely on handcrafted biomarker extraction [10, 11, 12], convolutional neural networks (CNN)- -bsed mod-
els [13, 14, 15, 16], graph convolutional networks (GNNs)- bassed models[ 17, 18], or combinations of CNNs and self-attention modules[19] 20]. Notably, effective transformer-basee models for MedTS classification remain underexplored.
Transformers have demonstrated strong performance in time series representation learning across tasks such as forecasting [21, 22, 23), classification [24, 25], and anomaly detection [26, 27], with
These authors contributed equally to this work.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

predominant focus on forecasting. While these methods are applicable to MedTS classification, their design motivations and mechanisms may not fully align with the unique requirements of this domain. For example, as shown in Figure 1, models like Autoformer [28] and Informer [29] adopt the token embedding approach from the vanilla transformer [30], using a single cross-channel timestamp as an input token. This strategy struggles to capture coarse- -aarneegaiiedd temporal features. In contrast, iTransformer [31] encodes the entire series from one channel as an input token, which can overlook fine-grained temporal features while focusing on multi-channel correlations. Additionally, PatchTST [32] embeds a sequence of timestamps from one channel as a patch for self-attention, limiting the model's 's capacity to learn cross crasnel relationships.
These existing methods fail to fully exploit the distinctive characteristics of MedTS data, such as local temporal dynamics, inter-channel correlations, and multi scall--scaeeee feature analysis. First, effective capturing temporal dynamics demands multi-timestamp inputs to capture local temporal patterns, as highlighted in approaches like PatchTTST [32] and Crossformer [33]. Second, leveraging cross-channel information is critical; for example, multi-channel EEG data recorded following the International 10-20 system [34] monitors the brain activities, with each electrode/channel corresponding to specific brain regions. Since brain functionn are integrated, inter-channel correlations (e.g., brain connectome) are crucial in EEG analysis [35, 36, 37j. Third, representation learning across multiple temporal scales and periods is vital to uncover a broad range of health patterns, as certain disease indicators may only appear within specific frequency bands [10, 12].
To bridge this gap, we propose Med- former a multi-granularity patching transformer designed explicitly for MedTS classification. Our approach introduces three mechanisms to enhance learning ca- pacity. First, we propose a novel token embedding method using cross-channel patching, effectively capturing both multi- rimestamp and cross-channel features. To the best of our knowledge, this is the first application of cross-channel patch- ing for transformer embedding in time series analysis. Second, rather than us- ing fixed-length patches, we employ multi- granularity patching with a list of patch lengths, enabling the model to capture features in different scales. This multi- granularity approach could simulate dif- ferent frequency bands, capturing band- specific features without relying on hand- crafted up'downsampling and band filters. Third, We introduce a two-stage (intra- and inter- granularittt) self-attention mechanism to capture features within individual granularities and correlations across granularities, enabling complementary information integration across scales.
Transformer
b) iTransformer
PatchTST
www Amaas.ad
www Timesiamp Mmm
www
All-Tmerstmp
www
Mutl-Tmmeetmmp
www
Single Charnel
wwm
Single Channel
d) Medlfotmer(Ours)
4-4 b 2 Mm M Mwww
2 2s
Mm
aow/mm/mm wwww
Tuls-Timestmmp Mult- Granulrarlllrlly
wm www
Musi-Grartatt utaily Bull-Alantion
Token embedding methods. Vanilla transformer, Autoformer, and Informer [30, 28, 29] employ a single cross- channel timestamp as token; iTransformer [31] utilizes an entire channel as token; and PStchTST and Cross- former [32, 33] adopt a patch of timestamps from one channel as a token. For MedTS classification, we propose Medformer considering inter-channel dependencies (cross- choosel tem- poral properties (multi-timestle timlstapimmetmmp,, and multifaceted scale of temporal patterns amulti-gramularity.
Figure 1:
We conduct extensive experiments using ten baselines across five public datasets, including three EEG datasets and two ECG datasets, focused on detecting diseases such Alzheimer's and cardiovascular conditions under both subject-dependent and subject-independen setups (Figure 2). Results show that Medformer achieves the highest average ranking across all six evaluation metrics and five datasets (Figure 4), highlighting its superior effectiveness, stability, and potential for real-world applications.
Related Work
Medical Time Series. Medical time series refers to specialized time series data collected from the human body, commonly used for disease diagnosis [3, 71, health monitoring [6. 1], and
interfaces (BCIs) [39. 2]. Different MedTS modalities include EEG [40, 41, 42], ECG [7, 8, 9], EMG [43, 44], and EOG [45, 46), each offering distinct capabilities for various medical applications.
Wee note that this name has been previously used in other domains [38].

---

For example, EEG and ECG data are instrumental in assessing brain and heart health [40, 7]. Recent BCI research explores using EEG to control objects, providing functional support to individuals with disabilities [2, 47J. Unlike general time series research, which predominantly focuses on forecasting tasks [48, 49], MedT'S research is centered around signal decoding, which involves classifying hidden information within MedT'S sequences. Current approaches often rely on biomarker identification and deep-learning models utilizing CNNs, GNNs, or hybrid models combining CNNs with self-attention modules. For instance, band features such as relative band power and inter-band correlations [11, 50] have proven effective in EEG-based A Alzheimer's disease diagnosis. Deep-learning models like EEGNet [14], BEG-Conformer [20], and TCN [51, 13] have also shown strong performance across various MedTS classification tasks.
Transformers for Time Series. Transformer has demonstrated its strong learning and scaling-up ability in many domains, including natural language processing [30, 57] and com- puter vision [58, 59]. Existing transformer- based methods for time series analysis can be categorized into two main directions: mod- ifying token embedding methods and self- attention mechanisms, or both. For exam- ple, PatchTST [32] uses a sequence of single- channel timestamps patch for token em- bedding. Methods like Autoformer [28]. Informer [29], Nonformer [54], and FED- former [52] develop new self-attention mech- anisms or replace the self-attention module improve learning ability and reduce complexity.
Table I: Existing methods do not fully utilize all poten- tial characteristics in MedTS.
Models
Multi- Timestamp Channel
Cros-
Multi- Gramularity
Granularity- Interaction
Autoformer [28]
Crussormer [35]
FEDformer [52]
Informer [29]
iTramformer [31]
MIST [53]
Nonformer [54]
PatchTST [32]
Pathformer [55]
Reformer [56]
Transformer (30)
Medfnrmeev(uuss)
Crossformer [33] and iTTrasffrmer [31] modify both token embedding methods and self-attention mechanisms. Patching. Patch embedding has been widely used in time series transformers since the proposal of PatchTST [32]. Existing methods of patching, such as Crossformer [33], CARD [23], and MTST [53]. inherit from PatchTST [32] and utilize a sequence of single-channel timestamps for patching. This channel- -ineneeliddeeendentt patching might benefit learning ability in time series forecasting but may not be effective in MedTS classification.
Kuanu-Granullrity. Existing methods such as Pyraformer [21], MTST [53]. Pathformer [55], and Scaleformer [60], utilize multi-granularing embedding to capture features different scales, allowing models learn both fine- one-grained and coarse- grense-g patterns. We discuss the differences between our method and existing multi multi-granularity
approaches in Appendix G.I.
Medformer includes both novel token embedding and self-attention mechanisms. Figure and Table I present comparison of token embedding methods and feature utilization between our method and
existing methods. The components of our method can be easily incorporated into existing methods to improve classification learning ability. For example, cross-channel guati-granullrtty patching can be integrated with methods that modify self-attention mechanisms, such as Autoformer [28] and Informer [29], for token embedding. Similarly, the two-stage multi-granularity self-attention can be combined with existing multi- tultiiigraularttyy methods, like MTST [53] and Pathformer [55], to enhance the learning of nnter-granularig features.
3 Preliminaries and Problem Formulation
Disease Diagnosis with MedTSS Medical time series data typically exhibit multiple hierarchical levels, including subject, session, trial, and sample levels [13]. In discase diagnosis tasks using MedTS, each subject is usually assigned single label, such as indicating the presence or absence of Alzheimer's ' disease. However, multi-labeling may be necessary when a subject has co-occurring con- ditions [61]. Notably, a subject's medical physiological state remains relatively stable over time (or within short periods without significant change). For instance, a subject diagnosed with Alzheimer's disease is expected to retain that diagnosis for many years. If the subject also has Parkinson's 's disease, a multi- learning approach is required, which essentially conducts classification tasks for each disease independently if they are not mutually exclusive.
Since long sequences of time series data (e.g., trials or sessions) are often segmented into shorter samples for deep learning training, all samples from single subject should idcally retain the same
3

---

medical condition label. Thus, each MedTS sample typically includes a class label indicating specific disease type and a subject ID indicating its original subject. Given the ultimate goal of diagnosing whether a subject has particular disease, experimental setups must be carefully designed to reflect real-world clinical applications. Diverse setups can yield markedly different outcomes, potentially leading to misleading conclusions. Here, we introduce two widely used setups in MedTSS classification and clarify their distinctions. Figure 2 provides a simple illustration of these two setups.
Subject-Dependere In this setup, the division into training, validation, and test sets is based on time series samples. All samples from various subjects are randomly shuffled and then allocated into the respective sets. Consequently, samples with identical subject IDs may be present in the training. validation, and test sets. This scenario potentially introduces "information leakage, " wherein the model could inadvertently learn the distribution specific to certain subjects during the training phase. This setup is typically employed to assess whether a dataset exhibits cross-subject features and has limited applications under real-world Med'I S-bssed disease diagnosis scenarios. The reason is simple: we cannot know the label of unseen subjects and their corresponding samples during training. Generally, the results of the subject-dependent setup tend to be notably higher than those from the uubject-independent setup, often showing the upper limit of a dataset's learning capability.
In this setup, the divi- sion into training, validation. and test sets is based on subjects. Each subject and their cor- responding samples are exclusively distributed into one of the training, validation, test sets. Consequently, samples with identical subject IDs can only be present in one of these sets. This setup holds significant importance in dis- ease diagnosis tasks as it closely simulates real-world scenarios. model on subjects with known labels and sub- sequently evaluate its performance on unseen subjects; in other words, evaluate if subject has a specific disease. However, this sctup poses significant challenges in MedT'S classification tasks. Due to the variability in data distribution and the potential presence of unknown noise within each subject's 'ss data, capturing general task-related features across subjects becomes challenging [62, 13, 63, 64]. Even if subjects share the same label, the personal noise inherent in each subject's ' data may obscure these com- mon features. Developing a method that effectively captures common features among subjects while disregarding individual noise remains an unsolved problem.
Training
Testing 8
52
S:
8
Subject- Dependent
S1
S
enables us to train
Subject- Independent
Figure 2: dubbett-dgendeennnddeeeeeen setups (fig- ure adopted from our previous work [13]). the subject-dependent setup, samples from the same subject can appear in both the training and test sets, causing information leakage. In subject-indepenneet setup, samples from the same subject are exclusively in either the training or test set, which is more challenging and practically meaningful but less studied.
In this work, evaluate our model mainly in the subject-independe real-world applications, aiming to draw attention within the time series research community to the substantial challenges posed by this setup. While our model is not specifically tailored to address subject-indeperdeee problems, it integrates multi-timestamp. cross-channel, and multi- uulli-granularity features MedTS, enhancing its capacity to capture subject-invariant representations. Consequently, our model is well-equipped to tackle the subject-independer challenge to a certain extent, and our results (Section 5) confirm such capability of Medformer.
setup better align with
We next present the problem formulation for multivariate medical time series classification in the context of disease diagnosis.
Problem (MedTS Classification), Consider an input MedTS sample Din E TT where T denotes the number of timestamps and C represents the number of channels. Our objective is to learn an encoder that can generate representation h, which can be used to predict the corresponding label y E K of the imput sample. Here, K denotes the number of medically relevant classes, such as various disease types or different stages of one disease.

---

a)
Muiti-Cramularity Self- AiffAtteenion
Output
Cranniacity 1
Projection
Gounuasity
Gratmlarity
Mx
LayerNorm
Routers
Feed Forward
Totes Geaoriaeity
Tabe: Gesorbirity
LayerNorm
b)
Cross-Channel Multi- Haatt-Gramularity Patch Embedding
Patchas
Patch Embooklings
Augmented Embeddings
Grannlurity 1.
Attention
Grannlattyy La
CL
ea
Embedding
ea
MV
Wwn
Geaordaatty u, Trp 32
CLM/W/// www. awd/m/ www. aammm Mwwm
Toput Medical Series zis
Figure 3: Overview of Medformer. a) Workflow. b) For the input sample was we apply n distinct patch lengths in parallel create patched features (9) where i ranges from to n. Each patch length represents unique granularity. These patches are then projected (!) and subsequently augmented to form a(!) c) obtain the final embedding x by combining the augmented a( with both the positional embedding Wpes and the gramularity embedding w Additionally, granularity-speeife router (+) is designed capture integrated information for each respective granularity. We then perform intraggramularity self-attention, focusing on individual granularitics, and inter-gramularity self-attention, using the routers to facilitate communication across different granularities.
Method
In this section, we first describe the cross-channel multi multi-granularig patching mechanism for learning spatial-temporal features in different scales. Next, we analyze the two- sto-stage multi tiltti-grruuarttyy self- attention mechanism, which leverages features within the same granularity and correlations among different granularities. The architecture of the proposed Medformer is illustrated in Figure 3.
Multi-Granularity Patch Embedding. From the medical perspective, the brain heart functions as a cohesive unit, suggesting a naive assumption that there are inherent correlations among different channels in MedTS [35, 36, 37J, as each channel represents the activities of distinct brain or heart regions. Motivated by the above assumption, we reasonably propose multi-channel patching for token embedding, which different from existing patch embedding methods that embed patches in Shlannel-indeeedder manner and fail to capture inter-channel correlations [32, 33, 23]. Figure/ provides an overview comparison of existing token embedding methods and ours. Additionally. existing research on EEG biomarker extraction has shown that certain features are linked different frequency bands, such as a, 3, and y bands [10, 12]. This motivates us to embed patch tokens in a multi-gran granuligraull way. Instead of using traditional methods like up/downsampling or handcrafted band filtering, granularaanllritty patching automatically corresponds to different sampling frequencies, which can simulate different frequency bands and capture band-relate features.
5

---

Given the above rationales, we propose novel token embedding approach: cross-channed multi- granularity patching. Given an input multivariate MedTS sample Fin E Txx and a list of different patch lengths {I1, I.,,..., , Tn J. For the i-th patch length I, denoting granularity i, we segment the input sample into N, cross-channel non-overlapping patches z (N, Zero padding is applied to ensure that the number of timestamps 7 is divisible by L,, making N; [T/Ls]
The patches are mapped into latent embeddings space using linear projection: x (9) (6)
W(O,
where x: E IR NRD and ( E (((,C))D,, Inspired by the augmented views contrasting in the contrastive learning framework [65, 13, 66], we further apply data augmentations such as masking
and jittering on ( to obtain augmented embeddings Te E RNNxD. We assume the augmentation can improve the learning ability in the following granularttt self-attention stage by foreing different granularities to learn and complement information from each other.
A fixed positional embedding W pos R is generated for positional encoding [30], where G' is very large number. We add [oss[1 N.] RNxDD, the first N; rows of the positional embedding Woss. and a learnable granularity embedding w gr E IIID, to obtain the final patch embedding for the i-th granularity with patch length Le:
x =
W pos N. W
(1)
where (0) E NRDD Note that the granularity embedding w( aims to distinguish among granularities and is broadcasted to all N, embedding rows with D dimension during addition.
To reduce time and space complexity, we initialize a router to be used in the multi-granularity self-attention (as described later) for each granularity:
u
W pu (NV + +1] W()
where u W N: + I]. W
E RIID Here, pess + 1] is not used for positional embedding but to inform the router about the number of patches with the current L, granularity, and W contains the granularity information. Both components help distinguish the routers from one another.
Finally, we obtain a list of final patch embeddings (x(1), xx2), ..... xx()) and router embeddings {u()), (2))..... ..... (") } for different granularities with patch lengths {L1, L2...., Ln} We feed the embeddings to the two-stage gulti-granularity self-attention.
Granllaryyyyy Self-Attention. Our goal is to learn multi multi-granularity features and granularity interactions during self-attention. naive approach to achieve this goal is to concatenate all the patch embeddings {a((1, (2) x (*)) into large patch embedding X E R NN
and perform self-attention on this new embedding, where denotes the number of different granularities.
However, this results in a time complexity of O (EE=,,)) N.))) which is impractical for large n.
To reduce the time complexity, we propose a router mechanism and split the self-attention module into two stages: a) intra-granularity self-attention and b) inter-granularit self-attention. The intra- granularity stage performs self-attention within the same granularity to capture the distinctive features of each granularity. The inter-granularity stage performs self-attention across different granularities to capture their correlations.
a) Grana-Grayulyrity Self-Attention, For the i-th patch length L, denoting granularity we verti- cally concatenate the patch embedding (i()) E NRN and router embedding (()) E RIxD to form
an intermediate sequence of embeddings (i) R [N+++)xxD
(8) x (3)
(3)
where |-||-] denotes concatenation. We perform self-attention the new 20)) for both the patch embedding x! and the router embedding (
(1)
x Attn Intra (z(0) 2(1) z(0)
Intra
(4)
(1) <- Altn Intra
Intra
(9) (1)) z(1)
(1,)2()))

---

where Attn (Q, K, V) denotes the scaled dot- not-product self-attention mechanism in [30]. Note that the router embedding u is updated concurrently with the patch embedding x+ to maintain consistency, ensuring that the router effectively summarizes each granularity's features in the current training step and is ready for the subsequent inter-granunn sieer-ganularity self-attention. The intra granularity self-attention mechanism allows the model capture temporal features within single granularity, facilitating the extraction of local features and correlations among timestamps at the same scale.
b) Graeularynylrrity Self-Attention. We concatenate all router embeddings (u(), u()) u2))................... u u(n) to form sequence of routers U E IR"XD:
U
u
[
(5)
where n is the number of different granularities. For granularity with patch length Li, we apply self-attention the router embedding (+) E 121x with all the routers U:
u ( + Attn Inter (u (ul),,,)
(6)
Each router contains global information specific one granularity by doing intra- graruuaaaullrrty self- attention. By performing self-attention among routers, information can be exchanged and learned across different granularities, effectively capturing features across various scales. Additionally, the use of the router mechanism successfully reduces the time complexity of the naive approach from o (EE
N.)") (==:N))) to O (E N? + ==²) Given that N; S T, the worst-case time complexity for our self-attention mechanism is O (n1² n²). However, a reasonable choice of patch lengths as power series, i.e., L; 2 leads to time complexity of O(T²). To further reduce complexity and memory consumption, we apply shared layer normalization and feed-forward layers across all granularities. See appendix F for more details about complexity analysis.
Summary. Our method utilizes the standard transformer architecture shown in Figure 3. For given sample Lin. after M layers of self-attention learning, we obtain a list of updated patch embeddings (x()), (2) x (n)) ). which we concatenate them to form a final representation h that can be used to predict label y E R in downstream classification task. Note that although we discuss multi- granularity here, our method is flexible and can be easily adapted variants such as Linle-granularity or even repetitive same granularities. See Appendix D.2 for more details.
5 Experiments
We compare our Medformer with 10 baselines across 5 datasets, including 3 EEG datasets and 2 ECG datasets. Our method is evaluated under two setups (Section 3): subject-dependent and bobjectindeeeendet. In the subject-dependent setup, training, validation, and test sets are split based on samples, while in the subject-independent setup, they are split based subjects.
Table 2: The information of processed datasets. The table shows the number of subjects, samples, classes, channels, sampling rate, sample timestamps, modality of McdTS, and file size. Here, #Timestamps indicates the number timestamps per sample.
Datasets
#-Subject
# Sample
#-Class
# Channel
# TTTmestamps
Sampling Rate
Modality
File Size
APAWA
5,967
256
256Hz
EBG
186MB
ADFTD
69,752
256
256Hz
EEG
2.52GB
TDBrain
72
6,240
33
256
256Hz
BBG
571MB
PTB
198
64,356
15
300
250Hz
BCG
2,15GH
PTB-XL
17,596
191,400
12
250
250Hz
ECG
4,28GB
Datasets. (1) APAVA [67] is an EEG dataset where each sample is assigned binary label indicating
whether the subject has Alzheimer's disease. (2) TDBRAIN [68] is an EEG dataset with a binary label assigned to each sample, indicating whether the subject has Parkinson' 's disease. (3) ADFTD [69, 19] an EEG dataset with three-class label for each sample, categorizing the subject as Healthy, having Frontotemporal Dementia, or Alzheimer's discase. (4) PTB [70] is an ECG dataset where each sample
7

---

Table 3: Results of Subject-Dependent Sctup. The training. validation, and test split based on samples
according to a predetermined ratio. Results the ADFTD dataset under this setup are presented here.
Datasets
Models
Accuracy
Precision
Recall
score
AUROC
AUPRC
Autoformer
87.8311.66
87.63_166
87.22.1.97
87. 1..3811.79
93.82.11.44 96,5910.s
Crossformer
89.35
89.00-1M
88.7911.37
88.8811.40
97.52.0..8
95,4511.08
FEDformer
77.63+2.37
76. 76.76=211
76.68=22448
76.60=2.6
91.67=1334
84.94=2111
Informer
90.93
90.74=071
10.50:1.14
90.60
98,19 19
96.51
iTransformer
64.90+0.25
62.53=a27
62.21 +0.26
62.25
81 -1.52=0.29
68.87++0.9
ADFTD
MTST
65.08+0.69
63.8 63.85=000
62.7 +0,64
+
63.03=a.58
81 $1.33600.56
69.34=0089
(3-Classes)
Nonformer
96. 16.47
95, 0.556
95.99
95,9610.47
99.59 10,599.0. 10.09
99.08
PatchTST
66.26:10.90
65.08=041
64.97
64.95
83 =3.07
71.70
Reformer
11.51
91.15=11.9
91.65
11.14
98. 20.35
97.88
Transformer
97.00
96.87
96,8610.8
96.8610.44
99.75 10.00
99 10.07
Medformer (Ours)
97.62-20.34
97.53-0.3
97.48
97.50=0.36
99,83=0005
99.62=t011
labeled with a binary indicator of Myocardial Infarction. (5) PTB-XL [71] is an ECG dataset with five-class label for each sample, representing various heart conditions. Table provides information on the processed datasets. For additional details on data characteristics, train-validation-test splits under different setups, and data preprocessing, please see Appendix B.
Baselines. We compare with 10 state-of-the-ar time series transformer methods: Autoformer [28], Crossformer [33], FEDformer [52]. Informer [29], iTransformer [31]. MTST [53], Nonformer [54],
PatchTST [32], Reformer [56], and vanilla Transformer [30].
Implementation. We employ six evaluation metrics: accuracy. precision (macro- aveerred, recall (macro- averaged), F1 score (maero-averaged), AUROC (macro-averaged), and AUPRC (macro- averaged). The training process is conducted with five random seeds (41-45) on fixed training, validation, and test sets to compute the mean and standard deviation of the models. All experiments are run on an NVIDIA RTX 4090 GPU and a server with 4 RTX A5000 GPUs.
For data augmentation methods, we provide six widely used methods time series augmentation [72, 66, 73, 61]. For more details about these six methods, see Appendix A. For the parameter tuning in our method and all baselines, we employ layers for the encoder, set the dimension D to 128, and the hidden dimension of feed-forward networks to 256. We utilize the Adam optimizer with a learning rate of le-4. The batch size is set to [32,32, 128,,128,.12 PTB, and VTB-XL, respectively. The training epoch is set to 100, with early stopping triggered after 10 epochs without improvement the Fl score on the validation set. We save the model with the best Fl score on the validation set and evaluate it on the test set. See Appendix C for any additional implementation details of our method and all baselines.
for datasets APAVA, TDBrain, ADFTD,
5.1 Results of Subject-Dependeri
Setup. In this setup, the training, validation, and test sets split based on samples. All samples from all subjects are randomly shuffled and distributed into the training, validation, and test sets according predetermined ratio, allowing samples from the same subject to appear in three sets simultancously. As discussed in the Preliminaries section 3, this setup has limited applicability for MedTS- discase diagnosis in real-world scenarios. It is usually used evaluate whether the dataset exhibits cross-subject features quickly. The results obtained from this setup are typically much higher than those from the ienpeettindeeenneent setup, showing a dataset's 's upper limit of learnability.
Results. We evaluate the EEG dataset ADFTD using this setup to provide a direct comparison of results with the subject-independent setup. The results are presented in Table 3. Our method outperforms all the baselines, achieving the top-1 results in all six evaluations, with an impressive Fl score of 97.50%. Notably, baseline methods like Informer, Nonformer, Reformer, and Transformer also demonstrate strong performance, achieving Fl scores exceeding 90%. The overall results indicate the presence of discernible and learnable features related Alzheimer's Disease within this dataset.
5.2 Results of Subject-Independen
Setup. In this setup, the training, validation, and test sets are split based on subjects. All subjects and their corresponding samples are distributed into the training, validation, and test sets according to a predetermined ratio subject IDs. Samples from the same subjects should exclusively appear in one
8

---

Table 4: Results of Sunjeet-Indeeendet Setup. training, validation, and test sets are distributed based on subjects according to predetermined ratio/IDs. Results of the APAVA, TDBrain, ADFTD, PTB, and PTB-XL. datasets under this setup are presented here.
Datasets
Models
Accuracy
Precision
Recall
F1 score
AUROC
AUPRC
Autoformer
68.64 18.64.1.82
68.48_2.10
68.77
68.00 68.061194
15.94
74.38 14.05
Crossformer
13.77
79.29-436
68.8621.70
68,93
72.39
72.05+366
FEDformer
74.9412.15
1..5
73.5613.55
33.5
83.72.1197
82.9412.77
Informer
73.11+4.00
75.17=606
69.17+4.56
69.47=506
70.46=491
70.75+5.27
iTrrasformer
1.655
74.77=2.10
71.76+1.72
72.30=1.79
85.59=1.55
84.39
APAVA
MIST
1..14
79.30=097
65.277+2.28
64.01
68.87
71.06
(2- (-Classes)
Nonformer
71.89
71.80=4.58
69.44
69.74
70.78 70.55
PatchTST
6..03
78.76=12x
59,91
55.97+310
65.65=0025
67. +0.76
Reformer
78.7012.00
82.50_335
75.00
15.93
73.94
76. 1..04
Transformer
76.3014472
59.64_595
73.0915.01
73.75
73.23 12.50.16.00
Medformer (Ours)
78.74:0.44
81.1 1 =084
75.40
26.31
83.20
83.662092
Autoformer
87.33+3.79
88.06=356
87.33+4.79
87. 87.26=384
=3.81
93.32+2242
Crossformer
11.5612.19
212
81.56.1219
81.501220
91.20
91.51
FEDformer
18.13
181
78.13
78.04
16.56
86.48
Informer
89.02.+255
89.43=2.14
89.02-12.50
88,982254
96.64-0068
96.75-106
iTransformer
1..67
1.6
74.67.1..6
1.06
13.37
13.73
TDBrain
MTST
76.9643.76
77.24=3.99
76.96+3.76
76.88+383
85.27=4446
82.81+5.64
(2-Classes)
Nonformer
87.88+2488
88.86=184
87.88+2.45
87.78+2.6
97.05=0.68
96.99=0.68
PatchTST
79.25
79.60=4.99
79.25
79.20
87.95
86.36+6.67
Reformer
87.92+2m1
88.64=1440
87.92+2001
87.85
96.30=a054
96.40#0.45
Transformer
87.1711.67
87
87.17
87.101168
96.2810.92
96.341081
Medformer (Ours)
89.62.1081
89.68_0.7
89.62
89.621081 1081
96.41
96.51
Autoformer
45.25
43.67=194
42.96++2.18
42.59
61,02
43.10#22.0
Crossformer
50.45
45.57=143
45.88+1.2
45.50=1.0
66.45=2m
48.33+2215
FEDformer
46.3010.59
46.05L0.76
41.22.11.8
43.91
62.62.1175
46.11
Informer
48.4511.96
16.54_118
46.0611.84
45.74
65.87
47.6011.30
iTrrnsformrr
$2.60
46.79=127
4..28
46.79=113
67.26:1116
4..53+1.21
ADFTD
MTST
45.60
44.70=133
45.05
14.
62.50=2081
45.161085
(3 (3-Classe))
Nonformer
49.95
47.71=097
47.46+1.50
46.96=135
66.23
47.33+1178
PatchTST
44.37=005
42.40-1.13
42,06+1.45
41.97=137
60.08=1.50
42.49+1.79
Reformer
10.78
1..64
49.89
47.94
19.17
11.73
Transformer
50.4742.14
49.13=183
48.01
48.09
1.39
48.93++202
Medformer (Ours)
53.27:1.54
1.50
10.51
10.65
1.19
51.21
Autoformer
73.35+2.10
72.11 2299
63.24
63.69=384
38.54
74.25
Crossformer
80.17:3.79
85.04-183
71.25
72.75=7.19
88.55=345
87.31 +3.25
FEDformer
76.05
37.58=301
66.10+3.55
67.14
84.93
82.59+3.42
Informer
78.69
82.87=102
69.19+2.00
70.84=347
92.09
90.02++0.01
[Transformer
83.8910.71
88.25 1.18
76.3911.01
1.00
11.18
90.93
PTB
MIST
1..59
66.31 190
67.381371
86.86.12.75
83.75
(2-Classes)
Nonformer
78.66:0.49
-2.77=086
69.11220.87
10.90=100
89.37
86.67
PatchTST
1..7
155
63.89
64.3623.88
88.79.1091
83.39
Reformer
77.9642.1
81.72-161
68.20+3.35
69.65
91.13=0074
88.42+1.30
Transformer
1..37+1.02
81.84=066
67.14+1.80
68.47+219
1..0
87,22+1.68
Medformer (Ours)
83.50
85.19
77.11
79.18
92.81
90.32
Autoformer
61.68
11.60=164
49.10 19.10
48.85
82.04=1144
11.93
Crossformer
73.30=a.14
65.06=035
61.23
62.59
90.02-0006
67.43++0.22
FEDiormer
47.20
02
49.04447.6
47.89=844
82, 13.417
52.31 #7.08
Informer
70.43
59.12
60.44
88
64.76
TTransorrme
69.28
49,59=045
54.62+0.15
56.20=a19
86.71
60.27 +021
PTB-XL (5-Classes)
MTST
72.1410.27
63.84_.072
60.01
61.43
10.97
65.83
Nonformer
10.56
61.57_0066
57.75 10.72
59.101.066
88.32.10.6
63.40
PatchTST
73.23
65.70=0064
60.82+0.76
62.61:0.44
89.74-0.19
67.32+0m2
Reformer
71.72-10:4
63.12=10
59.2022075
18
88. 08.80
64.7210.7
Transformer
70.59+0.44
61.57=as5
57.62+0035
59.05=az5
88.21-9.16
63.36+029
Medformer (Ours)
72.87+0.23
64.14=042
60.60+0.46
62.02+a.77
89.66-8.13
66.39+am

---

of these three sets. This setup simulates real-world MedTS- based disease diagnosis, aiming to train model on subjects with known labels and then test it on unseen subjects determine if they have specific disease. The challenges associated with this setup are discussed in section 3. All five datasets are evaluated using this setup.
Results. Table 4 presents the results of the subject-independent setup. Our method achieves the top-1 Fl scores on 4 out of 5 datasets. Overall, our method achieves 15 top-1 and 30 top-3 rankings out of 30 eval- uations conducted across 5 datasets and 10 baselines, considering 6 different metrics. Fig- ure 4 provides an overview heatmap table of average rank across 5 datasets on 6 met- rics for all methods. Lower rank numbers indicate better results, with rank represent- ing the best performance among all methods. Our method demonstrates the best average rank among all methods across the 6 met- rics. Additionally, it is notable that the re- sult for ADFTD is a 50.65% F1 score un- der the subject-independen setup, which is significantly lower than the 97.50% F1 score achieved under the subject-dependet setup. This comparison highlights the challenge of the subject-independent setup.
Average Rank of Models (Lower is better)
Autaformer
9.2
9.6
9.0
9.0
8.4
8.6
Crossformer
4.4
5.2
5.4
5.6
5.0
FEDrormer
8.2
8.8
8.2
8.4
7.8
7.8
Informer
5.2
5.2
Irranformer
5.6
6.8
6.4
6.4
5.6
4.8
MTST
Nonformer
6.2
5.8
5.2
5.0
5.6
6.2
-5
PatchTST
8.4
7.0
8.4
8.2
7.8
8.4
Reformer
3.8
3.8
3.6
3.8
4.0
3.8
Transformer
5.4
5.6
5.6
5.2
5.4
5.8
Medformer
1.6
1.8
1.4
1.4
2.4
Accuracy
Precision
Recall
score
AUROC
AUPRC
Figure 4: Average Rank of Jebeeet-Independent Setup. The heatmap table shows the average rank of Medformer and 10 baselines across datasets using the subject- independent setup. A lower number indicates better results. The average rank is calculated across the 5 datasets to ab- tain the overall average rank.
5.3 Ablation Study and Additional Experiments
Ablation Study. 1) Module Study: We conduct a module study to evaluate each proposed mechanism in our method (Appendix D.1). 2) Patch Length Study: We perform parameter tuning on the patch lengths to evaluate the effectiveness of multi-grann culti-granularitiss (Appendix D.2). Additional Experiments. We also perform experiments two human activities recognition datasets [74, 75] demonstrate the learning ability of our model on general time series with potential channel correlations (Appendix E).
Conclusion and Limitations
Conclusion This paper presents Medformer, uulti-granuulaitty patching transformer tailored for MedTS classification. We introduce three novel mechanisms that leverage the distinctive features of MedTS. These mechanisms include cross-channel patching capture multi-timestamp and cross- channel features, multi caalti-gaaulariity embedding to learn features at various scales, and a two-stage multi- quatilgranullarity attentttenionn mechanism to extract features both within and across granularities. Ex- perimental results across five datasets, evaluated against ten baselines under the setup, demonstrate the effectiveness of our method, showing its potential for real-world applications.
Limitations and Future Work The design of Medformer allows for inputting various patch lengths, offering opportunities and challenges. While varying patch lengths have been shown to outperform uniform lengths in many cases (see Appendix D.2 and Appendix C), not all patch length combi- nations yield optimal results. Some combinations may perform worse than uniform patch lengths, necessitating careful tuning of patch lengths. Future work could explore mechanisms for automat- ically selecting patch lengths. Additionally, developing modules that decompose subject-specific features from rusateelated features could further enhance learning under the subject-independen setup, presenting an intriguing direction for future research.
Acknowledgments and Disclosure of Funding
This work is partially supported by the National Science Foundation under Grant No. 2245894. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funders.
10

---

References
[1] Yara Badr, Usman Tariq, Fares Al-Shargie, Fabio Babiloni, Fadwa Al Mughairbi, and Hasan Al-Nashash. A review on evaluating mental stress by deep learning using eeg signals. Neural Computing and Applications, pages 1-26, 2024.
[2] Hamdi Altaheri, Ghulam Muhammad, Mansour Alsulaiman, Syed Umar Amin, Ghadir Ali Altuwaijri, Wadood Abdul, Mohamed A Bencherif, and Mohammed Faisal. Deep learning techniques for classification of electroencephallgramn (ceg) motor imagery (mi) signals: review. Neural Computing and Applications. 35(20): (220:)::1611212220 2023.
[3] Xinwen Liu, Huan Wang. Zongjin Li, and Lang Qin. Deep learning in ecg diagnosis: A review. Knowledge- Based Systems, 227:107187, 2021.
[4] Fatma Murat, Ozal Yildirim, Muhammed Talo, Ulas Baran Baloglu, Yakup Demir, and U Ra- jendra Acharya. Application of deep learning techniques for heartheats detection using eeg signals-analysis and review. Computers in biology and medicine. 120:103726, 2020.
[5] Aniqa Arif, Yihe Wang, Rui Yin, Xiang Zhang, and Ahmed Helmy. Ef-net: Mental state recognition by analyzing multimodal eeg-fnirs via enn. Sensors, 24(6): 2024.
[6] Mahboobeh Jafari, Afshin Shoeibi, Marjane Khodatars, Sara Bagherzadeh, Ahmad Shalbaf, David López Garcfa, Juan M Gorriz, and U Rajendra Acharya. Emotion recognition eeg signals using deep learning methods: A review. Computers in Biology and Medicine, page 107450, 2023.
[7] Qiao Xiao, Khuan Lee, Siti Aisah Mokhtar, Iskasymar Ismail, Ahmad Lugman bin Md Pauzi, Qiuxia Zhang, and Poh Ying Lim. Deep cearning-based ecg arrhythmia classification: A systematic review. Applied Sciences, 13(8):4964, 2023.
[8] Zekai Wang, Stavros Stavrakis, and Bing Yao. Hierarchical deep learning with generative adversarial network for automatic cardiac diagnosis from ecg signals. Computers in Biology and Medicine, 10664, 2023.
[9] Dani Kiyasseh, Tingting Zhu, and David A Clifton. Clocs: Contrastive learning of cardiac signals across space, time, and patients. In International Conference on Machine Learning, pages 5606-5615. PMLR, 2021.
[10] Katerina D Tzimourta, Vasileios Christou, Alexandros T Tzallas, Nikolaos Giannakeas, Loukas G Astrakas, Pantelis Angelidis, Dimitrios Tsalikakis, and Markos G Tsipouras. Ma- chine learning algorithms and statistical approaches for alzheimer's disease analysis based on resting-state eeg recordings: A systematic review. International journal of neural systems. 31 31(05):213000,, 2021.
[11] Golshan Fahimi, Seyed Mahmoud Tabatabaci, Elnaz Fahimi, and Hamid Rajebi. Index of theta/alpha ratio of the quantitative electmoencephallgran in alzheimer's disease: case-control study. Acta Medica Iranica, pages 502-506, 2017.
[12] Salah Al-Zaiti, Christian Martin-Gill, Jessica K Zegre-Hemsey, Zeineb Bouzid, Ziad Fara- mand, Mohammad O Alrawashdeh, Richard E Gregg, Stephanie Helman, Nathan T Riek, Karina Kraevsky-Phille et Machine learning for eeg diagnosis and risk stratification of occlusion myocardial infarction. Nature Medicine, 29(7):1804-1
2023.
[13] Yihe Wang, Yu Han, Haishuai Wang, and Xiang Zhang. Contrast everything: A hierarchical contrastive framework for medical time-series. Advances in Neural Information Processing Systems, 36, 2024.
[14] Vernon Lawhern, Amelia Solon, Nicholas R Waytowich, Stephen M Gordon, Chou Hung, and Brent Lance. Begnet: compact convolutional neural network for eeg-based brmin-computer interfaces. Journal of neural engineering, 15(5):056013,, 2018.
[15] Hisaki Makimoto, Moritz Höckmann, Tina Lin, David Glöckner, Shqipe Gerguri, Lukas Clasen, Jan Schmidt, Athena Assadi-Schmidhe Alexandru Bejinariu, Patrick Müller, et al. Performance of a convolutional neural network derived from an eeg database in recognizing myocardial infarction. Scientific reports, 10(1):8445, 2020.

---

[16] Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
[17] Xiaocai Shan, Jun Cao, Shoudong Huo, Liangyu Chen, Ptolemaios Georgios Sarrigiannis, and graph convolutional network for alzheimer classification based on brain functional connectivity imaging of electroencephalogran. Human Brain Mapping, (177)))5144-2222) 2022.
Yifan Zhao.
[18] Dominik Klepl, Fei He, Min Wu, Daniel Blackburn, and Ptolemaios Sarrigiannis. Adaptive gated graph convolutional network for explainable diagnosis of alzheimer's 's disease using eeg data. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023.
[19] Andreas Miltiadous, Emmanouil Gionanidis, Katerina D Tzimourta, Nikolaos Giannakeas, and Alexandros T Tzallas. Dice-net: a novel convolution-transfooner architecture for alzheimer detection in eeg signals. IEEE Access, 2023.
[20] Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. Eeg conformer: Convolu- tional transformer for eeg decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710-719, 2022.
[21] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021.
[22] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Expo- nential smoothing transformers for time-series forecasting, arXiv preprint arXiv:2202.01381, 2022.
[23] Xue Wang. Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. Card: Channel aligned robust blend transformer for time series forecasting. In International Conference on Learning Representations, 2023.
[24] Minghao Liu, Shengqi Ren, Siyuan Ma, Jiahui Jiao, Yizhou Chen, Zhiguang Wang, and Wei Song. Gated transformer networks for multivariate time series classification. arXiv preprint arXiv:2103.14433 2021.
[25] Zekun Li. Shiyang Li, and Xifeng Yan. Time series as images: Vision transformer for irregularly sampled time series. Advances in Neural Information Processing Systems, 36, 2024.
[26] Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. International Conference on Learning Representations, 2021.
[27] Junho Song, Keonwoo Kim, Jeonglyul Oh, and Sungzoon Cho. Memto: Memory-guidee transformer for multivariate time series anomaly detection. Advances in Neural Information Processing Systems, 36, 2024.
[28] Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with cuto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419-22430 2021.
[29] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.
[30] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[31] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. International Conference on Learning Representations, 2024.
12

---

[32] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.
[33] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizzig cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations, 2022.
[34] Uwe Herwig, Peyman Satrapi, and Carlos Schönfeldt-Lecuome Using the international 10-20 ceg system for positioning of transcranial magnetic stimulation. Brain topography, 16:95-99, 2003.
[35] Vincent Bazinet, Justine Y Hansen, and Bratislav Misic. Towards biologically annotated brain connectome. Nature reviews neuroscience, 24(12):747-760, 2023.
[36] Saeid Sanei and Jonathon A Chambers. EEG signal processing. John Wiley & Sons, 2013.
[37] Selcan Kaplan Berkaya, Alper Kursat Uysal, Efnan Sora Gunal, Semih Ergin, Serkan Gunal, and M Bilginer Gulmezoglu. A survey ecg analysis. Biomedical Signal Processing and Control, 43:216-235, 2018.
[38] Yunhe Gao, Mu Zhou, Di Liu, Zhennan Yan, Shaoting Zhang, and Dimitris Metaxas. data-scalable transformer for medical image segmentation: architecture, model efficiency, and benchmark. arXiv preprint arXiv: 2203002003,0111, 2022.
[39] Elon Musk et al. An integrated brain-machine interface platform with thousands of channels. Journal of medical Internet research, 21(10)ee1619, 2019.
[40] Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang, Qianying Huang, Florian Dubost, Daniel Rubin, and Christopher Lee-Messer. Self-supervised graph neural networks for improved electroencephalographi seizure analysis. In International Conference on Learning Representations, 2021.
[41] Chaoqi Yang. M Brandon Westover, and Jimeng Sun. Manydg: healthcare applications. In The Eleventh International Conference on Learning Representations, 2023.
Many-domain generalization for
[42] Xiaodong Qu, Zepeng Hu, Zhaonan Li, and Timothy Hickey. Ensemble methods and Istm outperformed other eight machine learning classifiers in an eeg-based bei experiment. In International Conference on Learning Representations, 2020.
[43] Dezhen Xiong, Daohui Zhang, Xingang Zhao, and Yiwen Zhao. Deep leamiing for emg-based human-machine interaction: A review. IEEE/CAA Journal of Automatica Sinica, 833::512-533, 2021.
[44] Yuanchao Dai, Jing Wu, Yuanzhao Fan, Jin Wang, Jianwei Niu, Fei Gu, and Shigen Shen. Mseva: A musculoskeletal rehabilitation evaluation system based on emg signals. ACM Transactions on Sensor Networks, 19(1):1-23, 2022.
[45] Yingying Jiao, Yini Deng, Yun Luo, and Bao-Liang Lu. Driver sleepiness detection from eeg and eog signals using gan and Istm networks. Neurocomputing, 408:100-111, 2020.
[46] Jiahao Fan, Chenglu Sun, Meng Long, Chen Chen, and Wei Chen. Eognet: a novel deep learning model for sleep stage classification based on single-channel eog signal. Frontiers in Neuroscience, 15:573194, 2021.
[47] Saifuddin Mahmud, Xiangxu Lin, and Jong-Hoon Kim. Interface for human machine interaction for assistant devices: A review. In 2020 10th Annual computing and communication workshop and conference (CCWC), pages
JEEE, 2020.
[48] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, Timesnet: Temporal 2d- variation modeling for general time series analysis. 2023.
[49] Jiecheng Lu, Xu Han, Yan Sun, and Shihao Yang. Cats: Enhancing multivariate time se- ries forecasting by constructing auxiliary time series as exogenous variables. arXiv preprint #xXiv2403.001633 2024.
13

---

[50] Magali T Schmidt, Paulo AM Kanda, Luis FH Basile, Helder Frederico da Silva Lopes, Regina Baratho, Jose LC Demario, Mario S Jorge, Antonio E Nardi. Sergio Machado, Jéssica N lanof, et al. Index of alpha/theta ratio of the electroencephalogamm: new marker for alzheimer's disease. Frontiers in aging neuroscience, 5:60, 2013.
[51] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv: 803.01271, 2018.
[52] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International PMLR, 2022.
Conference on Machine Learning, pages
[53] Yitian Zhang, Liheng Ma, Soumyasundar Pal, Yingxue Zhang, and Mark Coates. Multi- resolution time-series transformer for long-term forecasting. In International Conference on Artificial Intelligence and Statisties, pages 4222-4230. PMLR, 2024.
[54) Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 15:9881-9899 2022.
[55] Peng Chen, Yingying ZHANG, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. The Twelfth International Conference on Learning Representations, 2024.
[56] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference on Learning Representations, 2019.
[57] Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. survey on large language model (Ilm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211, 2024.
[58] Alexey Dosovitskiy. An image is worth 16x words: Transformers for image recognition at scale. ICLR, 2021.
[59] Fudong Lin, Summer Crawford, Kaleb Guillot, Yihe Zhang, Yan Chen, Xu Yuan, Li Chen, Shelby Williams, Robert Minvielle, Xiangming Xiao, et Mmst-vit: Climate change-aware crop yield prediction via multi-modal spatial-temporal vision transformer. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pages 5774-5784, 2023.
[60] Amin Shabani, Amir Abdi, Lili Meng. and Tristan Sylvain. Scaleformer: Iterative multi-scale refining transformers for time series forecasting. arXiv preprint arXiv:2206 04038, 2022.
[61] Rushuang Zhou, Lei Lu, Zijun Liu, Ting Xiang, Zhen Liang, David A Clifton, Yining Dong, and Yuan-Ting Zhang. Semi-supervised learning for multi-label cardiovascular diseases prediction: a multi-dataset study. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[62] Joseph Y Cheng, Hanlin Goh, Kaan Dogrusoz, Oncel Tuzel, and Erdrin Azemi. Subject- contrastive learning for biosignals. arXiv preprint arXiv:2007.00 200,0087,, 2020.
[63] Isabela Albuquerque, João Monteiro, Olivier Rosanne, Abhishek Tiwari, -ean-François Gagnon, and Tiago H Falk. Cross-subject statistical shift estimation for generalized plectrencephalograppy--bbe
mental workload assessment. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pages 3647-3653. IEEF, 2019.
[64] Chaoqi Yang, M Brandon Westover, and Jimeng Sun. Manydg: Many-domain generalization for healthcare applications. In The Fleventh International Conference on Learning Representations, 2022.
[65] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International conference on machine learning, pages 1597-1607. PMLR, 2020.
14

---

[66] Zhihan Yue, Yujing Wang. Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 3980-8987, 2022.
[67] J Escudero, Daniel Abásolo, Roberto Hornero, Pedro Espino, and Miguel López. Analysis of electroencephallgrams in alzheimer's 's disease patients with multiscale entropy. Physiological measurement, 27(11):109) 2006.
[68] Hanneke Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns. The two decades brainclinics research archive for insights in neurophysiology (tdbrain) database. Scientific data, 9(1):333, 2022.
[69] Andreas Miltiadous, Katerina Tzimourta, Theodora Afrantou, Panagiotis Ioannidis, Niko- laos Grigoriadis, Dimitrios G Tsalikakis, Pantelis Angelidis, Markos G Tsipouras, Euripidis Glavas, Nikolaos Giannakeas, al. A dataset of scalp eeg recordings of alzheimer's disease, frontotemporal dementia and healthy subjects from routine eeg. Data, 8(6):95, 2023.
[70] PhysioToolkit PhysioBank. Physionet: components of new research resource for complex physiologic signals. Circulation, 101(2)):e21-=:220, 2000.
[71] Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima I Lunze, Wojeiech Samek, and Tobias Schaeffter. Ptb-xl, large publicly available electrocardiography dataset. Scientific data, 7(1):1-15, 2020.
[72] Johannes Pöppelhaum, Gavneet Singh Chadha, and Andreas Schwung. Contrastive learning based self-supervised time-series analysis. Applied Soft Computing, 117:108397, 2022.
[73] Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised contrastive pre-training for time series via time-freque frequeryyy consistency. Advances in Neural Information Processing Systems, 35:3988-445 0088-400,1100 2022.
[74] Prabhat Kumar and S Suresh. Flaap: An open human activity recognition (har) dataset for learning and finding the associated activity patterns. Procedia Computer Science, 212:64-73, 2022.
[75] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, Jorge Luis Reyes-Ortiz, et al. A public domain dataset for human activity recognition using smartphones. In Esann, volume page 3, 2013.
[76] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, Timesnet: Temporal 2d darvaiiatoon modeling for general time series analysis. In International Conference on Learning Representations, 2023.
[77] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.0077 2023.
15

---

Appendix A Data Augmentation Banks
In the embedding stage, we apply data augmentation to the patch embeddings. We utilize bank of data augmentation techniques enhance the model's robustness and generalization. During the forward pass in training, each patch will pick one augmentation from available augmentation options with equal probability. The data augmentation methods include temporal flipping. channel shuffling, temporal masking, frequency masking, jittering, and dropout, and can be further expanded to more choices. We provide detailed description of each technique below.
Temporal Flippling We reverse the MedTSS data along the temporal dimension. The probability of applying this augmentation is controlled by parameter prob, with default value of 0.5.
Channel Shuffling We randomly shuffle the order of MedTS channels. The probability of applying channel shuffling is controlled by the parameter prob, also set by default to 0.5.
Temporal Masking We randomly mask some timestamps across all channels. The proportion of timestamps masked is controlled by the parameter ratio, with a default value of 0.1.
Frequency Masking First introduced in [73] for contrastive learning, this method involves converting the McdTS data into the frequency domain, randomly masking some frequency bands, and then converting it back. The proportion of frequency bands masked is controlled by the parameter ratio. with a default value of 0.1.
Jittering Random noise, ranging from 0 to 1, is added to the raw data. The intensity of the noise is adjusted by the parameter scale, which is set by default to 0.1.
Dropout Similar to the dropout layer in neural networks, this method randomly drops some values. The proportion of values dropped controlled by the parameter ratio, with a default setting of 0.1.
Appendix B Data Preprocessing
B.1 APAVA Preprocessing
The Alzheimer's s Patients' Relatives Association of Valladolid (APAVA) dataset', referenced in the paper [67], is public EEG time series dataset with 2 classes and 23 subjects, including 12 Alzheimer's s disease patients and 11 healthy control subjects. On average, each subject has 30.0 + 12.5 trials, with each trial being a 5-second time sequence consisting of 1280 timestamps across 16 channels. Before further preprocessing, each trial is scaled using the standard scaler. Subsequently, we segment each trial into halr-onvrl samples, where each sample is a 1-second time sequence comprising 256 timestamps. This process results in 5,967 samples. Each sample has a subject II) to indicate its originating subject. For the training, validation, and test set splits, we employ the jubject-independeett setup. Samples with subject IDs 11,16,19,201 and 1,2,17,18 } are assigned to the validation and test sets, respectively. The remaining samples are allocated to the training set.
B.2 TDBrain Preprocessing
The TDBrain dataset", referenced in the paper [68], is large permission-accessibbee EEG time series dataset recording brain activities of 1274 subjects with 33 channels. Each subject has two trials: one under eye open and one under eye closed setup. The dataset includes a total of 60 labels, with each subject potentially having multiple labels indicating multiple diseases simultaneously. In this paper, we utilize a subset of this dataset containing 25 subjects with Parkinson's 's disease and 25 healthy controls, all under the eye-closed task condition. Each eye-closed trial is segmented into non-overlapping 1-second samples with 256 timestamps, and any samples shorter than 1 I-second are discarded. This process results in 6,240 samples. Each sample is assigned subject IDD indicate its originating subject. For the training. validation, and test set splits, we employ the subject-independent setup. Samples with subject IDs [18, 19,20,21,,4,,44 46,47,48,44 are assigned to the validation set, while are assigned to the test set. The remaining
samples with subject IDs samples are allocated to the training set.
"hapss//osfiiojbysssr

---

B.3 ADFTD Preprocessing
The Alzheimer's Disease and Fron'T Tronmooeplrral Dementia (ADFTD) dataset', referenced in the papers [69, 19]. a public EEG time series dataset with 3 classes, including 36 Alzheimer's disease (AD) patients, 23 Frontotemporal Dementia (FTD) patients, and 29 healthy control (HC) subjects. The dataset has 19 channels, and the raw sampling rate is 500Hz. Each subject has trial, with trial durations of approximately 13.5 minutes for AD subjects (min=5.1, max=21.3), 33), 12 minutes for FD subjects (min=7.9, max=16.9), and 13.8 minutes for HC subjects (min= 12.5, max=16.5). A bandpass filter between 0.5-45Hz is applied to each trial. We downsample each trial to 256Hz and segment them into non-overlapping I-second samples with 256 timestamps, discarding any samples shorter than second. This process results in 69, 19,752 samples. For the training, validation, and test set splits, we employ both the subject-dependent and subject-independet setups. For the subject-dependent setup, we allocate 60%, 20%, and 20% of total samples into the training, validation, and test sets, respectively. For the subject-independen setup, we allocate 60%, 20%, and 20% of total subjects with their corresponding samples into the training, validation, and test sets, respectively.
B.4
PTB Preprocessing
The PTB dataset', referenced in the paper [70], is public ECG time series recording from 290 subjects, with 15 channels and a total of 8 labels representing 7 heart diseases and health control. The raw sampling rate is 1000Hz. For this paper, we utilize subset of 198 subjects, including patients with Myocardial infarction and healthy control subjects. We first downsample the sampling frequency to 250Hz and normalize the ECG signals using standard scalers. Subsequently, we process the data into single heartbeats through several steps. We identify the R-Peak intervals across all channels and remove any outliers. Each heartbeat is then sampled from its R-Peak position, and we ensure all samples have the same length by applying zero padding to shorter samples, with the maximum duration across all channels serving as the reference. This process results in 64,356 samples. For the training, validation, and test set splits, we employ the subject-independen setup. Specifically, we allocate 60%, 20%, and 20% of the total subjects, along with their corresponding samples, into the training, validation, and test sets, respectively.
B.5 PTB-XL Preprocessing
The PTB-XL dataset', referenced in the paper [71], is large public ECG time series dataset recorded from 18,869 subjects, with 12 channels and labels representing 4 heart diseases and 1 healthy control category. Each subject may have one or more trials. To ensure consistency, we discard subjects with varying diagnosis results across different trials, resulting 17,596 subjects remaining. The raw trials consist of 10-second time intervals, with sampling frequencies of 100Hz and 500Hz versions. For our paper. we utilize the 500Hz version, then we downsample to 250Hz and normalize using standard scalers. Subsequently, each trial is segmented into non-overlapping 1-second samples with 250 timestamps, discarding any samples shorter than second. This process results in 190,400 samples. For the training, validation, and test set splits, we employ the subject-independent setup. Specifically, we allocate 60%, 20% and 20% of the total subjects, along with their corresponding samples, into the training, validation, and test sets, respectively.
Appendix C Implementation Details
We implement our method and all the baselines based on the Time-Series--ibrat project" from Tsinghua University [76], which integrates all methods under the same framework and training techniques to ensure a relatively fair comparison. The 10 baseline time series transformer methods are
Autoformer [28), Crossformer [33], FEDformer [52], Informer [29], iTransformer [31], MTST [53],
Nonformer [54], PatchTST [32], Reformer [56], and vanilla Transformer [30].

---

For all methods, we employ 6 layers for the encoder, with the self-attention dimension D set to 128 and the hidden dimension of the feed-forward networks set to 256. The optimizer used is Adam, with learning rate of le-4. The batch size is set to (32,32,128, 12,,2,128,,28,123 128] for the datasets APAVA, TDBrain, ADFD, PTB, and PTB-XI., respectively. Training is conducted for 100 epochs, with early stopping triggered after 10 epochs without improvement in the F1 score on the validation set. We save the model with the best Fl score on the validation set and evaluate it on the test set. We employ six evaluation metrics: accuracy, precision (macro-averagee averaged), AUROC (macro-averaged), and AUPRC (macro-averaged), Both subject-dependent and boojecttindependdent setups are implemented for different datasets. Each experiment is run with 5 random seeds (41-45) and fixed training, validation, and test sets to compute the average results and standard deviations.
recall (macro-averaged), Fl score (macro-
Medformer (Our Method) We use a list of patch lengths in patch embedding to generate patches with different granularities. Instead of flattening the patches and mapping them to dimension D during patch embedding, we use conv2d network to directly map patches into a 1-D representation with dimension D. These patch lengths can vary. including different numbers of patch lengths such as {2, 4, 8, 16), repetitive numbers such as {8, 8, 8, 8), or a mix of different and repetitive lengths such as {8, 8, 8, 16, 16, 16). It is also possible to use only one patch length, such as {8), which indicates a single gramularity. The patch lists used for the datasets APAVA, TDBrain, ADFD, PTB, and PTB-XL are (2, 2, 2, 4, 4, 16, 16, 16, 16, 32, 32, 32, 32, 32), {8. 8, 8, 16, 16, 16), {2. 4. 8, 8, 16, 16, 16, 16, 32, 32, 32, 32, 32, 32, 32, 32). and {2, 8, 8, 8, 16, 16, 16, 32, 32, 32, 32, 32, 32, 32, 32), respectively. The data augmentations are randomly chosen from a list of four possible options: none, jitter, scale, and mask. The number following each augmentation method indicates the degree of augmentation. Detailed descriptions of these methods can be found in Appendix A. The augmentation methods used for the datasets APAVA, TDBrain, ADFD, PTB, and PTB-XL are [none, drop0.35), (none, drop0.25), (drop0.5), (drop0.5), and [jitterO.2, scale0.2, drop0.5), respectively.
{2,4, 16, 16, 16, 32, 32, 32, 32,322, 32).
Autoformer Autoformer [28] employs auto-correlation mechanism to replace self-attention for time series forecasting. Additionally, they use a time series decomposition block to separate the time series into trend-cyclical and seasonal components for improved learning. The raw source code is available at httss///ittub.comm Sattumm/AAuofoooo Autoformer.
Crossformer Crossformer [33] designs a single-channel patching approach for token embedding. They utilize two-stage self-attention to leverage both temporal features and channel correlations. router mechanism is proposed to reduce time and space complexity during the cross-dimension stage.
The raw code is available at https://ittub..cmm/iie www.ooooooao-----. wwwinhaabb-ntt//rrraaaeooo SJTU/Crossformer.
FEDformer FEDformer [52] leverages frequency domain information using the Fourier transform. They introduce seguueccy-enaanee blocks and frequency- enhhnned attention, which are computed in the frequency domain. A novel time series decomposition method replaces the layer norm module in the transformer architecture improve learning. The raw code is available at hitps:
Informer Informer [29] is the first paper to employ a one-forward procedure instead of autoregres- sive method in time series forecasting tasks. They introduce ProbSparse self-attention to reduce com- plexity and memory usage. The raw code is available at http:/ghwwu..ooo/oooaaa/////noo/
iTransformer iTransformer [31] questions the conventional approach of embedding attention tokens in time series forecasting tasks and proposes inverted approach by embedding the whole series of channels into a token. They also invert the dimension of other transformer modules, such as the layer norm and feedffrrarrd networks. The raw code is available at https://github. wom/thuml//Transstt
MTST MTST [53] uses the same token embedding method as Crossformer and PatchTST. It high- lights the importance of different patching lengths in forecasting tasks and designs a method that can take different sizes of patch tokens as input simultaneously. The raw code is available at
Nonformer Nonformer [54] analyzes the impact of non-stationarity in time series forecasting tasks and its significant effect on results. They design a de-stationary attention module and incorporate normalization and denormalization steps before and after training to alleviate the oversstationairizaionn Transformers.
problem. The raw code is available at
3

---

Table 5: Module Study.
Datasets
Models
Accuracy
Precision
Recall
F1 score
AUROC
AUPRC
No Inter-Attention
16.90
78,08 12
73.87
74.59
81.32.1.37 80.29
APAVA
No Augmentation
2292
76.69-341
71.72-12:
32.30=3.46
a7.05
78.15
Single-Chaneel Patching Medformer
13.08
76.43=1146
68.62199
68. 22.37
29.5442044
69.4311
78.74.0044
81,11 0,84
75,40
76.31
83.20
83.661..9
No Inter-Attention
18.17
88.27=002
88.17=072
88. 18
96.062040
96.18
TDBrain
No Augmentation
88.56=4066
88.67
88. 88.56=0.66
88.55
96.11
96.20+0.39
Single- Ghngle-Channel Patching Medformer
80,94
81.84=1.55
80,94=095
80.81 =0.92
89. 99.65=005
89.48+0.91
89.62:201
89.68
89.62_0.81
89.62:0.81
26.41
26.51
No Inter-Attention
52. 32.14=1111
51.13=2557
46.15 16:15:00.6
45.59=1.15
67. 67.99
49.68
ADED
No Augmentation
49.99.1636
48 16616
44, 44.59
44,07 14.75
65.03..6.2
47.1 77.11.5..66
Single- Chagle-Channel Patching Medformer
47.09
45.42=:130
43.94 0.80
44.11=0.44
62.07
44.57
53.27:1154
51,02:1.57
50.71
50.65
7..93:1.19
:1.21
No Inter-Attention
78.02=2270
80.96=1.88
68.65=4558
69.97=5.27
22.94
90.19
No Augmentation
17.64::165
81.03=1
67.88
69.31
89.377
PTB
Chngle-Channel Patching Medformer
79.02.1162
81.14
70.43
22.24
85. 1.559
82.2311.49
83.50=2211
85.19=0094
77.11=3.99
79.18=331
92.81
9..32
No Inter-Atlention
=2.51
63.61=022
59.75=0.30
61.25=mo.2
89.48==ous
65.74+0.26
No Augmentation
72.68
63.99=0.22
59.73=0041
61.26=0034
89.49=0os
66.00=0.21
PTB-XL
Patching
72.79.0035
64.80
59.57
61.43
88 88.971..19
65.91
Medformer
72.87:0023
64.14
60.60=0.46
62.02
89.66=:011
66.39:0.22
PatchTST PatchTST [32] embeds a sequence of single-channel timestamps as patch token to replace the attention token used in the vanilla transformer. This approach enlarges the receptive field and enhances forecasting ability. The raw code is available at http:///iitbb.co/yygggiiee//Paat
Reformer Reformer [56] replaces dot-product attention with locality-sensitive hashing. They also use reversible residual layer instead of standard residuals. The raw code is available at https: //github.com//lciddeee htwpa//b..oomccccddaaneereoooooo pytorch.
Transformer Transformer [30], commonly known as the vanilla transformer, is introduced in the well-known paper "Attention is All You Need." It can also be applied to time series by embedding each timestamp of all channels as an attention token. The Py PyTorch version the code is available at 120/attention- is-all- all--ou- you- ceto--eed-pyyae pytorch.
Appendix D
Ablation Study
D.1 Module Study
To assess the efficacy of our proposed mechanisms iess--inteeeeee inter-granularigg self-attention, embedding augmentation, and multi-channel patching- conduct ablation studies on five datasets across three distinct settings: without inter inter-granule granularity attention, without embedding augmentation, and with single-channel patching. We maintain the other modules intact in each setting and fix all hyperparameters as described the implementation details C. Table 5 presents a comparison between our full Medformer model and these three variants. The complete Medformer model secures 28 top-1 and 30 top-2 rankings across 30 evaluations, demonstrating robust performance. We observe that each module significantly enhances performance: attention contributes to a 3.64% improvement in Fl score, embedding augmentation leads to 4. 4.46% increase and multi-channel patching results in a 6. 10% enhancement in Fl score. We find multi-channel patching particularly beneficial for results, especially in EEG data. Overall, these results underscore the critical role of each component in our design.
average, across the datasets, inter granulartty
D.2 Patch Length Study
To investigate the effects of gulti-granullrity and computational complexity. we conduct an empirical analysis using various patch lengths on the APAVA dataset. Table presents the evaluation results for different combinations of patch lengths. Initially, we compare the performance of models using single patch length against models using five identical patch lengths (e.g. (8) vs (8, 8, 8, 8, 8)). Our findings indicate that using repetitive patch lengths generally enhances performance, except when

---

Table 6: Patch Length Study
Datasets
Models
Accuracy
Precision
Recall
F1
AUROC
AUPRC
[2]
71.82 18.13
73.23 18.1
69.69 1631
69.95 17.08
69.34
14.72
69.18 15.17
[4]
75.72 +3.75
78.
#1.84
72.14 +3.39
72.83
+3.64
72.75 +4.76
73.84
#5.08
[8]
71.29 +3,02
72.83 +2.33
67.38 44.25
67.17 +4,9
76.12
+4.25
76.74 #4.29
[12]
69.77 14.01
69.72 +5.65
67.09 41,34
+3.33 67.36
75.19
+:300
75.36 #:.48
{16]
70.92 11.99
71.46
4:.09
67.67 42.26
67.81 #2.45
76.97
+:2.3
77.21 =2.87
[24]
71.68 12,44
74.26
1:3.9
67. 12.55
67.13 12.85
79.07
1:3.34
78.73
1.3.2
[32]
72.55 11.51
11.49 75.74
68,38 12.99
68, 13.23
79.17
12.117
78.44
11240
APAVA
[2,2,2,2.2]
65.52 18.24
65.97
17.41
64.14 16,06
63.71 17.23
63.15
13:43
61.84
1.4.8
[4,4,4,4,4]
76.91 +1.72
78.66
#1.20
73.71
#1.26
74,46 +1.42
+:121 74.90
76.36
#3.12
{8,8,8,888]
71.81 +381
74.25
16.34
67.72 4:45
67.89 13.68
1:534 74.95
75.59
1:5.63
112,12,12,12.12]
71.17 13,85
72.18
15.97
67.65
13.27
67.96 13,48
76.71
14.82
77.27
1:4.1
116.16.16.16.16,
71.13 13.33
72.14
15.69
67.82
12.45
68.13 12.58
14.52 76.34
76.39
144.4
(24.24.24.24.44)
73.18 12.15
75.72
13.46
68,98
12.11
69.27 12.33
81.10
1261
81.20
11.68
(22.22.32.3.332)
74.34 12.20
78.92
11.57
69.66
12.80
69.80
13342
81.11
11.10
80.69
11,02
(222,4.16322)
78.21 +2.00
80.82
44.30
74.92
+2,07
75.78
+2.31
80.73 +2.34
81.38 42.38
= suggesting that additional identical patch lengths can capture more information, analogous to multi-head attention mechanisms.
Furthermore, assess the performance of manually selected combination of varying patch lengths, specifically {2, 2, 4, 16, 32}. This configuration achieves the highest performance across all evaluated metrics, underscoring the effectiveness our designed attention module in accommodating multi- granularity patches. However, it is worth noting that mixing different patch lengths does not guarantee improved performance. See G for more detailed discussion.
Appendix E
Additional Experiments
Table 7: Additional Datasets and Methods
We selected three old baselines in the previous experiments that showed strong performance: Crossformer, Reformer, and Transformer. Additionally, we introduce three new baselines: TCN, MCdemnTCN, and Mamba. These six baselines experiments, TDBrain(6,22) 6,00 samples, 2 classes), and two HAR datasets: FLAAP (13, samples, 10 classes) and UCI-HAR (10,299 samples, classes). The bold number denotes the best result, and the underlined number denotes the second best.
evaluated one old dataset in the previous
TDBrain (2 Classes)
FLAAP (10 Classes
UCI-HAR (6 Classes)
Datasets
Metries
Models
Accuracy
Fl Score
Accuracy
F1 Score
Accuracy
Score
Crossformer
81.56+219
81,50+220
75.84 25.84
25.52+na6
89.74
39.70
Reformer
87 92.92
87.85
21.65
71.14:145
88 88.4413.
88. 14
Transformer
87.17+167
87.10=168
74.96+125
74.49+139
88. 19.866166
88. 18.80+167
TCN
80.92+294
80.82+30R
16.4821666
65.29
93.08:448
93.19:a88
ModernTCN
81.96+2112
81.79+223
84.80
74.35
91.44=110:
91.47+a9
Mamba
89.58+a24
89.58
64.87+275
64.14+270
87.78+1110
87.72+1.0
Medformer
89.62.na1
89.62:2011
76.44:004
76.25
91.61+a75
To evaluate the performance of our method on general time series, we test it on two human activity recognition (HAR) datasets: FLAAP [74] and UCI-HAR [75], which exhibit potential channel correlations inherently. Additionally, we compare our method with three other approaches: TCN [51], ModernTCN [16], and Mamba [77]. Our method achieves the highest top-1 accuracy and F1 score on TDBrain and FLAAP, and ranks second-best on UCI-HAR.
Appendix F Complexity Analysis
Let the number of timestamps T. and patch list (L,, I.y..., In be given, where the i-th patch length Ta produce N, [T/L.] number of patches. During intra-granularity attention, we perform self-attention among the patch embeddings within the same granularity. The total complexity is
5

---

O ( N²). During intra granularity attention, we perform self-attention among routers, with time complexity of Onn)). Therefore, the total time complexity is O (n² + > N?).
One potentially useful patch list is the power series {2 .2.....2. }" where 2nd < T. In this case, the complexity of intra-gramularity attention reduces as follows:
1
n
T
n
T
)1)
0
Σ
No
Σ
0O
Σ
2
2
Σ
11
T
+
T
+ 11)
=0
7
n
1
27
n
1
+
2
2
2
2
0 2T log ) -
The complexity of inter-granularity attention is O( < O(log T). Therefore, the total time complexity of the two-stage multi-granularity self-attention module is O(T²). which the same complexity as the vanilla transformer. This analysis demonstrates model's ability to incorporate different granularities without significantly increasing computational overhead.
Appendix G Discussion
G.1
Comparision with Other Frati-Granullaity Methods
MTST [53] and Pathformer [55] differ from our Medformer in three significant aspects: (1) Patch- ing & Embedding MTST and Pathformer utilize single-channel patching, presupposing channel independence. In contrast, Medformer employs multi-channel patching to capture potential channel correlations. (2) Granularity Interactions MTST assimilates sraiiirranulrrtyy information by concatenating outputs from different branches, while Pathformer uses adaptive pathways for weighted aggregation of these outputs without any inter-granularity interactions within the attention modules. In contrast, Medformer introduces a novel inter-granularity attention mechanism specifically designed for granularity interaction, thereby effectively integrating multi gualii-ranularityy information.
Scaleformer [60] operates as a agdel-agnostic structural framework that employs variable down- sampling and upsampling rates embeddings outside of attention modules. Although it integrates seamlessly with non-patching methods like Autoformer and FEDformer, its incorporation into patching methods is not straightforward and may result in sub-optimal patch representations [53]. Consequently, the design objectives of Scaleformer are largely orthogonal to ours, which concentrate on multi-granularity patching and attention mechanisms.
Appendix H Broader Impacts
Our proposed model demonstrates performance comparable to or surpassing state-of-the- art baselines on medical time series classification tasks. The model's design, which includes specialized patch- ing and self-attention mechanisms, specifically targets channel correlations and multi caaltiigraularttyy information. We anticipate our findings will encourage further research into effective strategies for capturing multi-scale information in medical time series data. Additionally, this work could broaden interest in medical time series classification, an area that remains less explored compared to time series forecasting.
Besides, different experiment setups based on medical perspectives, such as subject-dependent and subject-indeepnndet,, are evaluated to simulate real-world applications. On a societal level, our model has potential applications healthcare, such facilitating the diagnosis of diseases using medical time series data. For instance, it could be employed to detect neurological disorders through EEG data. However, practitioners should be cognizant of the model's 's limitations.

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in abstract and introduction are supported by the results in Section 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See Appendix G.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
7

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 5 and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
8

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
We include anonymous link in the Abstract providing source codes with full implementation details for our methods and all baselines. All five datasets used for evaluation are publicly available.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https//nips.cc/public
Sundess/oooSSummmiiiooooiigg for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (hitps: wwwabl/ceguiids/oodeeebiiseeeee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/ettails
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See Section 5 and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We run experiments over five random seeds and report the average value with the standard deviation. See Table 2-6.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section C.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The authors follow the NeurIPS Code of Ethics during the conduct of this research.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Appendix H.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
10

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not pose a high risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We properly cite baseline models and the code library used for implementation. See Appendix C.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswitheode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: This paper does not publish new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not include crowdsourcing experiments nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not include crowdsourcing experiments nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
12

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
13


Reviews:
[{"review_id": "PRfuMBzR7b", "reviewer": "Reviewer_AkiE", "summary": "In this paper, the authors introduced a multi-granularity patching transformer tailored specifically for medical time series classification. To leverage the characteristics of medical time series data, the model incorporated three unique features : cross-channel patching to leverage inter-channel correlations, multi-granularity embedding for capturing features at different scales, and two-stage (intra- and inter-granularity) multi-granularity self-attention for learning features and correlations within and among granularities. The model was well tested with several datasets and showed mixed results against other models.", "questions": "As mentioned in Appendix G, what domains or data sets should be tested next to test the ability of this approach?", "limitations": "Currently the proposed model was tested only in EEG and ECG domains, thus the scope is too narrow for the general audience of the NeurIPS.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Considering the charactaristics of time series data in medical domain, this approach is quite reasonable and is showing strong results especially in EEG datasets.", "weaknesses": "Because this model has been evaluated on very specific time series data, EEG and ECG, it is unclear whether this approach can be extended to many other time series data outside the medical domain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and concerns regarding our paper! We appreciate your thoughtful questions. Here are the reponses to your question and concern. If you feel our response does not fully justify a higher score, please let us know how we can further improve our work. Thank you again for your consideration.\n\n***\n\n**Q1**: Extension to general time series outside the medical domain.\n\n**A1**: Thank you for raising this concern. Our answer contains two parts.\n\n1) We agree our method is designed for medical time series due to the inherent correlations among channels from a medical perspective[4][5][6]. However, we believe our method has the potential to be extended to other domains as well. Specifically, our approach is particularly well-suited for multi-channel time series data where theoretical correlations exist among different channels. Our multi-granularity cross-channel patching effectively embeds these channel correlations in a single step. Potential domains for application include human vital signs for health monitoring and time series data collected from mobile sensors for human activity recognition.\n\n2) To further evaluate the generalizability of our method, we add three new datasets, two human activities recognition datasets FLAAP(13123 samples, 10 classes) and UCI-HAR(10299 samples, 6 classes), and one multi-modal vital signs(PPG and ECG) dataset MIMIC-PERform-AF(20400 samples, 2 classes) for atrial fibrillation classification. See R-Table 1 for the results in the pdf file attached to the general response. In this table, we compare with three old baselines with good performance so far, Crossformer, Reformer, and Transformer, and three new baselines TCN, ModernTCN, and Mamba. We evaluate our method and the six baselines on three new datasets and one old dataset, TDBrain. Our method demonstrates top-1 Accuracy and F1 score in three datasets and second-best in the remaining one.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PLJ046PQIM", "reviewer": "Reviewer_vj5L", "summary": "This paper proposes a multi-granularity patching Transformer for medical time series classification. The proposed model, Medformer leverages cross channel patching to learn inter-channel correlations. It utilizes multi-granularity embedding for learning temporal patterns. Finally, it takes 2 stage self-attention to aggregate information from multiple granularities. The experiments on 5 datasets show the effectiveness of Medformer.", "questions": "1.How about the performance of the medformer when it is applied to general time series classification? This experiments could demonstrate the advantages of the model.\n\n2.What are the new challenges of time series classification in medical datasets? How does Medformer resolve it?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.The paper is well structured and easy to follow.\n\n2.The problem of medical time series classification is important.\n\n3.The paper conducts several experiments on medical time series datasets and demonstrates the effectiveness of Medformer.", "weaknesses": "1.The novelty of this paper is not enough. The idea of leveraging multi-granularity and multi-channel features has been studied in time series field.\n\n2.Time series classification has also been well studied. The paper should clarify and focus the specific challenges for medical time series classification compared to general time series.\n\n3.For general time series classification, several new SOTA methods have been released, such as Time-LLM, ModernTCN, and Lag-llama, etc. Such recent studies should be compared in the experiments.\n\n4.The results of Medformer are not superior in several metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your suggestions on our paper! We are happy you feel our paper is well-structured and easy to follow. We respond to each of your questions and concerns. If you do not feel we have sufficiently justified a higher score, please let us know your further concerns and how to improve. Thank you again!\n\n***\n\n**Q1**: The idea of leveraging multi-granularity and multi-channel features has been studied in time series field.\n\n**A1**: We agree that some methods utilized multi-granularity or multi-channel features in the time series domain, as we compared or described in the related work. However, we are the first to combine multi-granularity with cross-channel patching to model cross-channel information across different levels of granularity. In particular, we design an embedding method for medical time series data, where different channels represent signals from different regions of the same organs (brain, heart, etc.). Tight correlations among the channels are evident and valuable from a medical perspective[4][5][6]. Therefore, our approach is tailored to capture these correlations effectively with one-step data embedding. In contrast, as we mentioned in the paper (lines 107-110), existing patching methods in time series transformers, such as those inherited from PatchTST [1][2][3], rely on single-channel patching. These methods are designed for time series forecasting in tasks like weather and energy prediction, where different channels may represent entirely different meanings and, therefore, follow a channel-independent strategy. Thus, our method exhibits a significant deviation from the other methods and is a rational design for the specific datasets and challenges.\n\n***\n\n**Q2**: The paper should focus on the challenges of medical time series classification. What are those challenges, and how does Medformer resolve them?\n\n**A2**: \n1) **Strong correlation between channels and features at a wide frequency range motivates our design of multi-granularity cross-channel patching**: Medical time series data are collected from human subjects using sensors and electrodes, usually resulting in multi-channel and multi-modal data. As we discussed in our previous response, different channels in medical time series often represent activities in various regions of the same underlying organs, e.g., brain, heart, etc., from a medical perspective. Therefore, strong inter-channel correlations exist naturally [4][5][6]. Moreover, biomarkers in these signals appear at a wide range of frequencies with a difference of up to 1~2 magnitude [6]. To utilize these features, we design multi-granularity cross-channel patching that enables one-step data embedding, followed by a two-stage intra-inter granularity self-attention mechanism to exploit these features further.\n\n2) **The correlation of samples from the same subject encourages us to explore a subject-independent training setup**. As we described in the preliminaries section(lines 124-135), for medical time series collected for disease diagnosis tasks, the ultimate goal is to predict the label of **Subject** instead of **Sample** to diagnose disease. In this case, samples from unseen subjects should not be utilized in the training to follow a real-world scenario. Many existing works do not consider subject ID information[7][8]. To better simulate real-world conditions, we categorize data splitting strategies into two setups: subject-dependent and subject-independent. For subject-dependent, samples are randomly split into training, validation, and test sets. For subject-independent, we split subjects into these three sets, ensuring that samples from the same subjects are exclusively contained within each set (See Figure 2). Tables 2 and 3 in the paper show how deceptively high results could be achieved with improper setup, leading to ineffective models in real-world applications.\n***\n\n**Q3**: Several new time series classification SOTA methods have been released, such as Time-LLM, ModernTCN, Lag-llama etc. These studies should be compared. \n\n**A3**: Thank you for recommending these exceptional works. We will review them and incorporate them into the related work section in our upcoming revision. We have included experiments for ModernTCN in Table R.1 of the attached PDF. Despite our best efforts, we could not obtain results from the two LLM-based models, Time-LLM and Lag-Llama, as they are primarily designed for time series forecasting. Fine-tuning these pre-trained models for classification tasks would require more computational resources than currently available. However, to ensure a comprehensive evaluation of our model, we add two additional methods, TCN[11] and Mamba[13], also in Table R.1.\n\n***\n\n**Q4**: How does the Medformer perform in general time series classification? \n\n**A4**: While our method is designed for medical time series using their characteristics, we believe it applies to general time series data, particularly multi-channel time series with intrinsic correlations among channels. To prove this, we add 3 new datasets to our experiments: FLAAP, UCI-HAR, and MIMIC-PERform-AF. The results are presented in R-Table 1, attached to the general response PDF. We evaluated our method and 6 baselines across the 3 new datasets and the previously used TDBrain dataset. Our method achieved the best F1 score on 3 datasets and the second-best on the remaining one.\n\n***\n\n**Q5**: Medformer's results are not superior in several metrics.\n\n**A5**: PatchTST outperforms our method on the PTB-XL dataset by 0.6 in F1 score. Based on the ablation study presented in Table 4, cross-channel patching does not provide as much benefit as other datasets. We guess that this might caused by the smaller number of channels (12) in PTB-XL and weak channel correlation, which limit the advantages of our cross-channel patching. However, we believe the other mechanism, such as intra-inter granularity self-attention, can be combined with PatchTST to improve the result on PTB-XL.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ca9hNJs1fy", "reviewer": "Reviewer_zGJA", "summary": "The authors have created a model that seems to perform especially well for high-frequency waveform classification. They have benchmarked their model over several datasets, and its overall rank is higher than that of other models.", "questions": "The authors use two types of vital signs to benchmark their model. Is there enough reason that we could assume the superior performance also translates to other vital signs and, more broadly, medical time-series prediction tasks?", "limitations": "Tasks could be extended to (cheap to collect) PPG signals. Additionally, one could think of a multi-model setup as we often have more (static or lower frequency time-series) data on specific test subjects.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The model successfully combines three (seemingly pre-existing ideas). The model seems to perform reasonably well in the provided benchmarks. The subject task distinction is an interesting addition and shows that thought has been put into the design of the tasks. The code is provided together with some documentation, significantly improving reproducibility.", "weaknesses": "The limitations and discussion should at least be briefly discussed in the paper itself, not just in the appendix. The compared models, while plentiful, seem only transformer-based; it would be interesting to see a comparison to different architectures, especially since TCN is also mentioned in the related work (e.g., try the Mamba model https://arxiv.org/pdf/2312.00752). The improvements over existing transformer models seem marginal. \nThe efficiency of the models should be benchmarked (e.g., training time). \nTable 3: some results are tied as they lie in the range of the standard deviation of the best model. It would be good to highlight these results as well.\nLine 724: Comparison misspelled.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your suggestions to our paper! We are happy you interested in our design for experiments and carefully review our paper and code. Here are the detailed response for your questions and concerns.\n\n***\n\n**Q1**: The limitations should be discussed in the main paper, not appendix.\n\n**A1**: Thanks for the suggestion. We will move the limiation to main paper in our revised version.\n\n***\n\n**Q2**: Baselines are all transformer-based methods. It would be interesting to compare with different architectures (e.g., try the Mamba model).\n\n**A2**: Thanks for raising this concern. We add three more baselines that are not transformer-based for comparision: TCN, ModernTCN, and Mamba. The results are presented in R-Table 1 in the attached pdf file in general response. We evaluated our method and 6 baselines across the 3 new datasets and the previously used TDBrain dataset. Our method achieved the best F1 score on 3 datasets and the second-best on the remaining one.\n\n***\n\n**Q3**: The improvements over existing methods seems marginal. Some results are tied as they lie in the range of the standard deviation of the best model. It would be good to highlight these results as well.\n\n**A3**: Thanks for raising this concern. PatchTST outperforms our method on the PTB-XL dataset by 0.6 in F1 score. As we observe from the ablation study, PTB-XL does not benefit from cross-channel patching too much; we guess the reason is the smaller number of channels(12) of PTB-XL. In other words, we believe our model is more suitable for datasets that demonstrate more channel correlations. We will discuss and analyze more on results in our revised version of paper.\n\n***\n\n**Q4**: The efficiency of the models should be benchmarked (e.g., training time).\n\n**A4**: We have included a table (Table R.2) in the attached PDF that presents the runtime performance of several older baselines known for their strong performance, as well as the three new baselines, on the newly added dataset: FLAAP (13,123 samples, 10 classes). In general, CNN-based methods are fast, and the Mamba is extremely slow for unknown reasons. We guess this is because Mamba's new architecture does not match the GPU hardware optimization well. Our method achieves close runing time to other transformer-based methods.\n\n***\n\n**Q5**: Can the superior performance also extends to other vital signs and medical time-series prediction tasks? For example, PPG data and other multi-modal datasets.\n\n**A5**: Yes, we believe our model is adaptable to various types of medical time series, particularly those with multi-channel correlations. For multi-modal learning of medical time series collected simultaneously (e.g., PPG, EEG, fNIRS, ECG), we see three potential research directions for applying our method: 1) Interpolating and Concatenating: Interpolating all modalities to the same length and vertically concatenating them into a multi-channel time series. 2) Individual Representation Learning: Using separate models to learn individual representations for each modality, then concatenating these representations for downstream tasks. 3) Granularity-based Modality Correspondence: Applying different granularities to correspond to different modalities during the learning process. We plan to explore these directions in future work.\n\nAdditionally, we train our method on a new dataset, MIMIC-PERform-AF, which uses PPG and ECG data to distinguish atrial fibrillation patients (20,400 samples, 2 classes). The results, presented in R-Table 1, show that our method outperforms the second-best result by approximately 3% in F1 score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E1W8bNCQJN", "reviewer": "Reviewer_ApjC", "summary": "This paper introduces Medformer, a transformer variant designed to learn complex dynamics from medical signals. This is achieved in three key ways : \n1. Cross-channel patching for token embedding;\n2. Multi-length patching for coarse and fine feature processing;\n3. Multi-granularity self-attention for information granularities integration;\n\nThe proposed model is then extensively validated through comparison to ten baselines on five datasets.", "questions": "- For clarification, does the random seed only affect the initialization of the model? Are the training/validation/test splits the same for all five runs?\n- What was your reasoning behind the different augmentation techniques used depending on the dataset?", "limitations": "The authors have addressed the technical limitations of their work, albeit not in the main body of the article. However, they have not addressed the societal impact of their model, were it to be adopted in a healthcare setting : steps should be taken to either prevent or disclose clearly whether demographic biases (for example) found in the data are also found in the model's predictions.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Originality : The model proposed in this paper is the first to integrate both cross-channel patching with varying patch lengths as an embedding method, and a new multi-granularity self-attention mechanism.\n- Quality : The performed tests are extensive (five different datasets, ten competing baselines). Reproducibility is largely facilitated as all datasets and the full code are publicly available. To validate all components, ablation studies are provided. Since the introduction of multi-granularity could seem like it would increase complexity, care has been put so that this is not the case (e.g. with the two-stage router-based self-attention).\n- Clarity : The paper is well-written and easy to follow, and the figures are intelligible and informative. All equations and steps are clearly explained.\n- Significance : Improving diagnosis performance is doubly useful in the medical domain, as it directly aids healthcare, and indirectly eases the adoption of models by clinicians. On the technical side, this paper opens the door for multi-granularity transformers able to learn from both fine and coarse-grained features.", "weaknesses": "This paper suffers from one main weakness : processing medical time series without providing explainability studies undermines its usability and transparency. This could be addressed in the future by adapting one of the techniques developed to interpret predictions made by transformer models. \n\nThe provided code could also benefit from more comments, for example in the `exp/*` files.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback on our work！ We appreciate you carefully reading our method, equations, figures, and even the code! We are happy to receive your endorsement and feel that our paper is easy to follow! Again, thank you for your comments. Here are the responses to your questions and concerns.\n\n***\n\n**Q1**: Processing medical time series should provide explanability and transparency.\n\n**A1**: Thanks for raising this concern. We agree explanability is crucial to medical tasks, therefore we further clarify how we can provide explanations from our trained model with an example from the TDBrain dataset. This is a dataset of EEG signals for prediction on Parkinson's disease, which is suitable for exploring if our model is utilizing the correct channels' information and the signals at the right frequency for a reasonable diagnosis. To find out important channels contributing to our model's prediction, we first mask out the rest of the brain regions and only use certain combinations of them to perform predictions. Important channels contribute most to the prediction accuracy, as shown by Fig R.1 (b) in the attached PDF. In Fig R.1 (b), we can find Central sulcus contributes most to the prediction accuracy, followed by Frontal, Occipital and Parietal lobes, and plural available regions are generally better than single region. This aligns with the central sulcus's role in movement and its close anatomy relationship with Basal Ganglia, which is the primary brain region indicating Parkinson's disease [9], meaning that our model has successfully learned to pick up informations from brain regions that matters much more. Furthermore, Fig R.1 (a) shows how different routers are attending to different segment of the same signal (deeper red means that the router attends more to this patch). This demonstrates that other than capturing features at the 32Hz and 16Hz scaffolded by the patch length $L=8$ and $L=16$ for 256Hz EEG signals, the model is as well focusing on brain waves at 8~10Hz due to which the model is \"downsampling\" the tokens with attention scores in stripe patterns. This precisely aligns with the interval at which the EEG signals differs the most between healthy individuals and patients [10] and therefore is crucial in the diagnosis process of Parkinson's disease. In conclusion, our method is capable of focusing on important channels as well as time-frequency information which is well-supported by literatures and may give inspiration for future studies and applications.\n\n***\n\n**Q2**: The provided code could benefit from more comments.\n\n**A2**: Thanks for raising this concern. We will update and add more comments to our code in our next version.\n\n***\n\n**Q3**: Does random seed only affect the initialization of model? Are the training/validation/test splits the same for all five runs?\n\n**A3**: Yes. The random seed only affects the model's initialization, while the train-test splits remain consistent across the 5 runs. To satisfy the subject-independent setup, where the train-test split are based on subjects, we fixed the subjects used in each set. This means that the subjects included in the training, validation, and test sets remain the same, and only the model's initialization is randomized for each run.\n\n***\n\n**Q4**: What was your reasoning behind the different augmentation techniques used depending on the dataset?\n\n**A4**: We did some preliminary analysis and evaluations to choose augmentation methods. Our goal is to select augmentation methods that do not change the label for classification while introducing some variance in the samples. In general, timestamp masking is the safest meth for classification tasks, followed by jittering. Timestamp masking involves randomly masking certain timestamps in the time domain without damaging the semantic information. Jittering, when applied with a low ratio of random noise, also preserves the semantic information in the raw data. We also experimented with frequency band masking but found that it sometimes negatively impacted performance. We believe this could be due to the semantic information in some datasets being concentrated in specific frequency bands, which this type of augmentation may disrupt.\n\n***\n\n**Q5**: Considering the societal impact of their model, steps should be taken to prevent the disclosing of demographic information in the dataset. \n\n**A5**: Thank you for raising this concern. During data preprocessing, we manually removed the demographic information in the dataset and did not incorporate it into model training. We strongly agree that protecting sensitive information in the data is important when dealing with medical data. To further protect the demongraphic information, we may incorporate techniques such as differential privacy in future works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the thoughtful feedback provided by all the reviewers. We appreciate reviewer ApjC for recognizing the novelty of our work in being the first to integrate cross-channel patching and a new multi-granularity self-attention mechanism. In response to the reviewers' insightful comments, we have conducted additional experiments (Table R.1 in the attached PDF), including three new baselines, TCN[11], ModernTCN[12], and Mamba[13], and three new datasets, including two human activities recognition datasets, FLAAP(13123 samples, 10 classes)[14] and UCI-HAR(10299 samples, 6 classes)[15], and one multi-modal vital signs(PPG and ECG) dataset MIMIC-PERform-AF(20400 samples, 2 classes) for atrial fibrillation classification[16]. We will incorporate the newly added contents into the next version of our paper.

Thank you again for your suggestions and feedback! We work hard to refine our paper, and we sincerely hope our responses are informative and helpful. If you feel we have not sufficiently addressed your concerns to motivate increasing your score, we would love to hear from you further on what points of concern remain and how we to improve our work. Thank you again!

Due to space limitations, we include **all the references** here in the global response and **all the new tables and figures** in the attached PDF file . All the new tables and figures are start with **R** to distinguish from the submitted paper verion. The experiment results of the ablation study in R-tables 1 and 2 and R-Figure 1 a) and b) can also be found in the attached PDF file.

***

## **References**
[1] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.

[2] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. ICLR, 2023.

[3] Yitian Zhang, Liheng Ma, Soumyasundar Pal, Yingxue Zhang, and Mark Coates. Multi-resolution time-series transformer for long-term forecasting. In International Conference on Artificial Intelligence and Statistics, pages 4222–4230. PMLR, 2024

[4] Vincent Bazinet, Justine Y Hansen, and Bratislav Misic. Towards a biologically annotated brain connectome. Nature reviews neuroscience, 24(12):747–760, 2023.

[5] Imperatori, Laura Sophie, et al. "EEG functional connectivity metrics wPLI and wSMI account for distinct types of brain functional interactions." Scientific reports 9.1 (2019): 8894.

[6] Singh, A., Dandapat, S. (2015). Two-Dimensional Processing of Multichannel ECG Signals for Efficient Exploitation of Inter and Intra-Channel Correlation. In: Bora, P., Prasanna, S., Sarma, K., Saikia, N. (eds) Advances in Communication and Computing. Lecture Notes in Electrical Engineering, vol 347. Springer, New Delhi. 

[7] Cosimo Ieracitano, Nadia Mammone, Alessia Bramanti, Amir Hussain, and Francesco C Morabito. A convolutional neural network approach for classification of dementia stages based on 2d-spectral representation of eeg recordings. Neurocomputing, 323:96–107, 2019

[8] Fangzhou Li, Shoya Matsumori, Naohiro Egawa, Shusuke Yoshimoto,Kotaro Yamashiro, Haruo Mizutani, Noriko Uchida, Atsuko Kokuryu, Akira Kuzuya, Ryosuke Kojima, et al. Predictive diagnostic approach to dementia and dementia subtypes using wireless and mobile electroencephalography: A pilot study. Bioelectricity, 4(1):3–11, 2022

[9] Rahimpour, S., Rajkumar, S. and Hallett, M. (2022) “The Supplementary Motor Complex in Parkinson’s Disease,” Journal of Movement Disorders.

[10] Helson, P. et al. (2023) “Cortex-wide topography of 1/f-exponent in Parkinson’s disease,” npj Parkinson’s Disease.

[11] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.

[12] Luo, Donghao, and Xue Wang. "Moderntcn: A modern pure convolution structure for general time series analysis." ICLR. 2024.

[13] Gu, Albert, and Tri Dao. "Mamba: Linear-time sequence modeling with selective state spaces." arXiv preprint arXiv:2312.00752 (2023).

[14] KUMAR, PRABHAT; SURESH, S. (2022), “FLAAP: An open Human Activity Recognition (HAR) dataset for learning and finding the associated activity patterns ”, Mendeley Data, V1, doi: 10.17632/bdng756rgw.1

[15] Reyes-Ortiz,Jorge, Anguita,Davide, Ghio,Alessandro, Oneto,Luca, and Parra,Xavier. (2012). Human Activity Recognition Using Smartphones. UCI Machine Learning Repository. https://doi.org/10.24432/C54S4K.

[16] P. H. Charlton et al. (2022), “Detecting beats in the photoplethysmogram: benchmarking open-source algorithms,” Physiological Measurement.

Meta-review: The paper presents a new transformer model for medical time series. The reviewers are unanimously positive about the paper, appreciating the combination of ideas that went into the models. Questions were raised about comparisons to state of the art models, which the authors successfully addressed during the rebuttal phase. The authors also added several datasets to demonstrate the generalizability of the method, and also explained how their method could be used to derive explanations. Overall, this is a strong paper and ready to be presented.
Decision: Accept (poster)