Efficient Personalized Adaptation for Physiological Signal Foundation Model
Authors: Chenrui Wu
Haishuai Wang
Xiang Zhang
Chengqi Zhang
Jiajun Bu
Venue: ICML 2025 poster (2025)
Keywords: Physiological Signal
Time Series Classification
Foundation Model
Personlaization
Primary area: applications->time_series

Abstract: Time series analysis is crucial across various fields like energy, environment, transportation, finance and health. Deep learning has significantly advanced this field, particularly, the Time Series Foundation Model (TSFM) excels in multiple domains due to extensive pre-training. In this work, we focus on TSFM's challenges in medical practice: limited computing resources and medical data privacy. TSFM variants include fine-tuned models and those pre-trained for rapid deployment on diverse data. There may not be enough computing resources to train physiological signals locally in hospitals, and generalized TSFM is still inferior to task-specific methods on private, imbalanced local data. To address this, we propose PhysioPFM, a framework for efficiently personalizing TSFM. Our approach involves low-rank pre-training on public datasets, generator training by trained LoRA weights, and efficient weight generation via local data. Experimental results demonstrate that integrating generated models with TSFM enhances performance, and transferability, and reduces the need for additional sensitive data training.

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Chenrui Wu 12
Haishuai Wang'
Xiang Zhang?
Chengqi Zhang +
Jiajun Bu'
Abstract
(a) TSFM Local Adaption
(b) TSFM for Generic Pretraining
Time series analysis erucial across various fields like energy, environment, transportation, finance and health. Deep learning has significantly ad- vanced this field, particularly, the Time Series Foundation Model (TSFM) excels in multiple do- mains due to extensive pre-training. In this work, we focus on TSFM's challenges in medical prac- tice: limited computing resourees and medical data privacy. TSFM variants include fine-tuned models and those pre-trained for rapid deploy- ment on diverse data. There may not be enough computing resources to train physiological signals locally in hospitals, and generalized TSFM is still inferior to task-specific methods on private, im- balanced local data. To address this, we propose PhysioPFM, a framework for efficiently person- alizing TSFM. Our approach involves low-rank pre-training on public datasets, generator training by trained LoRA weights, and efficient weight generation via local data. Experimental results demonstrate that integrating generated models with TSFM enhances performance, and transfer- ability, and reduces the need for additional sensi- tive data training.
Pretrzined
Pretrained LM
United Model
Fine-cuning Testing
Pre- tree-training
Testing
Targot daca
Targer
Foreign dormain
Foreign
Figure 1. Illustration of existing two categories of time series foundation model. (a) represents the methods applying pre-trained LLM to train target data, and conducting tests on target data. (b) demonstrate the methods that pre-train on a large and multi-domain dataset, then test the target data for the zero-shot prediction.
(EEG), cardiotocography (CTG), electrooculogram (EOG), galvanic skin response (GSR), electrocardiogram (ECG), electromyogram (EMG), and others (Thapa et al., 2024; Wang et al., 2024b; Zhang et al., 2024; Al-Sacgh et al., 2021). These physiological signals are usually quantitatively measured by medical devices and then analyzed by doctors specialists to evaluate the patient's 's current state and make data-driven decisions, which shows great significance for health monitoring, disease diagnosis, and treatment (Liu et al., 2023). Specifically, we focus on various medical time series classification (TSC) tasks based on the physi- ological signals, including emotion recognition (Li et al., 2024a), sleep stage detection (Supratak et al., 2017; Dong et al., 2017), neurological disorder classification (Yang et al., 2022b; Zhang et al., 2022), etc.
1. Introduction
The recent trend of integrating deep learning algorithms on advanced wearable sensors and fixed medical equip- ment has catalyzed massive amounts valuable medical data (Spathis et al., 2020; Che et al., 2017). These recorded medical time series data continuous observations related to human health, usually including electroenceepallogram
However, while the community has benefited greatly from the large amount of new data collected by professional med- ical devices or ubiquitous wearable devices, analyzing phys- iological signals with existing deep-learning methods still faces inherent challenges in practice. First, the amount of data available for each signal is unbalanced. Most existing studies focus on EEG and ECG data. In contrast, other physiological signals have minor data available, making it challenging to establish unified and generic models for all signals. Furthermore, the sampling frequency and duration of different signals may also vary, further causing the diver-
'Zhejiang Key Laboratory of Accessible Perception and In- telligent Systems, College of Computer Science and Technology, Zhejiang University. 'School of Computing Science, Simon Fraser University. 'Department of Computer Science, The University of North Carolina Charlotte. 'Department of Data Science and Artificial Intelligence, Hong Kong Polytechnic University. Corre- spondence to: Haishuai Wang <haishuai, @ zju.edu.cn>>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
gence of data features and labels. Therefore, compared with time series in conventional domains, medical time series also have unique challenges in imbalanced distributions.
S
To effectively capture the complex trends of time series, the foundation model has recently attracted the attention of researchers. With pre-training and success of the language model, diverse time series foundation models (TSFM) have been proposed (Zhou et al., 2023; Jin et al., 2024; Rasul et al., 2023; Liu et al., 2024b; Goswami et al., 2024; Ansari et al., 2024; Garza & Srngenthaler-Canseee 2023a). TSFM mainly includes two categories as shown in Figure 1. The first category adopts language model parameters as pre- training parameters and performs fine-tuning and testing on target data locally (Zhou et al., 2023; Jin et al., 2024; Rasul et al., 2023; Ansari et al., 2024). Another category pursues generalization performance, is pre-trained with large-scale time series data in various fields, and achieves universal per- formance of train once for all (Liu et al., 2024b; Goswami et al., 2024; Garza & Mergenthaler-Canseoo 2023a).
DOO
BBARE
BBBITI
Privacy preservation Limited comp. resources Insufficient specialization
Figure 2. The challenges of implementing two types of TSFM to clinical practice. The first shows that private data cannot be uploaded for pre-training. Local fine-tuning on private data may cause huge computational overhead and impractical in some conditions. Applying foundation model pre-trained by multi-domain time series may not well with the features and distribution of local physiological signals without adaptation.
Specifically, in the data preparation phase, we first train the TSFM with massive public physiological signal data by Low-Rank Adapter (LoRA) (Hu et al., 2021; ; Zhou et al., 2024) and store the map of adapters and data sources. To effectively learn diverse features, we adopt the neural col- lapse (Papyan et al., 2020) to tackle the imbalanced data distribution. In the generator training phase, we use dif- fusion transformer (DiT) (Peebles et al., 2022) as a robust generator to synthesize LoRA weights. To bridge the gap between time series and adapter parameters, we capture the representative subsequence of time series shapelets (Wang et al., 201 8b; Li et al., 2021; Le et al., 2022; 2024) as the input conditions for DiT training. In the local personal- ized inference phase, given the local data's 's shapelets, DiT could generate customized LoRA weights to combine with generic TSFM. A personalized TSFM is obtained for the lo- cal medical expert. Only limited computing resources of the generator and TSFM inference are required. We summarize our contributions as follows:
Yet, several challenges are aroused by implementing the existing two types of TSFM, shown in Figure 2. For physi- ological signals, patients' sensitive data is prevented from being uploaded to foundation model service providers for pre-training due to recent laws like EU GDPR for patients' privacy. To adapt local physiological signals, for the first category, in the face of clinical diagnosis scenarios with lim- ited computing capacity, local fine-tuning will cause huge computing overhead, which is quite impractical. If applying the generic pre- pre-trained model, although various foundation models have been proven to perform well in multiple fields and have general feature extraction capabilities, some stud- ies have shown that they perform poorly on specific tasks, such as in fields where publicly available healthcare data is scarce and imbalanced (Glocker et al., 2023; Gupta et al., 2024; Wang et al., 2018a).
While several sask-specific TSFMs have been crafted for particular domains, such as Brant-X (Zhang et al., 2024), SleepFM (Thapa et al., 2024) and ECG-FM (McKeen et al., 2024). They aim to fit one task with massive public and private data pre-training These approaches also face the issue of access to private data for training and the challenge of transferring to unseen physiological signal tasks.
We novelly propose personalized approach to transfer the time series foundation model to clinical physiologi- cal signal tasks with lower computing costs and privacy guarantees.
We propose specialized LoRA generator of DIT. bridging the time series and model by discriminant subsequences shapelets.
In this work, we propose PhysioPFM, a Personalized Foundation Model approach for Physiological signal to effi- ciently and lightweight produce a customer model for clin- ical practice with privacy guarantee. Inspired by the con- cept
Extensive experiments demonstrate the robustness and efficiency of our PhysioPFM.
Personalization (Chen et al., 2023; Li ct al., 2024c), our overall goal is to adapt a TSFM to personalized clinical tasks without extra training during de- ployment. Based on pre-trained and generalized TSFM, we aim to synthesize customized Low-Ram adapter parameters locally with lightweight cost and privacy guarantee.
2. Related Work
2.1. Time Series Foundation Model
Recently, the development of large language models (LLMs) has revolutionized the field of natural language process-
2

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
ing (NLP) (Chang et al., 2024), demonstrating multimodal adaptability beyond text (Qin al., 2024; Zeng al., 2023; Jiang et al., 2024b). Time series have practical value com- parable to natural languages, and in essence, they exhibit similar sequentiality in generative modelling (Gruver et al., 2024). This unique feature has prompted much research to adapt LLM to time series (Li et al., 2024b). Current time series foundation models (TSFM) can be divided into two types. The first type utilizes language model parameters to fine-tune target data. Ilm4ts (Chang et al., 2023) uses fine-tuned transformer modules and positional encodings in GPT-2 to align pre-trained LLMs with time series data for forecasting tasks. One-Fits-All (GPT4TS) (Zhou et al., 2023) freezes the multi-head attention and FFN layers, fine- tunes input, positional embeddings and layer norm blocks, based on GPT-2 parameters. Time-LLM (Jin et al., 2024) introduces reprogramming architecture, aligning LLM's word embedding with time series embedding, with frozen LLama backbone (Touvron et al., 2023a;b).
the optimal network architecture for EEG-based emotion recognition; OMHGL (Pan al., 2023) proposes an online multimodal hypergraph learning method for emotion recog- nition based on multimodal hypergraph fusion and online hypergraph learning. For sleep stage detection, in (Kong et al., 2023), a bilevel optimization approximation for EEG- based sleep stage classification is proposed with a neural architecture search framework; NeuroNet (Lee et al., 2024) proposes self-supervised learning framework for sleep state detection by integrating contrastive learning tasks and masked prediction tasks. To uniformly solve the physiologi- cal signal classification, Bran-X (Zhang et al., 2024) utilizes a pre-trained brain signal foundation model, Brant-2 (Yuan et al., 2024) with 2 TB private data, then aligns other signals to brain signals in two-level patching. In this work, we aim to efficiently transfer the general knowledge of TSFM to physiological signals with public datasets. Our focus is to achieve specialization through openness and protect privacy without exposing patients' private data. We also consider the classification challenges of noisy and imbalanced time series in physiological signal data.
Another type targets generalization with large-scale data pretraining for few-shot or zero-shot tasks. LLMTimee (Gru- ver et al., 2024) investigates inputting time series as text with different preprocessing to LLM for zero-shot forecast- ing. TimeGPT-1 (Garza& & Srrgenthaler-Canseer 2023b) is a commercial time series LLM based on the transformer architecture and pre-trained with more than 100 billion data points for zero-shot prediction. Timer (Liu et al., 2024b) proposes a pre-training dataset (UTSD) including multiple domains with 1 billion time points and trains a GPT-style (decoder-only) model to predict. MOMENT (Goswami et al., 2024) also collects large public dataset: Time Series Pile. then proposes learnable masked patches to train a trans- former. These pre-trained TSFMs focus on generalization with a wide domain of time series, where their special- ized abilities in medical time series may not be satisfactory. Therefore, we propose lightweight TSFM personalization scheme to tackle the specialization ability.
3. Preliminaries
3.1. Problem Setup
We consider multi-class time series classification tasks un- der a train-once- personalization scenario (Chen et al., 2023; Li et al., 2024c). There is a primitive time series foundation model Wo pre-trained by large-scale public datasets D. Each local clinical expert/user k holds pri- vate dataset Dr = {(a} y² F where at is the input of the training sample, corresponding vy denotes the given label, and the sum of local data does not belong to pre- training datasets U Dj DD Ø. For each physiological signal E IR M x. includes M channels with a length of L timestamps. The optimization objective is to obtain a best personalized model W/ for each user based on private dataset Dr., formulated as:
2.2. Time Series Classification
min L(W) = (W,, Dx)),
(1)
In general, time series analysis involves four fundamental downstream tasks: forecasting, imputation, classification and anomaly detection. Current TSFMs mostly emphasize forecasting ability or general ability. However, time se- ries classification (TSC) has some unique challenges like heterogeneous timestamp length (Chen et al., 2024), noisy labelling (Liu et al., 2024c), and domain gap from different sensors (He et al., 2023; Wu et al., 2024), aside from only training robust feature-extracting ability or replacing a prediction head. Therefore, more studies have focused TSC, especially the applications of physiological signals. For emotion detection, TNAS (Li et al., 2022) combines the Transformer model with NAS and searches through a multi-objeccive evolutionary algorithm (MOEA) to obtain
where S denote the personalization operation function based on the generative network Go.
3.2. Low-Rank Adaptation
The size of the parameters of existing LLM usually starts from billions. Many works (Li al., 2018) have proved that matrices for deep learning are often paraparaaeterizzd Low-Rank Adaptation (LoRA) (Hu et al., 2021) allows in- direct training of some dense layers in neural network by optimizing the rank factorization matrix of the dense layers that change during the adaptation process while keep- ing the pre-trained weights unchanged. Hence, LoRA im- proves efficiency for fine-tuning large pre-trained language
3

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
models. In detail, LoRA first fixes the pre-trained LLM Wo Raxk and introduces two trainable low-rank ma- matrices A E I and
(within-class feature means). Formally, the covariance Ei -> 0, and Dg == Re E (ha.: b ( he)² where nhe is the index of samples in e-th class, h F(0; 2;,) is the feature of the i-th sample of c-th class,
trices: dimensionality--edueae dimension-raiee raising matrix B E Edxr, where < min(d, k). The LoRA's forward pass equation is denoted as:
and h = 1 Σi= has is the within-class mean of class e features, that is the prototype.
Wo + AW* Wo + BAx,
(2)
NC2 (Convergence to Simplex ETF); The class proto- types will collapse to simplex ETF and be maximally separated. h. = (he h - be Il. Vc E [C]] sat- isfies Eq. (4), where be refers the global mean of all the features.
where AW = BA is the parameter needing to be updated, is the same input for Wo and AW. The more concen- trated and stable parameter distribution in LoRA helps im- prove generalization by reducing the risk of overfitting. Dur- ing the inference process, LoRA also introduces almost no additional inference latency. Only the calculation of W = Wi + AW is required. In this work, we aim to synthesize LoRA parameters to directly obtain customized LLMs based on target data in a local private diagnosis.
NC3 (Convergence to Self-Duality): The classifier weights will collapse to the same simplex ETF of the corresponding class. h. = w woll where w. is the classifier weight vector of class c.
NC4 (Simple Decision Rule): Under the conditions of (NCI)-(NC3), the classifier's prediction can be simplified to the nearest class centers based on Eu- clidean distances, denoted as: argmax; (h, we) arg ming ||h well, where h represents the last-layer feature of a sample used for classification.
3.3. Neural Collapse
Neural collapse is a phenomenon in which the feature pro- totypes and weights vectors of the classifier will gradually converge to Simplex Equiangular Tight Frame (ETF) struc- ture at the terminal phase of balanced data training (Papyan et al., 2020; Li et al., 2023; Yang et al., 2023; 2022a). We note the definition of the ETF and Neural Collapse as fol- lows:
4. Proposed Method
Definition 3.1 (Simplex Equiangular Tight Frame). A set of C-dimensional vectors w; E R4, where i E [C] and d C - 1, forms a simplex equiangular tight frame (ETF) if satisfies the condition:
4.1. Overview
As shown in Figure we describe the detailed process of the proposed physiological signal foundation model per- sonalization method: PhysioPFM. We consider three main processes: debiased alignment as data preparation, shapelet- to-LoRA model training, and lightweight custom model generation. First, we adopt public physiological signals to fine-tune the original time series foundation model in a LoRA way. Then we the map of the low-rank adapter and the corresponding data to train the generative model. Fi- nally, the user could apply the generative model to produce a customer low-rank adapter with private data. Combined with the time series foundation model, a customized physi- ological signal foundation model obtained with only the extra computing costs of generation.
C C W = V V 10-
(1066)
(3)
where W = [WI,.... wcl E Rdxc U Rdxc (d > C) is a partial orthogonal matrix satisfying UTU = Ic. Ic E CRC is a identity matrix and lc is a vector of ones. Any column vectors in W has the same l norm, and the pair-wise angles between any pair of vectors are maximized by the simplex ETF, denoted as:
C
1 - Vi.j e [C],
w wr C-1
(4)
4.2. Data Preparation
where it takes the value of if the indices i and j are equal, while yielding 0 in all other cases. The optimal condition could be obtained when Vi ス j, the maximal equiangular separation is w." w, --
Specifically, with the wide success of Low-Rank Adapter (LoRA) fine-tuning in foundation models (Hu al., 2021; Zhou et al., 2024), we first pre-train the time series foun- dation model with plenty of public physiological signal datasets in a LoRA way and store the map of adapters and data sources. To synthesize learnable LoRA parameters, we consider building a map between LoRA and input time series by training the generative model. We adopt multiple public physiological signal datasets Doub and decompose
The phenomenon of Neural Collapse encompasses four dis- tinct properties, which we highlight key manifestations:
NC1 (Within-Cha Class Variability Collapse): All the last-layer features will collapse to the class prototypes
4

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Data Preparation
Personalized Inference
Customized LoRA
EEG ECG
Pre-trained
CE Loss
TSFM
GSR
Private Data
Randam Nevs
Transformer
AC Loss
Shapeler Diccovery
Pre-trained
TSFM
Public Signals
LoRA
ETF Classifier
Generator Training
Fattee
Augmantation
Add Notoe
Talantbing
Predie
Took Dara
Sampling
1000
1ill
LoRA weights
Nossed model
Difussion Trandormer
Encoder
Pooden
G000
Shspelet D Diccoerr
Predicted LoRA
Taak LoRA
Correspondigg Task Signa
Figure 3. The framework of the proposed PhysioPFM. The training procedure follows: ! In the data preparation phase, we pre-train a time series foundation model by public physiology signals in the cloud, obtain the map of data and LoRA weights. In the generator training phase, we train robust weights generator by the data as a condition and LoRA weights as output. @ In the personalized inference phase, adopt local data and the trained generator synthesize customer LoRA weights for the TSFM.
them into more sub-tasks Drub to expand the amount of training tasks. For each task, we can obtain a personalized LoRA. Given the forward pass of LoRA in Eq. we adopt two classification losses to update the original TSFM and get customized LoRA for every task tg-
a random synthesis of simplex equiangular tight frame P E RdxC Any pair of prototypes vectors p:. Py P satisfy maximal pair- wise separation P Pj C-1+ Vi + j. j E [C]. Therefore, we add the classifieranchhored loss regularize the training, defined as:
Cross-Entrope Loss. The vanilla cross-entrops loss Lee is used to minimise empirical risks in a classification task:
Tly explx; Pv./T)
Cac = - log E Egj By exp(x; Py, Py./T)
(6)
where ny, is the number of samples for class ys. T is temper- ate hyper-parameter, we set r generally. Through these losses, the TSFM could be adapted to various physiological signal sub-tasks with debiased classifier. The overall loss function align the generic TSFM to physiological signals by low-rank adaptation formulated as:
Cee
ye 2o)))(()
DD
where Diuhh is the subset of the public physiological signal dataset Drub enrich the training data size, each individ- dataset and its subset spatial classes can Doub The specific public physiological signal datasets we collected are provided Appendix A.6.
L
Loe + Lac-
Anchored Classifier Loss. Due to the class-level distribu- tion imbalance and length heterogeneity of the collected physiological signal dataset, the number and series length may differ across classes and samples. We aim to update the TSFM debiased way. first define the prototype p; as the mean of embedding of the class E {1, 2....... C'} (Snell al., 2017). According Definition 1, we can derive optimal condition of class prototypes:
After LoRA training on each sub-task of public datasets, we define the sub-task of Drub as tg. The overall demonstration of the obtained data sample map can be expressed as = {AWW, & K 1 T3 T, where AW; BB
4.3. Generator Training
train the parameter generator, model generation with advances in diffusion process (Peebles et al., 2022; Jiang et al., 2024a) recently stands from traditional techniques like network architecture search (Annavajjala al., 2024; Cai et al., 2018) and neural network compression (Han et al., 2015; Wang et al., 2022), beyond image, video and other multimodal tasks. Different from generating a whole backbone model like a CNN and an MLP (Peebles et al., 2022: et al., 2024c), LoRA's size of parameters shows a
Proposition 4.1. In a multi-class classification problem, if prototypes P1 ....... PC E R are fixed and satisfy yi - Vi j. i [C), the maximum of angular between any two class prototypes will be reached, where the decision bound among classes is optimal.
This proposition provides the optimality condition of vectors maximize the minimal sample margin. Thus, we initialize
5

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
minor feature space and is easier to compress and predict by controlling the rank of the adapter r. As a more efficient paradigm for generating weights, we present the details of DiT training.
simplified variational lower bound, denoted
min L()) ΣV
Σ KEK JEJ
IAW (S(()(() (II)
Input learnable parameters. Through the data preparation in Section 4.2, we could obtain the pairs Tj- = (AWx, fx). We set LoRA weights AW& as input, flatten all parameters into a one-dimensional vector and chunk/tokenize the pa- rameters in each layer. We add augmentation and random noise while inputting.
where G3. is the generator model, S(tx))is the shapelets of task &2 inputted as conditions, noised LoRA weights is AWA- j E [J] denotes the timestep in the diffusion forward noising process. The learning objective of diffusion is to minimize the simplified variational lower bound, which reduces to predicting the denoised model parameters.
Input conditions. Existing works (Chen et al., 2023; Li et al., 2024c) build the connection between model weights and textual prompts for image classification tasks by pow- erful pre-trained CLIP (Radford et al., 2021). However, physiological signals are chronological data points and no such existing encoder bridges the text-to-time relationships as CLIP. To compress the input space and bridge the time se- ries with LoRA weights, we propose shapelet prototypes as a substitute. Shapelet is set of discriminative subsequences of a time series (Ye & Keogh, 2009), each of which ex- pected to represent class best. Therefore, shapelet can help to better understand the meaning of the time series and is widely used in the field of time series classification (Li et al., 2021; Zhang& & Sun, 2022; Liu et al., 2024d). Following the existing works (Le et al., 2022: 2024), we slice the time series by the Offline shapelet Discovery (OSD) method to get the shapelets (t,)), presented in Appendix A. For each class, we set a uniform amount of shapelets.
4.4. Personalized Inference
Given the trained generator network GG, we could deploy on local medical users. The user provides raw data to the encoder obtain shapelets as conditions. With the input conditions, the generator G. could synthesize suitable low- rank adapter weights AW" Combined with the original TSFM, the expert could conduct a diagnosis of disease by updated TSFM, which only requires the cost of the gen- erator and foundation model inference. The advantage of our personalized inference lics in these aspects: 1) Lower computing overhead, compared with other works, in local deployment, we only add the cost of the generating pro- cess instead of costly computation of foundation model fine-tuning. 2) Privacy preservation, need to upload sensitive data to the server for trainining, and the training for the DiT is also based on public physi- ological signals. DiT's powerful generalization capability enables the entire personalized model generation process to be performed locally. 3) The best of two worlds with public and private data. Our data preparation and generator training depend on public data, but our method can adapt to private data. The full utilization of both data boosts the overall performance.
method does not
Diffusion training. We adopt DiT as the generator to train. We present the basic training process of the DiT model as:
Diffusion forward pass. The forward process is adding noise to the initial sample from It-1 to Fx, by condition c. The probability of transitioning is modeled as a Gaussian distribution:
(( = (are: V Bext-1, B,I)
5. Experiments
5.1. Experimental Setup
where 3g are the timestep-dependent noise levels, and I rep- resents the identity matrix. The complete forward process is denoted as:
Datasets and tasks To fully evaluate performance, we con- sider four typical physiological signals classification tasks, covering different signals. We randomly sample 60% of the data for training, 20% for validation, and 20% for testing, following the existing work (Zhang et al., 2024), for all evaluation tasks. The specific datasets are as follows, and the introduction of each task shown in Appendix A.2.
T ((ITTIZ c) = II
Diffusion reverse process. The reverse process reconstructs the original sample from the noisiest state xr with condition c:
Sleep-state detection. Sleep-EDF dataset (Kemp et al., 2000) is public dataset containing multiple sleep records. These records are collected through multi-channel physiological signals including EEG, EOG, and EMG. For Sleep-EDF, we consider five class sleep stages: Wake, N1 (Light sleep stage 1), N2 (Light sleep stage 2), N3 (Deep sleep stage), and REM (Rapid Eye Movement) sleep. Parts of previous work
Pe Po c) = (0 Es t, c))
(10)
Diffusion training objective. The training objective to predict corresponding LoRA weights to minimize the
6

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Table Results in terms of test accuracy (% performance on Steep-state detection. Blue/bold fonts highlight the best baseline/our method. We evaluate classification accuracy, macro Fl score and kappa value.
Method
Overall Metrics
Perclass Fl Score
Metrics
Acc.
MF1
6
Wake
N2
N3
REM
Informeraz22
-1:91=1x:
69.93
55.28+12s
65.63-1m
+2.85+3m
52.37+1as
53 53.82+230 66.76+
FEDformeggoz
50.35 36.58_23
51.78
20.50
40.89
55.53
46.522215 1
SimMTM 2023
65.39::..
55.53
52.47
63.90=2m
38.44
50.73
28.4
66.89:220
TimesNetauza
77.95=207
70.28#24t
68.36
80.48
57.24
70.45
68.39
75.73
PatchTST 2023
14.51
68. 18.191
66.87
77.24
16.04
70.35 66.30
71.24
TTranformerau24
76.43=224
63,49+1.7
68, 68,20+29:
79.38=2x8
53,05
65,32+277
60.45+1.59
58 $8.06=3301
OneFits
65.04 16 16
63.28
80.24_1m
49.17
68.4913.
62.47..m
66.98
Time-LI.Mgoan
71.22.134 m0
72.63
82.56_2as
59.38
65.82
73.40
26.38
MOMENT 2024
81.90=142
72.72++2x
73.12
78.62-2x
54.69
77.07+
76.34+221
75.05
SleepDGa024
79,48
71.95=230
73,43a244
81.29
64.52
72.30
69.24
69.75
SleepFMggos
11.65_2
74.2611.
72.03
82.55_2w:
11.24
76.5211ax
76.93 14.38
Our PhysioPFMM
86.39 2554
76.27 +2.94
77.36+314
83.04-275
66.71+2a1
76,43+1..4
77.29+1..0
78.38
adopt 30-minute data before and after in,,ed, for fairness, we do not adopt this additional cropping, just raw data.
Implement details. For generative model architecture, we adopt GPT-2 (Radford et al., 2019) as the diffusion trans- former with 12 layers. During training, we use AdamW with batch size of 64, a learning rate of 4e-4, 1000 diffusion steps, and linear noise scheduler ranging from 0.0001 to 0.012. we divide the LoRA weights into chunks by layer, and the size of each chunk is 576. We set the rank of the adapter as 4. For the pre-trained time series foundation model, we adopt the 6-layer GPT2-based backbone (Rad- ford et al., 2019; Liu et al., 2024b), pre-trained by UTSD datasets (Liu et al., 2024b). We collect public physiological signal datasets for training in the data preparation process, shown in Table 6. The target task's data will be removed from generator training. We further prepare subsets and sub-classes of datasets to expand the number of tasks. We adopt the average result of 3 times.
Emotion detection. DREAMER (Katsigiannis & Ramzan, 2017) a multimodal physiological signal library consisting of EEG and ECG signals recorded during emotional arousal via audiovisual stimulation. Signals were recorded from 23 participants, along with the participants' self-assessment of their affective state after each stimulation, including valence, arousal, and dominance.
Arrhythmia diagnosis. The MIT-BIH arrhythmia dataset (Moody & Mark. 2001) includes more than 4,000 24-hour periodic dynamic ECG signals of 47 test individual units, 48 recording files with a duration of approximately 30 minutes, and a total of 109,500 heartbeats, of which abnormal heartbeats account for approximately 30%.
Freezing of Gaits Detection. The FOG dataset (Li, 2021) collects multimodal data including EEG, FMG, HCG, skin conductivity (SC), and acceleration (ACC) during walking tasks using high-quality hardware system that integrates commercially available and self-designed sensors. A total of 12 PD patients completed standard hospital experiments.
5.2. Main Results
In Table 1-4, we evaluate the performance of our proposed method against diverse baseline methods including the gen- eral deep learning time series model, time series foundation model with local training, pre-trained time series model for zero-shot deployment and specialized methods for down- stream tasks. The results show that PhysioPFM consis- tently outperforms all baseline methods on all four tasks, demonstrating the remarkable personalization ability of the proposed method. Specifically, for sleep state detection in Table 1, PhysioPFM achieves 4.49% surpassing the best baselines MOMENT. PhysioPFM also leads with the best performance the F1 score and is stable in the classifica- tion ability for each class, even in the tough class N1 light sleep stage, with 2.2% advantage.
Baselines. We take three groups of methods as base- lines. (1) vanilla deep learning technique for time series: SimMTM (Dong al., 2023). TimesNet (Wu al., 2023), iTtransformer (Liu et al., 2024a). PatchTST (Nie et al., 2023), FEDformer (Zhou et al., 2022), Informer (Zhou et al., 2021). (2) time series foundation model: OneFit- sAll (Zhou et al., 2023). Time-LLM (Jin et al., 2024), MO- MENT (Goswami et al., 2024). (3) for each downstream task, we adopt specialized methods: SleepFM (Thapa et al.,
2024) and SleepDG (Wang et al., 2024a) for sleep state de- tection, LSTM-MLP (Wang et al., 2023) and OMHGL (Pan et al., 2023) for emotion detection, DeepArr (Midani al., 2023) for arrhythmia detection, Extra Tree Classifier (Goel
For the evaluation of DREAMER datasets in Table 2, our PhysioPFM method achieves the best results in emotion recognition with the averaged accuracy of 73.97%, 82. 19%
et al., 2023) for FoG task.
7

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Table Results terms of test accuracy (%) performance on emotion detection. Blue/bold fonts highlight the best baseline/our method. We evaluate the accuracy. Fl score and AUC.
Dataset
Valence
Arousal
Dominance
Metrics
Acc.
F1
AUC
Acc.
F1
AUC
Acc.
F1
AUC
Informer2022
62.93
70.38
6
15.20 80.87.1.x
17.39.447
36 85.42
83.25
PEDformer2022
55.30+175
63.29+1as 60.53+299
70.34+434
75.85+2.1
72.56+356
72.26+2.1
81.19+3.4
80.04+336
SimMTM 2023
60.89+254
75.53+2.40 67.20+392
76.64+2a>
83.49
77.24+3.1
76.44+2m
86,56+1a7
82.17+21
70.32+2a7 78.98+1as 75.35
78.95
83.67
78.19
81.26*2.4
85.07
84,72+336
PutchTST 202a
62.35 72.80 66.73+227
71.29
82.24+1..0
75.54
75.47=2as
82.54
79.93+441
iTranformeraoz4
67.93
77.42 75.60
80.59
84.05++s4s
80,82+3.6
80.20
83.91 +2.10 82.94+33
OneFits 2022
64.04
73,43
64.23
75,03 81.29
22.39
77.46
85.09
83,40
Time-LLMaaue 20233
67.49
70.35
a6
77.32
84.
74.21
80.49
83.35
14.3
MOMENT 20144
69.20
77.34:240
74.22.22..
80.51
12:9
83.46
75.32
79.39
86,56,3204
86,90
LSTM-MLP 22025
67.
67.60
78.36s481
62.21 +2000
77.78+337
84.90+1.2
78. 28.24
79.11+229
89, 99.63
82.55+3.10
OMHGI 22023
21.29
66.20gass
70.09.2.7
79.53
1942
78.831474
17.4
83.5214.0
10.69.1.1
Our |MPsi) 79.25 79.41 1.94
82.19 86.65+44a5 80.84+316
84.45 :4.35 91.53 89.61+311
Table 3. Results in terms of model performance on arrhythmia diagnosis. We evaluate accuracy, precision, recall and F1.
Table 4. Results of model performance on Freezing of Gaits
Detection. We evaluate accuracy, precision, recall and FI.
Methods/Metries
Acc.
Prec.
Rec.
F1
Methods/Metrics
Acc.
Pree.
Rec.
F1
Informer
27.54 74.20=279 42 =2
Informer
64.25_1 is 66.34 74.21=2. 70.05
FRDformer
60.35
45 60.92=211 58.0622.5
FEDformer
59.80-1,44 57.67 68.15-4.0
62.47+3345
SimMTM
80.27 82.37=2a5 79.50+352 80,90+219
SimMTM
70.61=1.31 69.25 13:1
72.47 70.82
TimesNet
81.33++.u8 78.93=267 82.45=1772 80.65=200
TimesNct
12.19-1.x 70.88
73.79=2.44 72.30=347
PatchTST
13.53 73.65
67.45.426 70.41 1288
PatchTST
66.90__4e
67
64.75=150
65.9622
Tranformer
75.56
78.95
a..61 76.61 1::4
iTranformer
74.14-3.58 75.76+240
75.60=254
75.67+338
OneFitsAll
75.63 71.90=234
74.45 73. 15
OneFitsAll
62.52 60.94
71.30=271 65.71
Time-LLM
82.91 84,22.3345 81 81.0212251 82.5812.00
Time-LLM
71.62-1.66 72.56
71.11=423
71.82#321
MOMENT
83.25
86.92#2.31
83.57 85.21
MOMENT
73.03_22: 74.55
76.3627as 75.442
DeepAm
86.55 83.34
85 84.62=271
Extra Tree Classifier 73.95 14.09 25.35_2.44
74.71
Ours PhysioPFM 81_32_ 80.77 78.24_.711 79.48:2301
Ours PhysioPFM 89.94 88.42 3.19 90.72+2x1 89.55
the setting of ablation, we first test the validation of the cncsored-classifier loss. Then we use public data with the same task as Shapelet (if we have the same domain task, else average with 5 random ones), instead of the local data's 's shapelets. We also evaluate the raw TSFM without generating customized LoRA weights. The results show that ETF classifier has a great impact on model performance. A suitable temperature for contrastive learning also contributes with 1.87% to 4.63% advantages. We can also find that if the training is allowed, pre-trained TSFM could show great power in most cases after fine-tuning. Considering the vari- ant without LoRA, we could acknowledge that the generated LoRA weights could provide a comparable improvement with direct training. The improvement from the generated LoRA enhances the pre-trained TSFM and distinguishes ours from the generic TSFM. Although shapelet has minor impacts compared with other modules, it still illustrates the importance of personalization.
and 72.3% of valence, arousal and dominance recognition tasks, respectively. Our method benefits from the quick adaptation ability to different sub-tasks with pre-trained knowledge. In some conditions, baselines relying on large language model parameters like OneFitsAll and Time-LLM possess less knowledge of specialized downstream tasks. So, with an LLM weight as starting point, they are inferior to other transformer-basee methods such as iTtransformer.
We further investigate the performance of arrhythmia diag- nosis and FoG detection on Table 3,4. For arrhythmia diag- nosis, PhysioPFM leads with an average score of 89.94%, which is notable increase over the second-best performing method by 3.39%. For FoG detection, PhysioPFM achieves SOTA performance compared to all the baselines. It shows that PhysioPFM could adapt to different downstream tasks without heavy local training.
5.3. Ablation Study
Impact of Rank. We evaluate the impact of LoRA rank on two downstream tasks in Figure 4(a). The results show that increasing r from to 4 brings significant performance improvements while increasing r from 8 to 16 provides no further enhancement, It shows that a larger r is not neces-
To further show the effectiivnees of each proposed com- ponent, we conduct a comprehensive ablation study on PhysioPFM. The results shown in Table 5 demonstrate that all components contribute to the overall performance. For
8

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
1.00
1.00
0.90
rank=1
rank-8
50
200
0.95
rark=4
rank=16
0.95
100
1000
ACC.
0.90
0.90
ACC
0.80
Shapelet
0.75
Endrre Series
0.85
0.85
Text
0.80
0.80
0.70
100
Sleep-EDF
MIT-BIH
Sleep-EDF
MIT-BIH
size of input prompts/conditions
(a) Impact LoRA rank
(b) Impact of DIT training sample size
(c) Different prompts and sizes.
Figure 4. PhyisoPFM capability analysis. Experiments of (a) and (b) are conducted on sleep state detection and arrhythmia diagnosis.
(c) conducted on sleep state detection.
investigate the different prompts as input in Table 4(c). In lower prompt size, shapelets have comparable performance with the whole time series sequence. While the whole time series representation may be more complex, and inputting more prompts could have a negative impact on learning the mapping relationship between time series and LoRA weights. For texts, without robust pre-trained language encoders for time series like CLIP, the difficulty of learning is greater for textual prompts.
0.80
OrefitsA8
0.75
NONENT
PatcITIST
Merrory Consumption
PEDformer
5000
5.4. Adaptation Efficiency
0.55
Adaption (stepoch)
As critical issue for medical practice, conduct puting consumption study for baselines in the FoG task. In Figure 5, we observe diree-dimensional performance: accu- racy, adaptation time, and memory costs. Here, we define the adaptation time. For local training- methods, the training time and inference time are counted for adapta- tion time. For other methods, only counts the inference cost. We can find that FEDformer has the most time and computation costs. Although OneFitsAll achieves notable accuracy, training local data on the GPT-2 backbone also time-consuming. Some lightweight models could train and predict quickly, but lose accuracy. Our PhysioPFM conducts generator training on the server, which requires about 20GB of GPU memory, which is feasible for servers with suffi- cient computing power. In local adaptation, DiT inference only occupies 3 GB of GPU memory, reaching remarkable balance between accuracy, speed, and Consumption.
Figure Model performance on efficiency comparison. The adaptation time represents the sum of training time (if needed) and inference time. We the FoG task.
Table 5. Ablation study of PhysioPFM's modules in terms of accuracy.
Methods/Dataset
Sleep-EDF
MIT-BIH
FoG
TSFM with local training
87.15
88.59-134
83.71=225
Ours wlo Las
82,47
84,302152
78.56237
Ours Cac; r
14.6411.
86.10
79.45
Ours Cucs 10
81.762731
85.52=2mi
78.37-1
Ours wlo local shapelets
85.94
17.43
79.96
Ours w/o local LoRA
82.08+237
84.17=242
76.45+334
PhysioPFM
86.39+254
89.94=135
81,32=216
sarily better, and the size and difficulty of the dataset matter. Smaller r' can also achieve acceptable performance. While a larger r represents the increase in the size parameters that LoRA needs to train and generate. rank that balances computational overhead and performance is better.
6. Conclusion
Impact of training samples. We consider different sizes of samples for the generator training. Here, the training samples denote the sub-task number, i.e., the number of the map of the task and LoRA weights. The result in Table 4(b) shows that as the number of training samples increases, the performance of the model also improves. More training bringing the link between conditions and weights con- tributes to the ability of the diffusion transformation. How- ever, the number of map items relies on collecting public physiological signals.
In this work, we introduce PhysioPFM, a physiological sig- nal foundation model for train-once-for-all personalization. PhysioPFM obtains powerful neural network diffusion per- formance by pre-training on public medical time series. By simply inputting local private sensitive medical time series, customized LoRA weights can be generated without train- ing, supporting the efficient personalization of time series foundation models. With the advantages of low overhead, privacy protection and efficient performance, PhysioPFM proposes framework for clinical diagnosis of medical physiological signals.
Training text and series as prompts. the original design of PhysioPFM, the shapelets are used for the prompts. We

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Acknowledgments
evaluation of large language models. ACM Transactions Intelligent Systems and Technology, 15(3):1-45, 2024.
This work was supported by the National Key R&D Program of China (2022DD0160703), the National Natural Science Foundation of China (62202422 and 62372408).
Che, Z., Cheng, Y., Zhai, S., Sun, Z.. and Liu, Y. Boosting deep learning risk prediction with generative adversarial networks for electronic health records. In 2017 IEEE International Conference on Data Mining (ICDM), pp. 787-792. IEEE, 2017.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here
Chen, H.-Y., Li, Y., Cui, Y., Zhang, M., Chao, W.-L.,, and Zhang, L. Train-once-for-all personalization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11818-11827, 2023.
References
Chen, J., Cao, T., Xu, J., Li, J., Chen, Z., Xiao, T., and Yang, Y. Con4m: Context-aware consistency learning framework for segmented time series classification. arXiv preprint arXiv:2408. 00044, 2024.
Al-Saegh, A., Dawwd, S. A., and Abdul-Jabbar, J. M. Cut- cat: An augmentation method for eeg classification. Neu- ral Nenworks, 141:433-444 443,44432222 2021.
Alvarez-Estevez, D. and Rijsman, R. M. Inter-database validation of a deep learning approach for automatic sleep scoring. PloS one, 16(8):e025611) 2021.
Clifford, G. D., Liu, C., Moody, B., Li-wei, H. L., Silva, I., Li, Q., Johnson, A., and Mark. R. G. Af classifica- tion from a short single lead ecg recording: The phys- ionet/computing cardiology challenge 2017. In 2017 Computing Cardiology (CinC), pp. 1-4. IEEE, 2017.
Annavajjala, A., Khare, A., Agrawal, A., Fedorov, I., Lat- apic, H., Lee, M., and Tumanov, A. Deps: Delayed c-shrinking for faster once-for-all training. In European Conference on Computer Vision, pp. 315-331. Springer, 2024.
Dong, H., Supratak, A., Pan, W., Wu, C., Matthews, P. M., and Guo, Y. Mixed neural network approach for temporal sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(2):324 2017.
Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403 200.0:24003.7815, 2024.
Dong, J., Wu, H., Zhang, H., Zhang. I.., Wang, J., and Long, M. Simmtm: A simple pre-training framework for masked time-series modeling. Advances in Neural Information Processing Systems, 36, 2023.
Ayata, D., Yaslan, Y., and Kamasak, M. E. Emotion recog- nition from multimodal physiological signals for emotion aware healthcare systems. Journal of Medical and Bio- logical Engineering, 40: 10:1499557 2020.
Garza, A. and Mergenthaler-Canseey M. Timegpt-1. arXiv preprint arXiv:2310.03550 2023a.
Blankertz, B., Muller, K. K.-R., Krusienski, D. J., Schalk, G.,
Garza, A. and Mergenthaler-Canssco,, M. Timegpt-1, arXiv preprint arXiv:2310.03589, 2023b.
Wolpaw, J. R., Schlogl, A., Pfurtscheller, G., Millan, J. R., Schroder, M., and Birbaumer, N. The bci competition ii:: Validating alternative approaches to actual bci problems. IEEE transactions on neural systems and rehabilitation engineering, 14(2):53--155 2006.
Glocker, B., Jones, C.,, Roschewitz, M., and Winzeck, S. Risk of bias in chest radiography deep learning foun- dation models. Radiology: Artificial Intelligence, 5(6): e230060, 2023.
Cai, H., Chen, T., Zhang, W., Yu, Y., and Wang. J. Efficient architecture search by network transformation. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Goel, K., Sood, N., and Saini, Ensemble technique based parkison's 'ss disease detection from fog and eeg signals. In 2023 World Conference on Communication Computing (WCONF), pp. 1-5. IEEE, 2023.
Chang, C., Peng, W.-C., and Chen, T.F.. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. arXiv preprint arXiv:2.388.088 08466, 2023.
Goldberger, A., Amaral, I.., Glass, I.., Hausdorff, J., Ivanov, P. C., Mark, R., Mietus, J., Moody, G., Peng, C., Stanley, H., et al. Physionet: Components of a new research resource for complex physiologic signals. cire. 101 (23): e2155e2200/crrcllation electronic pages. 2000. june 13.
Chang, Y., Wang, X., Wang, J., WU, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. survey on
10

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Goldberger, A. L., Amaral, L. A., Glass, L.,, Hausdorff, J. M., Ivanov, P. C.,, Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., and Stanley, H. E. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 123(: c215- 2000.
wireless low-cost off-the-shelf devices. IEEE journal of biomedical and health informatics, 22(1):98-107, 2017.
Kemp, B., Zwinderman, A. H., Tuk, B., Kamphuisen, H. A., and Oberye, J. Analysis of sleep-dependen neuronal feedback loop: the slow-wave microcontinuity of the eeg. IEEE Transactions on Biomedical Engineering, 47(9):
Goswami, M., Szafer, K., Choudhry, A., Cai, Y., Li, S., and Dubrawski, A. Moment: A family of open time- series foundation models. In Forty-first International Conference on Machine Learning, 2024.
1 1185-1194, 2000.
Khalighi, s., Sousa, T., Santos, J. M., and Nunes, U. Isruc-sleep: A comprehensive public dataset for sleep researchers. Computer methods and programs in biomedicine, 124:180-1922 2016.
Gruver, N., Finzi, M.. Qiu, S., and Wilson, A. G. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024.
Koldijk, S., Sappelli, M., Verberne, S., Neerinex, M. A., and Kraaij. W. The swell knowledge work dataset for stress and modeling research. In Proceedings of the 16th international conference maltimodal interaction, pp. 291-298, 2014.
Gupta, D., Bhatti, A., Parmar, S., Dan, C., Liu, Y., Shen, B., and Lee, S. Low-rank adaptation of time series foun- dational models for out-of- of--omainn modality forecasting. arXiv preprint arXiv:2405. 10216, 2024.
Kong, G., Li, C., Peng, H., Han, Z., and Qiao, H. Beg-based sleep stage classification via neural architecture search. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:1075-1000 2023.
Han, S., Mao, H., and Dally, W. J. Deep compres- sion: Compressing deep neural networks with pruning, trained quantization and huffman coding, arXiv preprint arXiv:1510.00
2015.
Le, X.-M., Tran, M.-T., and Huynh, V.-N. Learning percep- tual position- aware shapelets for time series classification. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 53-69, Springer, 2022.
He, H., Queen, O., Koker, T., Cuevas, C., Tsiligkaridis, T., and Zitnik, M. Domain adaptation for time series under feature and label shifts. In International Conference Machine Learning, pp. 12746-12774, PMLR, 2023.
Hu, E. J., Wallis, P., Allen-Zhu, 7., Li, Y., Wang. S., Wang. L., Chen, W., et al. Lora: Low-rank adaptation of large language models. In International Conference Learn- ing Representations, 2021.
Le, X.-M.,, Luo, L., Aickelin, U., and Tran, M.-T. Shape- former: Shapelet transformer for multivariate time series classification. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1484-1494, 2024.
Ichimaru, Y. and Moody, G. Development of the polysomno- graphic database cd-rom. Psychiatry and clinical neu- rosciences, 175-1777 1999.
Lee, C.-H., Kim, H., Han, H.-j., Jung, M.-K., Yoon, B. C., and Kim, D.-J. Neuronet: A novel hybrid self-supervised learning framework for sleep stage classification using single-channel eeg. arXiv preprint arXiv:2404.1 17585, 2024.
Jangg Y., Xia, L., Wei, W., Luo, D., Lin, K., and Huang, C. Diffmmm: Multi-modas diffusion model for recommen- dation. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 7591-7599, 2024a.
Li, C., Farkhoor, H., Liu, R., and Yosinski, J. Measuring the intrinsic dimension of objective landscapes. arXiv preprint arXiv:1804.088: 2018.
Jiang. Y., Yang, Y., Xia, L.., Luo, D., Lin, K., and Huang, C. Recim: Recommendation instruction tuning. arXiv preprint ar arXiv:2412. 29..02222..999..., 2024b.
Li, C., Zhang, 7., Zhang, X., Huang, G., Liu, Y., and Chen, X. Eeg-based emotion recognition via transformer neural architecture search. IEEE Transactions on Industrial Informatics, 19(4):6016-6025, 2022.
Jin, M., Wang, S. Ma, L., Chu, Z., Zhang, Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., et al. Time- Ilm: Time series forecasting by reprogramming large lan- guage models. In The Twelfth International Conference on Learning Representations, 2024.
Li, C., Wang, F., Zhao, Z., Wang, H., and Schuller, B. W. Attention- -aaeed temporal graph representation learning for eeg-based emotion recognition. JEEE Journal of Biomedical and Health Informatics, 2024a.
Katsigiannis, S. and Ramzan, N. Dreamer: A database for emotion recognition through eeg and ecg signals from

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Li, G., Choi, B., Xu, J., Bhowmick, S. S., Chun, K.-P., and Wong, G. L.-H. Shapenet: A shapelet-neural network approach for multivariate time series classification. In Proceedings of the AAAI conference on artificial intelli- gence, volume 35, pp. 8375-8383, 2021.
McKeen, K., Oliva, L., Masood, S., Toma, A., Rubin, B., and Wang, B. Beg-fm: An open electrocardiogram foun- dation model. arXiv preprint arXiv:2408.00 05188, 2024.
Midani, W., Ouarda, W., and Ayed, M. B. Deeparr:
investigative tool for arrhythmia detection using a contex- tual deep neural network from electrocardiograms (ecg) signals. Biomedical Signal Processing and Control, 85: 104954, 2023.
Li, H. Multimodal dataset of freezing of gait in parkinson's discase. Mendeley Data, 3:2021, 2021.
Li, M., Liu, K., Chen, H., Bu, J., Wang, H., and Wang. H. Tsinr: Capturing temporal continuity via implicit neural representations for time series anomaly detection. arXiv preprint ar www.v:2411.11666 2024b.
Moody, G. B. and Mark, R. G. Development and evalu- ation of 2-lead eeg analysis program. Computers in cardiology, (1982):39 44, 1982.
Li, Z., Shang, X., He, R., Lin, T., and Wu, C. No fear of classifier biases: Neural collapse inspired federated learn- ing with synthetic and fixed classifier. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5319-5329, 2023.
Moody, G. and Mark, R. The impact of the mit-bih arrhythmia database. IEEE engineering in medicine and biology magazine, 20(3):45-50, 2001.
Moody, G. B., Muldrow, W., and Mark, R. G. A noise stress test for arrhythmia detectors. Computers in cardiology, 113)338,,334, 1984.
Li, Z., Gao, L.., and Wu, Text-to-model: Text-conditioned neural network diffusion for train-once-for-all personal- ization. arXiv preprint arXiv:2405. 14132, 2024c.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Leurning Representations, 2023.
Liu, F., Liu, C., Zhao, L., Zhang, X., Wu, X., Xu, X., Liu, Y., Ma, C., Wei, S., He, Z., et al. An open access database for evaluating the algorithms of electrocardiogram rhythm and morphology abnormality detection. Journal of Med- ical Imaging and Health Informatics, 8(7): 1373:336-, 2018.
Obeid, I. and Picone, J. The temple university hospital eeg data corpus. Frontiers in neuroscience, 10:196, 2016.
Pan, T., Ye, Y., Cai, H., Huang, S., Yang, Y., and Wang, G. Multimodal physiological signals fusion for online emotion recognition. In Proceedings of the 31 ACM International Conference on Multimedia, pp. 5879-5888, 2023.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers effec- tive for time series forecasting. In The Twelfth Interna- lional Conference on Learning Representations, 2024a.
Papyan, V., Han, X., and Donoho, D. L. Prevalence of neural collapse during the terminal phase of deep learn- ing training. Proceedings of the National Academy of Sciences, 117(40):2462-24463. 2020.
Liu, Y.. Zhang. H., Li, C., Huang, X., Wang, J., and Long, M. Timer: Generative pre-trained transformers are large time series models. In Forty-first International Conference Machine Learning, 2024b.
Peebles, W., Radosavovic, I., Brooks, T., Efros, A. A., and Malik, J. Learning to learn with generative mod- els of neural network checkpoints. arXiv preprint 1rX2: 2022.
Liu, Z., Alavi, A., Li, M., and Zhang, Self-supervised contrastive learning for medical time series: A systematic review. Sensors, 23(9):4221, 2023.
Liu, Z. Chen, D., Pei, W. Ma, Q., et al. Scale-teaching: ro- bust multi-scale training for time series classification with noisy labels, Advances in Neural Information Processing Systems, 36, 2024c.
Pham, T. T., Moore, S. T., Lewis, S. G., Nguyen, D. N., Dutkiewicz, E., Fuglevand, A. J., McEwan, A. L., and Leong, P. H. Freezing of gait detection in parkinson's disease: A subject-indeppnden detector using anomaly scores. IEEE Transactions on Biomedical Engineering, 6A(11):2719-2728, 2017. doi: 10.1109/TBMF.20 2665438.
Liu, Z., Pei, W., Lan, D., and Ma, Q. Diffusion language- shapelets for semi- cuiissupeeeiee time-series classification. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 38, pp. 14079-14087, 2024d.
Qin, J., Wu, J., Chen, W., Ren, Y., Li, H., Wu, H., Xiao, X., Wang, R., and Wen, Diffusiongpt: Llm- driven text-to eext-to-image generation system, arXiv preprint arXiv:2401.1006 2024.
Markova, V. Database for cognitive load affect and stress recognition, 2020. URL https: doi www.dooi.oo..eoooo.....oo.....ee org/o0/gl0oo0 1 21227 hw////bbwww5..
12

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Qiu, Y., Lin, F., Chen, W., and Xu, M. Pre-training in medical data: A survey. Machine Intelligence Research, 20(2):477179, 2023.
Terzano, M. G., Parrino, L., Sherieri, A., Chervin, R., Chokroverty, S., Guilleminault, C.,, Hirshkowitz, M., Ma- howald, M., Moldofsky, H., Rosa, A., et al. Atlas, rules, and recording techniques for the scoring of eyclic alter- nating pattern (cap) in human sleep. Sleep medicine, 2 (6):537-554, 2001.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Thapa, R., He, B., Kjaer, M. R., Moore IV, H., Ganjoo, G., Mignot, E., and Zou, J. Sleepfm: Multi-modal repre- sentation learning for sleep across brain activity, ecg and respiratory signals. In Forty-first International Confer- ence on Machine Learning, 2024.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pp. 8748-8763. PMLR, 2021.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., al. Llama: Open and efficient foundation lan- guage models. arXiv preprint urXiv: 13971, 2023a.
Rasul, K., Ashok, A., Williams, A. R., Khorasani, A., Adamopoulos, G., Bhagwatkar, R., Biloš, M., Ghonia, H., Hassen, N., Schneider, A., et al. Lag-Ilama: Towards foundation models for time series forecasting. In RO- FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023.
Touvron, H., Martin, L.,, Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.0923 2023b.
Schalk, G., McFarland, D. J., Hinterberger. T., Birbaumer, N., and Wolpaw, J. R. Bei2000: a general-purpose brain- computer interface (bci) system. IEEE Transactions on biomedical engineering, 5166):1034-1043, 2004.
Täujan, A.-M., Andrei, A.-G,, and Ionescu, B. Freezing of gait detection for parkinson's 's disease patients using accelerometer data: Case study. In 2020 International Conference on e-Health and Bioengineering (EHB), pp. 1-4, 2020. doi: 10.1 11000/EEBB0000000000002222
Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., and Van Laerhoven, K. Introducing wesad, a multimodal dataset for wearable stress and affect detection. In Pro- ceedings of the 20th ACM international conference multimodal interaction, pp. 400-408, 2018.
Wagner, P., Strodthoff, N., Bousseljot, R.-D., Kreiseler, D., Lunze, F. L., Samek, W. and Schaeffter. T. Ptb-xl, large publicly available electrocardiography dataset. Scientific data, 101):1-15, 2020.
Snell, J., Swersky, K., and Zemel, R. Prototypical networks for few-shot learning. Advances in neurul information processing systems, 30, 2017.
Wang, H., Cui, Z., Chen, Y., Avidan, M., Abdallah, A. B., and Kronzer, A. Predicting hospital readmission via cost-sensitive deep learning. IEEE/ACM transactions on computational biology and bioinformatics, 15(6):1968- 1978, 2018a.
Spathis, D., Perez-Pozuelo, I., Brage, S., Wareham, N. J., and Mascolo, C. Learning generalizable physiological representations from lcrle wearable data. arXiv preprint arXiv:2011.0460) 2020.
Wang. H., Wu, J., Zhang, P., and Chen, Y. Learning shapelet patterns from network-based time series. IEEE transac- tions on industrial informatics, 15(7):3864-382 3876, 2018b.
Subramanian, R., Wache, J., Abadi, M. K., Vieriu, R. L., Winkler, S., and Sebe, N. Ascertain: Emotion and person- ality recognition using commercial sensors. IEEE Trans- actions Affective Computing, 9(2):147-160, 2016.
Wang, J., Zhao, S., Jiang, H., Li, S., Li, T., and Pan, G. Gen- eralizable sleep staging via multi-level domain alignment. Proceedings of the AAAI Conference on Artificial In- telligence, volume 38, pp. 265-273, 2024a.
Supratak, A., Dong, H., Wu, C., and Guo, Y. Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg. IEEE transactions on neural systems and rehabilitation engineering, 25(11):1998 20088,00002220 2017.
Wang, L., Dong, X., Wang, Y., Liu, L., An, W., and Guo, Y. Learnable lookup table for neural network quantization. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 12423-12433, 2022.
Taddei, A., Distante, G., Emdin, M., Pisani, P., Moody, G., Zeelenberg, C., and Marchesi, C. The european st-t database: standard for evaluating systems for the anal- ysis of st-t changes in ambulatory electrocardiooraapyy European heart journal, 1669):166-1172. 1992.
Wang. X., Zhang, J., He, C., Wu, H., and Cheng, L. A novel emotion recognition method based on the feature fusion of single- eeg and ecg signals. IEEE Internet Things Journal, 2023.
13

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Wang, Y., Han, Y., Wang, H., and Zhang, X. Contrast every- thing: A hierarchical contrastive framework for medical time-series. Advances in Neural Information Processing Systems, 36, 2024b.
Zhang, J. and Wu, Y. A new method for automatic sleep stage classification. IEEE transactions on biomedical circuits and systems, 1155:1097-1110, 2017.
Zhang, L., Walter, S., Ma, X., Werner, P., Al-Hamadi, A.,
Wu, C., Wang. H., Zhang, X., Fang, Z., and Bu, J. Spatio- temporal heterogeneous federated learning for time series classification with multi-view orthogonal training. In ACM Multimedia 2024, 2024.
Traue, H. C., and Gruss, S. "biovid emo db": A multi- modal database for emotion analyses validated by subjec- tive ratings. In 2016 IEEE Symposium Series Compu- tational Intelligence (SSCI), pp. 1-6. IEEE, 2016.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. Timesnet: Temporal 2d-variation modeling for gen- eral time series analysis. The Eleventh International Conference on Learning Representations, 2023.
Zhang, N. and Sun, Multiview unsupervised shapelet learning for multivariate time series clustering. IEEE Transactions Pattern Analysis and Machine Intelli- gence, 45(4):4981 -4996, 2022.
Yang, Y., Chen, S., Li, X., Xie, L., Lin, Z., and Tao, D. Inducing neural collapse in imbalanced learning: Do we really need a learnable classifier at the end of deep neural network? Advances in neural information processing systems, 35:37991-38003 2022a.
Zhang, W., Yang, Z., Li, H.,, Huang, D., Wang. L., Wei, Y.. Zhang, L., Ma, L., Feng, H., Pan, J., et al. Multimodal data for the detection of freezing of gait in parkinson's disease. Scientific data, 9(1):606, 2022.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 35, pp. 11106-11111 1111,111,,,,,001 2021.
Yang. Y., Truong, N. D., Eshraghian, K., Nikpour, A., and Kavehei, O. Weak self-supervised learning for seizure forecasting: a feasibility study. Royal Society Open Sci- ence, (8):220374 2022b.
Zhou, T. Ma. Z., Wen, Q. Wang, X., Sun, L., and Jin, R. Fedformer: Frequency enhanced decomposed trans- former for long-term series forecasting. In Interna- tional conference on machine learning, pp. 27268-27286 PMLR, 2022.
Yang. Y. Yuan, H., Li, X., Lin, Z., Torr, P., and Tao, D. Neural collapse inspired feature-classffier alignment for few- maas-innrrementl learning. In The Eleventh International Conference on Learning Representations, 2023.
Zhou, T., Niu, P., Sun, L.,, Jin, R., et al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 01443322-433555 2023.
Ye. L. and Keogh, E. Time series shapelets: a new prim- itive for data mining. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discov- ery and data mining, pp. 947-956, 2009.
Zhou, Y., Li, H., Du, S., Yao, J., Zhang, Y., and Wang, Y. Low-rank knowledge decomposition for medical founda- tion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11611- 1 11620, 2024.
Yuan, Z., Zhang, D., Chen, J., Gu, G., and Yang, Y. Brant- 2: Foundation model for brain signals. arXiv preprint 1022,1 2024.
Zeng, Y., Zhang, X., Li, H., Wang, J., Zhang, J., and Zhou, W. X 2-vlm: All-in-one pre-trained model for vision- language tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Zhang, D., Yuan, Z., Chen, J., Chen, K., and Yang. Y. Brant- x: A unified physiological signal alignment framework. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4155 4166, 2024.
Zhang, G.-Q., Cui, L., Mueller, R., Tao, S., Kim, M., Rueschman, M.. Mariani, S., Mobley, D., and Redline, S. The national sleep research resource: towards a sleep data commons. Journal of the American Medical Informatics Association, 23(10):1351-333 2018
14

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
A. Appendix
A.1. Offfine Shapelet Discovery
Considering that most existing shapelets mining methods require online training, that is, training the classification model while improving the mining quality, and our method does not need directly use shapelets as a training feature but only as an input condition for the generator, we adopted an existing Offline Shapelet Discovery (Le et al., 2022: 2024)
In the shapelet extraction stage, OSD identifies shapelet candidates using Perceptually Important Points (PIPs), It starts by including the first and last points of the time series in the PIPs set. Then, it repeatedly adds the point with the highest reconstruction distance to the set. Each time a new PIP is added, new shapelet candidates are created using three consecutive PIPs. The number of PIPs determines the size of candidates.
In the shapelet selection stage, OSD selects shapelets by ensuring an equal number is chosen for each class. For each shapelet candidate, we calculate its Pairwise Shapelet Distance (PSD), denoted as:
T-I+1 PSD(X,S.) = min (CID + 11.
(12)
where X is the given series of length T, a subsequence S = 81, ...., 81 of length I, with | < T, CID is the complexity- invariant distance. This distance is used to compute the "information gain," which measures how well a shapelet separates classes. The shapelets with the highest information gain are selected and added the final shapelet pool. To compress the input dimension, we set fixed number for all classes {S Vcc E [C] as shapelet prototypes.
A.2. Downstream Tasks
Sleep-state defection. Sleep is crucial for human health. It accounts for one-third of human lifetime. Effective diagnosis and treatment of patients with sleep-related disorders is currently pressing and intensively researched topic in the healthcare community (Zhang& & Wu, 2017; Dong et al., 2017).
Emotion detection. Emotion detection is the process of identifying human emotions and has been applied in many fields such as healthcare, safe driving, and the metaverse (Ayata et al., 2020). With the rapid development of portable wearable physiological signals have been conveniently monitored and analyzed online throughout the day (Pan et al., 2023).
Arrhythmia diagnosis. Cardiovascular disease has become the leading cause of death from nom-commu diseases worldwide, with more people dying from cardiovascular disease each year than from any other cause of death. Arrhythmia is common cardiovascular syndrome. Abnormal origin of cardiac electrical stimulation, abnormal conduction sequence, and frequency changes can all cause arrhythmia, which is a major challenge currently faced by the clinical treatment of cardiovascular disease. Different types of abnormal heart rhythms will show different states in the electrocardiogram waveform and frequency. Studying electrocardiogran signals can diagnose a variety of arrhythmia symptoms.
Freezing of Gaits Detection. Freezing of gait (FoG) a common symptom of Parkinson's 's disease (PD) and usually involves an inability to cope with concurrent cognitive, limbic, and motor inputs, resulting in movement disruption (Täuţan et al., 2020). Current clinical FoG assessments are self-report diaries by patients and manual video analysis by experts (Pham et al., 2017).
A.3. Detailed Algorithm
We describe the detailed algorithm of the diffusion transformer training process, as followed Algorithm 1:
A.4. Details of Baselines
We introduce the basclines for comparison:
SimMTM (Dong et al., 2023), a pre-trained framework time series for recovering masked time points via weighted aggregation of multiple neighbours outside the manifold.
TimesNet (Wu et al., 2023), analyzes time series changes from a multi-periodicity perspective, expands one-dimensional time series data to wwo-dimensional space, and uses advanced visual backbone networks for feature extraction.
iTtransformer (Liu al., 2024a), this method embeds the entire time series of each variable into tokens independently.
15

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Algorithm 1 LoRA generator training
1: Input: Number of training runs N. training sample map T [(AW,, 4)) diffusion process length J, diffusion cumulative variance schedule a.
2: Initialize: Learnable parameters & for G
3: for 1, ..,, do
4: Sample a mini-batch of data from data map
5: (AWe, 4x) T
6: Noise LoRA parameters
~ U(11,...,
8:
AW2 ~ ~NVTAAWW, (1- T)
9:
Compute the predictions
10:
AW; +- (a())t)), AW
11:
# *Cmpptee the loss
12:
loss + 1&W4- AW
13:
# #Update DiT's parameters
14:
Φi+1 +- applateloss;;; σ)
15:
end for
PatchTST (Nie et al., 2023), processes each dimension of the multivariate time series separately, that is, inputs each dimension into the Transformer Backbone separately, and then splices the obtained prediction results along the dimension direction.
FEDformer (Zhou et al., 2022), reduces the distribution shifts between input and output through seasonal-tree decomposi- tion; and proposes model structure that applies attention mechanism in the frequency domain to increase robustness to noise.
Informer (Zhou et al., 2021), proposes ProbSpare Self-Attention effectively replaces the traditional Self-Attention
OneFitsAll (Zhou et al., 2023), trains time series data on the pre-trained GPT-2 weights.
Time-LLM (Jin al., 2024), freezes the LLM backbone, and then uses two learnable modules (Patch Reprogramming and Output Projection) reprogram the input and output respectively.
MOMENT (Goswami et al., 2024). collects large and diverse collection of public time series, Time-series Pile. Then it uses lightweight reconstruction head to reconstruct the embedding of the input time series for pre-training.
SMeepFM (Thapa et al., 2024), curates a large polysomnography dataset of multi-modal sleep recordings and trains the model by contrastive learning.
SIeepDG (Wang al., 2024a), designs an epoch-level feature alignment to align the feature distribution of each single sleep epoch between different domains and designs sequence-level feature alignment to minimize the difference in sequence features between different domains.
LSTM-MLP (Wang et al., 2023), proposes an emotion recognition method based on feature fusion of single-lead EEG and ECG signals using various time domain, frequency domain and nonlinear features.
OMHGL (Pan et al., 2023), includes multimodal hypergraph fusion and online hypergraph learning. Multimodal hypergraph fusion can fuse multimodal physiological signals and effectively obtain emotion-related information. Online hypergraph learning aims to learn new information from online data by updating hypergraph projection.
DeepArr (Midani et al., 2023), combines feed-forward and recurrent deep neural networks using a sequential fusion approach to exploit relevant feature representations of arrhythmias in electrocardiogram (ECG) signals.
Extra Tree Classifier (Goel et al., 2023), combines the integration technology of multiple prediction methods to improve the performance of the EEG signal gait freezing detection model
A.5. Evaluation Metrics
In this section, we describe the evaluation metrics we adopted in the experiments for classification tasks. We first introduce the concept of a confusion matrix. TP (True Positive): correctly predicted positive example. That is, the true value of the
16

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
data is positive example, and the predicted value is also a positive example; TN (True Negative): correctly predicted negative example. That is, the value of the data is negative example, and the predicted value is also negative example; FP (False Positive): wrongly predicted positive example. That is, the true value of the data is negative example, but it is wrongly predicted positive example; FN (False Negative): a wrongly predicted negative example. That is, the true value of the data is positive example, but it is wrongly predicted as a negative example.
Accuracy, Accuracy refers to ratio correctly classified samples to the total number of samples.
TP+TN
Accuracy TP
(13)
Precision. Precision refers to the proportion samples that are positive among the samples predicted be positive.
TP
Precision
(14)
TP FP
Recall. Recall refers to the ratio of the actual number of positive samples in the predicted positive samples the total number of positive samples in the samples.
TP
Recall
(15)
FN
F1. The FI score is a weighted average of precision and recall.
Precision x Recall
= Precision + Recall
(16)
Macro F1. The Macro F1 score refers to the average of each class's Fl score.
Macro F1
Σ F1
(17)
Kappa coefficient. k is method used to evaluate the consistency of classification models.
TP+TN Po TP + FP+FP- TN FN
Accuracy
P. (TP (TP+FP) X (TP + FN) + (FN + +NN (FP+ TN)
(18)
(TP (TP FNY
Po P K 1 1-P P
AUC. AUC (Area Under the Curve) is an indicator for evaluating the performance of binary classification models, usually used the area under the ROC curve (Receiver Operating Characteristic curve). AUC represents the ability of the classifier to rank positive examples before negative examples.
AUC //''
Precision (u))du (w)))all
(19)
A.6. Public Datasets
In Table 6, we provide statistical information collected public physiological signals, mainly from (Zhang al., 2024; Qiu et al., 2023) and PyhsioNet (Goldberger et al.). We exclude the datasets for experiment evaluations from pre-training.
B. Theoretical analysis
The proof for Proposition 3.1 is given as:
17

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Table Statisties of the physiological benchmark datasets.
Task
Nume
Modulities
# of subjects
Sampling
MIT-BIH arhythmia dataset (Moody & Mark, 2001)
ECG
360Hz
PTB-XL (Wagner et ul., 2020)
ECG
71
500Hz
Arhythmmia Diagnosis
European ST-T database (Taddei ct al., 1992)
BCG
2
250Hz
AF clussification challenge 2017 (Clifford et al., 2017)
BCG
300Hz
PTB diagnostic ECG (Wagner et al,, 2020)
BCG
9
N/A
AHA (Moody & Mark. 1982)
ECG
8
250Hz
CPSC2018 (Liu et al., 2018)
ECG
8
500Hz
Denoise
MIT-BIH noise stress test (Moody et al., 1984)
BCG
360Hz
CLAS (Markova, 2020)
BCG.BVP.EM,.OM GG,R
60
256HZ
SWELL (Koldijk et al., 20114)
ECG.SC
2048HZ
Emotion Detection
ASCERTAIN (Subramanian al., 2016)
EEG, ECG, EDA
58
N/A
BIO-VID EMO DB (Zhang et al., 2016)
ECG, EMG, SC EEG, ECG
86
N/A
DREAMER (Katsigiannis & Ramzan, 2017)
23
256Hz
Seizure Detection
Hospital (TUH) (Oheid & Picone, 2016)
ERG
315
200Hz
Sleep-EDF (Kemp et al., 2000)
EEG, EOG, EMG EEG. EOG, EMG
22
100Hz
SHHS (Zhang al., 2018)
5804
12550Hz
Sleep-state Detection
ISRUC (Khalighi et 2016)
EEG, ECG. EOG, EMG
118
200Hz
HMC (Alvarez-Estevez Estevez & Rijsman, 2021)
EEG, ECG, EOG, EMG
151
256Hz
BCT Competition IV (Blankertz et al., 2006)
BEG, FOG
9
250Hz
Motor Imagery
EEG Motor Movenen//Imagery Dataset (Schalk et al., 2004)
EEG, EOG
109
160Hz
Freezing of Gaits
Li et, af (1.i, 2021)
BHG, EMG, ECG, SC, ACC
12
1000Hz
Cyclic Alteraatnn Pattern
CAP (Terzuno et ul., 2001)
EEG, ECG, EOG, EMG
108
512Hz
MIT-BIH PSG (Ichimaru & Moody. 1999)
EEG, ECG, EOG. EMG
18
250Hz
Sleep Apoca
UCDDB (Goldberger et ul., 2000)
EEG, BCG, BOG, EMG
25
128Hz
Stress and Affect Detection
WESAD (Schmidt et al., 2018)
ECG, EMG
700Hz
Proof. Suppose C' class prototype vectors P1, ,2,...,PP
PC E Rd satisfy: Unitization: Ip:l = 1, symmetric inner product
constraint: 2 P) = - Vi * j. Define the Gram matrix G RCXC where:
ifi J if
Gig pf
The matrix has the following properties: the diagonal elements are and the off-diagonal elements are - -C-1- The matrix rank is d, and it is symmetric semi-positive matrix (because the vector is in d-dimensional space). The above Gram matrix corresponds to the Simplex Equiangular Tight Frame. Its core features are: all off-diagonal elements are equal, that is, the angles between vectors are consistent, the vectors are evenly distributed in the feature space, and the minimum interval between classes maximized. This structure is the only configuration that satisfies symmetry and minimizes the similarity between classes. The angle between any vectors 0 satisfies:
cos = p.. Pj
C -
In classification problems, the decision boundary is determined by the geometric relationship the prototype vectors. For a linear classifier, the decision boundary for two categories and] j is:
Ilp,
1|x112 11
Rd (p: pj)"
Since ||p.]| ||p,]| = 1, the boundary is simplified to:
(pr -P)))T> r = 0.
The margin between the two boundaries is:
2
Margin
||p: pj
18

---

Efficient Personalized Adaptation for Physiological Signal Foundation Model
Calculate lpp, pyll = 2(1 p,) = 2 (1 + ot.)) ) so:
2
2(C 1)
Margin
C
2 2 )
When all class prototypes meet the symmetry condition, the intervals between all classes consistent and reach the maximum possible value, so the decision boundary is optimal.
19


Reviews:
[{"review_id": "3VcopA2vs9", "reviewer": "Reviewer_Q918", "summary": "This paper proposes a novel approach to achieve efficient adaptation for physiological signal foundation modals to private datasets. The main idea is to use Low-rank Adaptation (LoRA). However, unlike existing methods that train LoRA weights for adaptation, it utilizes a diffusion model to generate the LoRA weights. To this end, the paper first prepared 30 datasets and obtained LoRA weightings for them. They are then used to train a diffusion model with a Transformer as the backbone. The shapelet prototypes are extracted from the datasets and used as conditions in the diffusion generation. The trained diffusion model can then be used to generate a LoRA weight for private data. Empirical evaluations are performed on four typical physiological classification tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for recognizing the value of our work. We are grateful for your thorough feedback, especially considering the massive requirements of the review. We hope the following comments could address your questions:\n\n**W1:** Relation to hypernetwork.\n\n**For W1:** In a macro sense, our method and the hypernetwork both provide adaptive parameters for another backbone network. Specifically speaking, according to Figure 1(b) of the reference, the hypernetwork also needs to be trained together with the main model, calculate the performance loss of the main model, and perform gradient backward propagation on the hypernetwork. Unlike the hypernetwork, we do not update the hypernetwork based on the performance of the backbone model, which is costly in our context. Our paper directly maps the data feature space to the parameter space, aiming to establish a mapping between LoRA parameters and data features. Our method pursues better generation quality and aims to learn an effective and representative latent space. This is logically different from the traditional hypernetwork and is designed to meet the challenges of our scenario to achieve local training-free adaptation. In general, our method has similar macroscopic goals to hypernetworks, but their specific implementations are quite different. We will include a discussion of this aspect in the main text and provide more detailed related work.\n\n**W2:** Discussion of applying to multi-task scenarios.\n\n**For W2:** Thank you for raising this important concern. We focus on general physiological signals classification in this work. When the target task changes, e.g. transferring to a time series forecasting task, our approach may need to adjust the pre-training on the generator to build a new ability for new tasks. Another possible way is to expand the type of input condition, which could be a significant future work to extend our methods to multi-task scenarios. \n\n**W3:** The impact of training sample size for the generator.\n\n**For W3:** We kindly note that we have included the impact of training sample size on the generator in Section 4.3, Impact of training samples. To fully enhance the ability of the generator, we adopt multiple ways to expand the training sets in the data preparation phase, including partial dataset, partial class, and random selection of subjects. Given the different input of data, the generalized ability of the generator could be more diversified.\n\n**W4:** Definition of the adaptation time.\n\n**For W4:** We apologize for any confusion caused. For our proposed method, Figure 5 includes the time to generate LoRA weights and TSFM inference. For baselines, the adaptation time refers to the local training time. Because our method does not need to fine-tune the foundation model, while the baselines need to train and update the model. Backpropagation on a large model is time-consuming. We only have a single-step inference to generate LoRA, which takes little time compared to multiple-round training.", "claims_and_evidence": "The paper claims that the proposed PhysioPFM is more robust and efficient than existing approaches utilizing a time series foundation model for physiological signal tasks. The claim is well supported by the empirical evaluations. Comparison with recent SOTA methods indicates the effectiveness of the proposed approach. The efficiency comparison also shows that the proposed PhysioPFM is more efficient in terms of memory consumption and adaption time.", "methods_and_evaluation_criteria": "The proposed method is technically sound. The use of a diffusion model to generate LoRA weight is quite interesting and novel. The benchmark datasets used are appropriate, and the evaluation metrics are also suitable.", "theoretical_claims": "The Proposition 3.1 follows directly from Definition 2.1. No other theoretical claims are made.", "experimental_designs_or_analyses": "The experimental settings are sound. Sufficient details of the experimental design are included, and the baselines compared are appropriate. The evaluation metrics used are suitable. An ablation study is also performed.", "relation_to_broader_scientific_literature": "Fine-tuning foundation models using LoRA has been intensively explored in the literature. However, using a generative model to generate LoRA weights is a novel approach. The framework overall is similar to the hypernetwork approach [1]. The authors should also discuss the connections with the hypernetwork methods.\n\n[1] Chauhan, V.K., Zhou, J., Lu, P., Molaei, S. and Clifton, D.A., 2024. A brief review of hypernetworks in deep learning. Artificial Intelligence Review, 57(9), p.250.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n- The proposed method of generating LoRA weights is novel and interesting.\n- The paper is organized, well presented, and easy to follow.\n- The experiments are well executed, and the results are pretty promising.\n\nWeaknesses:\n- The diffusion model takes only the shaplets as a condition to generate LoRA weights. This seems to be a bit restrictive for multi-task scenarios since the same LoRA weights will be generated for different tasks given the same physiological signal input.\n- The diffusion model is trained using only 30 public physiological signal datasets; the impact of the number of datasets used in training the diffusion model is unclear.", "other_comments_or_suggestions": "Please see my comments above.", "questions_for_authors": "Usually, the inference of diffusion models is quite time-consuming. Does the adaptation time reported in Fig.5 of PhysioPFM correspond to the time needed for the diffusion model to generate the LoRA weight?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ruF9TSOKtN", "reviewer": "Reviewer_FR4e", "summary": "The paper provides a personalized approach to transfer the time series foundation model to clinical physiological signal tasks. The main constraints are the lower computing costs and privacy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate for your comprehensive and valuable review, particularly given the meticulous standards during the review of this venue. We appreciate the opportunity to address your concerns.\n\n**W1:** Clarification on the privacy issue.\n\n**For W1:** We apologize for missing a detailed discussion on privacy preservation. Privacy constraint is one of our contributions. A generic pre-trained time series foundation model is hard to fit the diverse local tasks and data. Uploading patients’ data to the cloud for robust pretraining may raise privacy concerns. Our proposed method achieves this goal by data isolation, that is, sensitive patients’ data are preserved locally.  Due to the removal of data exposure, privacy issues could be solved. The most related work includes some time series foundation models and federated learning. Brant-X [1] adopts the EEG foundation model Brant-2 as a basis, which is pre-trained on 4TB of private brain signal data. This approach requires private data and large-scale pre-training on it. If the medical entity has sufficient computing resources, it could be reasonable, otherwise, it may need to transfer private data to a robust cloud. On the other hand, federated learning preserves privacy by exchanging models, not data, which is similar to our paradigm.  [2] considers using FL to train a foundation model for medical time series. Multiple rounds of exchange on the large models may lead to remarkable communication costs, and models may be attacked by model poisoning attacks from history gradient updates. While our work focuses on the personalized TSFM with a privacy guarantee.  Due to the length limitation of the rebuttal, we promise to add a more comprehensive privacy-preserving literature on related work and our method.\n\n[1] Zhang, Daoze, et al. \"Brant-X: A Unified Physiological Signal Alignment Framework.\" KDD 2024.\n\n[2] Ali, Mahad, et al. \"Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting.\" IEEE EMBC 2025.\n\n**W2:** Definition 2.1 and Proposition 3.1.\n\n**For W2:** Thanks for your valuable comments. Definition 2.1 follows the standard definition of neural collapse. It is not a contribution, but an illustration. \n\nThe proof for Proposition 3.1 is given as:\n\n$\\textit{Proof.}$ Suppose $C$ class prototype vectors $p_1, p_2, \\dots, p_C \\in \\mathbb{R}^d$ satisfy: Unitization: $|p_i| = 1$, symmetric inner product constraint: $p_i^T p_j = -\\frac{1}{C-1}, \\forall i \\neq j$. Define the Gram matrix $G \\in \\mathbb{R}^{C \\times C}$, where:\n$$\nG_{ij} = p_i^T p_j =\n\\begin{cases}\n1 & \\text{if } i = j, \\newline\n-\\frac{1}{C-1} & \\text{if } i \\neq j.\n\\end{cases}\n$$\n\nThe matrix has the following properties: the diagonal elements are 1 and the off-diagonal elements are $-\\frac{1}{C-1}$. The matrix rank is $d$, and it is a symmetric semi-positive matrix (because the vector is in d-dimensional space). The above Gram matrix corresponds to the Simplex Equiangular Tight Frame. Its core features are: all off-diagonal elements are equal, that is, the angles between vectors are consistent, the vectors are evenly distributed in the feature space, and the minimum interval between classes is maximized. This structure is the only configuration that satisfies symmetry and minimizes the similarity between classes.\nThe angle between any two vectors $\\theta$ satisfies:\n$$\n\\cos\\theta = p_i^T p_j = -\\frac{1}{C-1}.\n$$\nIn classification problems, the decision boundary is determined by the geometric relationship of the prototype vectors. For a linear classifier, the decision boundary for two categories $i$ and $j$ is:\n$$\nx \\in \\mathbb{R}^d \\mid (p_i - p_j)^T x + \\frac{\\|p_j\\|^2 - \\|p_i\\|^2}{2} = 0.\n$$\n\nSince $\\|p_i\\| = \\|p_j\\| = 1$, the boundary is simplified to:\n$$\n(p_i - p_j)^T x = 0.\n$$\n\nThe margin $\\gamma$ between the two boundaries is:\n$$\n\\gamma= \\frac{2}{\\|p_i - p_j\\|}.\n$$\n\nCalculate $\\|p_i - p_j\\|^2 = 2(1 - p_i^T p_j) = 2\\left(1 + \\frac{1}{C-1}\\right)$, so:\n$$\n\\gamma = \\frac{2}{\\sqrt{2\\left(1 + \\frac{1}{C-1}\\right)}} = \\sqrt{\\frac{2(C-1)}{C}}.\n$$\nWhen all class prototypes meet the symmetry condition, the intervals between all classes are consistent and reach the maximum possible value, so the decision boundary is optimal.\n\nWe will add this proof to the appendix in the revised manuscripts.", "claims_and_evidence": "Not always. A main issue is that it is not clear how the authors are addressing the privacy protection in the paper. The paper would have benefitted from a clear positioning within the related literature. See comments bellow for more details.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "There is no proof given for Proposition 3.1.", "experimental_designs_or_analyses": "The experimental designs and the conducted analysis seem to be acceptable, and the ablation study demonstrates the relevance of the chosen techniques.", "relation_to_broader_scientific_literature": "The paper roughly integrates several concepts within a single foundation model to pre-train/fine-tune it. This paper provides some interesting results, and the ablation study confirms the relevance of the chosen techniques, such as low-rank adapter and neural collapse.", "essential_references_not_discussed": "There are some missing related work. Since this paper seeks to address privacy issues, it would have been relevant to include some related work on this topic.", "other_strengths_and_weaknesses": "A major issue in this paper is the lack of clear description of the contributions within the privacy constraint. The paper states that its major contribution is privacy preserving. However, there is no much information on the privacy, and how it is clearly preserved. Moreover, the paper does not cite any related work from the privacy preserving literature.\n\nIt is not clear from the paper that Definition 2.1 is a contribution  or not. \nMoreover, the paper and appendix do not present the proof of Proposition 3.1. It is not clear how it was established.\n\nThe two main ingredients in this work are (i) to train the foundation model with massive public physiological time series by using a low-rank adapter, and (ii) to use a neural collapse in order to combat the imbalance in the data distribution.\nMoreover, a diffusion transformer is used as a robust generator to synthesize the low-rank adapter weights.", "other_comments_or_suggestions": "-", "questions_for_authors": "-", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dcwpVDmhFZ", "reviewer": "Reviewer_qTAF", "summary": "* This paper studies medical time series classification based on physiological signals. \n* ML models for prediction from medical time series is challenging since we often have:\n    * Unbalanced amount of data for each signal\n    * Varying sampling frequency/duration\n* Time series foundation models (TSFMs) can be effective, however they can be expensive to adapt to individual medical centres, and we may not want to upload data from a given centre to a server for finetuning.\n* This work proposes an approach where we:\n    * Learn a dataset of LoRA weights for a TSFM \n    * Learn a diffusion transformer to output LoRA weights for a pretrained TSFM, using the LoRA weights found above as a training set, with the DiT input being a condensed representation  of the input dataset (shapelet transformed)\n* At inference time, for a new dataset, can run inference with shapelet + diffusion model to get LoRA weights, then adapt the TSFM without any further training cost.\n* The method is evaluated on four classes of physiological signal datasets: Sleep state detection, emotion detection, arrhythmia diagnosis, and freezing of gait detection. \n* The proposed model performs well, improving on baselines.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your valuable comments, and we are grateful for the time and effort you have invested in reviewing our work. Below, we provide a point-by-point response to address each of your concerns:\n\n**W1:** Clarifying the data partition.\n\n**For W1:** Thanks for your valuable comments. It is at the patient level. Physiological signal data are collected from individual subjects. The experiments are conducted on corresponding signals in a subject-independent setting. We divide the subjects into training, validation, and test sets in a 3:1:1 ratio. One subject’s data won’t appear across train/val/test sets. It may arouse concerns about how it can be considered a personalized adaptation (for baselines) without training on the data of a specific local patient. In practice, local test data must be unlabeled, and the model training is through existing labeled data, and then the model is applied to the patients who need to be diagnosed. So, this setting is reasonable and in line with the existing related works' paradigm.\n\n**W2:** Hyperparameter search.\n\n**For W2:** We test the best learning rate, diffusion steps, chunk size and rank of the adapter for our methods. For baselines, we generally follow the original hyperparameters and keep necessary settings fair, such as data partition.\n\n**W3:** Discussion on the setting and selection of the generator.\n\n**For W3:** The architecture of DiT has great expressive power in diffusion tasks, especially in conditional cross-modal applications such as text-to-image and text-to-video, where the original DiT is applied. Small models like MLP have difficulty in cross-modal generation and conditional generation. In difficult tasks such as parameter generation, the latent space generated by MLP may be poorly representative. Therefore, how to achieve a trade-off between the lightness and performance of the generator is a future work worth exploring. In this work, we mainly applied the SOTA cross-modal generator DiT to achieve high-quality parameter generation. At the same time, the transformer complies with the scaling law, a feature that will help the model to generalize only by amplifying parameters. We conduct an analysis of the generation quality on weights, considering the Euclidean distance between input weights(noised) and generated weights. As demonstrated in the following results, at the end of the training of DiT, the results of distance reached 2.32 to 4.10 for different tasks, proving superior generating performance than conditional GAN and MLP. In addition, MLP and CGAN require huge numbers of rounds to converge, more than 1000 epochs, while DiT does not.\n| Similarity   | DiT | MLP | CGAN |\n|---| --- | --- | --- |\n| Task 1     | 4.10    |  153.52   |  79.63  |\n| Task 2 |  2.32     |  139.46   |  75.49  |\n\n\n**W4:** Detailed ablation study on model fine-tuning.\n\n**For W4:** We sincerely thank you for your insightful feedback.  We evaluate the TSFM with local LoRA fine-tuning, and the results are in the following table.  We also add the existing results from Table 5, which includes ours without LoRA generation, and our method. It can be seen that training TSFM on local data could achieve superior performance in most cases. While our generated LoRA weights could also provide a comparable improvement to direct training. Without training or generated weights, the pre-trained TSFM is hard to fit the specific domain knowledge well.\n\n| Method   | Sleep-EDF | MIT-BIH | FoG\n|---| --- | --- | --- |\n| TSFM with local training     | 87.15    |  88.59   |  83.71   |\n| Ours w/o local LoRA | 82.08 |84.17| 76.45 |\n| Ours | 86.39| 89.94 | 81.32  |\n\n\n**W5:** Typo and presentations. \n\n**For W5:** We appreciate the reviewer's keen eye and thorough review. We have carefully revised the typos and will provide a more rigorous expression of Figure 3 in the revised version of our work.\n\n**W6:** Suggestions on related work.\n\n**For W6:** Thanks for your constructive suggestions. Due to the length limit, we have to adopt the current organization. It is truly significant to place the related work section in the main body for better understanding by the readers. We promise to reorganize it. We will also be adding more extensive related work, such as privacy protection-related work.", "claims_and_evidence": "Overall reasonable when compared to baselines.", "methods_and_evaluation_criteria": "Evaluation datasets are sensible; methods could use further investigation, see below.", "theoretical_claims": "Not major focus of paper.", "experimental_designs_or_analyses": "Positives:\n* Diverse range of datasets and tasks studied \n* Good selection of published baselines\n* Ablations generally thorough\n* Overall, encouraging results compared to the baselines.\n\nAreas for improvement/questions:\n* I would like more detail on the evaluation datasets. How are these split into train/val/test? Is it patient level? Without this, it’s hard to interpret the results well.\n* What kind of hyperparameter search happened with your method, vs the baselines? \n* Why diffusion for the LoRA learning? Could the standard MLP used in eg hypernetworks work, especially given small size of parameters? Would that be easier to train? \n* What happens if you take your pretrained TSFM and do local LoRA without your diffusion model at all? I understand this is computationally expensive, but it’s interesting to understand the effect of generative model for LoRA vs just learning the weights directly.", "relation_to_broader_scientific_literature": "The related work in the main paper is inadequate. A more detailed related work in main body would add a lot of value, in addition to what is in the appendix.", "essential_references_not_discussed": "Not familiar enough with literature to comment here.", "other_strengths_and_weaknesses": "* The writing needs work overall — there are a number of typographical errors, referencing a Table instead of a Figure, for example\n* The overall contribution of the method was not that clear. Rewording to more closely represent the Figure 3 would be quite valuable.", "other_comments_or_suggestions": "Overall, I think this is a good contribution, but the lack of related work in the main body makes it hard to contextualise the work. I also have some questions above that would be great to get clarity on.  If these are answered, I would be inclined to increase my score.", "questions_for_authors": "Please see above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vVZjIGjVbq", "reviewer": "Reviewer_57t7", "summary": "This work proposes a new method to adapt physiological signal foundation models using DiT to generate LoRA weight matrices.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your extensive feedback. We especially appreciate your effort to meet the conference's rigorous review criteria. We kindly address your questions as follows.\n\n\n**W1:** Scope of adaptation time.\n\n**For W1:** Thanks for your thoughtful comments. We separated the two phases of pre-training and adaptation, considering a practical cloud-edge scenario. If utilizing a traditional pre-trained TSFM, the TSFM is first pre-trained with massive data in a venue with sufficient resources (cloud). Then, the TSFM is used to be trained on local physiological signals, and its training time is the adaptation time we defined. We did not count the original pre-training of TSFM into its adaptation time. Similarly, for our method, pre-training can be achieved using public physiological signals and a venue with sufficient resources. Our adaptation time includes generator inference and TSFM inference. Therefore, we respectfully disagree that LoRA dataset preparation and DiT training are part of adaptation time.\n\n\n**W2:** Discussion of important ablation study and baselines.\n\n**For W2:** We sincerely thank you for your detailed suggestions.  We would like to address your concerns by showing additional ablation results for our method. We adopt the TSFM with local LoRA fine-tuning, TSFM with a prototype-based classifier, ECG-FM on arrhythmia diagnosis, along with two given row results in Table 5 as references. From the results, we can find that if the training is allowed, pre-trained TSFM could show great power in most cases after fine-tuning. Considering the variant without LoRA, we could acknowledge that the generated LoRA weights could provide a comparable improvement with direct training.  A prototypical network is also a valuable approach as it could surpass general baselines but still be lower than the adapted weight-based strategy (Ours, fine-tuning TSFM). It may require more computing cost in similarity(metric) calculation, and could be easily affected by imbalanced data. More alignment or neural-collapse tricks may help to improve this approach. Metric learning is a promising future work to explore. We will also add the complete results of ECG-FM to Table 3 in the revised manuscripts.\n\n| Method   | Sleep-EDF | MIT-BIH | FoG\n|---| --- | --- | --- |\n| TSFM with local training     | 87.15    |  88.59   |  83.71   |\n| TSFM with Proto-classifier |  82.24     |   81.06   |  74.56  |\n| ECG-FM | -  | 84.90  | - |\n| Ours w/o local LoRA | 82.08 |84.17| 76.45 |\n| Ours | 86.39| 89.94 | 81.32  |\n\n**W3:** Clarifications on generator selection.\n\n**For W3:** In parameter generation tasks, on the one hand, these small models may not be able to fully demonstrate their generalization ability when applied to more complex tasks and parameter spaces. On the other hand, previous methods do not support conditional high-performance parameter generation, and the novel DiT-based conditional neural network diffusion has better generation results. The architecture of DiT has great expressive power in diffusion tasks, especially in conditional cross-modal applications such as text-to-image and text-to-video. Unlike hypernetwork, which takes the model parameters as input and generates parameters, we directly map the data feature space to the parameter space. In addition, the hypernetwork needs to perform backpropagation based on the loss of the backbone network, which will be costly in large model scenarios. We conducted a validation experiment, taking the condition and noised model parameters as input, and measuring the Euclidean distance between the output model parameters and the original input parameters. The results show that the generation effect of the method based on conditional GAN or MLP is far behind DiT. As the SOTA cross-modal generator, we adopted DiT. It is also worthwhile to find the trade-off between cost and quality in future work.\n| Similarity   | DiT | MLP | CGAN |\n|---| --- | --- | --- |\n| Task 1     | 4.10    |  153.52   |  79.63  |\n| Task 2 |  2.32     |  139.46   |  75.49  |\n\n**W4:** Typos. \n\n**For W4:** Thank you for this astute observation. We apologize for any confusion caused and have carefully revised it.\n\n**W5:** Clarifying the data partition and definition of personalization.\n\n**For W5:** The experiments are conducted on corresponding signals in a subject-independent setting. We assign subjects to train/val/test partitions. One subject’s data won’t appear across train/val/test sets. The inference of the generator is based on testing sets, where the train and val sets are not used for our method but for baselines, for fairness. Compared with the generalized TSFM, our personalization refers to obtaining a model parameter that adapts to local data for new local tasks and data features. Because the generalized model may not work well on specific tasks, and fine-tuning a large foundation model will also be costly, realizing model personalization in a lightweight way is meaningful.", "claims_and_evidence": "The authors claim that the proposed method transfer physiological foundation model to different tasks with lower computing costs. While it is true that the proposed method claims to not need any training during adaptation stage, the proposed method requires significant compute to obtain the LoRA dataset and train the DiT. On the other hand, the baseline methods do not need any training between the pre-training and adaptation stage. In essence, the proposed method requires front-loading the adaptation compute to the stage between pre-training and adaptation. I think the author should tune down the claim for lower adaptation costs and be clear about the extra training compute required to obtain the LoRA dataset and train the DiT. Furthermore, it could be preferable to compare the total compute used after pre-training stage, it can be argued that the LoRA dataset preparation and DiT training is part of adaptation.", "methods_and_evaluation_criteria": "The benchmarks are reasonable.", "theoretical_claims": "No theorem or proof was provided.", "experimental_designs_or_analyses": "How about a baseline that fine-tunes the pre-trained TSFM (6-layer GPT2-based backbone). While I understand that this violate data privacy, all the baseline methods use different backbone networks. This will help to understand how much of the base performance comes from the pre-trained TSFM and how much improvement the DiT generated LoRA provides.\nAslo consider a prototypical network as baseline? During adaptation, one can use only the pre-trained TSFM (6-layer GPT2-based backbone), without finetuning LoRA, to calculate the prototypes, and make predictions via metric-based classifier. This is very compute efficient adaptation. This will also give an idea of how good the pre-trained TSFM is, as well as how much improvement the DiT generated LoRA provides.\nECG-FM was mentioned in related works, but not benchmarked for ECG arrhythmia task.", "relation_to_broader_scientific_literature": "The idea of using one neural network to generate weights for another neural network (hypernetworks) is not itself novel, but it has not been done with DiT to the best of my knowledge.", "essential_references_not_discussed": "What about GAN-based or hypernetworks for generating the LoRA weights? Why was DiT selected for generating the weights of LoRA when GAN and hypernetworks can achieve a similar purpose. Prior works have shared the similar idea of generating weights using a neural network for adaptation, perhaps these are relevant baselines to consider, since the core contribution of this work is generating LoRA weights using DiT.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "Table 1 Weak\nSection 3.3 multimodel \nSection 4.2 depp learning", "questions_for_authors": "Section 4.1 Experimental setup, it was claimed that 60:20:20 train:val:test split was performed. How is the 60% training data used for conditioning the DiT, i.e. were all 60% of data used for shapelet discovery? How is the validation set used, if at all?\nThe 60:20:20 split was randomly sampled, data from one patient can be present in both the training set and validation/test split. What is the “personalized” adaptation? It appears to be capable of adapting to unseen dataset.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel approach to achieve efficient adaptation for physiological signal foundation models to private datasets. The main idea is to using Low-rank Adaptation (LoRA). However, different from existing methods that train LoRA weights for adaptation, this paper utilizes a diffusion model to generate the LoRA weights, which is novel and interesting. All the four reviewers are positive to the work, although they also raised a lot of comments that can help to improve the work. The authors should improve the paper when preparing for the final version of the paper.
Decision: Accept (poster)