Title: Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals
Venue: NeurIPS 2025
Accepted: no
Keywords: Foundation Model

Signal Processing

Time Series

Wearable Sensing

Digital Health
Abstract: Time-series foundation models excel at tasks like forecasting across diverse data types by leveraging informative waveform representations. Wearable sensing data, however, pose unique challenges due to their variability in patterns and frequency bands, especially for healthcare-related outcomes. The main obstacle lies in crafting generalizable representations that adapt efficiently across heterogeneous sensing configurations and applications. To address this, we propose NormWear, the first multi-modal and ubiquitous foundation model designed to extract generalized and informative representations from wearable sensing data. Specifically, we design a channel-aware attention mechanism with a shared special liaison [CLS] token to detect signal patterns in both intra-sensor and inter-sensors. This helps the model to extract more meaningful information considering both time series themselves and the relationships between input sensors. This helps the model to be widely compatible with various sensors settings. NormWear is pretrained on a diverse set of physiological signals, including PPG, ECG, EEG, GSR, and IMU, from various public datasets. Our model shows exceptional generalizability across 11 public wearable sensing datasets, spanning 18 applications in mental health, body state inference, vital sign estimation, and disease risk evaluation. It consistently outperforms competitive baselines under zero-shot, partial-shot, and full-shot settings, indicating broad applicability in real-world health applications.

Abstract

Time-series foundation models excel at tasks like forecasting across diverse data types by leveraging informative waveform representations. Wearable sensing data, however, pose unique challenges due to their variability in patterns and frequency bands, especially for healthcare-related outcomes. The main obstacle lies in crafting generalizable representations that adapt efficiently across heterogeneous sensing configurations and applications. To address this, we propose NORMWEAR, the first multi-modal and ubiquitous foundation model designed to extract generalized and informative representations from wearable sensing data. Specifically, we design a channel-aware attention mechanism with a shared special liaison [CLS] token to detect signal patterns in both intra-sensor and inter-sensors. This helps the model to extract more meaningful information considering both time series themselves and the relationships between input sensors. This helps the model to be widely compatible with various sensors settings. NORMWEAR is pretrained on a diverse set of physiological signals, including PPG, ECG, EEG, GSR, and IMU, from various public datasets. Our model shows exceptional generalizability across 11 public wearable sensing datasets, spanning 18 applications in mental health, body state inference, vital sign estimation, and disease risk evaluation. It consistently outperforms competitive baselines under zero-shot, partial-shot, and full-shot settings, indicating broad applicability in real-world health applications.

1 Introduction

Mobile and wearable sensors have been shown to be valuable for the field of healthcare by passively and continuously tracking physiological signals such as photoplethysmography (PPG) for pulse, electrocardiography (ECG) for heart activity, galvanic skin response (GSR), and electroencephalography (EEG) for brain activity. These time series signals are beneficial for early diagnosis, personalized health insights, and remote patient monitoring (Zhang et al., 2024a).

Recently, several foundation models have emerged for time series modeling, including Ansari et al. (2024); Abbaspourazad et al. (2023); Woo et al. (2024); Foumani et al. (2024). Another common approach for signal modeling involves converting raw signal series into 2D images or spectrograms, using fixed-size sliding windows, followed by the use of visual encoders like Vision Transformers (ViT) to extract representations for making inferences (Semenoglou et al., 2023; Wimmer & Rekabsaz, 2023; Vishnupriya & Meenakshi, 2018; Chun et al., 2016; Krishnan et al., 2020; Dosovitskiy et al., 2020). These works have significantly advanced the field and provided valuable insights, yet two main issues still exists which need further exploration to fully understand their potential in wearable scenarios. First, contrastive learning-based foundation models (Abbaspourazad et al., 2023) rely on a predefined set of input signal types, making them unsuitable when transferring to scenarios with different types and numbers of sensors. Second, while both time series foundation models (Ansari et al., 2024; Zhang et al., 2022; Woo et al., 2024) and spectral-based approaches (Semenoglou et al., 2023; Wimmer & Rekabsaz, 2023) attempt to address this issue by training a generic encoder that

EEG T3

EEG T4

Task 1

PPG

Task 2

ECG

Task 3

EEG F7

EEG F8

EEG Cz

Task 4

ACC X

GSR

ACC Y

PPG

… …

Pre-Training

State Inference

Physical

Activity

Mental

State

Fatigue

Detection

Risk Evaluation

Hypertension

Diabetes

Brain

Abnormality

Heart

Abnormality

Muscle

Abnormality

EEG Main Tasks

Seizure

Tumor area

Eye state

Vital Sign Estimation

Blood

Pressure

Hemoglobin

Heart Rate

Adaption

Varied number and type of sensor channels

Reconstruction

Masking

Lightweight

Decoder

Number of

Input Channels

Patching and initial

embedding

Intra-channel Encoder

Inter-channel Encoder

Intra-channel Encoder

Intra-channel Encoder

Inter-channel Encoder

Intra-channel Encoder

…

Layer 1

Layer 2

Layer 3

Layer 4

Continue stacking

Signal Embedding

Text Embedding

Similarity

Match

1)

Zero-shot through

representation alignment and

matching of sentence similarity

2)

Linear Probing

A

B

C

Application

D

NORMWEAR

↑

can handle type-agnostic series, they remain limited to processing only univariate series. Because 39 of this constraint, these previous works fail to account for the heterogeneity of multivariate input 40 data; specifically, they do not capture the complex relationships between signals from sensors located 41 on different body parts. These two limitations of recent approaches hinder their generalization and 42 usefulness for wearable health monitoring. 43

Moreover, Wearable-based multimodal physiological signals present unique challenges that distin44 guish them from general time series data, such as stock prices or weather patterns. Wearable signal 45 modalities, such as PPG and EEG, vary in characteristics like dimensionality, sampling rate, and 46 resolution, often requiring modality-specific preprocessing. Existing methods tokenize raw signals 47 (Ansari et al., 2024; Zhang et al., 2022) or convert them into image or spectral representations (Wu 48 et al., 2023; Mathew et al., 2024; Vaid et al., 2023). While effective for specific tasks, these ap49 proaches lack generalizability and fail to provide a consistent preprocessing pipeline across multiple 50 modalities. A consistent framework that accommodates diverse signal requirements is essential for 51 training deep learning-based foundation models and advancing multimodal signal analysis. 52

In this work, we present NORMWEAR, a normative foundation model, aiming to learn effective 53 wearable sensing representations, addressing the above-discussed research gaps. NORMWEAR has 54 been pretrained on more than 2.5 million multivariate wearable sensing segments, comprising total of 55 14,943 hours of sensor signal series, using publicibly avaliable datasets. We evaluated NORMWEAR 56 on 18 public downstream tasks against competitive baselines across zero-shot, few-show, and full-shot 57 settings. Overall, our contributions with the proposed NORMWEAR healthcare modeling framework 58 can be summarized as follows: 59

60

61

62

To our knowledge, we are the first to develop a foundation model specifically designed for wearable sensing data, capable of processing arbitrary configuration of multivariate signals from sources such as the heart, skin, brain, and physical body.

63

64

65

66

67

68

69

70

71

72

73

74

75

76

NORMWEAR comprises novel methodologies built upon the advanced practice in both the fields of signal processing and deep learning, including (a) continuous wavelet transform (CWT) based multi-scale representations for modality- and number-agnostic tokenization, (b) channel-aware attention layer that enables the model to process arbitrary multivariate inputs, and (c) a human sensing adapted fusion mechanism that enabled NORMWEAR to achieve zero-shot inference on health related wearable sensing tasks.

We are also the first to integrate and process a comprehensive wearable signals dataset with varied number of input channels for training self-supervised learning algorithms, with thorough downstream evaluation. These datasets cover key health applications, including mental and physical state inference, vital sign estimation, and disease risk evaluation.

Our proposed NORMWEAR aims to provide a generalized data representation solution for smart health monitoring, benefiting the general public, and serving as a fundamental tool for researchers and professionals to address future healthcare challenges. We made the code and cleaned data to be publicly available to spur reproducible research.

2 Related Work 77

Foundation models have emerged as a transformative paradigm in machine learning, enabling 78 generalizable and reusable representations across diverse downstream tasks (Bommasani et al., 2022). 79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

In the time series domain, recent works (Ansari et al., 2024; Foumani et al., 2024; Abbaspourazad et al., 2023; Narayanswamy et al., 2024) have demonstrated success in tasks such as forecasting, classification, and anomaly detection. However, their generalizability to health-related wearable signals remains limited due to the lack of in-depth evaluation, reliance on specific sensor types (Wang et al., 2025; Jiang et al., 2024; Yang et al., 2023) and univariate data (Pillai et al., 2024; McKeen et al., 2024), as well as the inability to handle the heterogeneity of multivariate wearable signals. In contrast, NORMWEAR builds upon these principles by introducing a modeling framework that is agnostic to the sensor modality and number of input channels, as stated in section 1, and is presented in details in section 3. NORMWEAR has been evaluated on 18 digital healthcare tasks and demonstrate peak performance against solid time series modeling baselines, including common statistical approach, SoTA model in time series with self-supervised learning (Zhang et al., 2022), SoTA spectrum based modeling approach (Wu et al., 2023), and SoTA time series forecasting model (Ansari et al., 2024). Our work not only generalizes to arbitrary sensor configurations but also ensures compatibility across multivariate data, addressing key limitations of earlier approaches.

3 Method

3.1 Dataset construction for model pretraining and downstream evaluation

We curated a collection of 9 publicly available datasets (Table 5) exclusively for model pretraining, resulting in approximately 230,962 multivariate time series segments, comprising 4,294 hours of total sensor signal series, across various modalities, including PPG, ECG, EEG, GSR, PCG, and inertial measurement unit (IMU) data. To address the dataset size limitation, we then applied herustic data augmentation (algorithm 1) to expand the pretrain dataset to 2.5 million segments, comprising 14,943 hours of total sensor signal series. Notably, each sample segment may contain a variable number of input channels depending on the sensor signals provided by the respective datasets. This input configuration aligns seamlessly with our model's design, which is optimized to flexibly handle arbitrary numbers and configurations of sensor signal inputs.

To prevent potential data leakage in downstream tasks, we evaluate our model's transferability using an additional 11 publicly available datasets encompassing 18 modeling tasks, which include affective state classification, physical state recognition, biological estimation, and disease risk evaluation. Details about the datasets is presented in Table 4.

3.2 Tokenization

Tokenization is a fundamental term widely used in natural language processing. In the context of wearable sensing, we leverage this term to represent the stage of signal processing before sending the processed data to the deep learning-based encoder. Spectral methods, which utilize the short-time Fast Fourier Transform (FFT) (Brigham, 1988) with a sliding window to compute spectrograms, are widely regarded as the benchmark approach for tokenization. However, due to the inherent trade-off between time and frequency resolution, the spectral representation with a fixed window size cannot be generalized. This is because the window size has to be modulated accordingly when the modality varies. To enhance transferability, we propose a well-designed signal processing pipeline that preserves information in both the frequency and time domains across multiple scales. We begin by calculating the first and second derivatives for each single signal series, as suggested by Slapniˇ car et al. (2019), followed by computing the continuous wavelet transform (CWT) on both the raw and derivative series, resulting in three scalograms. Then, we stack the three scalograms to form data in RGB-image-like format. The derivatives capture the rate of signal change at different moments, while the wavelet transform provides a multi-resolution encoding that preserves information from both the time and frequency domains Torrence & Compo (1998). For the wavelet transform, we use the Mexican Hat wavelet for signal convolution, as recommended by previous studies (Burke & Nasor, 2004; Hosni & Atef, 2023; Hassani, 2021; Negi et al., 2024; Nedorubova et al., 2021b). We apply scales ranging from 1 to 64, following the guidance of (Sengupta et al., 2022; Nedorubova et al., 2021a), which sufficiently covers most frequency bands of interest for physiological signals. Finally, this RGB-like scalogram is divided into patches, which is treated in the same way as tokens in an ViT (Dosovitskiy et al., 2020). In this way, this tokenization approach can be applied to various types of sensing signals without sensor-specific adjustments or reconfigurations.

3.3 Share-weighted Encoder

Rather than concatenating tokens from all channels into a single long sequence and processing them with a full attention transformer, we treat each channel of the multivariate signal as an independent

Intra-channel Encoder

)

Fusion Block (

Inter-channel Encoder

)

Light-Weight Decoder

Input

Target

Patch Projection

Tokenization

(a) Masking Strategies

(b) Channel-Aware Fusion

Signal

Encoder

x

12

(1) Temporal +

Scale

Original

Spectrogram

(2) Scale only

(3) Temporal

only

(4) Unstructured

Yield the best

performance

(1) All-Attention

O(d

ᐧ

(L

ᐧ

C) 2

)

Concat

Fusion Block

(2) Cross-Attention

O(d

ᐧ

L

ᐧ

C 2

)

Fusion Block

Fusion Block

(3) [CLS]-Attention

O(d

ᐧ

C 2

)

Fusion Block

(4) Mean-Pooling Attention

O(d

ᐧ

(L

ᐧ

C+C 2 ))

Fusion Block

CLS Token

Patch token

ACC_X

ACC_Y

PPG

Yield the best performance

Remarks

:

(L >> C)

-

d

: embedding size

-

L

: number of patches

-

C

: number of channels

······

······

.··.··

······

::

.·····

M

input stream. Although all channels share the same transformer backbone, the forward pass is executed 135 separately for each one. This design allows the model to first learn the temporal characteristics of each 136 sensor without interference from others. It not only reduces computational cost but also increases 137 flexibility. Because each channel is processed independently, the model can be pretrained on datasets 138 with varying numbers or types of sensors and later fine-tuned on a target task with a different sensor 139 configuration. 140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

3.4 Channel-Aware Attention with Liaison Special Token

Following the tokenization step, we adopt common reconstruction-based pretraining strategies from Masked Auto Encoder (MAE) (He et al., 2021; Huang et al., 2023; Zhang et al., 2023), where input tokens are randomly masked and the model is trained to reconstruct the original time series using mean squared error (MSE) loss. Inspired by Huang et al. (2023), we experiment with four masking strategies, as shown in Figure 2 (a), including masking on (1) temporal and scale, (2) scale only, (3) temporal only, and (4) unstructured axes. We observe that the temporal and scalar masking yields the best performance for the downstream tasks. For the model architecture, we construct the backbone of our proposed framework with a convolutional patching layer followed by 12 standard Transformer blocks (Vaswani et al., 2023). For the same reason, NORMWEAR uses a lightweight decoder consisting of 2 Transformer blocks, combined with a linear projection layer and a convolution layer to reconstruct the raw physiological signals both temporally and spatially. We also prepend a [CLS] token to each signal channel, following standard practice in transformer models, for learning a global representation of the input sequence for that channel.

Another important point to consider is that although empirical studies (Nie et al., 2023; Abbaspourazad et al., 2023) show that channel-independent structures effectively capture local patterns, they fail to account for relationships across channels. To address this, we use the [CLS] token from each signal channel as a liaison token, allowing them to exchange information through the channel-aware fusion layer afrer every other encoder block. We explore several fusion approaches and different design of liaison token as shown in Figure 2 (b), with each method described below:

All-Attention Fusion: This approach involves concatenating all tokens from each modality without considering their individual properties and fusing the information through a self-attention module. However, this method requires quadratic computation time, as every token passes through the self-attention module, making it impractical for real-world applications.

(2) Cross-Attention Fusion: In addition to the cross-attention mechanism used in Cross-ViT (Chen 165 et al., 2021), we introduce a slight modification to fit in our problem setting. We propose a symmetric 166 fusion method, using the [CLS] token from each modality as an intermediary to exchange information 167 between the patch tokens of another modality, then projecting the information back to its original 168 modality in the subsequent Transformer layer. While this strategy is efficient, it restricts the model 169 to handling only two time series signals or modalities, which deviates from our goal of building a 170 general model capable of processing an arbitrary number of channels. 171

172

173

174

175

176

Mean

STDEV

Sampling

Fused

Embedding

Y

Backbone Encoder

ACC_X

PPG

Patches

Key

Value

Question of Interest, e.g.

What is the state of heartbeat abnormalities?

Text Encoder

Aggregation Module

Query

Recency Score

X

Importance Score

+

Weighted

Average

Likelihood

Parameter

Sampling

Classes, e.g.

-

Heart rate at normal state.

-

Heart rate at abnormal state.

… …

-

High risk of hypertension

Text Encoder

S T C

1

S T C

2

S T C

3

C

1

C

2

C

3

Similarity

Prediction

Inference

Backprop

Pre-trained

Freeze

＾

(3)

[CLS]-Attention Fusion

The [CLS] token serves as an abstract global representation for each signal modality. Here, we propose a hybrid fusion approach. We stack the [CLS] tokens from all

signal modalities and perform feature fusion using a self-attention mechanism. The fused [CLS] token is then reattached to its original channel, enabling the newly learned information to be propagated to

each patch token in subsequent transformer encoder layers.

(4) Mean-Pooling Fusion Similar to the [CLS]-Attention Fusion approach, we employ mean-pooling 177 within each channel instead of using the [CLS] token as an abstract global representation. 178

179

180

181

182

183

184

185

Our empirical results show that [CLS]-attention fusion achieves the best downstreaming performance for our proposed NORMWEAR model. Details of all the ablation studies are reported in Appendix C. Beyond accuracy, we want to emphasize that the [CLS]-Attention Fusion design is highly flexible. This flexibility arises from the fact that self-attention is length-flexible and permutation-invariant (Vaswani et al., 2023). Consequently, it integrates naturally with our shared-weight encoder, allowing the model to accommodate a variable number of sensor channels presented in any order. We provide additional empirical evidence of NormWear's permutation invariance in Table 12, Appendix C.

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

3.5 Sensor-Semantic Representation Alignment

Zero-shot inference is an important aspect to evaluate foundation model. We evaluate our model in this setting by retrieving the closest text-derived label for each unseen task in the shared embedding space. Specifically, to unify information across diverse modalities, we incorporate a representation alignment objective that encourages the embeddings of physiological sensor data to reside in the same latent space as paired textual descriptions. Once this shared space is established, it naturally supports zero-shot inference by allowing unseen sensor inputs to be interpreted through their proximity to text-derived anchors, without additional task-specific training. Several important works in this direction focusing on domains of vision-language Radford et al. (2021), audio-language Wu et al. (2023), and motion-language (Zhang et al., 2024b). These works leverage end-to-end training to bind their modality of interest into semantic space. In this work, we extend this methodology to explore NORMWEAR's ability to generalize across unseen datasets and tasks.

Building on prior work in representation alignment, we notice that in healthcare-related tasks where flexible inference across diverse scenarios is often required, the ground truth labels often have substantial overlap. For instance, depression is inferred from stress levels (LeMoult, 2020), and running and cycling produce similar IMU signals (Li et al., 2019). Due to these nested relationships, it create potential challenge to representation alignment when using contrastive learning, which requires clearly defined positive and negative pairs. To address this, we first propose a novel way to fuse the signal representations together with improved qualities, then align the representation with vector distance as an auxiliary loss for contrastive learning method. In addition, to reduce computation cost and counteract the issue of catastrophic forgetting (Li et al., 2023), we use off-the-shelf frozen encoders for both signal and text modalities.

Human physiological signals are task-specific, dynamic, and often weakly labeled (He et al., 2018; Kim et al., 2022; Qian et al., 2021; Ma et al., 2021). To address these characteristics, we introduce three complementary scoring mechanisms during feature aggregation: relevance scores prioritize patches aligned with the task objective (e.g., IMU for activity recognition), guided by query sentences such as ' What activity is the subject doing? '; recency scores emphasize recent segments to better reflect the current physiological or emotional state (Roelofs, 2017; Chowdhury et al., 2020; Chaudhury et al., 2021); and importance scores weigh signal segments that contain meaningful or transient

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

patterns often buried in weakly labeled sequences. Together, these scores guide the MSiTF fusion module to generate compact, task-aware representations. This design is inspired by memory-stream retrieval mechanisms (Park et al., 2023) and is tailored to the demands of human-centered sensing tasks such as risk assessment, affect detection, and activity recognition.

Memory Stream inspired Temporal Fusion (MSiTF). Our Aggregation or Fusion Module, MSiTF, is designed to addresses the above-discussed three challenges through three scores discussed below. Specifically, we denote f as the function that takes the semantic embedding of query sentence q and backbone output H ∈ R P × E as input, where P is the patch size and E is the embedding size, thus having the final fused representation f ( q, H ) = ˆ Y ∈ R E .

We define the Relevance score as the cross attention between the key representations of each sensor time step and the query sentence embedding, obtained from a pretrained language model (Clinical TinyLlama (Muzammil, 2021)). This mechanism allows the model to identify distinct but contextually relevant segments in the sensor input. For the Recency score, we use an exponential decay function to reflect the intuition that recent time steps are more important than earlier ones. Finally, we consider the importance score IMP in this case to be whether to keep the representation at each time step or not. In order to achieve this, we assign binary parameters to each time step, denoted as θ t = p ( v t ) ∈ R 2 where v t ∈ R E is the representation vector at time step t and p is a trainable linear transformation function which will be optimized during pretraining. We then have the importance score for each patch defined as

where ϵ is the noise term sampled from Gumbel distribution (Jang et al., 2017), and τ is the temperature controlling the sharpness of the softmax function. Because arg max is not a differentiable function, we will directly take the resulting probability corresponding to index at j = 1 to be the importance score, with τ being set to a small number to push the result closer to one hot vector from the softmax function. As a result, this logit function will determine to what extent to activate the gate during forward pass on each patch of the input signals. The final score for each patch is the summation of the three scores as described above. This score will be treated as the weight for aggregating the representations from all the patches to form the fixed length embedded output (vector with size of 768 in our case).

Once the signal embeddings are aggregated, we adopt a variational-inspired approach (Kingma & Welling, 2022). This design injects stochasticity into the representation, encouraging the model to explore and capture nuanced variations in semantic representations. Finally, we leverage contrastive learning with auxiliary loss on vector distance to train the MSiTF module with a projection layer to text representation on the pretraining datasets. The sentence template formation and training details are presented in Appendix B.5.

4 Experiments

NORMWEAR is pretrained exclusively on the data shown in Table 5. In this section, we present a comprehensive evaluation across 11 downstream publicly available datasets, focusing on 18 widelyrecognized digital healthcare tasks. We evaluate the methods following order of zero-shot capability, partial-shot learning, and full-shot learning.

4.1 Selection of baselines covering representative modeling strategies

Modeling multivariate wearable signals with arbitrary input channels and sensor types, such as those capturing activities of heart, brain, and body physical motions, presents unique challenges, as no universally recognized open-source baseline or state-of-the-art (SoTA) model exists in this domain. To evaluate our approach, we selected diverse and representative baselines (as shown in Table 3).

In the literature, various modeling strategies have been proposed. Firstly, early approaches involved handcrafting statistical features, which was a widely adopted practice in signal processing (Yan et al., 2023a; Reyes-Ortiz et al., 2012; Mikelsons et al., 2017). We include this simple baseline as sanity check. Secondly, since sensory data can be naturally represented as time series (Woo et al., 2024; Semenoglou et al., 2023), we benchmarked our model against Chronos (Ansari et al., 2024) , as well

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

0.831

0.859

0.731

0.802

0.807

State

EEG

Disease

Micro

Macro

0.762

0.668

0.517

0.626

0.649

State

EEG

Disease

Micro

Macro

0.753

0.792

0.815

0.793

0.787

State

EEG

Disease

Micro

Macro

NormWear (Ours)

TF-C

CLAP

Chronos

Statistical

AUC ROC

AUC PR

as a self-supervised framework TF-C (Zhang et al., 2022). Finally, the spectrum-based modeling methods (Vishnupriya & Meenakshi, 2018; Chun et al., 2016; Krishnan et al., 2020) are widely used for signal modeling. Therefore, we incorporate CLAP (Wu et al., 2023) into baselines that has demonstrates SoTA performance in spectrogram-based modeling. Regarding the comparison with concurrent works proposing foundation models for a specific sensor modality, we leverage PaPaGei (Pillai et al., 2024) for PPG datasets, ECG-FM (McKeen et al., 2024) for ECG datasets, and CBraMod (Wang et al., 2025) for EEG datasets. These baselines span distinct paradigms, providing a solid foundation to demonstrate the strengths of our model in wearable signal tasks. For uni-modal baselines like Chronos and CLAP, we feed each signal separately into model and concatenate their representations after the forward pass. This ensures that all models have the same field of view, making the comparison fair.

4.2 Zero-shot Evaluation

We achieve zero-shot inference by pretraining our proposed novel temporal fusion module on the task of representation alignment. We include the SoTA spectral-based model CLAP Wu et al. (2023) as a baseline to provide a more comprehensive comparison of the results. For CLAP, we experimented with both Manhattan distance (MD) and dot product (DP) as similarity metrics during inference. We observe that there are no statistically significant differences in performance when using MD and DP for label retrieval in CLAP. From table 1, we could observe that overall, NORMWEAR equipped with MSiTF outperforms the baselines. We compare NORMWEAR with a few ablations by removing importance score (w/o IMP) and removing text augmentation (w/o text aug). We can observe that performance drop after removing each of the above components, verifying their respective importance in improving generalization across various downstream tasks. We present this outcome to demonstrate the zero-shot capability in the wearable signal domain, an aspect not present in recent studies. We also hope this outcome could potentially provide a new perspective that can help drive progress in this direction within the field.

Model

WESAD

UCI-HAR

DriverFatigue

GAMEEMO

Epilepsy (health area)

PPG-BP (HTN)

PPG-BP (DM)

PPG-BP (CVA)

PPG-BP (CVD)

ECG-Abnormal

PhysioNet EMG

Macro Avg.

CLAP - MD

45.3

62.8

58.5

53.1

44.9

45.1

47.6

30.5

84.9

59.4

41.8

46.0

57.4

22.9

55.4

50.4

51.2

CLAP - DP

50.7

52.3

61.1

51.6

54.4

41.9

58.6

46.4

74.3

52.2

41.4

50.6

58.9

42.7

38.3

51.7

52.2

before bind

44.1

48.2

52.1

48.4

54.1

62.6

53.9

52.5

24.6

48.8

49.6

46.3

56.8

54.3

48.2

49.6

49.4

NORMWEAR

w/ MSiTF

55.8

71.2

57.2

51.0

55.7

61.3

67.6

55.8

66.0

57.1

62.5

70.0

59.0

63.1

70.1

61.6

61.5

- w/o IMP

56.2

70.3

55.4

49.8

54.0

56.5

66.9

57.3

52.9

56.5

54.3

61.7

60.7

73.4

65.2

59.4

59.6

- w/o text aug

54.8

65.8

55.2

49.2

31.0

58.4

58.6

32.8

58.1

50.2

52.6

50.8

50.6

47.7

33.6

50.0

51.4

- w/o refine

59.5

72.8

42.7

57.3

50.6

69.0

43.3

50.5

74.8

48.3

38.8

44.6

44.1

72.4

75.7

56.3

56.6

4.3 Partial-shot and Full-shot Evaluation

We evaluate the learned representations using linear probing through supervised training on each downstream dataset, and report performance on the corresponding held-out test set. To ensure

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

fair comparison, we use a unified evaluation protocol with identical hyperparameter settings and implementation across all models and the dataset (Yuan et al., 2024). This design ensures that performance differences are not due to variations in learning rate, regularization, or data augmentation (Oliver et al., 2018). Specifically, the classification tasks, using logistic regression, are solved by Newton's method with conjugate gradient, with AUC ROC being reported as main metric. The regression (vital signs) tasks, using ridge regression, are solved by Cholesky's method with closed form solution, with relative accuracy being reported. For partial-shot evaluation, we leverage 10% of the training data for the linear probing, and detailed performance result is presented in Table 11. The full-shot evaluation results is presented in Table 2. All scores are the higher the better.

From Figure 4, Table 2, and Table 15, we observe that NORMWEAR consistently achieves peak performance across all task groups, including activity recognition, EEG signal analysis, disease risk evaluation, and vital sign estimation. Furthermore, its leading performance remains consistent across various evaluation metrics. Based on the macro-averaged total score across task groups, NORMWEAR delivers a 3.9% improvement over the state-of-the-art (SoTA) time-series self-supervised learning framework (Zhang et al., 2022), a 5.3% improvement over the SoTA spectrum-based modeling method (Wu et al., 2023), a 6.1% improvement over SoTA time-series forecasting models with LLM backbones (Ansari et al., 2024), and a 5.2% improvement over standard statistical baselines. On larger datasets, NORMWEAR significantly outperforms the statistical baseline by 9.0% and 7.5% for activity recognition and EEG brain activity monitoring tasks, respectively. On smaller datasets, it still achieves peak performance in disease risk evaluation. For vital sign estimation, all methods yield comparable results, suggesting inherent challenges in these regression tasks that warrant further investigation but are beyond the scope of this study.

When comparing with recent modality specific foundation models, NormWear's main benefit is 314 that it capture cross-modal relationships, making it more versatile for wearable health tasks. While 315 it sacrifices modality-specific optimization for adaptability, this may slightly reduce performance 316 in highly specialized tasks. Single-signal models excel in their domains due to deeper modality317 focused training. Instead of maximizing single-modality data, we prioritize signal diversity for better 318 generalization. Benchmarking shows that NormWear, trained on a smaller dataset than EEG-only 319 models, still achieves competitive results, highlighting the effectiveness of our pre-training approach. 320 These findings illustrate NORMWEAR's capacity to balance consistency and adaptability across a 321 diverse range of tasks and conditions. By excelling across standard benchmarks while addressing the 322 intricacies of varied applications, NORMWEAR exemplifies the philosophy of a foundation model: a 323 reliable generalist capable of performing robustly across both typical and challenging scenarios. 324

Statistics

Chronos

CLAP

TF-C

NormWear

Approaches

55

60

65

70

75

80

Summarized Performance

+0.6%

+6.8%

+4.5%

+7.4%

+7.0%

Zero-shot

Partial-Shot

Full-Shot

(a) Adaptation summary

0

1

2

3

4

% Cumulative Performance Change

Baseline.

+Raw-input

(CA+Mask).

+CWT (Best, Final NormWear).

Switch to Cross Attn Fusion.

Switch to Mean Pool Fusion.

Switch back to [CLS] Liaison Attn. (Best repeat).

Switch to Unstructured Masking.

Switch to Time Mask.

Switch back to Structued Masking. (Best repeat).

5 Conclusion and Discussion

Conclusion. In this work, we mainly propose a foundation model for wearable physiological signals. NORMWEAR is a practical tool that could serve as a starting point for researchers and clinicians when tackling a problem with wearable based signal data. Our proposed model could extract informative representations from raw signal series, which can be leveraged for further machine learning modeling, clustering, embedding vector-based information retrieval, and deployment of real-time health states monitoring with minimal tuning. We've justified the utilizability and generalization of NORMWEAR through an extensive evaluation of various ubiquitous health applications. As for future works, it is important to leverage our framework on larger scale clinical applications and explore the applicability of embedding vectors as state representations for intervention modeling problems that comprise the decision-making process.

Limitation and Future Work. We acknowledge several limitations to be addressed in future work. (1) The representation alignment component is currently trained on a limited set of healthcare-related objectives, and expanding the pretraining corpus with more diverse semantic labels may improve generalization. (2) While our design supports classification tasks well, adapting the framework for regression remains an open challenge, and future work may explore alternative formulations beyond label discretization. (3) NormWear currently focuses on physiological signals with relatively narrow frequency bands; extending its applicability to higher-frequency modalities such as audio or lower-resolution clinical summaries is a promising direction.

Broad Impact. NORMWEAR is the first foundation model tailored for multivariate physiological signals that supports a wide range of wearable health tasks across sensor modalities, device types, and clinical applications. Through a unified CWT-based tokenization pipeline and a channel-aware fusion mechanism, it enables robust, modality-agnostic representation learning. Our extensive evaluation across zero-shot, partial-shot, and full-shot settings demonstrates NormWear's strong generalizability and practical relevance. We believe NormWear provides a valuable resource for advancing foundation modeling in digital health and promoting more unified benchmarks in the community.

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

Ethics Statement

This study contains applications in the field of healthcare. We ensured that all the data being used during pretraining and evaluations were made publicly available by the original authors, and all these works were cited properly.

Reproducibility Statement

The full code base is submitted in supplementary material referred to as NormWear_main.zip , comprising all the scripts for exploratory data analysis and preprocessing, model construction, pretraining, downstream evaluation, result analysis, and all the visualizations that are described in this paper. The GitHub repository containing all the documentation will be published simultaneously with the paper.

References

Abbaspourazad, S., Elachqar, O., Miller, A. C., Emrani, S., Nallasamy, U., and Shapiro, I. Large-scale training of foundation models for wearable biosignals. arXiv preprint arXiv:2312.05409 , 2023.

Abuzairi, T., Vinia, E., Yudhistira, M. A., Rizkinia, M., and Eriska, W. A dataset of hemoglobin blood value and photoplethysmography signal for machine learning-based non-invasive hemoglobin measurement. Data in Brief , 52:109823, 2024. ISSN 2352-3409. doi: https://doi.org/10.1016/j.dib. 2023.109823.

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023.

Alakus, T. B., Gonen, M., and Turkoglu, I. Database for an emotion recognition system based on eeg signals and various computer games-gameemo. Biomedical Signal Processing and Control , 60: 101951, 2020.

Alzahab, N. A., Di Iorio, A., Apollonio, L., Alshalak, M., Gravina, A., Antognoli, L., Baldi, M., Scalise, L., and Alchalabi, B. Auditory evoked potential eeg-biometric dataset, 2022.

Andrzejak, R. G., Lehnertz, K., Rieke, C., Mormann, F., David, P., and Elger, C. E. Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state [dataset]. Physical Review E , 2023. doi: 10.34810/data490. URL https://doi.org/10.34810/data490 .

Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815 , 2024.

Bajaj, N., Carrión, J. R., and Bellotti, F. Phyaat: Physiology of auditory attention to speech dataset. arXiv preprint arXiv:2005.11577 , 2020.

Beh, W.-K., Wu, Y.-H., and Wu, A.-Y. A. Maus: A dataset for mental workload assessment on n-back task using wearable sensor, 2021. URL https://dx.doi.org/10.21227/q4td-yd35 .

Bhaskaran, A., J, S. K., George, S., and Arora, M. Heart rate estimation and validation algorithm for fetal phonocardiography. Physiological Measurement , 43(7):075008, jul 2022. doi: 10.1088/ 1361-6579/ac7a8c. URL https://dx.doi.org/10.1088/1361-6579/ac7a8c .

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D. E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P. W., Krass, M., Krishna, R., Kuditipudi, R., Kumar, A., Ladhak, F., Lee, M., Lee, T., Leskovec, J., Levent, I., Li, X. L., Li, X., Ma, T., Malik, A., Manning, C. D.,

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

Mirchandani, S., Mitchell, E., Munyikwa, Z., Nair, S., Narayan, A., Narayanan, D., Newman, B., Nie, A., Niebles, J. C., Nilforoshan, H., Nyarko, J., Ogut, G., Orr, L., Papadimitriou, I., Park, J. S., Piech, C., Portelance, E., Potts, C., Raghunathan, A., Reich, R., Ren, H., Rong, F., Roohani, Y., Ruiz, C., Ryan, J., Ré, C., Sadigh, D., Sagawa, S., Santhanam, K., Shih, A., Srinivasan, K., Tamkin, A., Taori, R., Thomas, A. W., Tramèr, F., Wang, R. E., Wang, W., Wu, B., Wu, J., Wu, Y., Xie, S. M., Yasunaga, M., You, J., Zaharia, M., Zhang, M., Zhang, T., Zhang, X., Zhang, Y., Zheng, L., Zhou, K., and Liang, P. On the opportunities and risks of foundation models, 2022. URL https://arxiv.org/abs/2108.07258 .

Bousseljot, R., Kreiseler, D., and Schnabel, A. Nutzung der ekg-signaldatenbank cardiodat der ptb über das internet. In PTB-XL, a large publicly available electrocardiography dataset , 2009. URL https://api.semanticscholar.org/CorpusID:111121953 .

Brigham, E. O. The fast Fourier transform and its applications . Prentice-Hall, Inc., 1988.

Burke, M. and Nasor, M. Wavelet based analysis and characterization of the ecg signal. Journal of Medical Engineering & Technology , 28(2):47-55, 2004.

Carmona, C. U., Aubet, F.-X., Flunkert, V., and Gasthaus, J. Neural contextual anomaly detection for time series, 2021. URL https://arxiv.org/abs/2107.07702 .

Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. Unsupervised learning of visual features by contrasting cluster assignments, 2021. URL https://arxiv.org/abs/2006. 09882 .

Chaudhury, S., Yu, C., Liu, R., Kumar, K., Hornby, S., Duplessis, C., Sklar, J. M., Epstein, J. E., and Reifman, J. Wearables detect malaria early in a controlled human-infection study. IEEE Transactions on Biomedical Engineering , 69(6):2119-2129, 2021.

Chen, C.-F., Fan, Q., and Panda, R. Crossvit: Cross-attention multi-scale vision transformer for image classification, 2021.

Chowdhury, M. H., Shuzan, M. N. I., Chowdhury, M. E., Mahbub, Z. B., Uddin, M. M., Khandakar, A., and Reaz, M. B. I. Estimating blood pressure from the photoplethysmogram signal and demographic features using machine learning techniques. Sensors , 20(11):3127, 2020.

Chun, S. Y., Kang, J.-H., Kim, H., Lee, C., Oakley, I., and Kim, S.-P. Ecg based user authentication for wearable devices using short time fourier transform. In 2016 39th international conference on telecommunications and signal processing (tsp) , pp. 656-659. IEEE, 2016.

Dar, M. N., Rahim, A., Akram, M. U., Khawaja, S. G., and Rahim, A. Yaad: young adult's affective data using wearable ecg and gsr sensors. In 2022 2nd International Conference on Digital Futures and Transformative Technologies (ICoDT2) , pp. 1-7. IEEE, 2022.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Esmaili, A., Kachuee, M., and Shabany, M. Nonlinear cuffless blood pressure estimation of healthy subjects using pulse transit time and arrival time. IEEE Transactions on Instrumentation and Measurement , 66(12):3299-3308, 2017.

Fekri Azgomi, H., Branco, L. R. F., Amin, M. R., et al. Regulation of brain cognitive states through auditory, gustatory, and olfactory stimulation with wearable monitoring. Scientific Reports , 13:12399, 2023. doi: 10.1038/s41598-023-37829-z. URL https://doi.org/10.1038/ s41598-023-37829-z .

Foumani, N. M., Tan, C. W., Webb, G. I., and Salehi, M. Improving position encoding of transformers for multivariate time series classification. Data Mining and Knowledge Discovery , 38(1):22-48, 2024.

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

Freepik. Hypertension; blood pressure gauge; motion sensor; student sleeping in class; diabetes; blood cells; edge computing; galvanic skin response; motion sensor; accelerometer sensor; eeg, n.d. URL prefix: https://www.flaticon.com/free-icon/ , IDs: hypertension_4939229; blood-pressure-gauge_3184052; motion-sensor_2818201; student-sleeping-in-class_43739; diabetes_2750352; blood-cells_3400003; edge-computing_11068838;galvanic-skin-response_11228469; motion-sensor_17881894; accelerometer-sensor_11330476; eeg_9851782.

Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., and Stanley, H. E. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation , 101(23):e215-e220, 2000. Circulation Electronic Pages: http://circ.ahajournals.org/content/101/23/e215.full PMID:1085218; doi: 10.1161/01.CIR.101.23.e215.

Hassani, T. Federated emotion recognition with physiological signals-gsr, 2021.

He, J., Zhang, Q., Wang, L., and Pei, L. Weakly supervised human activity recognition from wearable sensors by recurrent attention learning. IEEE Sensors Journal , 19(6):2287-2297, 2018.

He, K., Chen, X., Xie, S., Li, Y ., Dollár, P., and Girshick, R. Masked autoencoders are scalable vision learners, 2021. URL https://arxiv.org/abs/2111.06377 .

Hosni, A. and Atef, M. Remote real-time heart rate monitoring with recursive motion artifact removal using ppg signals from a smartphone camera. Multimedia Tools and Applications , 82(13): 20571-20588, 2023.

Hu, K., Ivanov, P. C., Chen, Z., Carpena, P., and Stanley, H. E. Effect of trends on detrended fluctuation analysis. Physical Review E , 64(1):011114, 2001.

Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., and Feichtenhofer, C. Masked autoencoders that listen, 2023. URL https://arxiv.org/abs/2207.06405 .

Jang, E., Gu, S., and Poole, B. Categorical reparameterization with gumbel-softmax, 2017.

Jiang, W.-B., Zhao, L.-M., and Lu, B.-L. Large brain model for learning generic representations with tremendous eeg data in bci, 2024. URL https://arxiv.org/abs/2405.18765 .

Jolliffe, I. T. and Cadima, J. Principal component analysis: a review and recent developments. Philosophical transactions of the royal society A: Mathematical, Physical and Engineering Sciences , 374(2065):20150202, 2016.

Kachuee, M., Kiani, M. M., Mohammadzade, H., and Shabany, M. Cuffless blood pressure estimation algorithms for continuous health-care monitoring. IEEE Transactions on Biomedical Engineering , 64(4):859-869, 2016.

Kazemnejad, A., Gordany, P., and Sameni, R. EPHNOGRAM: A Simultaneous Electrocardiogram and Phonocardiogram Database (version 1.0.0), 2021. URL https://doi.org/10.13026/ tjtq-5911 .

Kim, D., Lee, J., Cho, M., and Kwak, S. Detector-free weakly supervised group activity recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 20083-20093, 2022.

Kingma, D. P. and Welling, M. Auto-encoding variational bayes, 2022. URL https://arxiv.org/ abs/1312.6114 .

Krishnan, P., Yaacob, S., Krishnan, A. P., Rizon, M., and Ang, C. K. Eeg based drowsiness detection using relative band power and short-time fourier transform. J. Robotics Netw. Artif. Life , 7(3): 147-151, 2020.

LeMoult, J. From stress to depression: Bringing together cognitive and biological science. Current Directions in Psychological Science , 29(6):592-598, 2020.

Li, H., Derrode, S., and Pieczynski, W. An adaptive and on-line imu-based locomotion activity classification method using a triplet markov model. Neurocomputing , 362:94-105, 2019.

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

Li, J., Li, D., Savarese, S., and Hoi, S. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pp. 19730-19742. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/li23q.html .

Liang, Y., Chen, Z., Liu, G., and Elgendi, M. A new, short-recorded photoplethysmogram dataset for blood pressure monitoring in china. Scientific data , 5(1):1-7, 2018. doi: 10.6084/m9.figshare. 5459299.v5.

Ma, H., Zhang, Z., Li, W., and Lu, S. Unsupervised human activity representation learning with multitask deep clustering. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , 5(1):1-25, 2021.

Mathew, G., Barbosa, D., Prince, J., and Venkatraman, S. Foundation models for cardiovascular disease detection via biosignals from digital stethoscopes. npj Cardiovascular Health , 1(1):25, Oct 2024. ISSN 2948-2836. doi: 10.1038/s44325-024-00027-5. URL https://doi.org/10. 1038/s44325-024-00027-5 .

McKeen, K., Oliva, L., Masood, S., Toma, A., Rubin, B., and Wang, B. Ecg-fm: An open electrocardiogram foundation model, 2024. URL https://arxiv.org/abs/2408.05178 .

Mikelsons, G., Smith, M., Mehrotra, A., and Musolesi, M. Towards deep learning models for psychological state prediction using smartphone data: Challenges and opportunities. In ML4H Workshop at 31st Conference on Neural Information Processing Systems (NIPS) , 2017. URL https://arxiv.org/abs/1711.06350 .

Min, J., Wang, P., and Hu, J. The original EEG data for driver fatigue detection. figshare.Dataset. , 7 2017. doi: 10.6084/m9.figshare.5202739.v1.

Muzammil, M. Finetuning endevsols/tinyllama-2.5t-clinical model on clinical dataset., 2021. URL https://huggingface.co/muzammil-eds/tinyllama-2.5T-Clinical-v2 .

Narayanswamy, G., Liu, X., Ayush, K., Yang, Y., Xu, X., Liao, S., Garrison, J., Tailor, S., Sunshine, J., Liu, Y ., Althoff, T., Narayanan, S., Kohli, P., Zhan, J., Malhotra, M., Patel, S., Abdel-Ghaffar, S., and McDuff, D. Scaling wearable foundation models, 2024. URL https://arxiv.org/abs/ 2410.13638 .

Nedorubova, A., Kadyrova, A., and Khlyupin, A. Human activity recognition using continuous wavelet transform and convolutional neural networks. arXiv preprint arXiv:2106.12666 , 2021a.

Nedorubova, A., Kadyrova, A., and Khlyupin, A. Human activity recognition using continuous wavelet transform and convolutional neural networks. arXiv preprint arXiv:2106.12666 , 2021b.

Negi, P. C., Giri, H., Sharma, S., Sharma, N., et al. A comparative study of scalograms for human activity classification. In 2024 IEEE 4th International Conference on Human-Machine Systems (ICHMS) , pp. 1-5. IEEE, 2024.

Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers, 2023. URL https://arxiv.org/abs/2211.14730 .

Oliver, A., Odena, A., Raffel, C. A., Cubuk, E. D., and Goodfellow, I. Realistic evaluation of deep semi-supervised learning algorithms. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_ files/paper/2018/file/c1fea270c48e8079d8ddf7d06d26ab52-Paper.pdf .

Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. Generative agents: Interactive simulacra of human behavior, 2023.

Pillai, A., Spathis, D., Kawsar, F., and Malekzadeh, M. Papagei: Open foundation models for optical physiological signals, 2024. URL https://arxiv.org/abs/2410.20542 .

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

Pimentel, M. A. F., Johnson, A. E. W., Charlton, P. H., Birrenkott, D., Watkinson, P. J., Tarassenko, L., and Clifton, D. A. Toward a robust estimation of respiratory rate from pulse oximeters. IEEE Transactions on Biomedical Engineering , 64(8):1914-1923, 2017. doi: 10.1109/TBME.2016. 2613124.

Qian, B. and Rasheed, K. Hurst exponent and financial market predictability. In IASTED conference on Financial Engineering and Applications , pp. 203-209. Proceedings of the IASTED International Conference Cambridge, MA, 2004.

Qian, H., Pan, S. J., and Miao, C. Weakly-supervised sensor-based activity segmentation and recognition via learning from distributions. Artificial Intelligence , 292:103429, 2021.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision, 2021. URL https://arxiv.org/abs/2103.00020 .

Reiss Attila, Indlekofer Ina, S. P. PPG-DaLiA. UCI Machine Learning Repository, 2019. DOI: https://doi.org/10.24432/C53890.

Reyes-Ortiz, J., Anguita, D., Ghio, A., Oneto, L., and Parra, X. Human Activity Recognition Using Smartphones. UCI Machine Learning Repository, 2012. DOI: https://doi.org/10.24432/C54S4K.

Roelofs, K. Freeze for action: neurobiological mechanisms in animal and human freezing. Philosophical Transactions of the Royal Society B: Biological Sciences , 372(1718):20160206, 2017.

Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., and Van Laerhoven, K. Introducing wesad, a multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM international conference on multimodal interaction , pp. 400-408, 2018.

Semenoglou, A.-A., Spiliotis, E., and Assimakopoulos, V. Image-based time series forecasting: A deep convolutional neural network approach. Neural Networks , 157:39-53, 2023. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2022.10.006. URL https://www.sciencedirect.com/ science/article/pii/S0893608022003902 .

Sengupta, R., Polian, I., and Hayes, J. P. Wavelet transform assisted neural networks for human activity recognition. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS) , pp. 1254-1258. IEEE, 2022.

Slapniˇ car, G., Mlakar, N., and Luštrek, M. Blood pressure estimation from photoplethysmogram using a spectro-temporal deep neural network. Sensors , 19(15):3420, 2019.

Thompson, J. M. T., Stewart, H. B., and Turner, R. Nonlinear dynamics and chaos. Computers in Physics , 4(5):562-563, 1990.

Torrence, C. and Compo, G. P. A practical guide to wavelet analysis. Bulletin of the American Meteorological society , 79(1):61-78, 1998.

Vaid, A., Jiang, J., Sawant, A., Lerakis, S., Argulian, E., Ahuja, Y., Lampert, J., Charney, A., Greenspan, H., Narula, J., Glicksberg, B., and Nadkarni, G. N. A foundational vision transformer improves diagnostic performance for electrocardiograms. npj Digital Medicine , 6(1):108, Jun 2023. ISSN 2398-6352. doi: 10.1038/s41746-023-00840-9. URL https://doi.org/10.1038/ s41746-023-00840-9 .

Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research , 9(11), 2008.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need, 2023. URL https://arxiv.org/abs/1706.03762 .

Vishnupriya, S. and Meenakshi, K. Automatic music genre classification using convolution neural network. In 2018 International Conference on Computer Communication and Informatics (ICCCI) , pp. 1-4, 2018. doi: 10.1109/ICCCI.2018.8441340. URL https://ieeexplore.ieee.org/ document/8441340 .

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

Wang, J., Zhao, S., Luo, Z., Zhou, Y., Jiang, H., Li, S., Li, T., and Pan, G. Cbramod: A criss-cross brain foundation model for eeg decoding, 2025. URL https://arxiv.org/abs/2412.07236 .

Wimmer, C. and Rekabsaz, N. Leveraging vision-language models for granular market change prediction, 2023. URL https://arxiv.org/abs/2301.10166 .

Wolf, A., Swift, J. B., Swinney, H. L., and Vastano, J. A. Determining lyapunov exponents from a time series. Physica D: nonlinear phenomena , 16(3):285-317, 1985.

Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. Unified training of universal time series forecasting transformers, 2024. URL https://arxiv.org/abs/2402.02592 .

Wu, Y., Chen, K., Zhang, T., Hui, Y., Berg-Kirkpatrick, T., and Dubnov, S. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 1-5. IEEE, 2023.

Yan, Y., Huang, Y.-C., Zhao, J., Liu, Y.-S., Ma, L., Yang, J., Yan, X.-D., Xiong, J., and Wang, L. Topological nonlinear analysis of dynamical systems in wearable sensor-based human physical activity inference. IEEE Transactions on Human-Machine Systems , 53(4):792-801, 2023a. doi: 10.1109/THMS.2023.3275774.

Yan, Y., Huang, Y.-C., Zhao, J., Liu, Y.-S., Ma, L., Yang, J., Yan, X.-D., Xiong, J., and Wang, L. Topological nonlinear analysis of dynamical systems in wearable sensor-based human physical activity inference. IEEE Transactions on Human-Machine Systems , 53(4):792-801, 2023b. doi: 10.1109/THMS.2023.3275774.

Yang, C., Westover, M., and Sun, J. Biot: Biosignal transformer for cross-data learning in the wild. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems , volume 36, pp. 78240-78260. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/ file/f6b30f3e2dd9cb53bbf2024402d02295-Paper-Conference.pdf .

Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., and Doherty, A. Selfsupervised learning for human activity recognition using 700,000 person-days of wearable data. npj Digital Medicine , 7(1), April 2024. ISSN 2398-6352. doi: 10.1038/s41746-024-01062-3. URL http://dx.doi.org/10.1038/s41746-024-01062-3 .

Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. mixup: Beyond empirical risk minimization, 2017.

Zhang, W., Yang, L., Geng, S., and Hong, S. Self-supervised time series representation learning via cross reconstruction transformer. IEEE Transactions on Neural Networks and Learning Systems , 2023.

Zhang, X., Zhao, Z., Tsiligkaridis, T., and Zitnik, M. Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems , 35: 3988-4003, 2022.

Zhang, X., Chowdhury, R. R., Gupta, R. K., and Shang, J. Large language models for time series: A survey. arXiv preprint arXiv:2402.01801 , 2024a.

Zhang, X., Teng, D., Chowdhury, R. R., Li, S., Hong, D., Gupta, R. K., and Shang, J. Unimts: Unified pre-training for motion time series, 2024b. URL https://arxiv.org/abs/2410.19818 .

A Datasets 626

Few openly accessible multi-channel or multi-device datasets for physiological signals exist, limiting 627 advancements in this field. To address this gap, we curated a dataset comprising approximately 628 385 hours of recordings. Using the augmentation algorithm described below, we expanded this 629 dataset to 4294 hours. The distribution of the pretraining dataset, as shown in Figure 6, reflects 630 the inherent diversity of the original recordings, ensuring balanced representation across channels 631 and devices. This curated and augmented dataset provides a critical resource for developing robust 632 models, facilitating progress in multi-channel physiological signal research.

GSR

SkinConductance

9.7%

9.7%

EEG

IMU

Brain

Physical

35.7%

15.9%

35.7%

15.9%

4.6%

PCG

15.8%

18.3%

38.7%

PPG

ECG

Heart

633

Table 4 overviews used dataset in our experiement along with the modality and task type. We will 634 gives further details for each dataset below: 635

WESAD (Schmidt et al., 2018) is a publicly available multimodal dataset used for wearable stress 636 and affect detection, formulated as a classification task with labels: neutral, stress, and amusement. 637

The dataset includes physiological and motion data collected from 15 subjects during a lab study, 638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

using a chest-worn RespiBAN device and a wrist-worn Empatica E4 device. From the chest device, we use electrocardiogram (ECG), galvanic skin response (GSR), and triaxial acceleration (ACC-X, ACC-Y, ACC-Z), all sampled at 700 Hz. From the wrist device, we use photoplethysmogram (PPG), galvanic skin response (GSR, 4 Hz), and triaxial acceleration (ACC-X, ACC-Y, ACC-Z, 32 Hz). The selected channels span multiple physiological and motion modalities from both chest and wrist sensors. Each data segment is labeled with one of the three affective states, serving as the target output for classification tasks.

UCI-HAR (Reyes-Ortiz et al., 2012) dataset is publicly available and is used for classifying human activities based on sensor data. It comprises data from 30 volunteers, aged 19 to 48, each performing six activities: walking, walking upstairs, walking downstairs, sitting, standing, and laying. During these activities, participants carried a waist-mounted smartphone equipped with embedded accelerometer and gyroscope sensors. The input channels consist of triaxial linear acceleration and triaxial angular velocity, totaling six channels. Each data segment is labeled with one of the six activities, serving as the target output for classification tasks. The sensors recorded data at a constant rate of 50 Hz.

Driver Fatigue EEG Dataset (Min et al., 2017) is a publicly available dataset used for detecting driver fatigue based on electroencephalogram (EEG) signals. EEG data were collected using a 40-channel Neuroscan amplifier. The recordings include EEG data corresponding to two states: alert and fatigued. Each data segment is labeled with one of these states, serving as the target output for classification tasks.

Epileptic Seizure Recognition (Andrzejak et al., 2023) dataset is publicly available and is used for classifying neurological and physiological states based on EEG signals. It comprises data from 500 subjects, each recorded for 23.6 seconds using a single EEG channel at a sampling rate of 178 Hz. Each sample is labeled with one of five brain states, allowing for the construction of multiple binary classification tasks that target different aspects of neurological assessment. Specifically, we formulated five tasks:

Eye Relaxation : Detects eye fatigue by distinguishing between relaxed and alert states based on EEG changes related to eye closure.

Health Area : Classifies brain regions as healthy or affected by neurological abnormalities.

Tumor Area : Detects EEG patterns indicative of tumor presence in specific brain regions.

Seizure : Identifies seizure activity from non-seizure states.

Eyes Open vs. Closed : Differentiates EEG signals associated with visual input states.

GAMEEMO (Alakus et al., 2020) is a publicly available dataset used for emotion recognition based on EEG signals. It comprises data from 28 subjects, each playing four emotion-inducing computer games (boring, calm, horror, and funny) for five minutes per game, totaling 20 minutes of EEG data per subject. EEG signals were recorded using the EMOTIV EPOC+ headset, which includes 14 channels (AF3, AF4, F3, F4, F7, F8, FC5, FC6, O1, O2, P7, P8, T7, and T8) positioned according to the 10-20 system. The signals were sampled at 128 Hz. After each gameplay session, subjects rated their emotional response using the Self-Assessment Manikin (SAM) form, providing continuous scores for arousal and valence. These scores were quantized into binary values using subject-specific median thresholds: arousal and valence ratings above the median were labeled as high, and those below or equal to the median as low. Combining the binarized arousal and valence ratings yields four discrete emotional classes: low arousal and low valence, low arousal and high valence, high arousal and low valence, and high arousal and high valence. Each data segment is labeled with one of these four classes, serving as the target output for four-class emotion classification tasks.

ECG Heartbeat Categorization (Bousseljot et al., 2009) is a publicly available dataset used for classifying heartbeat signals based on electrocardiogram (ECG) recordings. It comprises two collections of heartbeat signals derived from PhysioNet's MIT-BIH Arrhythmia Dataset and the PTB Diagnostic ECG Database. The first collection includes 109,446 samples categorized into five classes: normal (N), supraventricular ectopic (S), ventricular ectopic (V), fusion (F), and unknown (Q), with ECG signals sampled at 125 Hz. The second collection consists of 14,552 samples categorized into two classes: normal and abnormal, also sampled at 125 Hz. For our analysis, we restructured the dataset into a binary classification framework by consolidating the original categories into two classes: normal and abnormal heartbeats.

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

PPG-China (Liang et al., 2018) is a publicly available dataset used for classifying cardiovascular and metabolic conditions based on photoplethysmography (PPG) signals. It comprises 657 data records from 219 subjects, aged 20 to 89 years, including individuals with conditions such as hypertension and diabetes. PPG signals were recorded using a single channel at a sampling rate of 125 Hz. Each subject's data includes PPG waveforms and corresponding clinical information, facilitating the construction of multiple classification tasks focused on cardiovascular and systemic health monitoring. Specifically, we formulated four tasks:

PPG-HTN : Identifies stages of hypotension severity by classifying PPG signals into four levels.

PPG-DM : Detects diabetes by distinguishing between diabetic and non-diabetic individuals.

PPG-CVA : Identifies the presence or absence of cerebrovascular accidents (strokes) based on PPG patterns.

PPG-CVD : Assesses cardiovascular disease by classifying PPG signals into three cardiovascular health categories.

PhysioNetEMG (Goldberger et al., 2000) is a publicly available dataset used for classifying neuromuscular conditions based on electromyography (EMG) signals. It comprises single-channel EMG recordings from the tibialis anterior muscle of three subjects: one healthy, one with neuropathy, and one with myopathy. The EMG signals were recorded at a sampling rate of 4,000 Hz. Each recording was segmented into time series samples using a fixed-length window of 6 second. Each segment is labeled according to the subject's condition-healthy, neuropathy, or myopathy-serving as the target output for classification tasks.

Non-invasive Blood Pressure Estimation (Esmaili et al., 2017) is a publicly available dataset used for cuff-less blood pressure (BP) estimation. It comprises data from 26 subjects, each with recorded electrocardiogram (ECG) and photoplethysmogram (PPG) signals, sampled at 1,000 Hz. Reference BP measurements were taken during signal acquisition. Each subject's data also includes demographic information such as age, weight, and height. The dataset is structured to facilitate regression tasks aimed at predicting systolic and diastolic BP values.

PPG-HGB (Abuzairi et al., 2024) is a publicly available dataset used for non-invasive hemoglobin (Hb) measurement based on photoplethysmography (PPG) signals. It comprises data from 68 subjects, aged 18 to 65 years, with a gender distribution of 56% female and 44% male. PPG signals were recorded using the MAX30102 sensor, which emits red and infrared light. The sensor's analog-todigital converter (ADC) output data rate can be programmed from 50 samples per second (sps) to 3200 sps. Each subject contributed 12 sets of PPG signals, totaling 816 data records. We formulate regression tasks aimed at predicting Hb concertration levels.

Fetal-fPCG (Bhaskaran et al., 2022) is a publicly available dataset designed for estimating fetal heart rate (FHR) using fetal phonocardiography (fPCG) signals. It includes recordings from 60 pregnant women, aged 18 to 37 years, with gestational ages between 31 and 40 weeks. The recordings were collected at St. John's Hospital in Bangalore using an electronic stethoscope (SS30LA) connected to a Biopac MP36 data acquisition system. The stethoscope was placed on the lower abdomen of each subject to capture the fPCG signal, which was sampled at 2,000 Hz. The dataset supports regression tasks, where the goal is to predict continuous FHR values directly from the fPCG waveforms.

B Implementation Detail

B.1 Data Preprocess.

For the data preparation, we set the uniform sampling rate and interval length to 65 HZ and 6 seconds respectively. In our case, 65 Hz covers most of the frequency bands of interest such as heart activity, physical motions, and neuron activity up to the beginning of Gamma power (above 30 Hz). And a great amount of samples are less than 6 seconds such as (Reyes-Ortiz et al., 2012; Liang et al., 2018; Bousseljot et al., 2009). We conduct basic pre-processing for each signal with identical setting: (1) de-trended by subtract the result of a linear least-squares fit to series data from the raw time series, and (2) Gaussian smoothed with standard deviation of 1.3 (0.02 seconds), ensuring a highly consistent dataset for training.

Since the Transformer's computational requirements scale quadratically with input length, to release the full potential of our self-supervised algorithm, we segment our multivariate time series into

intervals with a uniform length and pad shorter samples with zeros. This approach not only enables 746 parallel processing of samples in large minibatches but also addresses variation in the length of 747 individual samples. 748

749

750

751

For the downstream task, we split the data into train and test sets for linear probing evaluation with portion of 80% and 20% correspondingly. The split is stratified on the anonymized subject ID if this information is provided by the dataset.

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

B.2 Data Augmentation.

Since there are very few publicly available datasets containing multiple devices or modalities, we aim to expand our curated training set to fully leverage the potential of self-supervised learning. Inspired by data augmentation techniques in computer vision and natural language processing (Zhang et al., 2017; Carmona et al., 2021), we adopt a heuristic approach to augment the dataset. Specifically, we augment each sub-dataset by a factor of 10. For each dataset, we sample two time series, randomly extract a segment from one, and substitute it with a transformed counterpart, as outlined in the pseudocode in Algorithm 1. As a result, our training set is expanded to 2,586,404 segments, corresponding to 4,294 hours of data.

Algorithm 1 Time Series Mixup Augmentation

Input:

Time series dataset X , number of augmentations n

Output:

Augmented Dataset

1: for i = 1 to n do

2: Sample two time series x (1) , x (2) ∼ X

3: Sample a chunk size λ ∼ U (0 , l )

4: Sample start indices s 1 , s 2 ∼ U (0 , l - λ )

5: Swap chunk from x (2) into x (1) :

6:

Append

x

(1)

into

˜

X

7:

end for

8:

return

˜

X

B.3 Pretraining Framework.

Normwear is derived from the Masked Autoencoder (MAE) (He et al., 2021). The detailed hyperparameter choice is descibe in 6. We use a Conv2D layer with a kernel size of (9, 5) and a stride of (9, 5), ensuring no overlapping patches. This layer takes input with 3 channels and projects it to 768 channels, matching the hidden size of our encoders. In Normwear, we apply structured masking independently to each variate along both the frequency and time axes, with respective masking ratios of 0.6 and 0.5. This results in an expected overall masking ratio of 0.8 for each variate. Only the unmasked tokens are passed to the encoder, reducing computational complexity. To enhance representation learning, we introduce six additional transformer blocks as fusion layers, interleaved with the original 12 encoder blocks, creating a total of 18 blocks. Each transformer block has a hidden dimension of 768 and uses LayerNorm as in the original MAE. The latent embeddings obtained from the encoder are projected from 768 to 512 dimensions. Learnable masked tokens are reinserted at their original positions, and positional embeddings are added to guide the decoder in reconstructing the input series. The lightweight decoder consists of two transformer blocks with a hidden dimension of 512, followed by two Conv1D layers. The first Conv1D layer maps from the flattened multivariate signal embedding to an intermediate dimension, and the second Conv1D layer maps from this intermediate dimension back to the original multivariate signal space. A GELU activation function is used between these layers, with BatchNorm applied to the input. The decoder reconstructs the original input series, and the model is trained using Mean Squared Error (MSE) loss on all data points. Our models are pre-trained for 45,000 steps with a batch size of 256, using the AdamW optimizer with a learning rate of 10 -4 . We did not perform on-the-fly data augmentation,

788

Ground Truth

CWTTransformation

Prediction

Masked

CWTTransformation

Ground Truth

Prediction

Masked

Ground Truth

CWTTransformation

Prediction

Masked

50

as suggested in the MAE framework, due to the high masking ratio. (An end-to-end example of the 782 input and output of this pretraining pipeline is illustrated in Fig. 7) 783

All the models are pretrained on 4 NVIDIA RTX 3090 graphical computing unit (GPU), with 24GB 784 of GPU memory on each card. 785

B.4 MSiTF. 786

For pretraining the representation alignment module, we have the training hyper-parameters in Table 787 7.

B.5 Aligner Module, Objective Function, and Pretraining. 789

The Aligner Module matches two vectors: the fused representation f ( q, H ) = ˆ Y ∈ R E with the 790 semantic embedding ( Y ) of ground truth sentence, which is obtained from prompting the ground 791 truth label using a template, for example, 'The subject is presently {activity_label}" . In the same 792 manner as the query embedding, the ground truth sentence is encoded using the same pre-trained 793 language model (Muzammil, 2021). At this stage, Y is leveraged to supervise the fused output ˆ Y . 794 The vanilla contrastive learning loss formula following Zhang et al. (2024b) is: 795

where N is the batch size and γ is the learnable temperature parameter. We denote this loss function 796 as contrastive loss with batch normalizer. We also leverage a refine process after contrastive learning 797 using simlarity loss with per sample normalizer, which is essentially cosine similarity loss, with 798 vector distance as supplemental penalty: 799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

where λ is hyper-parameters controlling the weight of the supplemental loss components.

B.6 Sentence template example for signal-sext alignment.

To enhance the expressiveness and diversity of supervision signals for our MSiTF alignment module, we convert categorical labels into natural language descriptions using varied prompt templates. We apply this strategy to several pretraining tasks. We present example sentence templates below for emotion recognition and activity recognition to demonstrate the general idea of how we derive text modality from the raw label:

For the emotion recognition task, we use:

For the activity recognition task, we use:

By exposing the model to multiple phrasings for the same label, this design helps it learn modalityinvariant representations that are more robust to linguistic variation and better aligned across modalities. Specifically, to increase the diversity of semantic representations of query and ground truth sentences in the pretraining signal corpus, we utilize large language models (GPT-3.5) (Achiam et al., 2023) to generate 20 alternative variations for each sentence, from which only one is randomly sampled during pre-training. During test-time inference on downstream datasets, each ground truth label is converted into a sentence (details in appendix B.6), which is transformed into a semantic embedding using a frozen text encoder. The sentence with the closest distance with the embedding from our foundation model is used as the final inferential result.

B.7 Statistical Feature List.

Our statistical baseline includes features extracted from both the time and frequency domains. In the time domain, we compute the mean, standard deviation, maximum, minimum, skewness, kurtosis,

25% quantile, median, and 75% quantile. In the frequency domain, we extract the spectral centroid, 823 spectral spread, mean frequency, peak frequency, as well as the 25%, 50% (median), and 75% quantile 824 frequencies. 825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

B.8 Radar Plot or Performance Trend.

To enhance the visual contrast between model performances across tasks, we applied the Softmax function to the raw performance scores. This transformation rescales the scores to a range between 0 and 1, accentuating relative differences between models. While the Softmax transformation emphasizes the relative improvement of our model over others, we note that the absolute scores may differ from those originally reported.

C Ablation Study

Due to computational constraints, we will conduct the ablation study on our smaller dataset (37k samples) to train and evaluate the model, establishing a proof of concept and demonstrating the effectiveness of our approach in a controlled setting.

Fusion Schemes. Table 8 shows the performance of different fusion schemes, including (1) no fusion, (2) cross-attention fusion, (3) [CLS]-attention fusion, and (4) mean-pooling fusion. We excluded all-attention fusion in our ablation study because it is computationally prohibitible. Among all the compared strategies, the [CLS] token fusion generally achieves the best accuracy with a minor increase in parameters.

Masking Strategies in Pre-training. We ablated our masking strategy introduced in Section 3.4. 841 Using a consistent mask ratio of 0.8 in all strategies, we found that applying masking along the scale 842 and time axes produced the best performance (details in Table 9).

843

844

845

846

847

848

849

850

851

852

853

854

855

856

857

858

859

Input Representations. Table10 compares the performance of two input representations: (1) CWT scalogram and (2) raw time series. The CWT scalogram converts the time series into a timefrequency representation, while the raw time series retains the original sensor data. Among the two representations, the model trained on CWT scalograms demonstrates better performance, suggesting that the time-frequency features enhance model accuracy.

Semi-Supervised Learning (Partial-shot). To evaluate the generalizability and quality of learned representations, we conducted a semi-supervised learning evaluation following the protocol established by prior self-supervised methods (Caron et al., 2021). Specifically, we assessed performance on the NORMWEAR dataset using frozen features and a limited labeled subset (10%). We deliberately excluded the commonly used 1% label evaluation due to the inherently small sample size of our downstream medical dataset. A 1% labeling scenario would provide fewer than ten labeled instances, rendering the results statistically unreliable and scientifically unjustified. Instead, we sampled 10% of the training data while preserving the original label distribution, and then trained a linear classifier atop the frozen NORMWEAR features for classification tasks and regression tasks. The results, summarized in Table 11, demonstrate the effectiveness of our method under realistic semi-supervised constraints.

860

861

862

863

864

865

866

867

868

869

870

871

872

873

Permutation-Invariant Input Channel Analysis. In many multimodal or multichannel sensing tasks, the input channel order is typically fixed and determined by hardware or preprocessing pipelines, limiting flexibility during deployment. This constraint raises the question of whether Normwear relies on a specific channel ordering to perform well. To examine this, we conducted an experiment on datasets with multiple input channels by circularly shifting the channel order by one position and evaluating the resulting model performance. As shown in Table 12, the model performance remains consistent across different permutations. These results suggest that our model does not rely on a fixed input channel configuration and is robust to variations in channel ordering, making it more applicable in practical scenarios where such inconsistencies may occur.

k -fold Analysis. To evaluate whether Normwear maintains consistent performance on datasets with limited subject diversity, we conducted 5-fold cross-validation stratified by subject ID. We applied this protocol to all downstream tasks containing 30 or fewer subjects to ensure a robust assessment. As shown in Table 13, our model consistently outperformed the baselines across all tasks, demonstrating the robustness of our evaluation metric.

Downstream Tasks

Statistical

Chronos

CLAP

TF-C

NormWear-L

(Ours)

WESAD

79.992

±

0.707

83.332

±

0.841

87.824

±

0.463

82.701

±

0.536

89.585

±

0.683

UCI-HAR

95.602

±

0.148

91.956

±

0.256

96.864

±

0.175

97.382

±

0.138

98.179

±

0.06

DriverFatigue

69.614

±

1.138

72.48

±

2.848

66.251

±

0.471

65.026

±

1.198

68.971

±

1.32

GAMEEMO

64.281

±

1.292

56.694

±

0.878

64.119

±

0.543

62.925

±

0.999

67.863

±

0.72

Noninvasive-BP

92.83

±

0.386

92.223

±

0.356

92.612

±

0.272

88.707

±

0.622

93.381

±

0.516

80.464

±

0.734

79.337

±

1.036

81.534

±

0.385

79.348

±

0.699

83.596

±

0.660

874

875

876

877

878

879

880

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

Demographic Anlysis. Several previous works (Abbaspourazad et al., 2023; Narayanswamy et al., 2024) have used learned representations to infer demographic labels. These results suggest that wearable signals do contain demographic information. In Table 14, we wanted to investigate that NormWear does not extract only demographic information (e.g. age, sex, height, etc. depending on what is available within each dataset), hence indicating that the representation that our proposed model extracted and the demographic could be used as complementary features to each other during downstream modeling. From Table 14, we observe that demographic information and wearable signal representations each excel at different tasks. In most cases, concatenating them improves overall performance. However, the occasional performance drop after concatenation suggests a confounding relationship between the two, implying that demographic data and NormWear's wearable representations capture different aspects.

D Statistical significance on the model comparison

We performed a statistical analysis to test the significance of the differences in model performance. First, we ran the downstream evaluations 100 times for each model on every task without fixing the random seed. The results remained consistent due to the stable optimization process.

Next, we applied a permutation test on the results from these 100 runs to determine whether NormWear's AUC ROC score is greater than that of the baselines. The reported p-value represents the probability of observing a test statistic as extreme or more extreme than the observed difference under the null hypothesis, which assumes that NormWear's score is not higher than the baseline. In nearly all cases, the p-value is less than 0.01, confirming the statistical significance and indicating the robustness and superiority of our approach. Table 8 presents the statistical test results across different task groups (as indicated by the color coding in the main tables) along with the overall average scores.

We also include a critical difference (CD) diagram to visually compare the performance of multiple models across datasets and highlight statistically significant differences. To generate the CD diagram, we first conducted a Friedman Chi-square test on the models' scores across all downstream tasks, which yielded a p-value of P < 0 . 001 , confirming that the models' performances come from different distributions. We then applied the Conover post hoc test to examine pairwise differences between model performances; the p-values for NormWear compared with the baselines are shown in the last row of Table 8. Finally, based on these results, we generated the CD diagram displayed in Figure 9. In this diagram, our proposed model, NormWear, is well separated from the others, indicating its statistical superiority over the competitive baselines.

908

Ours/Baselines

Stats

Chronos

CLAP

TFC

NormWear

- activity

P < .

01

P < .

01

P < .

01

P < .

01

NormWear

- eeg

P < .

01

P < .

01

P < .

01

P < .

01

NormWear

- risk

P < .

01

P < .

01

P < .

01

P < .

01

NormWear

- vital

P < .

01

P < .

01

P < .

01

P < .

01

NormWear

- micro avg.

P < .

01

P < .

01

P < .

01

P < .

01

NormWear

- macro avg.

P < .

01

P < .

01

P < .

01

P < .

01

Conover post hoc

P < .

001

P < .

001

P < .

001

P < .

05

E Supplementary Metrics 906

Normwear 's performance is summarized in Fig. 4 and detailed in Table 15. Normwear consistently 907 exceeds the baseline models by a wide margin, demonstrating a clear advantage.

0.50

0.55

0.60

0.65

0.70

0.75

0.80

0.85

stats (0.48)

chronos (0.5)

(0.83) normwear

(0.65) tfc

(0.53) clap

Critical difference diagram of average score ranks

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

F Scaling up the Pretraining Data Size

In addition to demonstrating that NormWear outperforms all strong baselines, we further investigate the effect of varying pretraining data size on the model's downstream performance to examine whether the scaling law applies to our proposed methodology. As shown in Figure 10, the overall performance (measured by accuracy) significantly improves as the pretraining data size increases from approximately 37k (62 hours) to nearly 2.5M (4000 hours) samples of wearable signal data. This observation indicates that our model adheres to the scaling law, highlighting its potential scalability and suitability for future large-scale applications.

G Channel Fusion Complexity analysis

When conducting multi-channel modeling, for example, when the input comprises an arbitrary number of signals, a fusion operation needs to be conducted across all channels in order to let the model extract correlation information. Because we will deploy the model on an edge device like Jetson Nano, other than empirical evidence of the performance, we also have to consider the computation complexity of different approaches. A brief visualization of the runtime complexity of different approaches is presented in figure 11. The detailed derivation is presented in the following subsections.

1e6

10

5

10

6

Dataset Size (Number of Samples)

73

74

75

76

77

78

79

Accuracy (%)

State Recognition

EEG Task

Diease Risk

0

2

4

6

8

10

Number of input channels.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Relative Big-O runtime complexity.

All-Attention

Pairwise-Cross-Attention

[CLS]-Attention

Mean-Pool-Attention

(a) Runtime Big-O when increasing the number of channels

0

20

40

60

80

100

Length of input signal series (in milliseconds)

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Relative Big-O runtime complexity.

All-Attention

Pairwise-Cross-Attention

[CLS]-Attention

Mean-Pool-Attention

(b) Runtime Big-O when increasing

the sequence length

G.1 All-Attention

For the approach of conducting self-attention by concatenating all the patches, we arrive the Big-O complexity expression as follows:

We denote C as the number of input channels, d as the embedding size, L as the number of patches convolved from the time series in each channel (proportional to sequence length), and x ∈ R C × L × d as the input data before feeding into the fusion block. We have a total of L · C patches.

When calculating the attention scores, dot products are computed for each pair of the patches, which results in the following calculation process: where "1), 2), 3)" represents the operations conducted at the first, second, and third rounds of entering the entire nested loops. The complexity for the first round of operation results in

Algorithm 2 All-Attention Complexity

a complexity of: 943

944

945

946

947

948

949

where in the case of multi-head attention, the dot product still has the complexity of O (2 d ) , and because the number of heads is a constant, the final complexity is equivalent to the result in equation 4.

Similarly, the softmax operation will result in a complexity of O (( L · C ) 2 ) , and the final weighted average operation will also have a complexity of O ( d · ( L · C ) 2 ) , which results in total complexity of

G.2 Cross-Attention 950

For the pairwise cross-attention approach following guidance of Chen et al. (2021), we have the 951 operation defined as 952

Algorithm 3 Cross-Attention Complexity

with the same notion in the previous subsection. The total complexity is 953

G.3 [CLS]-Attention 954

This is the approach that we adopted for the final version of our proposed foundation model. Only 955 the embedding corresponding to the [ CLS ] token of each channel is involved during the self-attention 956 operation. Therefore, the complexity is 957

G.4 Mean-pool Attention 958

959

960

961

For fusion with mean-pool attention, we first calculate the mean representation for each channel, resulting in a complexity of O ( C · L · d ) . And self-attention with Tese mean representations has the same complexity as [CLS]-attention, which is O ( d · C 2 ) . Thus, the total complexity is

H MSiTF Complexity analysis 962

Algorithm 4 MSiTF Runtime Complexity

Where d being the latent size, p being the number of total patches, c being the number of available ground truth choice, k and v being the key and value linear mapping, S ∈ R p × d as the signal embeddings, Q as the query sentence embedding, and C as the list of available answer choice sentences. The total runtime complexity is O ( d 2 + pd + p + cd ) . Since d is constant, we have runtime complexity of O ( p + c ) .

Regarding memory complexity of MSiTF, with m being the size of text encoder, w being the size of normwear, we have (i) Signal representations: O ( pd ) ; (ii) Text representations: O ( cd ) ; (iii) Total: O ( m + w + d ( p + c )) . Since m , w , and d are all constants, we have memory complexity of O ( p + c ) .

I Feature Visualization

Feature visualization serves as a tool to interpret and analyze the internal representations learned by the model. By examining activation patterns or embedding structures at various layers, we aim to understand how the model encodes input signals and whether these representations align with relevant semantic or structural information. This analysis provides insight into the effectiveness of the learned features and can inform architectural or training modifications to improve performance and generalization.

I.1 The model is agnostic to the input signals

This section investigates whether, without requiring the signal modality type information as input, NORMWEAR can effectively distinguish between different signal sources. We randomly sampled 500 samples for each sensor type and fed them into our pretrained model. We use t-SNE (Van der Maaten & Hinton, 2008), with PCA (Jolliffe & Cadima, 2016) initialization to visualize the learned representations corresponding to the [CLS] special token at the last layer. The PCA preserves the global structure, while t-SNE emphasizes local relationships in the data. From Figure 13(a), we observe that representations from sensors located at the same body position are clustered closely together, while representations from different body locations are clearly separated. This suggests that our model is signal-agnostic, as it can recognize the signal type differences, map their representations appropriately in the embedding space, and guide feature extraction within each Transformer block.

I.2 Waveform visualization

Figure 13 (b) under 'Feature Associations" shows the features extracted by our model. Each patch corresponds to a representation with a vector size of R 768 . When ordered by time sequence, these representations form 768 waveforms per layer, representing the model's extracted features. The figure

963

964

965

966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

991

992

displays 64 randomly sampled waveforms from a selected layer. The features highlighted in purple 993 and gray indicate the top 10 patterns positively and negatively associated with the target task (diabetes 994 classification, in this example), with associations determined by linear regression parameters during 995 linear probing. Additionally, our relevance-based fusion mechanism identifies the contribution of 996 each time step during inference, highlighted by red dots in the 'Time Step Relevance" section of 997 Figure 13 (b). 998

999

1000

1001

1002

1003

1004

Such a visualization pipeline can assist researchers and clinicians by offering insights into how the model reaches its final predictions. Given the millions of parameters and hundreds of waveform features per layer, visualizing these features individually is inefficient for understanding the overall behavior of the proposed foundation model. As a result, we use several techniques in nonlinear dynamic analysis (Thompson et al., 1990) to quantify the overall patterns of these extracted features, which are discussed in detail in section I.3.

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

I.3 Quantify the intrinsic behaviors: nonlinear dynamics analysis on the layer-wise waveforms

Understanding the representations extracted by intermediate layers is crucial to interpreting our model's behavior. To quantify the meaningfulness of these representations, we conducted a nonlinear dynamics analysis inspired by chaos theory. This method analyzes the features' intrinsic behaviors through metrics like the Lyapunov exponent (Wolf et al., 1985) (sensitivity to initial conditions), Hurst exponent (Qian & Rasheed, 2004) (self-correlation/seasonality), and persistence entropy (Yan et al., 2023b) (unpredictability in system states). We obtain the following key observations:

1. Deeper Layers Capture Higher-Order Complexity.

For signals such as GSR, EEG, and ACC, deeper layers show lower self-correlation (DFA (Hu et al., 2001)) and higher unpredictability (persistence entropy), indicating a transition to representations that are less periodic and more chaotic.

The decrease in the Lyapunov exponent across layers suggests reduced variation in extracted features, aligning with the idea that deeper layers capture more abstract, long-term patterns with broader receptive fields.

Modalities with Simpler Dynamics. In contrast, PPG and ECG signals, dominated by regular heart activity, exhibit more stable patterns across layers. This aligns with their simpler waveform structures and less complex dynamics compared to signals related to neural and physical activities.

These visualizations reveal that the model progressively transforms raw sensory data into representations aligned with the complexity of each signal. For GSR and EEG, deeper layers exhibit increased unpredictability and reduced periodicity, highlighting the extraction of nuanced, higher-order patterns critical for human sensing. In contrast, the stability of representations for PPG and ECG reflects their simpler dynamics, demonstrating the model's adaptability to varying signal characteristics. This analysis confirms that the intermediate representations are purposefully optimized to capture the temporal and structural nuances of each modality, supporting the conclusion that the model learns meaningful features tailored to human sensing tasks.

0

2

4

6

8

10

Layers

0.3

0.4

0.5

0.6

0.7

DFA

0

2

4

6

8

10

Layers

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Persistence Entropy H1

0

2

4

6

8

10

Layers

0.5

0.6

0.7

0.8

0.9

Persistence Entropy H0

0

2

4

6

8

10

Layers

0.20

0.25

0.30

0.35

0.40

0.45

0.50

0.55

0.60

Lyapunov Exponent

PPG

ECG

GSR

EEG_F

ACC_X

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

75

50

25

0

25

50

75

Component 1

75

50

25

0

25

50

75

100

Component 2

PPG

ECG

GSR

ACC_X

ACC_Y

ACC_Z

EEG_F

EEG_O

EEG_R

EEG_L

(a) T-SNE plot of the embeddings of the [ CLS ] special tokens, including signals of PPG, ECG, GSR, tri-axis accelerometer, and EEG at lobe of front, occiput, right, and left

Diabetes Patient

Feature Associations

Time Step Relevance

PPG

(b)

Visualization of features

extracted by the intermediate layer

I.4 T-SNE plot among classes

In this section, we present T-SNE plots of NormWear's embeddings across different classes to provide insights into their structure and assess their suitability for sample similarity-based information retrieval. It is important to note that these plots are exploratory in nature and do not serve as a claim of the embeddings' superiority. As shown in Figures 14a and 14b, clear class separations can be observed in certain scenarios. For example, EEG samples from seizure subjects and normal subjects are distinctly separated, and physical activity types are well-clustered. For ECG data, abnormal heartbeats tend to form cohesive clusters. However, it is essential to recognize that these T-SNE plots reduce the latent representations into a 2D space, which may not fully capture the inherent properties of the embeddings in their original high-dimensional form.





(a) Visualization of embedding on EEG signals.

Standing

Walking

Laying

Walking Downstairs

Walking Upstairs

Sitting

(b) Visualization of embedding on signals from IMU sensors.

I.5 Supplementary Qualitative Analysis of MSiTF 1041

0.0

1.0

Scores

Time Step

0

160

0

160

Q1: Is the person's eyes open?

Q2: Is the person under relexation?

Q3: What is the situation of the

person's brain?

Q4: Is this an area with tumor ?

Q5: Is the person not

experiencing a seizure?

Importance Score

-

Single Channel EEG

Time Step

To understand how each of our proposed gating modules in MSiTF-relevance, recency, and im1042 portance-select useful features for different tasks, we visualize the scores assigned to each time 1043 window. As shown qualitatively in Figure 15, the heatmaps reveal that both relevance and importance 1044 scores are sensitive to task differences. For example, in the eye closure detection task, the model 1045 focuses on the last few patches, whereas in the seizure detection task, it emphasizes patches with 1046 large fluctuations. A similar pattern is observed for the importance score, where patches are weighted 1047 differently across tasks. This suggests that our gating mechanism can adaptively select relevant 1048 features based on the task. We include a figure of the recency score (Figure 16) for completeness. 1049 Since the recency score is derived from a fixed decay function and is not learned, it remains the same 1050 across tasks. 1051

To improve visualization, we aggregated token scores using a window size of 9, which matches our 1052 tokenization patch size. We then applied Z-score normalization to ensure comparability across tasks. 1053 The sample was selected from the Epilepsy dataset due to its multiple and diverse task types.

0.0

1.0

Scores

Time Step

0

160

Recency Score

J Reconstruction Example 1055

Predicted

Predicted

Target

Target

EEG_F

EEG_F

EEG_L

EEG

EEG_O

EEG_O

EEG_R

EEG_R

GSR

GSR

PPG

PPG

Time Steps

Predicted

Farget

Target

Predicted

EEG_F

EEG_F

EEG_O

EG_O

EEG_R

}<

OSF

PPG

PPG

>

Time Steps

Predicted

Target

Chest_ACC_x

Chest_ACC_Y

Chest_ACC_Z

ECG

GSR

Wrist_ACC_x

Wrist_ACC_Y

Wrist_ACC_Z

PPG

GSR

Time Steps

Predicted

Target

Chest_AcC_x

Chest_ACC_Y

Chest_ACC_Z

ECG

GSR

Wrist_ACC_x

Wrist_ACC_Y

Wrist_ACC_Z

PPG

GSR

Predicted

Chest_ACC_x

Chest_ACC_Y

Chest_ACC_Z

ECG

GSR

Wrist_ACC_x

wrist_ACC_Y

Wrist_ACC_Z

PPG

GSR

Time Steps

Predicted

Chest_ACC_x

Chest_ACC_Y

Chest_ACC_Z

ECG

GSR

Wrist_ACC_X

Wrist_ACC_Y

Wrist_ACC_z

PPG

GSR

Time Steps

Time Steps

1056

1057

1058

1059

1060

1061

1062

1063

1064

1065

1066

1067

1068

1069

1070

1071

1072

1073

1074

1075

1076

1077

1078

1079

1080

1081

1082

1083

1084

1085

1086

1087

1088

1089

1090

1091

1092

1093

1094

1095

1096

1097

1098

1099

1100

1101

1102

1103

1104

1105

1106

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS Paper Checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

1107

1108

1109

1110

1111

1112

1113

1114

1115

1116

1117

1118

1119

1120

1121

1122

1123

1124

1125

1126

1127

1128

1129

1130

1131

1132

1133

1134

1135

1136

1137

1138

1139

1140

1141

1142

1143

1144

1145

1146

1147

1148

1149

1150

1151

1152

1153

1154

1155

1156

1157

1158

1159

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

1160

1161

1162

1163

1164

1165

1166

1167

1168

1169

1170

1171

1172

1173

1174

1175

1176

1177

1178

1179

1180

1181

1182

1183

1184

1185

1186

1187

1188

1189

1190

1191

1192

1193

1194

1195

1196

1197

1198

1199

1200

1201

1202

1203

1204

1205

1206

1207

1208

1209

1210

1211

1212

1213

1214

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

1215

Justification: [NA]

1216

1217

1218

1219

1220

1221

1222

1223

1224

1225

1226

1227

1228

1229

1230

1231

1232

1233

1234

1235

1236

1237

1238

1239

1240

1241

1242

1243

1244

1245

1246

1247

1248

1249

1250

1251

1252

1253

1254

1255

1256

1257

1258

1259

1260

1261

1262

1263

1264

1265

1266

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

1267

Justification: [NA]

1268

1269

1270

1271

1272

1273

1274

1275

1276

1277

1278

1279

1280

1281

1282

1283

1284

1285

1286

1287

1288

1289

1290

1291

1292

1293

1294

1295

1296

1297

1298

1299

1300

1301

1302

1303

1304

1305

1306

1307

1308

1309

1310

1311

1312

1313

1314

1315

1316

1317

1318

1319

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

1320

1321

1322

1323

1324

1325

1326

1327

1328

1329

1330

1331

1332

1333

1334

1335

1336

1337

1338

1339

1340

1341

1342

1343

1344

1345

1346

1347

1348

1349

1350

1351

1352

1353

1354

1355

1356

1357

1358

1359

1360

1361

1362

1363

1364

1365

1366

1367

1368

1369

1370

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human 1371 subjects 1372

1373

1374

1375

1376

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

1377

1378

1379

1380

1381

1382

1383

1384

1385

1386

1387

1388

1389

1390

1391

1392

1393

1394

1395

1396

1397

1398

1399

1400

1401

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: [NA]

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces normwear which is a multi-modal foundation model built to handle different wearable data, such as PPG, ECG, EEG, GSR, and IMU signals. It uses a channel-aware attention mechanism with a shared token to capture across-sensor patterns. The authors pretrained the model on a wide range of public datasets, and demonstrate generalization across 18 health-related tasks.\n\nSTRENGTHS AND WEAKNESSES:\n* The tokenization combines continuous wavelet transform (CWT) with signal differentiation, significantly increasing the dimensionality. It is unclear why the model needs explicit differentiation, as foundational models can learn such operations internally. This raises concerns about training and inference efficiency at scale.\n\n* The authors use CWT instead of windowing, arguing it avoids fixed window functions, but CWT introduces its own hyperparameters (e.g., mother wavelet, scaling). Given the distinct frequency profiles of signals like ECG and PPG, it is unclear why a uniform scaling is applied or why this choice is better than using STFT.\n\n* Despite time-series data’s similarity to audio, the authors do not benchmark STFT-based tokenization.\n\n* Although all channels share the same transformer backbone, the forward pass runs separately per channel, which can be computationally costly, especially for multi-channel signals.\n\n* Section 3.5 highlights challenges with nested relationships (e.g., stress vs. depression, running vs. cycling) in contrastive learning, but the paper does not clarify how the proposed approach (additional text modality without labels) solves this well-known problem. While CLIP-style alignment works well for visual-text pairs where the image-text correspondence is often rich and intuitive, it’s less clear why aligning time-series signals (like EEG, ECG, IMU) to natural language labels should yield better representations. Signals are often low-level, noisy, and not easily or inherently describable in human language, so this alignment may introduce unnecessary abstraction or noise. This setup also adds multiple layers, i.e., sentence template generation, large language model augmentation, and text embedding matching, which significantly increases complexity. The paper doesn’t clearly explain whether simpler label supervision (e.g., direct categorical labels) performs worse or why the extra linguistic layer is essential.\n\n* Standard deviation or variability across experiments is not reported, even though some baselines yield close performance, making it hard to assess statistical significance in specific datasets. While the authors provided statistical significance on the model comparison in Appendix D, it is not detailed for each application.\n\nOverall, the paper employs several techniques without fully explaining the motivations or providing ablation studies, adding to the method's complexity without clear justification.\n\nQUESTIONS:\n1) Line 801, B.6 Sentence template example for signal-sext alignment. Typo?\n\n2) PPG-DaLiA dataset: The original manuscript reports ~36 hours of data, but Table 5 lists 71 hours for pretraining. Could the authors explain this discrepancy?\n\n3) Appendix Fig. 15, Q4: The question asks, “Is this an area with tumor?” — is this meant to refer to a specific EEG channel or the full multi-channel EEG signal? Why do the authors assume a specific patch can reveal tumor-related features rather than considering the full temporal context?\n\n4) Sampling rate mismatch: Since the signals have different sampling rates and the tokenization does not standardize them, does the model learn features specific to each sampling rate?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 1, "ORIGINALITY": 1, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose the NormWear foundation model for data collected from wearable sensors, such as smart watches. The model is pretrained on large datasets comprising multiple modalities of data, including PPG, EEG, and accelerometer data. They consider zero-shot, partial-shot, and full-shot evaluation on downstream evaluation data (unseen during pretraining) comprising many different tasks ranging from health-related tasks such as prediction of disease to human activity recognition. They demonstrate impressive performance in all three evaluation settings against multiple types of baselines, including models trained on handcrafted features, deep time series models, and foundation models for specific sensing modalities, such as PPG.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- Very thorough evaluation and comparison across many different datasets, prediction settings, and baselines. It meets the evaluation standard that I would expect for a paper focused on foundation models.\n- Highly novel problem setting, as I am not aware of previous work on foundation models for multi-modal wearables data. I agree with the authors' argument on why this type of data is different from the type of data considered by deep time series models, as well as the need for modeling multiple modalities rather than having a foundation model for just a single modality.\n- The proposed zero-shot evaluation seems to be a new and challenging evaluation setting for wearables data.\n\n## Weaknesses\n- Accuracy in full-shot evaluation compared to single modality foundation models is mixed. The results only partially support the claimed benefit of having a foundation model that incorporates multiple modalities.\n- Novelty in the different components of the NormWear architecture is unclear. In the introduction, the novelty claim seems to suggest that the continuous wavelet transform-based transform, channel-aware attention, and human sensing adapted fusion mechanism are all new.\n- Checklist is not completed properly. All justifications are listed as [NA]. The checklist is especially important for a paper like this that involves many existing datasets for pretraining and creating new assets. For example, item 12 on licenses for existing assets is not satisfied.\n- Paper needs a thorough proofread for spelling and grammatical issues. The presentation is also unclear to me in many different areas.\n\nMinor issues:\n- Figure 5(b): Structued masking -> Structured masking\n- Line 801: signal-sext alignment -> signal-text alignment\n\nQUESTIONS:\n1. I have not seen zero-shot evaluation used on wearables data before and only have passing knowledge on its usage in other settings. What is an example of an application setting where zero-shot evaluation would be useful with wearables data?\n2. Why is the text augmentation so crucial for the performance of zero-shot evaluation? Without it, the AUC for NormWear is only 50-52%, which is in the same range as CLAP and is essentially a random guess. Could adding such a text augmentation into CLAP lead to similar results as NormWear for zero-shot evaluation?\n\nLIMITATIONS:\nLimitations are discussed in the conclusion. The discussion seems to be adequate.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, they propose a multimodal foundation model for sensor data. Unlike prior multimodal foundation models for sensor data, this does information sharing both between and within signals. Their approach has several steps: they curated a pre-training dataset from several sources, they develop a tokenization technique based on classical and recent approaches, they propose several approaches for sharing information within and between modalities and incorporate an alignment objective  to encourage alignment between sensor data embeddings and text descriptions. They show results, including zero-shot results on various downstream tasks with strong improvements over baseline.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n- This is actually a pretty comprehensive pipeline that seems relatively well thought out.\n- There is very much a need for more foundation model techniques for wearables/mHealth.\n- The combination of this pipeline and your application seem to be novel.\n- The figures are not bad and are helpful.\n- The results seem strong.\n\n# Weaknesses:\n- It would really help to have more equations, at least in the appendix. For instance, in 3.4 you mentioned several different fusion techniques. It would be much easier to understand them and what their stated limitations or benefits are if you gave some equations. You have a few e.g. equation 1 but far too few.\n- Many of the citations are not the standard ones e.g. the original transformer and VAE papers. You should find their actual conference publications.\n- You claim the first multi-modal and ubiquitous foundation model designed to extract generalized and informative representations from wearable sensing data. This is true if you only claim multimodal I believe, but you still need to cite [1].\n\n\nMinor point: line 159 has afer, should be after\n\nQUESTIONS:\n# Questions and comments:\n- The figures could really use captions. There is a lot going on in them and some hand-holding would be helpful. I get that there is a space limit, but in particular I look at the figures and if I want to skim them to get an idea of the paper, I don’t know what the most important things.\n- “Once the signal embeddings are aggregated, we adopt a variational-inspired approach” what does this mean exactly? Variational refers to optimization over function spaces, and VAE is doing that for approximate posterior inference for latent variables. I don’t see how optimization over function spaces inspires your approach. Am I missing something??\n- If I understand correctly, given your modality specific comment, are you beating e.g. PaPaGei (ICLR 2025)?\n\n[1] Xu, Maxwell A., et al. \"RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data.\" International Conference on Learning Representations (2025).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present NORMWEAR, a foundation model tailored for multivariate, multi-modal physiological signals captured via wearable sensors. NORMWEAR addresses two main challenges:\n1. Heterogeneous sensor configurations (varying modality, frequency, and number of channels),\n2. Generalization across diverse health tasks (zero-, partial-, and full-shot).\nThe model includes:\n* CWT-based tokenization for modality-agnostic preprocessing,\n* Shared-weight intra-channel encoders and a novel channel-aware [CLS] token-based fusion for inter-sensor relationships,\n* A memory stream inspired temporal fusion (MSiTF) module for semantic alignment with textual descriptions, enabling zero-shot inference.\nNORMWEAR is pretrained on 2.5 million sensor segments and evaluated on 18 tasks across 11 public datasets. It demonstrates state-of-the-art results over baselines in zero-, partial-, and full-shot settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The work is first of its kind to introduce a foundation model across heterogeneous wearable physiological modalities (ECG, EEG, IMU, etc.). The channel-aware [CLS]-token-based attention fusion and memory-inspired semantic alignment (MSiTF) are both well-motivated and technically novel.\n2. The design includes clear architectural innovations: Tokenization via CWT+derivatives, Independent per-channel encoding to address sensor variability, Attention-based fusion across signal modalities, Relevance/recency/importance scoring in the MSiTF module.\n3. Ablation studies and model variants (e.g., removing importance score, masking types, etc.) are thoroughly investigated.\n4. Evaluations cover a wide range of tasks, modalities, and settings. Both macro and micro metrics are reported across 18 tasks, showing consistent improvement. Comparison against state-of-the-art domain-specific (e.g., PaPaGei, CBraMod) and general models (Chronos, TF-C, CLAP) is extensive.\n5. The model supports arbitrary sensor setups without needing retraining. Zero-shot capability (via sensor-text alignment) is especially promising for real-world deployments where labeled data is sparse.\n\nWeaknesses\n1. While the architecture is sophisticated, there is little theoretical analysis of why the proposed fusion mechanisms work better. For instance, why [CLS]-based attention fusion outperforms mean-pooling is not analytically justified beyond empirical trends.\n2. Some sections (e.g., MSiTF scoring mechanisms, derivation of gating weights) are dense and not clearly explained. Equation (1) uses Gumbel-softmax without adequately discussing why this is necessary over standard soft attention.\n3. The use of a frozen TinyLlama clinical encoder is interesting, but not justified—would other encoders perform differently?\n\nQUESTIONS:\n1. Clarify theoretical motivations for the design of [CLS]-token fusion and gating in MSiTF.\n2. Provide qualitative examples of zero-shot predictions with corresponding textual queries for interpretability.\n3. Strengthen discussion on limitations of modality-specific vs generalist performance tradeoffs.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}