Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections
Authors: Berken Utku Demirel
Christian Holz
Venue: NeurIPS 2025 poster ()
Keywords: Unsupervised learning
contrastive learning
time series
Primary area: general_machine_learning

Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations without labeled data. Most SSL approaches rely on strong, well-established, handcrafted data augmentations to generate diverse views for representation learning. However, designing such augmentations requires domain-specific knowledge and implicitly imposes representational invariances on the model, which can limit generalization. In this work, we propose an unsupervised representation learning method that replaces augmentations by generating views using orthonormal bases and overcomplete frames. We show that embeddings learned from orthonormal and overcomplete spaces reside on distinct manifolds, shaped by the geometric biases introduced by representing samples in different spaces. By jointly leveraging the complementary geometry of these distinct manifolds, our approach achieves superior performance without artificially increasing data diversity through strong augmentations. We demonstrate the effectiveness of our method on nine datasets across five temporal sequence tasks, where signal-specific characteristics make data augmentations particularly challenging. Without relying on augmentation-induced diversity, our method achieves performance gains of up to 15--20\% over existing self-supervised approaches. Source code: \url{https://github.com/eth-siplab/Learning-with-FrameProjections}

Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections
Berken Utku Demirel
Christian Holz
Department of Computer Science ETH Zürich, Switzerland berken frrken.demirer felinff ethz. ch
Department of Computer Science
ETH Zürrch, Switzerland christian. Ahristian.holz@nnt holz@inf
ethz. ch
Abstract
Self-supervised learning (SSL) has emerged powerful paradigm for learn- ing representations without labeled data. Most SSL approaches rely on strong, well-establissed, handerafted data augmentations representation learning. However, designing such augmentations requires domain- specific knowledge and implicitly imposes representational invariances on the model, which can limit generalization. In this work, we propose an unsupervised representation learning method that replaces augmentations by generating views using orthonormal bases and overcomplete frames. We show that embeddings learned from orthonormal and overcomplete spaces reside on distinct manifolds, shaped by the geometric biases introduced by representing samples in different spaces. By jointly leveraging the complementary geometry of these distinct mani- folds, our approach achieves superior performance without artificially increasing data diversity through strong augmentations. We demonstrate the effectiveness of our method on nine datasets across five temporal sequence tasks, where signal- specific characteristics make data augmentations particularly challenging. Without relying on augmentation-induce diversity, our method achieves performance gains of up to 15-20% over existing sele-suue approaches. Source code: https: thub. cww wtt.m//ttssppab/aarri.ii. with- Fwr-Frrrerroectiioonn.
generate diverse views for
Introduction
Sample efficient unsupervised representation learning is a critical open challenge in deep learning. While recent supervieee techniques have shown strong performance in several tasks, they typically rely on handcrafted, aggressive data augmentations that expose the model to different versions of input samples in each training epoch to increase data diversity 1 Moreover, these techniques only perform well when augmentations are carefully optimized for the downstream task; otherwise, model performance drops significantly 51. Even when the downstream task known, designing effective augmentations can be challenging for data types lacking well-defined structure, i.c., text, tabular, signals [6,9], as the overly strong augmentations can cause model collapse [0].
Simply increasing sample diversity through random augmentations does not guarantee improved performance representation learning. Augmentations are only effective if the augmented views of sample have sufficient representational similarity with views of other intra-class samples [I] [2] Without adequate representational similarity, the model struggles to generalize across classes, leading to degraded performance on downstream tasks where intra-class variability is not fully captured through augmentations (13 Since achieving representational similarity in high-dimensional spaces is challenging [16]. key limitation of augmentation based self- self-supervised learning lies in its reliance on handcrafted transformations. These transformations can distort critical class-level structure, leading to severe performance degradation in complex heterogencous data regimes.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Another important limitation of augmentation-basee based SSL is the inductive bias and feature suppres- sion introduced by both the augmentation process and the optimization objective 18. When trained with strong augmentations, models tend to focus on a subset of predictive features, typically those aligned with the enforced invariances, while suppressing other features that may be critical for downstream performance [19]. This often leads to reliance on subset of features, harming generalization [7 20). Moreover, the inductive bias introduced by augmentations acts as double- edged sword: promoting invariance to certain transformations may benefit some tasks but harm others P1 For instance, rotation-basee augmentations are commonly used in activity recognition from inertial measurement units to promote robustness across sensor placements 23 However, they can obscure crennaatioo-depeddeee features needed to distinguish between fine aine-griied activities such standing and sitting, where subtle differences in sensor orientations are important. These effeets are more problematic in signals where the semantic relevance of augmentations varies across tasks.
In this work, we introduce novel SSL. method that generates views by projecting data onto an orthonormal base and an overcomplete frame, and then performs instance discrimination across these spaces. We demonstrate that the learned representations from instance discrimination lie distinct manifolds, each shaped by the inherent geometric biases its corresponding projections. Building on this observation, we propose to learn mapping functions that transform the original data's space into alternative latent representations. This mapping enables us to obtain multiple representations using a single encoder augmented with mapping functions use collections of manifolds.
Our method leverages the inductive bias introduced by projecting data into an orthonormal basis and an overcomplete frame to learn representations. We summarize our contributions as follows:
We propose novel self-supervised learning method that projects data into an orthonormal basis and an overcomplete frame to perform instance discrimination across these fixed transformations without increasing the data diversity using handcrafted augmentations.
We empirically and theoretically show that embeddings from these transformations lie on distinct manifolds shaped by domain-specific geometric biases. We then jointly leverage these complementary structures to improve performance on downstream tasks.
We demonstrate that our method achieves up to 15 15-20% performance gains over existing methods on nine datasets across five temporal sequence tasks while using fixed transforma- tions across datasets unlike existing approaches that rely on task-specific augmentations.
2 Method
2.1 Notations
We use lowercase letters (e.g., x) denote scalar quantities, and bold lowercase letters (e.g., to represent vectors, such as time series, while bold uppercase letters (e.g., X) are used for matrices. The parametric function is represented as fe(.) where 0 is the parameter. The discrete Fourier transformation is denoted as F(.), yielding a complex variable as Fx(k) E Ck where k is the frequency. The detailed calculations for each operation are given in the Appendix
2.2
Setup
We follow the common SSL setup. Given an unlabeled dataset D = {(x)){{ where each x; is a real-valued sequence of length L with C channels, the goal is to train learner fo that maps inputs to representations h; = fx(x... To evaluate the learned representations, we train a linear classifier on top of the frozen encoder using a labeled set D, y.) with M < K and y, (1,...; N).
2.3 Orthonormal Bases and Overcomplete Frames
Analyzing signals in different domains is useful for detecting desired patterns, as each domain is tailored to capture specific aspects of the data 24 25 Our method employs the Fourier, F= (k), and the Gabor wavelet transform, W>(a, b), construct, respectively, an orthonormal basis (using a tight frame) and an overcomplete frame. Equation defines their corresponding discrete transformations.
1 VT.
1 Σ x(n) - Wx(a.b) = Va Σ x(() 1b
Fx(1)
(1)

---

where v is the Gabor frame. We use these two transformations as they are complementary. Specifi- cally, the Fourier transform provides global overview of the signal's 's frequency content, while the Gabor wavelet enables localized frequency analysis by zooming specific time intervals [26].
2.4 Instance Discrimination
Data representations in the Fourier and Gabor wavelet domains are inherently unique, i.e., distinct samples yield distinct transforms. Consequently, the instance discrimination task between the views in different domains is well-defined. Moreover, since these transformations are isometric, the resulting views are not only unique for each sample but also preserve the underlying geometry. These properties offer significant advantages over existing SSL methods relying on strong augmentations, which may distort samples, causing different classes to appear similar or losing task-relevant information [27].
We the normalized lemperature-scaled cross-entropy (NT-Xent) loss 1
28 29 based cosine
similarity, with separate encoders and projection heads for each domain. For example, the Fourier branch takes the transformed input Fx and produces embedding via x (F((x((xx)) as shown The instance discrimination loss calculated across three domains is defined in Equation
in Figure T
((= , -2) (d')) = log exp(sim(z (d)
(d') (/7))//)
where d + d', and d, d. E (LFF,W,
eep((sim(2)
(4) (d')
2k )/7)
where t is the time domain. We compute the final loss as the unweighted sum of all pairwise terms between the time, Fourier, and wavelet domains over a batch of N samples, as shown Equation 3
LID
Σ
1 2N
Σ
1
2 (0))
d,d' (L.F,WW ded
2.4.1 Invariances
Strong augmentations in instance discrimination can cause encoders become overly reliant on subset of features or invariant to some transformations that discard task-relevant information [17 30)
Since the designed augmentations implicitly assume particular set of representational invariances (e.g., invariance to rotation), and can perform poorly when a downstream task violates this assumption (e.g., distinguishing sitting vs standing) g Proposition 2.1 shows that our method avoids this issue.
Proposition 2.1. Let få denote an optimal encoder under NT-Xent for domain d E {t, W). for some unintended transformation W, the encoder is invariant, i.e., fâ (Wxx) = fā (x), then for any anchor sample x the NT-Xent loss across domains is lower bounded by the number of negatives.
22) (
(d)
log(K 1)
where K 2 1 is the number of negatives that become near-positives due to the invariance.
Proof. At the NT-Xent optimum, positive pairs align perfectly (2]
fā(((x)) (x), Ad,
for the domain transformations T [t.F.W). If fā is also invariant to W, then at least K 2 negatives satisfy 2; fā (Wx,) with (x), 21) =s 1. Even if all other negatives dissimilar, the denominator of NT-Xeet contains at least K large terms, yielding
1/T
((x) 2 log
11ee = log( K +1)
(K
Thus the loss admits nontrivial lower bound in the presence of unintended invariance.
contrastive learning setups which may en- courage spurious invariances 8 30). our method ensures that such invariances cannot minimize the overall optimization objective. The detailed derivation of the proposition is provided in Appendix A
Proposition 2.1 states that, unlike augmentation

---

(a)
(b)
(c)
fx(()
(F)
gx(.)
NT-Xent mepfemnel (11)
F(x(())
Caplinus (2
mmmmmm x(t)
fx(.)
19
sc(.)
(1)
Mapping Latent space
0/ (=*)) 200
$ -W
$ -
Sw(-)
h
M
gv(.)
Input-output
6)
h = fx(x)
Figure 1: Overview of our method. (a) Original data and its transformed versions. The Fourier transformation is given in polar coordinates where we feed to the model magnitude and angle of each harmonic separately. (b) Representations h from each encoder lie on distinct manifolds M, with latent mappers $ translating across domain-specifie spaces. (c) Embeddings from each projection, with non-linear mappers used during pre-training to improve predictability across spaces.
2.5 Collections of Manifolds
Transforming data into well-known bases or overcomplete frames has clear benefits for pattern recognition, but doing this inference time is costly due to the extra encoders and transformations. When transformation operates in the same space as convolutional filters, prior works showed that applying transformations to the filters instead of the input reduces inference cost while still producing diverse representations (e.g., rotating filters instead of rotating the image and reprocessing it (33)..
However, our method transforms data into the complex space where applying equivalent transfor- mations to neural networks is not straightforward. We therefore propose lightweight latent space mappers & that transform the original representations into other spaces to leverage their geometry in downstream tasks. Specifically, we train two mappers, 4/ _!!" + M, where E {F, W). to approximate the representations produced by the corresponding nomain-specific encoders.
Our approach focuses on learning pairwise relationships, such as relative angles between samples across manifolds, rather than mapping individual points between latent spaces. Therefore, unlike prior methods based on affine latent-space mappings (34,, we use non-linear mappers. This is motivated by Proposition 2.2] which shows that in high dimensions, representations of the same sample across spaces can become orthogonal, while pairwise angle variation can span the full range. This suggests that preserving pairwise geometry is more challenging than aligning individual points.
Proposition 2.2 (Angle Concentration vs. Pairwise Spread), Let h (t) .) Unif(S where h0 = (F(x)) Although individual samples across latent spaces tend toward orthogonality, the pairwise angular difference Dij between distinct samples can span the full range up to
arccos (1h 12 (F)) 2
π
while arccos((h) (1) .h.. (1)))) arccos( (h (F) (F)))) Dij Sa (4)
Proof.
( (F) (F) dim (h2 (1LL (1)_]) h/ - - 21 = 3h ,1 ( (1 cos Vo E [0, (5)
Therefore, Aiyl 10 0/1 ST.
Figure 2 illustrates Proposition 2.2 by showing the angle densities between the same (i j. arccos (1,hh,)))) and different samples (i / j. | across domains. We also provide support- ing details in Appendix C and the full proof in Appendix A The key point of Proposition 2.2 is that preserving rear-orthogonality for many pairs does not guarantee global isometry between latent spaces. Because higher-order relations (e.g., triplet geometry and curvature) can change, the spaces can differ globally despite pairwise near-oothogonality.
In our method, aim to take advantage of these different geometries, shaped by inductive bias of data and architectures, to improve the performance in the downstream tasks. Therefore, we employ two mapper functions (d) to capture the geometry of latent spaces for other domains.

---

To improve predictability across latent spaces and reduce estimation enror while preserving distinct geometries, we em- ploy lightweight mappers over the em- beddings, "'-+/ _M + +M d E {FF W.. and optimize them jointly us- ing the loss defined in Equation 6
90'
30
15'
25
50
The overall pre-training loss for encoders the sum of map and LID. with no addi- tional weighting. After pre-training, freeze the encoders and train the latent space mappers using the same Lmpp loss, During inference, we only use the main encoder fx and the latent space mappers & excluding all projectors g(-), aux- iliary encoders (fr, fw). and mappers on projected embeddings $--+d. We pro- vide pseudocode implementation of our method in Appendix
-60
088
0.4
02000
05
Figure 2: Radial histograms illustrating angle distributions. (a) Top: Angle density of arccos ((h h ())): Bottom: Angle density A... (b) Same illustration from the Gabor wavelet h (W), In both cases, representations of the same samples across domains approach orthogonality, while pairwise angle differences remain widely distributed.
mmpp
1 N
(2())
-2
1 N
1 (2(1)
(6)
3 Experiments
3.1 Datasets
We conducted experiments on nine datasets across five tasks, including heart rate (HR) estimation from photoplethysmography (PPG), step counting and activity recognition using inertial measure- ments (IMUs). cardiovascular disease classification from electrocardiogram (ECG) and sleep stage classification from electroencephallgraphhh (EEG). We provide brief descriptions of each dataset below. Additional details, including pre-training and fine-tuning settings available in Appendix
Heart rate We used the IEEE Signal Processing Cup in 2015 (IEEE SPC) 35 and DaLia (36 for PPG-based heart rate prediction from wrist. We used the leave-one-sesiion-ou (LOSO) cross- validation, which evaluates models on subjects/sessions that were not used for training.
Activity recognition We used HHAR 37 and USC 38 for activity recognition from inertial mea- surement units from smartphones or wearable devices. We evaluated the cross-person generalization performance of the models, that is, the model was evaluated on previously unseen subjects.
Cardiovascular disease (CVD) classification We conducted experiments on China Physiological Signal Challenge (CPSC2018) 39) and Chapman University, Shaoxing People's Hospital (Chapman) datasets [40]. We selected the same four specific leads as [41 while treating each dataset as a single domain with small portion of the remaining dataset used for fine-tuning. We split the dataset for fine-tuning and testing based on patients (each patient's s recordings appear in only one set).
Step counting We used the Clemson dataset 422, which released for pedometer evaluation. We conducted experiments using wrist IMUs where labels are available through videos.
Sleep stage classification We used the Sleep-EDF dataset, from PhysioBank [43), which includes 197 wighe-night PSG sleep recordings, where used single EEG channel (i.e., Fpz-Cz) with sampling rate of 100 Hz, following the same setup as in while using only 10% for fine-tuning.
3.2 Baselines
Fundamentals We compare our method to core SSL approaches in the linear evaluation setting D.
These include SimCLR (. BYOL 2 VICReg B, and Barlow Twins 45). We also include CLIP (46), since data transformations in our method can be interpreted as different data domains.
5

---

Temporal sequences We also compare our method with SSL techniques for temporal data, in- cluding TS-TCC 44, TF-C 47), simMTM 48), and Vec 49. These methods are designed specifically for temporal sequences; for instance, TF-C uses Fourier encoder during both training and inference, while TS-TCC employs task-specific augmentations with transformer architectures.
3.3 Implementation
We employed ResNet [50] with eight blocks [5]]. the backbone for the encoder, with the projector consisting of two fully connected layers. For latent space mapping, we use a lightweight ID convolutional that downsamples and reconstructs the input with transposed convolutions, preserving dimensions while enabling non-linear transformations with fewer than 1 parameters. Similarly, we two convolutional encoders for the Fourier- and navelet-transformee inputs. To ensure fair comparison, baselines use 484-dimensional encoders as output, while ours uses 128 per encoder.
We train models with batch size of 1024 for 256 epochs and decay the learning rate using the cosine decay schedule. After pre-training, we train a single linear layer classifier on features extracted from the frozen pre-trained network. The models were optimized using Adam 52 with learning rate of 0.003, while the linear layer was fine-tuned with a learning rate of 0,03. Reported results are mean and standard deviation values across three independent runs with different seeds. For each dataset, we set the Fourier transform length equal to the signal length while excluding negative frequencies. For the wavelet transform, datasets. More details about the implementation and architectures are given in Appendix E.3
48 logarithmically spaced scales ranging from 1 to 128 for all
4 Results
We report the performance of our method against baselines across nine datasets spanning five tasks in Tables[] Overall, our approach achieves up to 20-30% improvements in some tasks, with an average gain of 10-15% across all datasets compared to both general SSL and sequence-specifi techniques. Moreover, unlike prior approaches that employ different augmentations, our method uses the same transformation across tasks without increasing the diversity of the training set.
Our main results show that our method outperforms previous techniques by a significant margin in several datasets. To further investigate, we explore whether prior methods can close this gap with comparable modifications. Specifically, we ask the following questions.
1. Our model includes lightweight mappers with additional backbones. We ask prior methods can match our performance by increasing backbone capacity, that is, by brute- moreeforce scaling.
2. Our model employs instance discrimination task with more than one view. Thus, we also ask existing techniques with multiple positive views can match our performance.
To answer the first question, increased the backbone capacity of previous fundamental techniques, especially SimCLR, BYOL and Barlow Twins as they represent unique approaches, by 2x compared to our method by adding additional residual blocks to the backbone. Results are given in Figure 3 We omit the error bars for figures where the standard deviations were negligible relative to the means.
Table 1: Performance comparison of our method with other methods for HR estimation
IEEE SPC12
IEEE SPC22
DaLiA
Method
MAE
RMSE
#1
MAB J
RMSE
#7
MAB J
RMSE
Pf
Supervised
FCN
15.1310.00 7008=020
21.6310.88
$2.09-3.43
16.571091
26.2011..0
55.98.10.78
12.4510112
18.3510.24
56.98110778
ResSee
13.60120.88
79.60=1.10
9.90a1477
16.67+1.60
67.58=2.98
5.50+0005
10.84:000
82:10::0:06
Self Supervised
SimCLR
12.42+0.05
20.96+0.30
73.62=0:52
16.41+0022
22.62+0.39
12.16111.2
16.88-0 19
22.64+40.22
56.37=0.21
RYOL
18.71+00.1
25.01+1.50
69:82:4.66
19.44-0055
26 16:66.00
46 56.74:5500
15.59:0.35
21.04-10.6
0..01
VICReg
13.17+0.82
20.38+1.27
73.65=0.02
16.78=0047
23. 10+00.55
54.10=1126
15.70=0.15
21,83++0.18
58. 55.32=00.6
Twins
13.22+0.34
20.42+0.88
67.51-201
22.08+0085
29.35+0.56
35.66533.40
11.87=0.57
19,20+0.29
62.20++0.40
CLIP
10.3110.35
17.10.0030
76.00-1.35
16.73.11008
26.39.1.64
21.59912.00
18.20.10.18
20.88.10.22
50.4211.40
TS-TCC
11.561:0.41
18.04110.66
78.38
16.52.0044
24.86.10.59
44.931..00
10.23
18.19410.44
62.77.10.04
SimMTM
13.20=:0.11
18.2740.22
73.78=1.10
17.03-6:.08
25. 18:4..9
51.33=2.62
13.5
10:1244007
55.47=0.09
TF-C
12.1010.115
20. 12.10.37
66.01_1.14
14.12.10.66
22.86.10.84
$2.74.1..10
16.15
23.47.10.13
50.12.11 10.12.11.45
TS2Vec
9,80=50499
16.6420.55
75.30=1..0
24.57:40.30
24.83:4..15
50.40=22.1
12.65=0032
20.04:10.33
58.1740.53
Ours
8.84-0.50
14,37+0.95
82.67-1.30
14.066=1.09
21.48+2.01
14.88=1.99
9.13++0.20
16.92+0.55
63.72=0006

---

Table 2: Performance comparison of our method with other techniques for Activity and Step
HHAR
Clemson MAPE
Method
Acc
W-FI
F
Acc f
W-FI
FL
MAE
RMSE 1
Supervized
14.21+1.56
22.88-2066
71,58+1881
48.87:+0744
46.02+0.95
45.33+0.82
5,02+0.26
2.86+8155
4.05+0.133
ResNet
69.85=2332
68.61=281
67.29:2.52
52.17=11.22
49.38=0084
48.01#1.22
6.55+2237
3.78+1.44
5.04+143
Self-Sxpervize
SimCLR
40.5510022
39.21
39.51
29.16110.69
29.02.0.67
28.9910.79
8.7010022
4.3610.13
6.3010.24
BYOL
49,6412248
48.63=2.75
48.02.2.59
28.4041.23
18.23+1.42
28.23
9.35
4.7240.12
6.79.:024
VICReg
38.05.13.01
37.12.12.66
37.38.13.02
23.75.11.00
23. 13.16.1.003
22.9211.21
10.87.0.61
5.47.0.35
7.78.10.14
Barlow Twins
38.97.0065
37.75=1.00
38.21
27.24:10.19
26.84
4.95
7.03:0221
CLIP
43.78:0089
42.53=0050
43.07:1095
25.55:10.63
25, 1.22
25:17:10
8.52:10.46
4.26::0.25
6,73:1033
TS: TSC
68.56+1.19
66.90-1..2
68.10+1.30
33.61++0.72
13.11+1.99
33.91+00.99
5.61++0.15
2.70+0:06
4.69+0.38
SimMTM
44.78:2062
42,48-0037
43.60:0022
22.34:10.28
25.68:0.41
29.72
8.77:10.15
4.63
6:90::018
TF-C
31.13=042
30.57=0440
31.00=0.31
30.78++0.39
28, 16.165
50.82+1.41
12.47+0.72
6.31
7.93++030
TS2Vee
13
56-66-01
64.13:00:1
35,40+0.96
32.17:11226
35.47:1.42
5,92:+0.3
3.01:+0:28
5.02:0042
Ours
70.67=006
67.74=029
68.79:0025
52.21+1.09
48.64=1.52
48.22#1.11
5.16+0.44
2.50
4.65-0045
Table 3: Performance comparison of our method with other techniques for CVD and Sleep
Method
Chapman
CPSC
Sleep
Acc 7
A
F1
Acc 1
AUC
Fl
Ace 1
W.FI
Kappa
Supervised
PCN
84.3+2213
95.40+0.57
82.1-2240
63.64+1122
91.30+0.22
60.43+1044
71.98-086
63.33+40.44
6.301
ResNet
93.161041
98.59110.5
92.02-0042
75.21
95.0210.03
71.70:1.90
76.94.1097
67,5211995
69.
(9.14110.6)
Seff-Supervised
SimCLR
75.28+0.57
93.55+0.25
0..04=0.50
50.10=0441
87.20+0.07
50,10=0024
72.45=2.32
58.93+1.59
$9.47=3.20
BYOL.
76.08+0440
93.54+0.18
74.80-0.45
51.90+0.30
87.05+0.22
50.89:0.35 08
70.77=0.17
58.23++0.55
55.90+1.20
VICReg
70.1011.90
89.3510.93
6..84_1.79
46.21.1129
84.70.0.50
42.51110.6
68.72.11.08
57.2411.04
17.1311442
Barlow Twins
72.43
91.17100000
70.42-1.53
48.6710.511
85.7810.19
44.57.0..33
70.10..:.22
57.72.10.81
57.8810.82
CLIP
$2.98+0.96
95. 15:0..2
81.00=1.03
50.01=00.89
86.40+0.32
47.99=0089
73.16-081
62.06:40..1
63.75=1.25
TS-TCC
73.5010.55
90.6510.07
71.10_0.57
51.5911122
86.3210.16
50.27:1132
12.8011.13
52.4331100
48.9811668
SimMTM
$4.29:1..99
95.87=0.18
83.31=1.25
51.70=:023
87 87.08%40.21
50.62:20.5
74.69=1844
63.53+1.21
65 352.1
TF-C
85.84+0.39
96. 10+0.11
84.71=044
47,86+0.49
86.27+0.05
45.42=0.66
64.50=1.80
56.77++2.11
52.61
TS2Vec
78.87:1100
90. 00.23:0.24
81 0::32:0047
48.73:10.85
85 1.0337
46.57:1110
65.71:11066
35.32:11.77
56.81:1.0
87.2110.80
96.5010.21
85.30_0..8
52.1010.900
87.1110.40
51.26.L.18
77.30
68.0510..6
69.16.1.322
(a) Performance (%)
(b)
Performance (%)
As shown in the results, simply increasing model size does not close the performance gap. In fact, larger models often exhibit sta- ble or decreased performance, particularly in heterogeneous low-data regimes, likely due to overfitting even with strong augmen- tations. In contrast, our method consistently outperforms them across all tasks, achieving 10-15% higher performance with approxi- mately half the number of parameters.
64.0
86.4
Curs
OuEs
85.0
62.5
TSTCC
TF-C
3WIW
RATM
82.5
60.0
SimCLK (2x)
S1mCT (2x)
80.0
57.5
RYGT.
BYOT. (2x)
77.5
SiMCLR
55.0
BYOL
75.0
RYOT (2x)
SimCLR
BWIN (2x)
52.5
725
Tsrcc
O
-F-C
One observation from this comparison is that TF-C performs worse than others for some datasets, despite using two encoders, one for the time and one for the Fourier domain. We hypothesize that this performance drop may stem from the strong augmentations applied in TF-C framework, which changes the magnitude of frequency components and degrade representations for noisy signals.
50.0
Size
70.0
BNTW
Size
0.4 0.5 (M)
0.3
0.4
0. (M)
Figure 3: Performance of our method on DaLiA (a) using correlation (p) and on Chapman (b) using Fl score, compared to other self-supervised learning tech- niques. Barlow Twins abbreviated as (BWTW), and backbone sizes shown in millions of parameters. The red circle in each plot denotes our method, which achieves higher performance with fewer parameters.
For the second question, we generated third view using task-specific random augmentation and applied the instance discrimination loss across all views, following the same setup as our method. Figure 4 presents the results, including comparisons with larger backbone for prior techniques.
These results show that adding a third view improves performance for prior methods, but they still fall short of ours by about 10%. Notably, introducing third view increases data diversity for these
7

---

methods, while in our case, using third transformation does not. This empirical evidence highlights the data- data-hungry nature of prior SSL approaches and the efficiency of our method further.
(a)
(b) Methods
(c) Methoos
Methods
singlo-view
multi
single-view
O mulli-viww
single-iew
multi-view
sincLe
SiMCLR
BLNCLR
aLmCLa (234)
BinCLa (2x4)
SinCta
BWTH
0-0
BWIW
MMIW
HWT IN (2x)
CUXS
BWIM (2x)
ENIW (2x))
Cara
Performance 0%)
Performance (1)
Performance 03)
58
80
64
41
45
49
53
70
75
80
95
Figure 4: Performance comparison across datasets, (a) DaLiA (p), (b) HHAR (Acc), and (c) Chapman (FI), when adding a third view for instance discrimination similar to our method. While prior methods benefit from the additional view, their performance still lags behind our approach by a large margin.
4.1 Ablation Experiments
Our method consists of multiple components, so we conduct many ablation studies to assess the contribution of each. First, we evaluate the impact of each data transformation by selectively removing them individually. We exclude the orthonormal basis transformation (w/o QB) and retain only the
overcomplete frame, then exclude the overcomplete frame transformation (w/o OF) and retain only
the orthonormal basis in the method. We present the results across tasks in Tables 4 5 and 6
Table 4: Ablation on proposed method in PPG datasets for HR estimation
Method
IEEE SPC12
IEEE SPC22
DaLiAerG
MAE 4
RMSE 4
P1
MAE
RMSE|
MAE
RMSE J
P1
Ours
8,84-0.00
14.37+095
82.67+1.30
14.06+11009
21,48+201
54.88=180
9.13-0.20
16.924056
63.72+000
wlo OB
11.23 (+2.39)
17.26 (+2.89)
(.6.60)
15.88 (+1.2)
24.82 (1334)
(3.96((0.92)
8.87 (.0.5)
16.13 (4.99)
(5.34
wlo OF
9.50
15.18 (-081)
81.19 (.49)
15.33 (-1.27)
21.29 (-281)
53.05
9:60 (+0.47)
17.31 (+0.39)
63.06 (0.66)
wlo 0%
9.62 (40.78)
15:46 (+1.09)
80.21 (:2.46)
14.16 (-0.10)
22.46 (40.8)
56.23 (42.35)
8.90 (.0.3))
16.27 (4065)
64.65 (40.03)
These results show that the best performance is mostly achieved when both transformations are used together, though the degree of performance change varies across tasks. One interesting result from our ablation study is that using only the Gabor wavelet transform sometimes outperforms the combination with Fourier transformation. Specifically, we observe that the Gabor wavelet is more effective for tasks involving sudden signal changes (e.g., abnormal heart rhythms, neural activity during sleep), while the Fourier transform better captures global structures such as periodic patterns in heart rate or step counting from inertial measurements. This empirical evidence supports our motivation for using complementary, principled transformations to jointly capture diverse signal characteristics.
Second, retain both transformations and their NT-Xent loss but exclude the latent space mappers - during inference, performing linear probing solely on the original latent space (0) Results are reported in the same tables with the first ablation experiment under "w/o & of-+d" As the results indicate, removing the mappers from our method degrades performance compared to the best case.
We conducted additional ablations to evaluate the performance of latent space mappers d/ and to assess the impact of embedding mappers -'dd dd on the performance. Results are given in Appendix D
4.2 Discussion of results
Do we need data augmentations? Data augmentations are widely viewed as essential for learning representations from unlabeled data 53]. A celebrated theory, InfoMin [27], argues that effective augmentations reduce mutual information between views while retaining task-relevant features. However, our method does not rely on asymmetric views but instead uses unitary transformations that preserve all information. Our results suggest that asymmetric views are not essential for SSL.
A recent study 12 highlights the importance of strong augmentations in instance discrimination by showing that augmentations can cause different intra-class samples to align when their augmented views overlap (i.e., two different cars appear similar when both are cropped to show only the wheels).
8

---

Table 5: Ablation on proposed method in IMU datasets for Activity and Step
HHAR
USC
Clemson
Method
Acc 7
W-FI
FL
Acc 1
W-FI
FL
MAPE L
MAE
RMSE
Ours
70.67.0.06
67.74..0.29
68.79_.0.25
52.2111109
48.6411.22
48.22.1.11
5.1610.44
2.50.0023
4.6510.550
w/o OB
65,10 (3.22)
(-200)
44,70 (:7.5)
41.20
41,95 (422)
(12.4)
3.65 (o125)
5.79 (+1.14)
w/o OF
69.74 (-0.93)
67.62 (-1.12)
68.01 (4078)
49.06 (-3:15)
(3.47)
45.51 (-2.1)
(.59())
2.72 (40.23)
5.07 (40.43)
67.11 (3.56)
66.24 (-1.0)
67.05 (-134) 51.26 (0.95)
47.81 (0.83)
(7.50
(+0.00
2.45 (0..5)
4.53 (..12)
Table 6: Ablation on proposed method in ECG and EEG datasets for CVD and Sleep
Chapmaa
Sleep
AUCC
AUC
Acc 1
Kappo (K)
Ours
87,21
96.50:00:1
85.30
52. 1,
87. 0,000
51,26.:1.15
3..30.0044
68.05
69.16:1132
86.79 0.42)
96.51 (+0.01)
85,90 (+0.60)
58.81 (+6.11)
89.35 (+2.24)
$5,91 (+4.5)
$1.15 (+3.85)
70.73 (-2.65)
74.37 (+5.21) 64.75 ( (441)
wil OF
81.82 (3.39)
94.15 ( (2335)
79.26 ( (6.04)
45.83 6.277)
84.47 (2669
(..18
(3.39)
63.98 (4.07)
w/o 2a
84,98 (3223)
95.70 (-0.0)
82,69 (2261)
$1.44
(6.5
49.63 (-1.3)
79.30 (+2.00)
68.40 (-0.35)
(-251)
However, this explanation does not extend to our method. Since our transformations are unitary, the views preserve the structure of the original sample without introducing overlaps across instances. self-supervised learning can succeed without
Our findings show that dittuancedddcrimmnttioon relying on hand-crafted strong augmentations for temporal signals. This opens the door for future work to test whether the same holds in other modalities, such as images and audio.
Implicit bias An interesting finding from experiments is that, although all encoders trained with the same loss at the same time and without dlop-gradien operations (commonly used to prevent collapse), the geometries of the learned representation differ significantly across encoders. We attribute this to the applied data transformations, which introduce strong implicit biases that shape each latent space in ways that emphasize different characteristies, such global or local features.
5 Limitations
While our work demonstrates that instance discrimination-base based SSL can be effective without aggres- sive data augmentations, we do not provide a theoretical explanation for the observed performance gains. We hypothesize that the representational improvements arise from implicit biases of the transformations, though this remains unverified by formal theoretical analysis.
In terms of scope, our experiments primarily focused on classification tasks, as this setting involves a wide variety of augmentations, including task-specific ones [5). making a natural testbed for our augmentation-fiet approach. Nevertheless, since our method leverages both global (FFT) and local (Gabor) representations for representation learning, it also holds potential for forecasting tasks where capturing both long-ran and localized temporal patterns is critical. Exploring this direction in both self-supervised and supervised paradigms remains an important avenue for future work.
Finally, while our method applies Fourier and wavelet transformations, it is worth noting that although the Fourier transform is computationally efficient (54), computing wavelet coefficients is expensive. In our experiments, we mitigated this by caching the coefficients after a one-time computation. We quantitatively evaluate the computational overhead of our method and other techniques in Appendix F showing that our approach maintains competitive runtime compared to state-of-the-al works.

---

Related Work
Data augmentation in SSL Learning representations through data transformations dates back to early self-supervised learning methods, which introduced pretext tasks that reformulated the problem into supervised one, such as predicting image rotations [55] or spatial contexts [56] 577 More recently, data augmentations have become central component of SSL, with stronger However, learning representations
transformations applied to boost sample diversity [I I2 44
58 Mainly, strong augmentations can alter the label of a sample [59), leading to model collapse (27]. and may suppress informative features [22]. This often causes models rely on subset of features aligned with augmentation- induced invariances, potentially ignoring others that are critical for downstream performance [19] 20 [2]] Motivated by these, propose method that replaces data augmentations with principled, well-understood transformations. Instead of relying on task-specific, hand-tuned augmentations, our approach performs instance discrimination using views generated from principled transformations.
with strong augmentations introduce new challenges [22
Mapping between latent spaces Mapping between different representations is growing area in deep learning as it enables the use of pre or co-trained models across domains [60 [1]. For instance, authors in [62 proposed zero-shot communication method between latent spaces by projecting them into a shared relative space, constructed from pairwise distances between anchor points. However, this approach relies anchor points to capture the structure of each latent space. More recent work focuses on directly translating between latent spaces using affine or orthogonal transformations [63] These methods are inspired by Procrustes analysis for latent space alignment [64 65], which has been applied in language processing 666 67]. Yet, orthogonal transformations preserve inner products and therefore cannot capture the full representational differences learned by distinct encoders an aspect central to our approach. Instead, our method estimates the geometry of the target latent space, shaped by domain-specific transformation biases, and enables its use in downstream tasks.
Multiview contrastive learning Learning representations with instance discrimination using multi- ple views without strong augmentations was proposed early in SSL (68 70). Earlier methods [68] used different channels (e.g., L and ab from an RGB image) as views, which act as implicit augmen- tations and may bias the model toward certain features. contrast, our method uses orthonormal and overcomplete transformations that are unitary or redundant by design, avoiding feature selection bias. Moreover, unlike prior work, we introduce lightweight mappers that learn the geometry of each representation space to better align them during inference to enable effective linear probing.
Implicit bias from frequency Prior work has leveraged frequency information for representation learning in temporal sequences (4 71 9. Some approaches, in line with ours, use the NT-Xent loss counterparts using the Fourier transform 477 or spectrograms [72 73). However, our method differs in two key ways. First, these methods rely on additional encoders for each transformed view [47 72)
to maximize agreement between time-domain inputs and their
and still require task-specific data augmentations. In contrast, we replace augmentations entirely with principled transformations that generalize across datasets without task-specific tuning. Second, previous work typically focuses on either global frequency features (via Fourier transform) [4 74
or localized frequency content (via spectrograms) [72 773. Our method integrates both by jointly using orthonormal and overcomplete representations while remaining more efficient using lightweight latent space mappers instead of using separate modality-specific encoders.
7 Conclusion
In conclusion, we have shown that principled geometric transformations in the form of orthonor- mal bases and overcomplete frames are effective for self-supervised representation learning. By generating views through unitary and frame-based based projections, our method uses complementary manifolds without perturbing or enlarging the data. Crucially, our approach achieves up to 15-20% performance gains across nine datasets in five tasks, without relying on larger backbone architectures or handcrafted, domain-specific augmentations. Our results underscore the importance exploiting intrinsic geometric biases in data representations, opening new avenue for SSL methods that prioritize mathematical structure over empirical trial and error. We believe this work paves the way toward more generalizable, augmentation-fiie self-supervision across wide range of domains.
10

---

References
[1] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020.
[2] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Ghesh- laghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent new approach to self-supervised learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS'20, 20, Red Hook, NY, USA, 2020. Curran Associates Inc.
[3] Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: Variance-iivarinne-ccvvaaaa regular- ization for self sule-suuer learning. In International Conference on Learning Representations, 2022.
[4] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[5] Berken Utku Demirel and Christian Holz. Finding order in chaos: A novel data augmentation method for time series in contrastive learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[6] Vikas Verma, Thang Luong, Kenji Kawaguchi, Hieu Pham, and Quoe Le. Towards domain- agnostic contrastive learning. In Marina Meila and Tong Zhang. editors, Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research. PMLR, 2021.
[7] Soyoung Yoon, Gyuwan Kim. and Kyumin Park. Ssmix: Saliency-based span mixup for text 2021.
classification. ArXiv,
[8] Yi Sui, Tongzi Wu, Jesse C. Cresswell, Ga Wu, George Stein, Xiao Shi Huang. Xiaochen Zhang, and Maksims Volkovs. Self-supervised representation learning from random data projectors. In The Twelfth International Conference on Learning Representations, 2024.
[9] Berken Utku Demirel and Christian Holz. An unsupervised approach for periodic source detection in time series. In Proceedings of the 41st International Conference on Machine Learning, ICML'22. JMLR.org, 2024.
[10] Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse in contrastive self-supervised learning. In International Conference on Learning Representations, 2022.
[11] Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro Astolfi, Kyunghyun Cho, and Yann LeCun. Slmathbb X sample contrastive loss: Improving contrastive learning with sample similarity graphs. In The Thirteenth International Conference on Learning Representations, 2025.
[12] Yifei Wang, Qi Zhang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Chaos is a ladder: A new theoretical understanding of contrastive learning via augmentation overlap. The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
[13] Yifei Wang. Qi Zhang, Tianqi Du, Jiansheng Yang. Zhouchen Lin, and Yisen Wang. A message passing perspective on learning dynamics of contrastive learning. In International Conference Learning Representations, 2023.
[14] Nikunj Saunshi, Jordan Ash, Surbhi Goel, Dipendra Misra, Cyril Zhang, Sanjeev Arora, Sham Kakade, and Akshay Krishnamurthy. Understanding contrastive learning requires incorporating inductive biases. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 2022.

---

[15] Lu Han, Han-Jia Ye, and De-Chuan Zhan. Augmentation component analysis: Modeling similarity via the augmentation overlaps. In The Eleventh International Conference on Learning Representations, 2023.
[16] Advait Parulekar, Liam Collins, Karthikeyan Shanmugam. Aryan Mokhtari, and Sanjay Shakkottai. Infonce loss provably learns pureserr-r representations. In Annual Confer- ence Computational Learning Theory, 2023.
[17] Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Darrell. What should not be contrastive in contrastive learning. In International Conference on Learning Representations, 2021.
[18] Ting Chen, Calvin Luo, and Lala Li. Intriguing properties of contrastive losses. In A. Beygelz- imer, Y. Dauphin, P. Liang. and Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[19] Joshua David Robinson, Li Sun, Ke Yu, kayhan Batmanghelich, Stefanie Jegelka, and Suvrit Sra. Can contrastive learning avoid shortcut solutions? In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances Neural Information Processing Systems, 2021.
[20] Zihu Wang, Yu Wang, Zhuotong Chen, Hanbin Hu, and Peng Li. Contrastive learning with consistent representations. Transactions on Machine Learning Research, 2024.
[21] Hankook Lee, Kibok Lee, Kimin Lee, Honglak Lee, and Jinwoo Shin. Improving transferability of representations via augmentation-aval self- seff-supervision. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances Neural Information Processing Systems, 2021.
[22] Yihao Xue, Siddharth Joshi, Eric Gan, Yu Chen, and Baharan Mirzasoleiman. Which features are leart by contrastive learning? On the role of simplicity bias in class collapse and feature suppression. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, pages 38938-38970. PMLR, 2023.
[23] Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, and Cecilia Mascolo. Exploring contrastive learning in human activity recognition for healthcare. ArXiv, abs/2011.11544 2020.
[24] Leon Cohen. Time-frequency analysis: theory and applications. Prentice-Hall, Inc., USA, 1995.
[25] F. Hlawatsch and G.F. Boudreaux-Bartels. Linear and quadratic fime-freyuency signal repre- sentations. IEEE Signal Processing Magazine, 9(2):21-67, 1992.
[26] Stéphane Mallat. A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way. Academic Press, Inc., USA, 3rd edition, 2008
[27] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc.
[28] Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016.
[29] Zhirong Wu, Yuanjun Xiong, X Yu Stella, and Dahua Lin. Unsupervised feature learning via nonpparametric instance discrimination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[30] Kimia Hamidieh, Haoran Zhang, Swami Sankaranarayanan, and Marzyeh Ghassemi. Views can be deceiving: Improved SSL through feature space augmentation. In The Twelfih Interna- tional Conference Learning Representations, 2024.
12

---

[31] AL Goldberger. Is the normal heartbeat chaotic or homeostatic? Physiology, 6(2):87-91, 1991. PMID: 11537649.
[32] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020.
[33] Taco Cohen and Max Welling. Group equivariant convolutional networks. In Proceedings of The 33rd International Conference on Machine Learning, 2016.
[34] Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, and Emanuele Rodolà. Latent space translation via semantic alignment. In Thirly-seventh Conference on Neural Information Processing Systems, 2023.
[35] Zhilin Zhang, Zhouyue Pi, and Benyuan Liu. Troika: A general framework for heart rate monitoring using wrist-type smotoplelhy--oorrppiic signals during intensive physical exercise. IEEE Transactions Biomedical Engineering, 62(2):522-53, 2015.
[36] Attila Reiss, Ina Indlekofer, Philip Schmidt, and Kristof Van Laerhoven. Deep ppg: Large-scale heart rate estimation with convolutional neural networks. Sensors, 19, 2019.
[37] Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Süiger Prentow, Mikkel Baun Kjær- gaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. SenSys, 2015.
[38] Mi Zhang and Alexander A. Sawchuk. Usc-had: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. Association for Computing Machinery, 2012.
[39] Eddie Y. K. Ng. Feifei Liu, Chengyu Liu, Lina Zhao, X. Zhang, Xiaoling Wu, Xiaoyan Xu, Yulin Liu, Caiyun Ma, Shoushui Wei, Zhiqiang He, and Jianqing Li. An open access database for evaluating the algorithms of electrocardiogram rhythm and morphology abnormality detection. Journal of Medical Imaging and Health Informatics, 2018.
[40] Jianwei Zheng, Jiamming Zhang, Sidy Danioko, Hai Yao, Hangyuan Guo, and Cyril Rakovski.
A 12-lead electrocardiograa database for arrhythmia research covering more than 10,000 patients. Scientific Data, 7(1):48, February 2020.
[41] Erick A Perez Alday, Annie Gu, Amit J Shah, Chad Robichaux, An-Kwok Ian Wong, Chengyu Liu, Feifei Liu, Ali Bahrami Rad, Andoni Elola, Salman Seyedi, Qiao Li, Ashish Sharma, Gari D Clifford, and Matthew A Reyna. Classification of 12-lead ecgs: the physionet/comput- ing in cardiology challenge 2020. Physiological Measurement, 12400) dec 2020.
[42] Ryan Mattfeld, Elliot Jesch, and Adam Hoover. A new dataset for evaluating pedometer performance. In 20/7 IEEE International Conference on Bioinformatics and Biomedicine (RIBM), pages 865-869, 2017.
[43] Ary L. Goldberger, Luis A. N. Amaral, Leon Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. Physiobank, physiotoolkit, and physionet. Circulation, 101(23):e215-<222, 2000.
[44] Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee Keong Kwoh, Xiaoli Li, and Cuntai Guan. Time-series representation learning via temporal and contextual contrasting. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI- 21, 2021.
[45] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stephane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 2230-. PMLR, 18-24 Jul 2021.
[46] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
13

---

[47] Xiang Zhang, Ziyuan Zhao. Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised contrastive pre- pre-training for time series via frequenrr consistency. In Proceedings of Neural Information Processing Systems, NeurIPS, 2022.
[48] Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. SimMTM: A simple pre-training framework for masked time-series modeling. In Thirty- seventh Conference on Neural Information Processing Systems, 2023.
[49] Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. Proceedings of the AAAI Conference on Artificial Intelligence, 36(8):8980-89)) 8987, Jun. 2022.
[50] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
[51] Shenda Hong, Yanbo Xu, Alind Khare, Satria Priambada, Kevin Maher, Alaa Aljiffry, Jimeng Sun, and Alexey Tumanov. Holmes: Health online model ensemble serving for deep learning models in intensive care units. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1614-1624, 2020.
[52] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[53] Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering,
2023.
[54] An algorithm for the machine calculation of complex fourier series. In Papers on Digital Signal Processing. The MIT Press, 11 1969.
[55] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
[56] Carl Doerseh, Abhinav Kumar Gupta, and Alexei A. Efros. Unsupervised visual representation learning by context prediction. 2015 IEEE International Conference on Computer Vision (ICCV), pages 1422-1430, 2015.
[57] Deepak Pathak, Philipp Krähenbühl, Jeff Donahue. Trevor Darrell, and Alexei A. Efros. Context encoders: Feature learning by inpainting. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2536-2544, IEEE Computer Society, 2016.
[58] Junbo Zhang and Kaisheng Ma. Rethinking the Augmentation Module in Contrastive Learning: Learning Hicrarchical Augmentation Invariance with Expanded Views. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
[59] Xiao Wang and Guo-Jun Qi. Contrastive learning with stronger augmentations. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 45()):5549-5560 2023.
[60] Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revisiting model stitching to compare neural representations. In Proceedings of the 35th International Conference on Neural In- formation Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc.
[61] Adrián Csiszárik, Péter Körösi-Szabó, Ákos K. Matszangosz, Gergely Papp, and Dániel Varga. Similarity and matching of neural network representations. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances Neural Information Processing Systems, 2021.
14

---

[62] Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. In The Eleventh International Conference on Learning Representations, 2023.
[63] Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, and Emanuele Rodolà. Latent space translation via semantic alignment. In Thirly-seventh Conference on Neural Information Processing Systems, 2023.
[64] Chang Wang and Sridhar Mahadevan. Manifold alignment using procrustes analysis. In Proceedings of the 25th International Conference on Machine Learning, 2008.
[65] Chang Wang and Sridhar Mahadevan. Manifold alignment without correspondence. In Proceedings of the s1 International Joint Conference on Artificial Intelligence, IJCAI'09, 2009.
[66] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:13 2017.
[67] Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. International Conference on Learning Representations, 2017.
[68] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding, In Andrea Vedaldi. Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors. Computer Vision ECCV 2020, pages 776-794, Cham, 2020. Springer International Publishing.
[69] Vladan Stojnic and Vladimir Risojevic. Self-supervised learning of remote sensing scene rep- resentations using contrastive multiview coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1182-1191, June 2021.
[70] Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. Dual contrastive prediction for incomplete multi-view representation learning. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(4):4447-4461, 2023.
[71] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. CoST: Con- trastive learning of disentangled seasonal-tren -trend representations for time series forecasting. In International Conference on Learning Representations, 2022.
[72] Luyu Wang and Aäron den Oord. Multi-format contrastive learning of audio representa- tions. ArXiv, abs/2103.06555 103.06508, 2021.
[73] Luyu Wang, Pauline Luc, Adrià Recasens, fieanBBaptise Alayrac, and Aäron van den Oord. Multimodal self-super learning of general audio representations. ArXiv, abs/2104.12807, 2021.
[74] Zhen Liu, Qianli Ma, Peitian Ma, and Linghao Wang. Temporal-frequencc co-training for time series semi-supervised learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37, 2023.
[75] Peter G. Casazza and Gitta Kutyniok, editors. Finite Frames: Theory and Applications. Applied and Numerical Harmonic Analysis. Birkhäuser, Boston, 2013.
[76] Tai Sing Lee. Image representation using 2d gabor wavelets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(10):959-97) 1996.
[77] Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4):983-1049, October 2016.
[78] A. N. Gorban and I. Y. Tyukin. Blessing of dimensionality: mathematical foundations of the statistical physics of data. Philosophical Transuctions of the Royal Society A: Mathematicul, Physical and Engineering Sciences, 376(21118220110233 2018.
15

---

[79] Isaac Chavel. Eigenvalues in Riemannian Geometry. Academic Press, November 1984.
Google-Bon Google-Bookss-D: OvIVITWuKGgC.
[80] Isaac Chavel. Isoperimetric Inequalities: Differential Geometric and Analytic Perspectives. Cambridge University Press, July 2001. Google-Boos Booee-Booolliiiio ID: LiPaxelSukoC.
[81] Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. CBramod: A criss-cross brain foundation model for EEG decoding. In The Thirteenth International Conference on Learning Representations, 2025.
[82] Jiexi Liu and Songean Chen. Timesurl: self-supervised contrastive learning for universal time series representation learning. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourleenth Symposium on Educational Advances in Artificial Intelligence, 2024.
[83] Leandro Giacomini Rocha, Dwaipayan Biswas, Bram-Ernst Verhoef, Sergio Bampi, Chris Van Hoof, Mario Konijnenburg, Marian Verhelst, and Nick Van Helleputte. Binary cornet: Accelerator for hr estimation from wrist-ppg. IEEE Transactions on Biomedical Circuits and Systems, 2020.
[84] Dani Kiyasseh, Tingting Zhu, and David A. Clifton. Cloes: Contrastive learning of cardiac signals across space, time, and patients. In International Conference Machine Learning, 2020.
[85] Hangwei Qian, Tian Tian, and Chunyan Miao. What makes good contrastive learning on small-scale wearable- based tasks? In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, KDD '22, page 3761-3771, New York, NY, USA, 2022. Association for Computing Machinery.
[86] Horace Barlow. Redundancy reduction revisited. Network: Computation in Neural Systems, 12(3):241, aug 2001.
[87] Dwaipayan Biswas, Luke Everson, Muqing Liu, Madhuri Panwar, Bram-Ernst Verhoef, Shrishail Patki, Chris H. Kim, Amit Acharyya, Chris Van Hoof, Mario Konijnenburg, and Nick Van Helleputte. Cornet: Deep learning framework for ppg-based heart rate estimation and biometric identification in ambulant environment. IEEE Transactions on Biomedical Circuits and Systems, 2019.
[88] Valentin Bieri, Paul Streli, Berken Utku Demirel, and Christian Holz. Beliefppg: uncertainty- aware heart rate estimation from PPg signals via belief propagation. In Proceedings of the Thirty-Nitth Conference on Uncertainty Artificial Intelligence, UAI '23. JMLR.org, 2023.
[89] Sergey loffe and Christian Szegedy. Batch normalization: Accelcrating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference Machine Learning, volume 37 Proceedings of Machine Learning Research, Lille, France, 07-09 Jul 2015. PMLR.
[90] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929-1999 1958, 2014.
[91] Viswanath Sivakumar, Jeffrey Seely, Alan Du, Sean R Bittner, Adam Berenzweig, Anuoluwapo Bolarinwa, Alexandre Gramfort, and Michael Mandel. emg2qwerty: A large dataset with baselines for touch typing using surface electromyography. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
[92] A Paszke al. Pytorch: An imperative style, perfopeeformance deep learning library. In Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019.
[93] Gregory Lee, Ralf Gommers, Filip Waselewski, Kai Wohlfahrt, and Aaron O'Leary. Py- Wavelets: A Python package for wavelet analysis. Journal of Open Source Software, April 2019.
16

---

[94] Jaemyung Yu, Jachyun Choi, Dong-Jae Lee, HyeongGwon Hong, and Junmo Kim. Self- supervised transformation learning for equivariant representations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[95] T. Anderson Keller, Xavier Suau, and Luca Zappella. Homomorphic self-supervised learning. Transactions on Machine Learning Research, 2023.
[96] Rumen Dangovski, Li Jing. Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. Equivariant self-supervised learning: Encouraging equivariance in representations. In International Conference on Learning Representations, 2022.
[97] Berken Utku Demirel and Christian Holz. Shifting the paradigm: diffeomorphism between time series data manifolds for achieving shift- infttiinvariancy in deep learning. In The Thirteenth International Conference on Learning Representations, 2025.
[98] Akira Hasegawa, Kazuyoshi Itoh, and Yoshiki Ichioka. Generalization of shift invariant neural networks: Image processing of corneal endothelium. Neural Networks, 9()):45-336, 1996.
[99] Namkyeong Lee, Junseok Lee, and Chanyoung Park. Augmentation-frre self-supervised learning on graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7):7372- 7380, Jun. 2022.
[100] Haifeng Li, Jun Cao, Jiawei Zhu, Qinyao Luo, Silu He, and Xuying Wang. Augmentation-free graph contrastive learning of invarian-diicrimiinatve representations. IEEE Transactions on Neural Networks and Learning Systems, 35 2024.
17

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have provided the empirical results with extensive ablations to show the contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

Justification:
Our theoretical results are given in Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided the implementation details the main manuscript Sec- tion 3.3 and Appendix E.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
For double-blind review, we include our code in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeeee //nips.com htte://lllllliiiiee////eeeaaaass ssiniiiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We have provided details in Appendix E. and E.2.2
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the standard deviation across three runs with different random seeds to reflect the variability and statistical reliability of our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided the computer resources in Appendix E
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http::///aaappp...oooooo//ooottr htw.biiietthhcccciideeeeeee
Answer: [Yes]
Justification: The research was conducted in accordance with the NeurlPS Code of Ethics, and no ethical concerns or violations were identified.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification; This work proposes general self ceff-suprviise learning framework for temporal data. It does not target specific application area, and as such, it does not have direct or immediate societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not include any pretrained language models, image generators or scraped datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have added the relevant references in the appropriate sections of the manuscript.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
22

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We release the source code accompanying method as part of the supple- mentary material under a CC BY-NC-SA: MuributiionNonoommnn Corammmria--S-aarAAAiii license. The code is documented and sufficient to reproduce the results presented in the paper. No new datasets or personally identifiable data are introduced.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does involve any crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our work does not involve any crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: No large language models (LLMs) were used in the development or implemen- tation of the core methods in this research. Any LLM use was limited to minor writing or editing support and did not impact the scientific contributions
paper.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy for what should or should be described.
24

---

Appendix
A
Theoretical Analysis
Here, present complete proofs of our theoretical study, starting with notations. We assume all the samples are absolutely summable, and finite.
A.1
Representations and Notations
A.1.1 Orthonormal Bases and Overcomplete Frames
An orthonormal basis in Hilbert space provides complete set of mutually orthogonal unit-norm vectors. The Fourier transform forms such a basis for square-integrable signals over finite intervals. For discrete signal x of length L, the discrete Fourier transform (DFT) is defined as:
L-1 Σ
(x)
x(n) e - Am
= VL n=U
where & indexes the discrete frequencies. this normalized form, the Fourier basis satisfies:
Σ
||
making it tight frame (also known Parseval frame (75). A frame can be generalization of basis that allows for redundancy. A set {Ør} C H is a frame for Hilbert space if:
A||x||2
Σ (x,
for all æ H and constants 0 < < B co. Frames provide stability and flexibility, especially for
non-stationary or noisy signals.
The Gabor wavelet transform is example of such redundant frame. It uses Gaussian modulated Gaussian-modulated si- nusoids to achieve localized tmme-frequency decomposition, trading off time and frequency resolution optimally (76]. The discrete Gabor wavelet transform for signal is defined in below.
(x
Σ
x
Va
a
where controls the scale and controls the translation (time shift); use 48 log-spaced scales and apply shifts at single time-step intervals. The detailed implementations for the wavelet calculations given in Appendix E.3.3 The Gabor wavelet (t) defined as:
(1l) = 25 (
where a controls the width of the Gaussian envelope and & the center frequeency. The Gabor wavelet captures localized oscillations and is well-suited for analyzing transient non-stationary features in wide-range of signals. In our work, the Fourier transform provides a global view of signal frequency content, while the Gabor wavelet transform enables fineggrined, localized analysis. Combining these complementary perspectives improves the expressiveness the learned representations.
The complete list of notations used throughout this manuscript is provided in Table A.1.3
A.1.2 Manifold
In our method, we define each latent representation space manifold M. similar to the manifold hypothesis [77 78), which states that high-dimensional data often lies on diwdeeennional structures
25

---

within the ambient space. Since we generate three different latent representations (from time, Fourier, and wavelet domains), we consider collection of manifolds, each corresponding transformation. In our implementation, each manifold is assigned a fixed latent dimension of 128, resulting total latent dimension of 384 across the three manifolds. For a fair comparison, we set the latent dimension to 384 for baseline methods, ensuring the linear classifier has the same number of parameters during fine-tuning.
specific
A.1.3 Notation List
Notation
Description
Temporal sequence represented as bold lowercase symbol
[x(
Fourier transformation of the temporal sequence with l frequencies
b)(a,))
Gabor wavelet transformation of the temporal sequence
f2()
The encoder to obtain representations for the temporal sequence
fr(.)
The encoder to obtain representations for the Fourier transformed temporal sequence
fw()
The encoder to obtain representations for the wavelet transformed temporal sequence The projector to obtain embeddings for the temporal sequence
(x(
g=(()
The projector to obtain embeddings for the Fourier transform of the temporal sequence
gw())
The projector to obtain embeddings for the wavelet transformation of the temporal sequence
(0
The representations obtained from temporal sequence, i.c., h (*) fx(x)
The representations obtained from the Fourier transformation of the temporal sequence
(W)
The representations obtained from the Gabor wavelet transformation of the temporal sequence
(0)
The embeddings of the sequence obtained from the projected representations, i.e. z (h')
(5)
The embeddings of the Fourier transformed temporal sequence
2 (W)
The embeddings of the wavelet transformed temporal sequence
6/-
The representation mapping function from time to Fourier domain
w/-
The representation mapping function from time to Wavelet domain
The embedding mapping function from time to Fourier domain
4. W
The embedding mapping function from time to Wavelet domain
(a,b)
The inner product of two vectors a and b
sim(2; : )2)))
(8)
(d))
Cosine similarity between the embeddings
Perpendicular
Temperature coefficient for NT-Xent loss
M
Manifold notation
A loss function, i.e., cross-entropy.
Table 7: Detailed list of notations used in this work
26

---

A.2 Proof for Proposition 2.2
Proposition A.1 (Angle Concentration vs. Pairwise Spread), Let h , ~ Unif(S where
h( = (F()) Although individual samples across latent spaces tend toward orthogonality, the
pairwise angular difference Dij between distinct samples can span the full range up to π.
arccos ((h h ())
π
while arccos( (h _ arccos (h (2) h ( Dij ST (7)
Proof. The first part follows the Chernoff on the spherical cap for angle concentration, Let (t) h hid Unif(Sd-1) and write = (h h )
1 Var[K]
E O,
(8)
d'
Lévy's concentration 779 80 gives, for every (0, 1),
Pr([k] > 2) S 2 exp
(d-2)e²
(9)
d
Hence & & O and, by continuity of arccos at 0,
arccos 5 & π
(10)
2
Let distinct indices + j while setting,
(ELL V := h h
101
dim V d 2--22 3d
(11)
Choose orthonormal u, uu E For any & E [0, T| define
(F) = u
hg (
cos Φ u sin u1
(12)
Then
(h h ()) = (h (1) n 1 = O.
(n( (1)) = cos Φ
(13)
Let
0/ arccos (, (,)))
0 arccos(h (F) 11) Φ.
(14)
Thus
( 500 d.
Aij
(15)
and varying & over [0, T] makes Aij sweep the full interval [0 - n. 0001 [[π,,]]] particular, or yields Agyl n.
Therefore, even though the angles from same samples concentrate at π 7/2, pairwise discrepancies reach any value up to the maximal n.
In practice, although (9) and (2) are coupled via the same loss, we conjecture that the high- dimensional geometry and symmetric repulsion of negatives make their joint distribution approximate to the independent uniforms on sd [32]. This justifies modeling both latent spaces as samples from Unif(Sd-)) when analyzing angular gaps between non-matched pairs.
27

---

A.3 Proof for Proposition 2.1
Proposition A.2. Let fä denote an optimal encoder under NT-Xent for domain E (t. (.J. If for some unintended transformation W the encoder is invariant, i.e., (W(x) fa (x), then for any anchor sample x the NT-Xent loss across domains is lower bounded by the number of negatives.
1(2: (1) (0 1(21010))
2 log(K + +1 0,
where 2 1 is the number of negatives that become near-positives due to the invariance.
Proof. For each domain E [1,F, W). fa : - Th be the encoder and Ia the projection head. We write the (unit-normalize)) embedding
(a(fa(x)) Smm!, and sim(u, v) = u'
(x) (l))(f((x))ll
(16)
For anchor in domain d and its positive view in domain d' =d (same underlying sample), the NT-Xent loss
exp (sim(z (1)))())))) (x), 2(d (x))
la (x) = log exp (sim(2(4)x)) (x), (())
exp 2()) (17)
with temperature r > 0. The full instance-discrimnnatie objective LID (Equation) 3 Ea over all ordered domain pairs and batch samples.
averages/sums
Suppose there exists an unintended transformation W and domain E (tFF,W) such that
fā(wx) = (x)
(18)
In batch, let 8 c {k + i} be indices near-positive negatives created by the invariance:
sim 2 (4) (1) (xx), 2 (dd') (xx)) 1 - for all & S,
(19)
with [S] = K 2 and some [0, 1). In the exact invariance case, 0.
Per-pair lower bound. Fix (d, d') and anchor sample We can define the positive similarity
between embeddings Spos sim(22 (d) 2 (d))). If we split the negative set into S (the K near-
positives) and R (the rest). From 17 and (99 we can write the loss function as in Equation 20
E /T
la.dr (x) - -log
log
efpea/t + EkES (z).)) T/T + ETER esim(()/T
@ppontt
K Kell-)/f
(20)
Factor e'cca/T from the denominator,
log 1 K 1-8-8pos
ls. (x) 2 log K
1-5-spax
At optimum, Spos = (or Spos 2 1 - in the approximate case). Since the RHS is decreasing in
Spos. the weakest bound (i.e., smallest lower bound that still holds) is obtained at Spoe 1, giving
(x,a 2 log (1+ KeKKee Ke-
(21)
In particular, for exact near weartpositives (6 = 0), la.a' 2 log(K + 1).
As is standard, assume that the NT-Xent optimum positive pairs align B1 32), i.c., sim(2(d)(xt (x)) 1. To extend this proof, we have also provided ers the approximate case. If spos 2 - E for some small E 2 0, then
which also cov-
la (x) 2 log (+Ke 2 log (1 + Ke (e-
because 2 0. Thus, Equation 21 still holds.
28

---

The bound in Equation 21 is tight when Spoe and exactly K negatives have similarity - 6 while all others contribute negligibly, so the denominator equals e//r Ke(1-5)/r
Extending to the full objective, recall that LID (Eq. B sums/averages la,a over all ordered domain
pairs (d, d') with d & d'. For a fixed anchor x, if invariance induces Ka, near-positive negatives with parameter da,d' for pair (d, d'), then applying Equation 21 each pair and summing gives
Σ Cl 2 d'dd
Σ d'ded
log + Ka e ou.wi)))
(22)
and by linearity of expectation,
LID
2 Ex
Σ digdt'
log + Ka e- (a
(23)
Asymptotically, for fixed T and bounded 88,d', each affected pair contributes (logg( + Ka,, Olog Ka,r)) If nonzero fraction p of batch of size B yields near-positives per affected pair (Ka,d' ((B),, then la,a (x) S?llg B) and consequently LTD Ω Eaya log B over those pairs. Thus any unintended invariance that produces even a single near-por near-positive per ordered pair enforces nontrivial lower bound; if collisions scale with batch size, the bound grows at least logarithmically in B.
Unintended invariances therefore inflate the NT-Xent denominator through near- sear-positive negatives, yielding the lower bound in Equation 21 We complete the proof by showing the objective cannot be minimized without bound when such invariances hold.
29

---

B Algorithm
In this section, we present the pseudocode for our method during pre-training and inference. Algo- rithm [ describes the training procedure, which takes sample, x, and model components as inputs, and outputs the trained encoder and latent space mappers.
Algorithm 1 Pre-training algorithm for the proposed method
1: Input: x, Fx. Wx. and the required models, i.e., fx(-)
FF, 2: Output: fx(),, " //-
The output of the pre-training is the single encoder with mappers
3: h fx(x). h)) ) = fx(Fx), (( = fw((.)
Obtain representations for each input
4: x =0x(h()) (5) = (h(h x x (w))
D Obtain embeddings for cach representation
5: LID Ed,d' Zे ΣN difid'
[( [(=([-) (1=k-1: = 1) {(2 = Instance discrimination loss
6: Rea
(F) -b-- (2) =ex (--
Obtain the estimated embeddings for both transformations
7: Eman N 1 (2(t) 2 ||, # - (z()) =w
Mapping loss for embeddings
Freeze (fx, fr, fw]: Omit, fox, ow, ow, th= +1++M Train (0h- "0
8: h (0) = fx(x), (F) fF((ss) h()) fw(W))
Obtain representations using trained models
9: h (F) 1// -/ (1( (1). (M = (1(
Obtain estimated representations for transformations
10: Cmap = 1/v |0
F (1(0) 13 ||, + N 1a (n))) 1 s0ml,, Mapping loss for representations
11: Return: fx(-), $ Φ/-
Algorithm 2 outlines the inference process, using only the main encoder and lightweight mappers. After applying the mappers, we concatenate the resulting representations for linear probing. During linear probing, both the main encoder and the mappers are kept frozen.
Algorithm 2 The proposed method for inference
1: Input: x, fx()). ",,",, and &
2: Output: h
The output of the pre-training is the representations
3: (0)) = fx(x)
Obtain representations for inputs
4: M (T) = T IJ (H), h = =====>/-,0 -t- (1)
Ohtain the estimated representations in other domains b Concatenate fcatures from all domains
5: h (h h h
(W)
6: Return: h
30

---

C Pairwise Distances
In addition to angular comparisons, we report the lg distances between sample pairs in the time, Fourier, and wavelet domains. Figures | [ 7 visualize these results. If pairwise distances were preserved across latent spaces, all points would lie along the y = z line-i.e., the distance between samples and j in the time-domain latent space h (9) would match that in the transformed domain h (d),, where d E {F, W.
200 ||x
hg 00
112
Count
200
|/x
h
10
112
Count
7000
175
6000
175
150
1.50
5000
5000
125
125
4000
4000
100
1000
3000
3000
075
075
0.50
2000
0.50
2000
0.25
1000
0.25
1000
0 0.25
0.50
075 100 125 150 175
2:00
0.25
0.50
0.75 100
12s 150 175
2.00
(F) h
la
| h (30
1|2
(a) Pairwise & distance comparison between time and Fourier domain latent spaces on IEEE SPC12
(b) Pairwise & distance comparison between time and wavelet domain latent spaces on IEEE SPC12
2.00
11
h
0000
2.00 |
175
175
75
150
20
150
70
125
125-
15
100
100
15
075
075
10
0.50
0.50
5
5
0.25
0.25
025 0g0 075 1:00
125 150 175 2.00
025 oggn 0/75 100
125 1500 175 2.00
1 (7)
R
lla
||4 (W) A
(12
(c) Pairwise & distance comparison hetween time and Fourier domain latent spaces on IEEE SPC22
(d) Pairwise &2 distance comparison between time and wavelet domain latent spaces on IEFE SPC22
Figure 5: Pairwise & distance comparisons across domains and datasets for heart rate estimation.
31

---

We conducted this investigation across tasks involving both single- and multi- temporal data. It is worth noting that while some datasets exhibit closer alignment between latent spaces, we observed consistent and non-negligible deviations across all tasks, indicating that latent space geometries differ across applications.
2.00
|1m 10
1/2
Count 0119
200
110 (9)
Gount (x10))
80
60
175
50
30
160
150
1.25
125
40
100
30
1.00
30
0.75
20
20
050
0.50
10
0.25
0.5
0
025 0.50
075 150 125 175 200
0.25
0.50
075 150
125
175
2:00
| k 112
||b h l1
(W)
09)
(a) Pairwise& & distance comparison between time and Fourier domain latent spaces on HHAR
(b) Pairwise &a distance comparison between time and wavelet domain latent spaces on HHAR
2.00
|1
(6
h
(4)
112
Count 6:30
2.00
1
(9)
h
112
Count bao
175
175
175-
475
150
150
150
125
125-
125
100
100
0.75
75
0.75
0.50
50
0.50
50
025
0.5
5 025 dago
075 100
1hs 150 175
200
025 050
075 16o
125
160
175
2.00
1 A 11
|| [16
(c) Pairwise & distance comparison between time and Fourier domain latent spaces on USC
(d) Pairwise & distance comparison between time and wavelet domain latent spaces on USC
Figure 6: Pairwise & distance comparisons across domains and datasets for activity recognition.
32

---

16
h
14
1/2
Count 1600P!
200
1
h
10
Count b000
70
175
150
150
125
30
125
40
100
100
20
0.55
0.50
0.50
10
0.55
025
0
0.25 0.00 075 100
125 150 175
2:00
0.25 0.00 075 100
125 150 175 2:00
|| h 4 l12
DM1 |/
h (1)
2
(a) Pairwise Eg distance comparison between time and Fourier domain latent spaces on CPSC
(b) Pairwise & distance comparison between time and wavelet domain latent spaces CPSC
1h
A
10
1/
Count 1x10
15
2.00
16
60
h
11
Count 6:10
175
175
150
150
-00
125
10
125
15
1.00
1.00
0.5
0.75
0.5
0.50
0.5
025
0.25
0.50
075 100
125 250 175 200
025 olso 075 100
1ás
1ão 175
200
||A h (77 lla
||4 (W)
A
lla
(c) Pairwise E distance comparison between time and Fourier domain latent spaces Sleep
(d) Pairwise & distance comparison between time and wavelet domain latent spaces Sleep
Figure 7: Pairwise & distance comparisons across domains and datasets for cardiovascular disease and sleep classification.
These figures reveal substantial deviations from y = x line, indicating that distances between samples (i * j) in the latent space vary across domains. Importantly, these discrepancies consistent despite all encoders being trained jointly with the same objective. These observations further support our motivation for leveraging multiple latent spaces to capture complementary structure in the data.
33

---

D Additional Experiments
D.1 Embedding mappers
We have employed embedding space mappers ($1-+d) ) after projection layers to improve predictability across latent spaces and reduce estimation error. In this section, we have presented the ablation experiments regarding the performance when the embedding mappers are excluded (w/o th from
our method. The results are given in Tables 8 9
and 10
Table 8: Further ablation on proposed method in PPG datasets for HR estimation
Method
IEEE SPC12
TEFE SPC22
DaliAppo
MAE
RMSE
Pt
P1
MAB :
RMSE
P7
Ours
8.8410.50
04.37:095
82.67:1.30
14.06:1.00
21.48:201
54.881189
9.1360220
16.9220.56
63 63.722006
w/o 4/-ed
9.87 (+1.08)
1585
79.78
(5.17)
24.75 (-2.23
53.19
(x0.00)
17.68 (+0.76)
(
Lin. 6%
9.77 (40.08g
16.27 (+1.90)
(-3.00)
15.61 (+1.55)
(-2.99)
(4.70
9:23 (40.
16.98 (-0.06)
63.67 (-005)
Non. Lin. 4/
10.11 (+1.27)
16.06 (11.9)
(-3.0))
(+.62
24.65 (-3.17)
53,03
(.92())1))
16.52 (-0.00)
64.29 (60.57)
Table 9: Further ablation on proposed method in IMU datasets for Activity and Step
Method
HHAR
USC
Clemson
W-F1
F
W-FI
F1
MAPEL
MAE
RMSE
Ours
70.67:00.66
67.74:0022
68.79-0025
$2:21
48 1::2152
48.22e1111
5.161004
2,50
4.65:10.5
w/o Wirnn
67.17
67.03 (4071)
67.01
52.09
49.31 (40.7)
49.25
5.31 (40.15)
2.57 (40.0)
4.76 (40.11)
Lin. 03
(2.54)
68:23 (+0.49)
67.08 (1171)
51.13 (108)
47.55 c (.09)
5.79 (-003)
2,55 (+0.05)
4.67 (+0.00
Non. 6//-
71.13
69.19 (+145)
70.35
53.20 (40
48.27 (-0.7)
(40.79)
6.40
(-0.6)
(40.8)
Table 10: Further ablation on proposed method in ECG and EEG datasets for CVD and Sleep
Method
Chopman
CPSC
Sleep
AUC
Acc
AUC
1
Kappa (4) 1
Ours
k7.21 $0.30
96.50
20:21
85.30 20.98
$2. 101000
87.01 10
$1.26 1118
77.30 =1.04
68.05 30.85
69 16 21.33 70.23 (+L.1)
w/o 4/-ed
86.43
96.31
84,40 (0.90)
52.03
(+0.9))
51,10 0.10)
(10966)
68.04 (0.01)
Lin. 4y
86.35 (-0.56)
96.06
(40444)
84.26 (-1.04)
(-.58
86.55 (-0-46)
48.78 (-.78(-1448)
(-1
(40.5)
68.61 (-0.56)
70.02 (-0.96)
Non. Lin. of
90.91 (V3/70)
97.96
89,76 (-4.46)
56.30 (-4.20)
88,09 (+1.00)
53.44 (32.18)
79.93 (+2.66)
70.27 (+2:22)
73.10 (-3.44
As can be seen these tables, removing the embedding space mappers (#L-+ consistently leads to decreased performance across all datasets. In few cases where the performance appears slightly better, the improvement remains within the range of standard deviation and is not statistically significant. These results support the role of embedding-level mapping in enhancing performance.
D.2 Latent space mappers
When using latent space mappers, we employed lightweight nonlinear convolutional networks (see Section E.3 for architectural details). Here, we report results from replacing the mapper with either a simple linear layer (Lin. - or nonlinear two-layer multilayer perceptron (MLP dd) We present the results in Tables
9
and I0
As shown in the tables, replacing the convolutional mapper with linear layer results in noticeable performance drop, suggesting that the relationships between latent spaces are too complex for simple linear mappings. Although non-linear MLP improves performance structure significantly increases the parameter count. To ensure a fair comparison with prior work while avoiding added computational overhead, we use our lightweight convolutional architecture.
some datasets, its two-layer
D.3
Using all encoders instead of mappers
We evaluate variant of our method that directly uses the original representations from all encoders. rather than mapping the time-domain representations into other domains. This setup yields 4-5% performance improvement over our default approach. However, significantly increases the number of parameters at inference time, making it less efficient. These results suggest promising direction for future work: leveraging principled transformations can substantially boost performance, though careful trade-off's with inference cost should also be considered.
34

---

D.4 Cross domain results
Cross-domain evaluations are commonly used to assess the generalization of self supervieed learning methods across datasets [47] [88.. Following this practice, we pretrain each model on a dataset and fine tune on another from a different domain, following the setup [48]. Both supervised models in our experiments (FCN and ResNet) are initialized randomly and trained. We present the result in Tables 11 and 12 Unlike linear probing, where only the linear classifier is trained with limited data, fine-tuning updates the entire encoder. To ensure consistency, we use the same limited data size as in linear probing, avoiding settings that resemble supervised training with abundant labeled data.
Table 11: Performance comparison of methods for Activity and Step in cross domain settings
HHAR
USC
Clemson
Method
Ace
W-FI
F1
Ace
W-FI
FL
MAPE
MAE
RMSE 1
Supervised
FCN
14.21=1.56
7288+2206
71,58:181
48.87=077
46:02:0.55
45.33+0052
5.02=0.26
2.86+0.15
4,05=0.13
ResNet
69.85:2.32
686112281
67,29:22552
52,17=1.22
49.38::0.44
1.001:122
6,55:2237
3.781144
5.04::144
Self- Sopervieed
SimCLR (in)
40.55=00.62
39.21:00.4
39.4110.66
29.16_00.69
29.02:0:87
28.9910.79
4.36.10.13
6.30=024
SimCLR (cross)
41.25=00.2
39.75
40.0550006
29.75=00.69
29.55
29.45:0779
8.55-00222
6.18=024
BYOL. (in)
49.64-2.44
48.63+2.77
48.02=2.99 59
28.40=1.23
28.23+142
28.23+0:66
9.35=0.19
4.72+0012
6.79=024
BYOL, (cress
50.10=2.48
49.05+2.75
48.60 59
28.95-1.23
28.70+142
28.65+0966
9.20=0.19
4.62+0.12
6.68-024
VICReg (in)
38.05:3.01
37.112:2.66
37.38:3.02
23.75=1.00 1.00
1.
22.92:1211
10.87:061
5:47:10.35
7,78=014
VICReg (cross)
38.55:13.01
37.5552.66
37.85
24.30_1100 1.00
23.7011103
23.4011:21
10.70.0061
5.35.10.35
7.65:00.14
Twins
38.97.00.65
37.7511.00
38.2111.12
27.21
26.81110200
26.2510.77
9.8910035
4.9510.11
7.03-021
Twins (cross)
39.60=0065
38.25::1000
38.8521.12
27.85
27.30...200
26.80.0077
9.7220:35
4.85
6.92=021
CLIP (in)
43.78=0099
42.53=00:0
43.07=005
25.55=0.63
25.78=1.25
25.1770075
8.52=0.46
4.26#0023
6.73=m63
CLIP (cross)
44.25=0.99
42.95=0:90
43.55=009
26.10=0.63
26.25+1.25
25.65+0.75
8.35=0.46
4.15#00.3
6.60=063
TS-TCC (in)
68,56=1119
66:90:11.22
68 10:00:3300
33.61=0722
33.11=1000
33.91
5.61:015
2:70+0006
4.69=033
TS-TCC (cross)
69.10:1.19
67.35:1.22
68 18.60:1130
34.15:0.72
33.60:1.00 1.09
34.3010.79
5.50:0.15
266:10066
4.60:0038
SimMTM (in)
1478.0062
42.4810.37
43.6010.62
22.34_.028
25.6810.41
29.72.1.778
8.77.0.18
4.61.10.32
6.90
SimMTM (cress)
45.52.0054
43.0110441
44.1010.8
22.90_.0228
26.10.10.41
30.0511.77
4.50.10.32
6.78-0.18
TF-C (in)
31.13=0.42
30.57=00.0
31.00=0.51
30.78=0.39
28. 26:10:5
30.82:141
12.47=0722
6.31
7,93=0330
TP. (cross)
31,70=0.42
31.05#0.40
31.55e0.51
31.30=0.39
28.75e00.3
31.25+141
12.25-0.72
6.15+00.7
7.75=0330
TS2Vec (in)
67.13=0.11
65.56+00.1
64, 23
35.40-0.96
32,17+1.26
35.771122
5,92-093
3.01+0026
5.02-0442
TS2Vec (cross)
67.70:0.11
66.05:0.21
64,70:0221
36.00=0:96
32,75+1.26
3608#112
5.85:0093
2.95:0028
4.95=042
70.67_00.06
67.74110.29
68.7910025
48 18.6411552
48.22.1.111
5.16.0.44
2.50.10.13
4.65..0.45
(cross)
71.20.00.06
68.20.10.29
69.30.0025
53.10..1.09
49.2011.52
48.8011.11
5.05.0..4
2.40.10.13
4.55..0.45
For experiments regarding the activity recognition and step counting tasks (Table 11), we have first used electrocardiogram signals to pretrain models and then fine tune on the specific dataset. Each method includes two rows: the first shows in-domain results (also reported the main manuscript), and the second shows cross-domain performance. Results are organized top-to-bottom per method to facilitate easier comparison.
For the cardiovascular disease classification (CVD) task (Table 12 we pretrain models on EEG signals and fine-tune them ECG datasets. For sleep stage classifiction, we pretrain on both ECG datasets and tne-tune on a small subset of the EEG dataset. The results are summarized in Table [2
35

---

Table 12: Performance comparison of methods for CVD and Sleep in cross domain settings
Chapman
CPSC
Sleep
Mothod
Ace
A
FI
Acc
AUC
F1
Acc 1
W-F1 1
Kappa T
Sapervierd
84.63=2.13
95.40+0557
82.41=2.40
63.64+1112
91.30=0022
60.43+104
71.98=0.66
63.33a084
62.01=1.30
16:0.6
98,59#005
92.02-0.42
75.21+1773
95.02-003
71.70+190
76.94=037
67.52+1.95
69.14+061
SimCLR (in)
75.28=0057
93,55+0025
74.04=0.50
50.10:041
87.20=047
50.10
72.45a2322
58.93+1.59
59,47=3.20
SimCLR (cross)
76.58=0.55
93.92+0022
05.39-0.45
51.03+0440
87,48-006
50.25+0:22
73.564225
19.84
60,41+3.10
BYOL (in)
27.08
93.7410018
0..5
52.9010::0
87.05
50.89
70.77..0.77
58.23
55,90:11.00
BYOL (cross)
77. 00:38
94.81:10:16
7..18-0042
52.95::0.28
87:32:0.00
51.35:0.36
71:82:0:25
59.29:0.52
56.80:11115
VICReg (in)
70. 10:1.00
89.35+0:3
67.84-1.79
46.21+11.9
84.70-0.50
40.51
68.72=1.08
57.24+1.04
57,13+142
VICReg (cruss)
1.33
$9.781090
68.75-1..70
47.3811126
85.09-0.47
43:25:0091
69.75.:1.00
58.10
58.0611.38
Barkow Twins (in)
12.43.1.44
91.1711060
1.54
48.67.10.51
03,78
44:57.0.53
70.1010.22
57.7210081
57,88
Barlow Twins (cross)
73.21
91,6110.57
1..38
19.32.10.18
86.1110.18
45.33.10.00
71.1310.00
58.5210.78
58.71
CLIP (in)
02.9800.96
95:15:042
81.00=1003
50,01:0:89
06:40:0.0
47.99:0.89
73.16:0811
62.0620.91
63,75
CLIP (cross)
83.69=0.94
95.39
81.85=1001
51.13+035
86.61=031
48.88+006
74.25=00.7
63.10=009
64.80=1.20
TS-ICC (in)
73.50
90.6510.07
01,50
51.5911122
86.32-0.16
50.2711.32
62.8011.133
52.4321.05
48.9811668
TS-TOC (cross)
74.06:0.53
91,00++0006
02.10-0.55
52.35
86,51=0.15
51.17:1.29
63,78:1.10
53.42$1.03
19.97:1.65
SimMTM (in)
84.29=1.29
95.87+0.18
83.31=1.25
51,70+0.23
87.08-0.21
50,62:0.55
74.69=1 184
63.53+1.21
65.31+2.76
SimMTM (cross)
85.21=1.25
96.03+0.17
84.20=1.20
52,34+0.22
87.33-0.30
51.5240.53
15.88+1.78
64.00+1.18
66.31+2.71
TF-C (in)
35.84
96.10=0::0
84.71=0.40
47.8644009
86.27=005
45.42:0066
64.50s1.50
56.77a221
52.61=2411
(cross)
80.67
96.281009
85.43.00.38
19.311106
86.55_0:04
16.2111064
57.8612.17
53.72.2.35
TS2Vec (in)
78.87:1.00
04123::0:44
81.32:0.47
45.731088
85.49=0.37
46.57:11.10
65.71::106
55.3241.77
56.81:1150
TS2Vec (cross)
80.03=1.00
90.48
82.13
49.81
85.76=0.35
47.28=1.07
66.94s1.04
56.34:1.74
57.84=1.87
Ours (in)
87.21-0.80
96.50++021
85.30-0.95
52.10++0:0
87.11-0.40
$1.26+1.18
72.30++104
68.05-0.6
69.16++132
Ours (cross)
88.03
96.78:10:00
86.01-0,95
53.111087
88.03-0.45
52.02:1.14
78.21
69.01
70.15:1.28
our method demonstrates strong generalization across domains, achieving the best performance in out of 18 metrics across datasets. This is particularly important because our approach employs specific transformations, such as Fourier and wavelet projections, which are inherently sensitive to the signal's 's sampling rate.
As shown in Tables 11 and 12
Despite variations in sampling rates between source and target datasets, our method consistently outperforms others. While similar resilience to sampling rate differences has been observed in prior works that employs frequency -omain representations (7). we hypothesize that the adaptability of our method stems from fine-tuning both the main encoder and the latent space mappers, which enables adaptation to cross-domain evaluation similar to previous methods.
D.5 Comparison with larger models
In our experiments, we aimed to ensure fair comparisons across methods by controlling for model capacity and training data. Therefore, we used the same or closely matched backbone architectures with the same amount of training data across methods, allowing us to isolate and evaluate the effect of the representation learning strategy itself rather than differences in model data scale.
Foundation models, while powerful, involve substantially larger architectures and extensive pretrain- ing massive datasets. As such, direct comparison in experiments is not straightforward. Nonetheless, for completeness, we conducted experiments with CBraMod [8]. one of the most recent of these foundation models for brain signals. Specifically, weights and evaluated on the Sleep-EDF dataset. Features were extracted using the pretrained model, followed by either (i) training only linear classifier or (ii) fine-tuning the full network the same training split. We have reported the results in Table 13 CBraMod employs a transformer layer
used the publicly available pretrained
Table 13: Comparison with the pretrained CBraMod on Sleep-EDF
Method
Linear Evaluation
Fine-tuning
CBraMod (pretrained)
65.03 + 0.63
72.98 + 0.47
Ours
69.19 + 1.32
71.13 + 0.84
36

---

for sequence-to seteeneeet--squeeeee sleep staging, whereas we applied a linear classifier for consistency with other baselines. It also worth noting that CBraMod was not originally trained on single-channel EEG, which may partly explain the lower evaluation performance. While fine-tuning increases the performance of CBraMod more compared to ours, is important to note that it has larger parameter count than our backbone, giving it greater capacity to adapt when all weights are updated according to the task. This makes direct comparison under fine-tuning less informative, as the performance difference can largely reflect model size rather than the quality of the learned representations.
By contrast, linear evaluation offers a fair basis for comparison, only a lightweight classifier is trained. Our results show that, despite its smaller design, our approach remains competitive.
D.6 Comparison with heavy specialized augmentations
To further assess the performance of method, we compared it against augmentation-hent baselines, including Times [82] and Finding Order in Chaos [5], which introduces frequency domain based mixup. Table reports results. For Finding Order in Chaos (FOC), we reproduced the results on SPC12 (where dataset splits differ slightly from our setup), and report the original numbers from the paper for SPC22 and DaLiA. Our method consistently outperforms augmentation-hean
baselines, with improvements of approximately 8-9% across all datasets. These results highlight that
Table 14: Performance comparison of our method with cetnneentation-haayy baselines
Method
IEEE SPC12
TEEE SPC22
DaLiA
MAB
RMSB J
MAE 1
RMSE
Pt
MAE L
RMSE J
Pt
Supervised
TCN
15,1340.50
2166-048
52.09=5.43
16.5740911
26. 26.20#0.60
55,98=0.75
12.45=0112
18.35#0.24
56.98+0.78
ResNett
7.08=020
13.60-0.38
79.60+1.10
9.90+147
16.67+1.60
67.58-2298
5.50+0.05
10.84+0.03
10+0000
Self- Supervised
SimCLR
12:42:0.05
20.96=030
73.6220.52
16.41
22.6210.39
52.16:1112
16.88=0.19
22.64110:22
56.3710.21
TimesURL
13.40=044
19.85-061
75.13=1:22
17.92+0.88
24.73+1.22
50.10=1:97
14.62=031
21.57+0.44
60.17+0.11
11.15.0.34
17,89_083
12.2510.47
10.57-055
20.37.0.73
62.83110.22
Ours
8.84=0.50
14.37=095
82.67=1.30
14.06
21.48:22.01
54.88=1.89
9.13=0020
16.92=0.36
63.72=0.066
our approach achieves stronger performance compared to methods based on heavy augmentations. While augmentations can be beneficial for certain datasets, their generalization is often limited due to the diverse characteristics of signals and the varying invariances required across tasks.
D.7 Significance analysis
In our experiments, some methods achieve strong results specific datasets; however, our approach consistently ranks highest across tasks. For instance, TF-C performs well on ECG (ranking second to ours in Table 3). but its accuracy falls to about 30% on the activity recognition (Table 2
We evaluate significance across 27 tasks (3 datasets metries for HR, 3x3 for Activity/Step, 3x3 for CVD/Sleep). After ranking, run the Friedman test and apply Nemenyi post-hoc comparisons. The critical difference diagram is given in Figure g
Overall (27 tasks) Critical Difference (Nemenyi. alpha-o.05)
5
Ours m3
(B) VICReg
1S-10C
(7) Barlow Twins
TS
TWTF-C
(53)
66 BYOL
CUIP6
SimMTM
Figure 8: The critical diagram for all tasks. Numbers show average ranks (lower is better); horizontal bars connect methods not significantly different. Our method achieves the best average rank.
37

---

E
Experiments
Here, we give a detailed description of datasets, architectures, metries, and training details for our experiments. We performed our experiments on NVIDIA GeForce RTX 4090 GPUs, involving training with three random seeds for all datasets, totaling approximately 680 GPU hours including ab- lation. All experiments fit within 24 GB of GPU memory, without requiring excessive computational resources. We reported the mean of three runs with the standard deviation.
E.1 Datasets
In this section, we give details about the datasets that are used during our experiments.
E.1.1 Human Activity Recognition and Step counting
The Clemson dataset has 30 participants (15 males, 15 females), where each participant wore three Shimmer3 sensors. We used the IMU sensor readings from non-dominant wrists predict step count where each sensor recorded accelerometer and gyroscope data at 15 Hz. We calculated the total magnitude of the accelerometer and fed it to the model as a pre-pre pre-processing without any filtering. We used window lengths of 32 seconds without an overlap in the regular walking setting. We conducted 10-fold validaani,,,,,,, with each fold consisting of 3 subjects. Pre-training performed using 9 folds, with the remaining fold held out for testing. The test fold is not used during either pre-training or linear fine-tuning.
Clemson
HHAR Heterogeneity Dataset for Human Activity Recognition (HHAR) is collected by nine subjects within an age range of 25 to 30 performing six daily living activities with eight differ- caailphoons--hhhuuuu
well, use data from smartphones- were kept in a tight pouch and carried by the users around their waists [37]. Subjects then perform 6 activities: "bike", 'sit', 'stairs down', 'stairs up', 'stand', and 'walk'. Due to variant sampling frequencies of smart devices used in HHAR dataset, we downsample the readings to 50 Hz and apply 100 (two seconds) and 50 as sliding window length with step size, the windows normalized to zero mean with unit standard deviation. We used the first four subjects (i.e., a, b, c, d) as source domains.
HHAR includes data from smartwatches
USC USC human activity dataset (USC-HAD) is composed of 14 subjects (7 male, 7 female, aged 21 49 with a mean of 30.1) executing 12 activities with a sensor on the front right hip. The data dimension is six (3-axis accelerometer, 3-axis gyroscope) and the sample rate is 100 Hz. 12 activities include walking forward, walking left, walking right, walking upstairs, walking downstairs, running forward, jumping up, sitting, standing, sleeping, elevator up, and elevator down. We used the pre-pro ree-processing technique with a smaller window size such that the input contains six channels with 100 features (it is sampled in a sliding window of I second and 50% overlap, resulting in 100 features for each window). The same normalization is also applied windows before feeding to models. We used the same setup with UCIHAR while source subjects are chosen as the last four this time.
E.1.2 Heart Rate Prediction
IEEE SPC This competition provided a training dataset of 12 subjects (SPC12) and a test dataset of 10 subjects (36. The IEEE SPC dataset overall has 22 recordings of 22 subjects, ages ranging from 18 to 58 performing three different activities [3 Each recording has sampled data from three accelerometer signals and two PPG signals along with the sampled ECG data and the sampling frequency is 125 Hz. All these recordings were recorded from the wearable device placed on the wrist of each individual. All recordings were captured with 2-channel pulse oximeter with green LEDs, tri-axial accelerometer, and chest ECG for the ground-truth HR estimation. During our experiments, we used PPG channels. We choose the first five subjects of SPC12 as source domains similar to activity recognition setup while the last six subjects of SPC22 are used for source domains to prevent overlapping subjects with SPC12.
DaLiA PPG dataset for motion compensation and heart rate estimation in Daily Life Activities (DaliA) was recorded from 15 subjects (8 females, males, mean age of 30.6), where each recording was approximately two hours long. PPG signals were recorded while subjects went through different
38

---

daily life activities, for instance sitting, walking, driving, cycling, working, and so on. PPG signals were recorded sampling rate 64 Hz. The first five subjects are used as source domains.
All PPG datasets are standardized as follows. Initially, a fourth-order Butterworth bandpass filter with frequency range of 0.5 Hz is applied PPG signals. Subsequently, sliding window of 8 seconds with 2-second shifts is employed for segmentation, followed by z-score normalization of each segment. Lastly, the signal is resampled frequency of 25 Hz for each segment.
E.1.3
Cardiovascular disease (CVD) classification
CPSC China Physiological Signal Challenge 2018 (CPSC2018), held during the 7th International Conference Biomedical Engineering and Biotechnology in Nanjing, China. This dataset consists of 6,877 (male: 3,699; female: 3,178) and 12 lead ECG recordings lasting from seconds to 60 seconds with Hz.. We use the original labelling 39 with one normal and eight abnormal types as follows: atrial fibrillation, first-degree atrioventricular block, left bundle branch block, right bundle branch block, premature atrial contraction, premature ventricular contraction, ST-segment depression, ST- elevated. We resampled recordings to 100 Hz and excluded recordings of less than 10 seconds.
Chapman Chapman University, Shaoxing People's Hospital (Chapman) ECG dataset which pro- vides 12-lead ECG with 10 seconds of sampling rate of 500 Hz. The recordings are downsampled to 100 Hz, resulting in each ECG frame consisting of 1000 samples. The labeling setup follows the same approach as in [40] with four classes: atrial fibrillation, GSVT, sudden bradycardia, and sinus rhythm. The ECG frames are normalized to have a mean of 0 and scaled to have standard deviation of 1. We split the dataset
60-20% for training and testing suggested 40)
We choose leads I, II, III, and V2 during our experiments for both ECG datasets. We followed a similar setup with prior works [84] and considered each dataset as a single domain different from previous tasks. The fine-tuning of the linear layer, which is added to the frozen pre-trained encoder, performed with 80% of the same domain.
E.1.4 Sleep stage classification
We used the Sleep-EDF dataset which has five classes: wake (W), three different non-rapid eye movements (N1, N2, N3), and rapid eye movement (REM). The dataset includes whole-night PSG sleep recordings, where we used single EEG channel (i.e., Fpz-Cz) with a sampling rate of 100 Hz. We followed the same data split TSTCC [44), with no additional pre-processing. The only difference is that, for linear probing, we used a random 10% subset of the unseen data rather than the full set, reflecting our setup where labeled data significantly smaller than unlabeled data.
E.2
Baselines
E.2.1
Supervised
We use 1D Fully Convolutional Network (FCN) that processes multichannel temporal inputs. The model consists of three convolutional layers with increasing filter sizes (32, 64, and 128), each followed by max pooling operations to progressively reduce the temporal resolution. A final linear layer maps the output class logits. We chose this architecture for the supervised and self-supervised learning paradigms as it was widely used before in the literature 5
FCN
We use the same backbone in the self-supervised methods for the supervised baseline, integrating a linear layer and training the model from scratch using random initialization.
ResNet
We perform a grid search over key hyperparameters for both networks, focusing on learning rate and batch size. The learning rate initialized at le-3 and reduced by half if validation performance does not improve for 15 epochs. The batch size is fixed at 64.
39

---

E.2.2 Self-Supervised
Fundamentals
SimCLR SimCLR ① introduces a contrastive learning framework for self-supervised visual repre- sentation learning. The method relies on maximizing agreement hetween differently augmented views of the same image via a contrastive loss in the latent space. We follow the previous implementations of SimCLR for time series [ 85].
BYOL For the BYL implementation, the exponential moving average parameter is set to 0.996 where the projector size set to 128. We set the learning rate to 0.03 similar to other SSL techniques. Following the original implementation, we use a weight decay parameter of 1.5e 6.
VICReg We follow the original implementation and set the coefficients for each loss term to 25 (A), 25 (w), and I (v), corresponding to the invariance, variance, and covariance terms, respectively, Although we conducted a search for these loss term values, no performance enhancements were detected across the tasks.
f= A + (c()) + + (/((2) c(z')),
(24)
where s the mean-squared Fuclidean distance, is a hinge function on the standard deviation of the embeddings along the batch dimension, c is the covariance regularization term as the sum of the squared off-diagonal coefficients
Barlow Twins Barlow Twins 45 86 presents objective function that naturally avoids collapse
for SSL by measuring the cross-correlation matrix between the outputs two identical networks fed with augmented versions of sample, and making it close to the identity matrix as possible. This causes the embedding vectors of augmented versions of a sample to be similar, while minimizing the redundancy between the components of these vectors. Following the original implementation, applied batch normalization to the extracted embeddings and set the hyperparameter À coefficient (in Equation 25 to 0.005.
L= 11- + - of
(25)
where C' is the cross-correlation matrix computed between the two sets of normalized embeddings.
Contrastive Language-Image Pretraining (CLIP) 46 learns joint representations by aligning paired image and text embeddings through contrastive learning. To compare method with this, we adapted a similar strategy for time series by treating the time-domain signal and its Fourier- transformed version separate encoders for the time and frequency domains and maximizing the similarity between their paired embeddings. During inference, we only used time encoder with linear probing. This setup mir- rors CLIP's approach to aligning image-text pairs, but instead aligns time-frequency representations of the same signal.
CLIP
modalities. Specifically, we applied CLIP-style objective by training
40

---

Temporal sequence
TS-TCC We follow the same architecture implementation with the losses, i.e., contextual and temporal contrasting. TS-TCC proposed applying two separate augmentations, one augmentation is weak (jiter-and-scale) and the other is strong (permutaiion-and-jitee)) The authors also change the strength of the permutation window from dataset to dataset. our experiments, we first used the original augmentations for each time series task, however, we observed performance decreases depending on the signal type. We, therefore, applied the specific augmentations for each time series, where we observed general performance improvement in other SSL techniques well.
TS2Vee Vec 49 is SSL method specifically designed for time series based on contrastive (instance and temporal wise) learning in a hierarchical way over augmented context views where the context is generated by applying timestamp masking and random cropping on the input time series. Following the original framework, hidden size of 64, which has a similar number of parameters with the architectures used by other SSL methods. The batch size is set to and the number of epochs to 120, following the original manuscript. Although we experimented with larger batch sizes, as SSL methods often benefit from them, observed no performance gains.
a dilated CNN architecture with depth of 10 and
TF-C
The Time- Feme-Frequency Consistency (TF-C) method 47 introduces a self-superrised learning framework for time series data by aligning representations from both time and frequency domains using the absolute Fourier transform. TF-C employs specific augmentations in both domains, such as jittering in the time domain, and perturbations like adding removing/decreasing frequency components in the Fourier domain, to create diverse views. During inference, TF-C utilizes both the time-domain and Trequency-domaal encoders, combining their outputs form the final representation, thereby integrating information from both domains for downstream tasks. In contrast, our method avoids this by requiring only single encoder at inference. We use the original implementation provided at grsthuu.com/miis--haae hwwhhaard/C----eeraraaaaa Prertriiiii
SimMTM SimMTM 48
presents a masked modeling framework tailored for temporal data. In time series, semantic information is heavily embedded in temporal variations, and random masking may disrupt critical patterns, making reconstruction unnecessarily difficult. SimMTM mitigates this by treating masked modeling as manifold learning problem. Instead of reconstructing masked points directly from nearby unmasked values, it recovers them via weighted aggregation from multiple complementary masked sequences. We use the same ResNet backbone for SimMTM to ensure fair comparison with other SSL methods. We also experimented with the original backbone proposed in the paper but observed no performance improvement. For our evaluation, we follow the original manuscript's hyperparameter settings for the masking ratio and the number of positive (masked) series. In addition, explore the effects of training dynamics by running SimMTM with our higher batch size (1024) compared to the paper's smaller batch sizes (128-256), and we test both short training epochs (40-50 used in the original) and longer schedules, treating these part of hyperparameter search. We observed performance degradation when reducing the number of epochs, so we run all baseline methods for the same number of epochs to ensure fair comparison. We use the official implementation provided at giigtii...oom.. wow.thum/SSSMMMMMMMMMMM
41

---

E.3 Implementation Details
Here, we have provided the details of the architectures, and hyperparameters. Primarily, we used the ID ResNet [51] implementation in the supervised settings. While some alternative deep learning models can perform better specific time series tasks such as the combination of convolutional and LSTM layers [87 88 we focused on residually connected convolutional architectures as backbones for representation learning.
E.3.1 Architectures
Here, we present the details of architectures that are investigated for the performance of shift-invariant techniques. Some details that are not given in the tables are as follows. Batch normalization [89] is applied after each convolutional block. ReLU activation is employed following batch normalization, line with [50]. We also applied Dropout [90] with 0.5 after each activation and before the convolutions. Finally, a global average pooling is implemented before the linear layers.
Table 15: Model architectures used in our experiments
(a) ResNet architecture for the main encoder
(b) Architecture for W(a,b)
# Blocks
Layer
Kernel
Output
Layer
Input + Output
Description
1
Input (C,T)
(C. T)
Input
(C,
Input
1
Conv
(5, 1)
(64, T/2)
AvgPool
(/.
Conv
(5. 1)
(128, T/4)
AvgPool2
+ (C.F/4,T/4)
8
Conv
(5, 1)
(128, T/4)
Conv1
(N,F,T)
Conv block
Conv2
+ *2N, F/2,T/2)
1
Linear
(n classes.)
Concatl
Pooll
Merge
# Parameters for dataser (C.F.T)
Conv3
7 (BN.F//,T//
IEEE SPC & Dalia (C=1, T=200)
2210k
Concat2
Pool2
Merge
Chapman & CPSC (C=4, T=1000)
s 197k
Conv4
7 (4N, F/N.F/8/T/8
Clemson (C=1, T=240)
2200
FC1
(1,25) 25)
Linear (time)
HHAR (C=6, T=51)
200k
USC (C=6, T=100)
200k
FC2
+ (128)
Final output
Sleep (C=1, T=3000)
2 10k
Parameters for dataser (C,F)
IEEE SPC Dalia (C=1, F=48)
500k
Chapman & CPSC (C=4, F=48)
5000
Clemson (C=1, F=48)
5500k
HHAR (C=6, F=48)
5500
USC (C=6, F=48)
5500k
(c) Architecture for F(x)
Sleep (C=1, F=48)
5500k
Branch
Layer
Output
(d) Architecture for mappers t--+d
ConvlD
(16, T)
Amplitude
Residual
Residual
(32, T/2)
(64, T/4)
Layer
Kernel Size
Output
Size
Linear
(64)
Input (1, L)
(1,
ConvlD
(16, T)
Conv ID
(3, 1), stride=2
(64, L/2)
Phase
Residual
(32, T/2)
ReLU
(64, L/2)
Residual
(64, T/4)
Conv Transpose
(3, 1), stride=2
(1, L)
Linear
(64)
All datasets
R:500
Output
Concatenate
(128)
All datasets
R:55k
Our latent space mappers are lightweight, with each containing approximately 500 parameters, resulting total of only additional parameters during inference. This keeps the overall inference cost of our method low. Moreover, even when baseline models are scaled up to match or exceed the total parameter count of all encoders, they still fall short in performance, highlighting the effectiveness of our approach over brute-force model scaling.
42

---

E.3.2 Augmentations
We applied commonly used augmentations for each task, with details listed in Table T6 In each epoch, two random augmentations were applied per sample and instance discrimination is applied. For the heart rate prediction task, we used: permutation noise, scale, and shift.
For activity recognilion and step delection from inertial signals, we used: permutation, noise, scale, shift, and rotation.
For cardiovascular disease and sleep stage classification, we used: resample, noise, scale, negate, and shift.
These augmentation sets follow prior work 5
ensuring task-relevant diversity.
Table 16: Common time series augmentations
Domain
Augmentation
Details
Noise
Add Gaussian noise sampled from normal distribution, M100 0.4)
Scale
Amplify channels by random distortion sampled from normal distribution (2, 1.1]
Negate
Multiply the
signal by a factor of
Split signals
than segments, then permute segments
Permute
and combine them into the original shape Interpolate the time-series to times its original sampling rate
Resample
and randomly down down-sample to its initial dimensions
Rotate the 3-axial (x, y, and z) readings of each IMU sensor by a random degree, which follows uniform around random axis
Rotation
space. (Only applied for Activity Recognition)
Time Flip
Flip the time series in time for all channels, i... XAngltt] x|-1]]
Shift
Apply circular shift with randorn amount Randomly chose section to zero out Combination Permutationn and Noise Combination Noise and Scaling
Random Zero Out
Permutation Noise
Noise Scale
Highpass
Apply highpass filter the frequency domain reserve high-frequency componeuts
Frequency
Lowpass
Apply lowpass filter the frequency domaii
low frequency components
Phase shift
Shift the phase of time- series data with randomly generalized number
Noise in Frequency
Add Gaussian noise, sampled from normal distribution A/0,, 0.5), to the frequency spectrum
Although we also experimented with frequency domain augmentations given Table 16 for each task, we observed consistent performance degradation. As result, we excluded these frequency augmentations from all baseline comparisons.
E.3.3 Transformations
Fourier transformation For the Fourier transformation, we compute the FFT using the full length of each input signal without applying any task-specific padding or optimization. Since the input signals rellevalued, we calculate only the positive (real) frequencies, leveraging the Hermitian symmetry 1 by setting norm=' ortho? norm='ortho in PyTorch's [92]
of the spectrum. We normalize the FFT using Vn torch. fft. rfft, ensuring the transformation is orthonormal.
Wavelet transformation For the wavelet transformation, we use the continuous Morlet wavelet with 48 logarithmically spaced scales computed via np. ge 128, num=48), without any task-specific tuning or dataset-dependent adjustments. This configuration is applied uniformly across all datasets to highlight the versatility and generality of our method. We use Py Wavelets [93 implementation.
43

---

F
Computational Overhead of Techniques
We analyze the computational overhead of SSL techniques designed for temporal data. We break down the main operations of TS2VEC 49]. TS-TCC
44 TF-C 47 and simMTM 48] and provide
empirical comparisons in Table 17
TS2Vec Vec samples overlapping subsegments for hierarchical contrastive loss. For each sample, it randomly selects a crop length [2u+1, 7", defines crop interval [c), c,] and an extended context interval [et, 6r), and applies per-sample random offsets to generate two augmented views. These views are passed through shared encoder, producing two embeddings. TS2 aligns their overlapping subsegments using temporal contrastive loss and recursively applies pooling to compute multiple loss terms at increasingly coarser resolutions. This multiscale loss is recomputed for every pair at each layer depth, leading to quadratic scaling with the number of layers. Moreover, because the crops are randomly sampled, TS2Vec forwards large sections of samples multiple times with different windows. Despite using shared encoder, this repeated window sampling and depth-wise contrastive loss accumulation make training slow, especially on long sequences.
TS-TCC In each batch, TS-TCC generates two augmented views using weak and strong augmen- tations, and processes both through a shared encoder to obtain temporal feature sequences. The temporal contrasting module performs cross-vies voos-viewwwn prediction task: it uses autoregressive model to summarize the past of one view into a context vector and predicts the future of the other view using linear projections. For each sample, this involves computing predictions for multiple future steps. comparing them against the target view, and accumulating a contrastive loss. This adds sequential dependency training, the model must first encode the past then perform multiple forward passes to compare predicted and true representations. While TS-TCC leverages efficient architectures like Transformers, the temporal prediction task and dual-view contrastive losses make less parallelizable than methods that process views independently or avoid cross-timestep dependencies.
TF-C In each batch, TF-C computes time and frequency representations in parallel using separate encoders and applies multiple contrastive losses. The input is first transformed via FFT to obtain trequency-domaie features. Two augmented views are generated: one in time via temporal augmen- tations (c.g., jittering, scaling), and one in frequency via spectral perturbations (adding/removing frequency components). Both views are processed by their respective encoders and projectors. TF-C then computes contrastive losses in three spaces: time-time, -ereuuecyy...ucccy.... and time-freque freeffeuu.ncc. The lime-freuuency consistency loss includes four cross-domain pairings (e.g.. original time vs. perturbed frequency) and is implemented using triplet-style margin loss. This results in eight forward passes per sample (two augmentations two domains two projections).
simMTM SimMTM introduces two coupled components: masked contrastive learning and masked reconstruction. In each batch, a contrastive similarity matrix is computed over samples and their masked variants, involving full pairwise dot- dot-product followed by calculating KL-divergence with soft targets. The reconstruction further adds cost by aggregating weighted representations using the similarity matrix and applying linear projection to reconstruct masked sequences. While these components operate in parallel, the simultaneous use large similarity matrices and reconstruction targets slows down training, especially with high masking rates or long windows.
Table 17 reports the average execution time per epoch for each method, measured using an NVIDIA GeForce RTX 4090 GPU with 24GB of memory. For each run, we timed only the core training operations using a unified timing function that accounts for both CPU and GPU execution. The table presents the mean and standard deviation across the five runs. For our method, the computation of wavelet and Fourier domain transformations for batch of samples takes z 2 seconds.
Table 17: Time taken (in seconds) for each SSL technique for a single epoch
Metric
Ours
VS2
TS-TCC
TF-C
simMTM
Execution Time (sec)
18.67=0.05
310=0.07
6.47:00.9
21.61+0.02
14:199
44

---

A key advantage of our method is that the drequency-domain preprocessing (FFT and Gabor trans- forms) is performed only once and cached, rather than recomputed every epoch. In contrast, traditional temporal augmentations (e.g., permutation, scaling with noise) must be regenerated each epoch, introducing repeated overhead.
To quantify this difference, we report the runtime for a batch of 1024 samples (each being univariate time series of length 200) measured on NVIDIA RTX 4090 (Table 18). FFT and Gabor transforms take 0.0075s and 3.59s, respectively, but these costs are paid once and do not scale with the number of epochs. By comparison, temporal augmentations scale linearly with training length: permutation requires 0.021 x epoch count seconds and scale+noise requires 0.010 x epoch count seconds. At 500 epochs- common for SSL methods our preprocessing yields about 4x speedup, and the advantage grows further for longer runs.
Table 18: Runtime of preprocessing steps (batch size 1024) measured NVIDIA RTX 4090. Unlike augmentations, FFT and Gabor transforms are cached after one-time computation.
Transform
epoch (s)
500 epochs (s)
FFT
0.0075
0.0075
Gabor
3.59
3.59
Permutation
0.021 epoch_coumt
10.5
Scale + Noise
0.010 x epoch_count
5.0
We note that our comparison focuses on the most commonly used temporal augmentations such as permutation and scaling with noise. we additionally implement and compare with the specialized time-series augmentation strategies [5], which often require auxiliary models or complex frequency- domain modifications, the computational gap becomes even larger. such cases, the advantage of
one-time preprocessing approach increases substantially, as these specialized augmentations add both runtime overhead and architectural complexity, whereas our method retains efficiency.
45

---

G Expanded Related Work
This section extends the related work discussed the main paper by highlighting methods that aim to eliminate reliance on augmentations in self- supe-supeeeieed learning, well as efforts to mitigate the representational biases introduced by common augmentations.
Representation invariance Representation learning methods have shown to be effective in several tasks. Several works explain this success by relating the success of learned representations to invariancy caused by data augmentations 17 21 94 30]. However, such invariance could be harmful to downstream tasks [95] 96] if they rely on the characteristics of the data augmentations, e-g.. location, pyplitudd-sensitivve One proposed solution to this limitations involves constructing separate embedding spaces, each invariant to all but one augmentation [77.. While this approach allows for disentangling the effects of different augmentations, it is constrained by the number of predefined embedding spaces and comes with increased computational cost. Another method introduces an sugmentation-await module that learns to predict the differences in augmentation parameters (e.g.. cropping positions) between two randomly transformed samples [21].
However, this technique has several drawbacks. First, it requires explicit parameterization of each augmentation, which is particularly challenging for temporal data where augmentation semantics (e.g., shifts, warps) are less structured than in images. Second, its effectiveness remains dependent on the choice and quality of augmentations used, reintroducing domain- speaiffc tuning.
In contrast, our method leverages principled, isometric transformations that preserve the geometry of the data, without enforcing explicit invariance. This allows the model retain signal characteristics that may be important for downstream tasks and would otherwise be suppressed by augmentations (17). For example, when shift invariance is beneficial in temporal tasks, our approach captures naturally through domain projections such as the Fourier transform
[97], without depending on handcrafted
augmentations or specialized architectures
98
Augmentation-frre SSL Learning representations without relying on augmentations has been explored domains where strong transformations risk distorting the data structure, such as graphs 999 [00) and time series | [74]. In graph learning, for instance, sugmentation-free ap- proaches generate alternative views by identifying nodes with similar local structures and global semantics [99]. However, these strategies are specialized for graph topologies and do not directly translate to temporal sequences. More recently, authors in [8] introduced random projections as modality and application-agnostic alternative for self-supervised learning. While this strategy pro- vides a generic way to avoid manual augmentations, in practice, domain-aware contrastive methods equipped with carefully designed augmentations still tend to outperform projection-basee approaches when reliable inductive biases about the data are available.
Our method goes a step further by eliminating the need for augmentations across diverse time-series applications. while still outperforming specialized augmentation strategies tailored to specific tasks, thereby highlighting its algorithmic superiority.
For time-series data, recent approach introduced a temporal-frequence co-training model for semi- supervised learning, using time and Fourier domain transformations to generate pseudo-labels (74). These methods generally overlook the non-stationary nature of temporal signals. Relying solely on Fourier transforms limits representation to global frequency patterns and may miss short-duration events, In contrast, our method incorporates wavelet frames capture local temporal variations, enabling finer resolution of transient signal components which are critical for representation learning.
Moreover, prior work often assumes that data should cluster similarly in time and frequency do- mains 74, implicitly treating the two latent spaces as geometrically aligned. Our empirical results challenge this assumption, showing that latent representations across domains differ. To address this, we introduce domain-specific latent space mappers that align and exploit the complementary structure of each space, enhancing representation learning.
46


Reviews:
[{"review_id": "QzTIACMc7N", "reviewer": "Reviewer_m1TD", "summary": "This paper notes that most SSL approaches require handcrafted augmentations, which need to be optimized for specific downstream tasks and are not necessarily generalizable. To address this they propose a novel approach where they do instance discrimination after a projection step. They use several representations of their data and train an encoder for each domain. I was actually a bit confused after that by Figure 1 and its description, but my understanding is that after multiple mappings they have a separate embedding for each branch. They then compute instancewise discrimination loss between the branches over a batch for neighboring points. Finally, they show that their method outperforms several well known classic SSL techniques on a number of benchmarks.", "questions": "# Questions\n- You mentioned in your related work several related approaches and fields, but you actually didn’t talk about how frame projections are used, if at all, in machine learning or contrastive learning. It would be good to mention it and contextualize how large the niche is where you’re novel (is it SSL views, something bigger)?\n- Could you add a little bit more intuition right before or after prop 2.1 involving a concrete example for what this means in e.g. a specific type of data type or representation? Not sure if this is possible.\n- Despite most of the paper reading quite clearly, I found figure 1 a bit difficult to parse.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 4, "strengths_and_weaknesses": "# Strengths\n- very clear. I am less familiar with frames and manifolds, so I appreciated how you provided a lot of intuition. I appreciated how e.g. proposition 2.1 had intuition before the statement of the proposition.\n- mathematically elegant solution to an important problem, previously mostly solved by heuristics. Further, it’s tied well to the applications.\n- One sees a large improvement over baselines.\n\n# Weaknesses\n- The baselines are more classic than SOTA and largely come from a few years ago. This makes the method potentially look better than it is. I’m forgiving a bit of this because the technique is elegant, but really, you should use harder baselines.\n- the datasets are also a bit older, but since there are a good number of them that is ok.\n\nOverall, I see this as worth publishing as it is an elegant, interesting method that performs at least reasonably against more classic baselines. Future work could be trying to push these sorts of ideas towards SOTA on competitive modern benchmarks.", "was_revised": true, "final_justification": "This is a nice, clean idea and it's cleanly written. While the baselines are not all SOTA methods, I don't believe that for this type of paper they need to be.  Further, I appreciated that they added additional experiments for their rebuttals for other reviewers.", "rebuttal": "We thank the reviewer for their time and for supporting our work. We have responded to each of the comments below.\n\n**Baselines and datasets**\n\nWhen selecting baselines, our goal was to provide a comprehensive comparison with recent methods. Therefore, we included approaches such as SimMTM (masking-based) and TF-C (frequency contrastive learning), which reflect two different directions in the field. Due to space constraints, we prioritized representative, recent and already established methods that are commonly used in the literature. That said, we would be happy to include any additional baselines the reviewer suggests in a revised version.\nRegarding datasets, we selected benchmarks that are well-established in the community and cover a broad range of real-life activities and conditions, including human activity recognition (HAR) and sleep monitoring. If the reviewer has suggestions for other datasets, we are happy to conduct further experiments and include them.\n\n**Questions**\n1) Frame projections are used in machine learning for capturing different patterns in data.\nHowever, to the best of our knowledge, our work is the first to apply known frame projections specifically for generating views in self-supervised learning without applying any augmentations.\n2) We thank the reviewer for this suggestion. We’ll include a clear statement right after Proposition 2.1: that while any two latent vectors may appear nearly orthogonal, the overall geometry of one latent space can differ radically from another. We will also point readers to Appendix C page 20 for further empirical validation after the proposition.\n3) We thank the reviewer for the feedback. We will revise Figure 1 to improve clarity and include additional explanations, particularly around the role of the mappers and the loss calculation, to make the figure easier to interpret.\n\n\nOnce again, we sincerely thank the reviewer for the thorough review and their time."}, {"review_id": "xHkqeh5dan", "reviewer": "Reviewer_NBuw", "summary": "This paper introduces a self-supervised learning framework that learns from three invertible transformations of each time-series example: raw signal, full FFT coefficients, and Gabor-wavelet frame, each encoded separately. Each pair of domains are applied with InfoNCE loss. Two tiny MLP mappers predict the frequency-domain embeddings from the time-domain embedding. In inference only the time encoder and the mappers are needed. On 9 ECG / EEG/ PPG / IMU datasets, the method surpasses SimCLR-style and other time-series-specific baselines by 30 %.", "questions": "- Can you add frozen-encoder baselines from LaBraM, EEGPT, and CBraMod, or explain why their checkpoints cannot run on your datasets?\n- Mapper ablation: what happens when each 500-parameter MLP is replaced by a linear layer?\n- Have you tried joint training in which mappers and encoders are optimised together? If so, what are the results?\n- Can the authors explain intuitively why this works? For instance, in a toy example where one view is cartesian coordinates and the other is polar, a polar-view encoder could trivially apply the known mapping from cartesian to polar. By analogy, why doesn’t your protocol collapse into each encoder simply inverting the other view? A concise, low-dimension explanation would help readers grasp where the learning signal comes from. Given that a theoretical explanation is not provided, I would hope to see an intuitive explanation.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The strengths are 1. The information-preserving design removes the need for hand-crafted augmentations. 2. The “coordinate consistency as (self) supervision” is novel and could potentially generalize to vision or audio.\n\n\n\nWeaknesses are \n1. Supervised references for EEG / ECG are outdated. Recent foundation models such as LaBraM, EEGPT, and CBraMod report strong numbers; comparison or discussion is required.\n\nJiang et al., 2024, “LaBraM: Large Brain Model for Learning Generic Representations with Tremendous EEG Data”.\nYue et al., 2024, “EEGPT: Unleashing the Potential of EEG Generalist Foundation Model by Autoregressive Pre-training”.\nWang et al., 2024, “CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding”.\n\n2. The paper misses the preprocessing time of the FFT + Gabor transforms. A simple wall-clock benchmark on the largest dataset—ideally side-by-side with the run-time of the common augmentation pipeline used by SimCLR-style baselines—would make the compute overhead clear.", "was_revised": true, "final_justification": "The additional experiments strengthened the paper. The authors also clarified several important points, such as the design decisions around fair comparisons. These clarifications helped resolve some of my earlier concerns.", "rebuttal": "We would like to thank the reviewer for their detailed feedback and for listing their questions, which we have addressed below. We also appreciate their recognition of the novelty and the significance of our proposed solution.\n\n**Supervised references**\nWe thank the reviewer for pointing this out. Our goal was to ensure a fair comparison by using baselines with similar model capacity and training data. Since our method does not rely on data augmentation, we also demonstrate that it performs well with effectively less data. In line with standard practice in self-supervised learning, we use the same backbone across methods to isolate the effect of architectures. While recent foundation models like LaBraM, EEGPT, and CBraMod show strong results, they typically rely on much larger architectures and huge pretraining datasets, making a direct comparison less meaningful in our setup. \n\nNonetheless, per the reviewer's suggestion, we have used the pre-trained weights (from Huggingface) of CBraMod (the most recent work) to evaluate on Sleep-EDF. We extract features using model’s pretrained weights, then either train only a linear classifier or fine‑tune the full network on the same training split of Sleep‑EDF. We reported the results below.\n\n| **Method**           | **Linear evaluation** | **Fine‑tuning** |\n| -------------------- | --------------------- | --------------- |\n| CBraMod (pretrained) | 65.03$\\\\pm$0.63   | **72.98$\\\\pm$0.47**  |\n| Ours        | **69.19$\\\\pm$1.32**  | 71.13$\\\\pm$0.84        |\n\nWe followed this evaluation scheme to isolate and compare the learned representations. It is also important to note that CBraMod was not originally trained on single‑channel EEG, which may contribute to observed performance discrepancies in linear evaluation. Additionally, CBraMod uses a transformer layer for sequence‑to‑sequence sleep staging, whereas we applied a linear classifier to ensure a fair comparison with other representation learning methods, such as TS-TCC. We will include these implementation details and the extended evaluation protocol in the revised manuscript.\n\n**Processing times**\n A key advantage of our approach is that the FFT and Gabor transform computations can be performed once and cached, enabling efficient training. In contrast, traditional augmentations are applied at every epoch, introducing repeated computational overhead. To clarify this, we report runtimes (measured on an NVIDIA RTX 4090) for a batch of 1024 samples using FFT (implemented with `PyTorch`), Gabor transforms (via the `pywt` library), and two common temporal augmentations (permutation and scale + noise, implemented with `NumPy`). Although we also experimented with implementing augmentations in `PyTorch` for speed, we did not observe significant differences in runtime.\n\n| **Transform** | **Runtime (s)---1 epoch**   | **Runtime (s)---500 epochs**  |\n| ------------- | ---------------------- | ---------------------- |\n| FFT           | 0.0075                 |    0.0075       |\n| Gabor         | 3.59                   |  3.59       |\n| Permutation   | 0.021 × *epoch\\_count* |      10.5     |\n| Scale + Noise | 0.010 × *epoch\\_count* |       5  |\n\nConsidering that SSL methods typically train for ~300-500 [1], up to 1000 epochs, our approach becomes significantly more computationally efficient than augmentations, with approximately a 4× speedup at 500 epochs, due to its one-time preprocessing cost.\n\n\n**Questions**\n\n1) We evaluated the EEG-based sleep classification task using the pretrained weights from one of the suggested foundational models. However, It is again important to note that the fairness of a direct comparison is controversial as both model size (i.e., CBraMod has 20x more parameters than our backbone) and training data of the foundational models are huge compared to our approach. In response to the reviewer’s suggestion, we have added a discussion and detailed comparison between this foundational model and our approach.\n2) We confirmed that the performance decreases significantly when we use linear mappers. We conducted this experiment and reported in Appendix D.2 on **page 23**.\n3) When we trained the mappers and encoders all together, we observed a decrease in the performance, and we hypothesized that the gradients from mappers could force encoders to a simple solution for making the mapping easier. We thank the reviewer for this question, we will include that ablation in Appendix as well.\n4) We thank the reviewer for raising this important question. The transformations we use, FFT and Gabor, are not straightforward for models to learn or invert, as they involve complex exponentials with high-frequency components. Specifically, in the case of the FFT, inverting the transformation requires a full linear mapping over all input dimensions; this is not achievable unless the encoder can represent a dense $D \\times D$ matrix, where $D$ is the input dimension. Multi-layer convolutional encoder with small receptive fields *cannot* implement such a transform unless its total receptive field spans the full input. Gabor transforms are even more complex, involving both time shifts and frequency modulations, which further increase the difficulty of exact inversion.\n\n\n    We conjecture that these transforms expose complementary structures in the data (e.g., global trends via FFT, localized patterns via Gabor), which encourages the model to learn representations rather than collapsing into trivial solutions.\n\n    We agree that a low-dimensional toy example could be helpful for intuition. However, due to the inherently high-dimensional nature of these transforms, such examples may oversimplify and fail to reflect the true dynamics. Thus, we will include a verbal explanation and expand on this intuition in the revision, as suggested.\n\n\n[1] A Simple Framework for Contrastive Learning of Visual Representations. ICML 2020.\n\n\nWe thank the reviewer again for their time and for helping us improve the clarity and depth of our work."}, {"review_id": "qlI4lNFY7l", "reviewer": "Reviewer_JywM", "summary": "The paper addresses the task of time series self-supervised learning (SSL). The authors propose a novel augmentation-free SSL method. Specifically, the method eliminates the need for traditional data augmentation by leveraging Fourier and Gabor wavelet transforms as additional views alongside the original input. In addition, the authors introduce a mapping function from the time domain to these transformed views during training to facilitate more efficient inference. The proposed method is evaluated on several time series classification tasks and datasets, where it outperforms existing augmentation-based SSL methods.", "questions": "- Could you discuss relevant works that are not covered in the paper?\n- What is the effect of the learned mapping function’s accuracy on the overall performance?\n- Does the proposed SSL method work on forecasting tasks?\n- Would it be possible to include fine-tuning experimental results as well?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The authors effectively highlight the challenge of designing hand-crafted augmentations, which makes the proposed augmentation-free approach highly relevant.\n- The paper is well-written, with particularly clear definitions of notations and methodology. The approach is easy to follow and understand.\n- To the best of my knowledge, this is the first work that utilizes Fourier and Gabor wavelet transforms for time series self-supervised learning (SSL), contributing new methods to the field.\n- Comprehensive Experimentation: The authors conduct an extensive set of experiments across a variety of tasks and datasets. The comparison between larger backbones and multiple views for the baseline methods (L198-L201) is insightful and provides a fair evaluation of the proposed method.\n\nWeakness\n- Some relevant works are missing: \n  - The paper does not discuss more recent work on time series contrastive self-supervised learning (SSL), such as Timesurl [1], which shows to outperform some of the included baselines. \n  - Other relevant research on augmentation-free SSL methods, such as [2] and [3], is not discussed or compared. \n- The mapping function is an important part of the work, but the effect of its accuracy is not discussed. \n- The paper lacks a discussion on the applicablity of the proposed method on ftime-series forecasting tasks.\n- The paper focuses evaluation on linear probing only without finetuning experiments, which is another popular way for testing representation quality.\n\n[1] Liu, J. and Chen, S., 2024, March. \"Timesurl: Self-supervised contrastive learning for universal time series representation learning.\" In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 12, pp. 13918-13926).\n[2]Verma, V., Luong, T., Kawaguchi, K., Pham, H. and Le, Q., 2021, July. Towards domain-agnostic contrastive learning. In International Conference on Machine Learning (pp. 10530-10541). PMLR.\n[3] Sui, Y., Wu, T., Cresswell, J.C., Wu, G., Stein, G., Huang, X.S., Zhang, X. and Volkovs, M., 2023. Self-supervised representation learning from random data projectors. arXiv preprint arXiv:2310.07756.", "was_revised": true, "final_justification": "In my original review, I raised several concerns regarding the completeness and positioning of the paper:\n- The experimental baselines were not sufficiently recent or competitive.\n- The related work lacked discussion of domain-agnostic or augmentation-free self-supervised learning (SSL) methods across modalities.\n- The method was only evaluated on time-series classification, with no discussion on its applicability to forecasting tasks.\n\nIn the rebuttal, the authors provided a thoughtful response and acknowledged these limitations. Specifically:\n- They committed to incorporating more advanced and recent baselines in future revisions.\n- They agreed to include and discuss augmentation-free SSL methods across modalities, both as baselines and in the related work.\n- They clarified that the proposed framework is not limited to classification and can generalize to forecasting tasks, and they will include this discussion in future versions.\n\nGiven these clarifications and commitments, I believe the paper shows stronger potential than initially assessed. While some improvements remain to be implemented, the response indicates a clear path forward. Based on this, I have raised my score.", "rebuttal": "We would like to thank the reviewer for taking their time and feedback. We have addressed each question below. \n\n**Relevant works**\n\nWe would like to thank the reviewer for pointing out Timesurl. In the limited time, we have evaluated its performance in challenging heart rate estimation datasets which include severe noise. Our method outperforms the timesurl by ~8–9%, as shown in the table below (using the correlation metric $\\rho$).\n\n| Method        | SPC12            | SPC22            | DaLiA            |\n|---------------|------------------|------------------|------------------|\n| Timesurl | 75.13$\\pm$1.22   | 50.10$\\pm$1.97 | 60.17$\\pm$0.11 |\n| **Ours**  | **82.67$\\pm$1.30** | **54.88$\\pm$1.89** | **63.72$\\pm$0.06** |\n\nWe observed that Timesurl applies significant modifications in the frequency domain for augmentations which can change the characteristics of the signal. We will extend the experiments to other datasets and include the results in revision.\n\n**Ablations on mapping function**\n\nWe have included several ablations about mapping functions in Appendix D on **page 23**.\n\n**Forecasting**\n\nWe would like to thank the reviewer for this question. We mainly focused on classification as the data augmentations are more diverse and challenging, where even specific augmentations are developed [4-5]. But, since our method uses global (FFT) and local (Gabor) representations together for learning, it can be useful for forecasting tasks as well. We will add this discussion to the manuscript. \n\n**Fine tuning**\n\nWe have provided the fine tuning results in Appendix D.4. on **page 24-25**.\n\n**Questions**\n\n1) We have included initial experiments related to the referenced Timesurl and we will expand them in the revision. Regarding [2], the authors used mixup instead of traditional augmentations, but prior work [4] has shown that simple mixup strategies often underperform on time series data. We also thank the reviewer for highlighting [3], which introduces random projection heads to generate pseudo targets and uses predictors trained with a Barlow Twins–style loss. While conceptually interesting, we found that Barlow Twins tends to perform poorly in noisy settings such as heart rate estimation, as its emphasis on decorrelating feature components can cause the model to amplify noise.\n\n     As a best practice, we aim to implement and compare this method directly with ours. However, due to its application-specific   design choices, reproducing it fairly across diverse settings made the process slow. We will include the complete comparison and discussion in the revised version.\n\n2) We have investigated the effect of mapping functions in Section 4.1 Ablations on **page 7-8** and on Appendix D.2. **page 23**.\n3) We thank the reviewer for asking this important question. Although we have not conducted experiments regarding forecasting, we believe our approach can also work in that application as it uses global (FFT) and local (Gabor) transformations to learn the representations, which can be quite useful for forecasting.\n4) We have included the fine tuning experiments on Appendix D.4. **page 24-25**, while following the same experimental setup with the previous works.\n\n\n[4] Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning. NeurIPS 2023.\n\n[5] Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency. NeuIPS 2022.\n\n&nbsp;  \nOnce again, we sincerely thank the reviewer for their comments and for highlighting the significance of our contributions."}, {"review_id": "pInEmVcPZh", "reviewer": "Reviewer_Pkg5", "summary": "The authors propose to use projections of the data into Fourier space and Gabor wavelet space as 'views' for self supervised learning, in-place of traditional augmentations. They show empirically that this method improves performance slightly across a range of time-series datasets and compared with a diverse set of SSL benchmarks.", "questions": "- The `t` domain is not defined prior to equation (2). \n- The authors note that SSL often results in models which are overly reliant on a subsset of features, or an undesirably invariant to a subset of features. How does using othonormal basis fix this other than the fact that the specific invariances may be harder to intuit now? \n- Can the authors justify the use of the term 'manifold'? It is not immediately clear that these representation spaces are manifolds in the formal sense. \n- A more extensive coverage of prior work specifically related to 'augmentation-free SSL', and an improved theoretical understanding of the proposed approach would be beneficial. For example, if the authors can formalize or empirically measure the types of invariances that their proposed projects would induce, this would greatly increase the impact of the work. \n- The proposed approach seems amenable to image SSL representation learning as well. Have the authors experimented with this?\n- The claim of 'up to 20-30%' performance increase seems a bit misleading given the difference in performance on many datasets is relatively small. I would encourage the authors to make the claim more quantitative and informative of the actual distribution of results.", "limitations": "The authors should mention if or why there method was not evaluated in image settings.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The proposed problem is important and well motivated. Indeed, data-specific augmentations are one of the big limitations of modern SSL methods, especially when applying them to data domains outside of images or text. Therefore, methods to alleviate this need for hand-crafted augmentations are highly valuable. Additionally, the overall insight that different frames can be used in place of these augmentations is an interesting and valuable one.\n- The experimental results are strong, showing broadly improved performance on many tasks, and increased performance at a range of model sizes. \n- The authors perform a set of ablations to validate their modeling choices, which is a welcome addition.\n\n\n**Weaknesses**\n- The theoretical understanding provided to justify the method is quite limited.\n- The experimental evaluation is focused on time-series modeling, while the most prominent SSL methods are used for vision and language -- yet the authors make no mention of these domains in the text. The authors should at least acknowledge this fact, and describe why or if their method does not work on such domains, or if they do not know, rather than ignoring this application. \n- It appears the authors re-trained all baseline methods from scratch in their reported results since the datasets they used are not the traditional SSL benchmarks the methods were developed on. It is not clear that in this process the baselines were tuned properly. There are many possible parameter settings, and with so many baselines, tuning would appear challenging. For Barlow Twins for example, the authors report using the hyperparameter lambda to 0.005 without any tuning. These details can have a significant effect on performance. If the authors could find any published results from prior work on these datasets that would be beneficial to increase confidence in their results.\n- The performance standard deviation on many experiments seem quite large relative to the differences in model mean performance, yet the authors have no statistical significance indications. \n- The figures 3 and 4 are missing error bars despite the tables having them.\n- The related work section is missing some related work which also studies self-superivsed learning without augmentations. One particularly related method is \"Homomorphic Self-Supervised Learning\" (Keller, Suau, Zappella, 2023 TMLR).", "was_revised": true, "final_justification": "The paper is an interesting method, applied to an interesting problem (augmentation-free SSL) that appears to work well. The main limitations of the paper are a lack of theoretical backing, and limited empirical evaluation (such as on vision tasks) that the authors have described as future work. We therefore think the reasons to accept the paper outweighs the reasons to reject marginally, since the paper is interesting, technically correct, and well written, but could have significant improvements with further development. .", "rebuttal": "We would like to thank the reviewer for their time and recognizing the importance of the problem. We have addressed each point below.\n\n**Theoretical justification**\n\nWe agree with the reviewer. Our Proposition, which shows the geometries of the latent spaces can be different, is the theoretical motivation for the proposed mappers but not the overall method.\nWe understand the importance of theoretical justification for SSL but the overall method is complex so we could not perform a rigorous mathematical verification. We can conjecture that our method works without collapsing because of the implicit bias exposed by the applied frame projections and their encoders. As per the reviewer’s suggestion, we have added a concise explanation at the end of our response to explain why NT‑Xent loss combined with transforms does not induce (undesired) invariances.\n\n\n**Vision-Language**\n\nWe thank the reviewer for raising this important point. Our method is motivated by the diverse data augmentation strategies in time series applications while depending on the task. In the vision domain, cropping and color augmentations are well established and widely used. Also, the design of encoders for frame projections requires additional careful modifications on the architectures. We acknowledge this and will mention it as an avenue for future work in the revised version.\n\n\n**Hyperparameters**\n\nWe thank the reviewer for pointing this out. We have tried several hyperparameters for Barlow Twins, however, we have observed no significant change in the performance of the models, and mostly got worse when the value increased. This is consistent with the original paper, where the authors conducted an ablation study and found the method to be stable across a range of hyperparameter settings.\n\nRegarding the other baselines, we use their original hyperparameters as specified for each application. For example, TS-TCC adjusts augmentation parameters depending on the task (e.g., activity recognition vs. sleep staging), and we follow those settings exactly. We believe this highlights the value of our approach as it performs consistently well across all datasets and applications without requiring task-specific hyperparameter tuning.\n\n\n**Standard deviation, significance**\n\nWhile some methods come close on individual datasets, our approach consistently achieves the top rank across all tasks. For instance, TF‑C performs strongly on ECG datasets (ranking second to our method) but its accuracy falls to about 30% on the activity recognition task. In response to the reviewer’s suggestion, we generated a critical difference (CD) diagram using the Friedman Chi‑Square test (significance at p < 0.05). For each task, we randomly sample from the confidence intervals 1,000 times, apply CD ranking, and average the results. The CD analysis confirms that our method is statistically superior to the alternatives. We thank the reviewer for the feedback to make the significance clearer, and we will include these new results in the revised manuscript.\n\n\n**Figures 3-4**\n\nIn Figures 3 and 4, the standard deviations were very small relative to the means, resulting in error bars that were not visually noticeable. For example, in Figure 3–4(a), the deviation was less than 0.1% of the mean, and around 1% in Figure 3–4(b). For clarity, we initially excluded them in figures. We will explain this exclusion in the revised script.\n\n**Related works**\n\nWe discuss augmentation-free SSL in Appendix G (Expanded Related Work, **page 33**). We will include the suggested reference in the revised version while noting that it relies on equivariant network architectures.\n\n**Questions**\n\n1. We thank the reviewer for the careful reading. We will add the definition in the revision.\n\n2. We thank the reviewer for this important question. While we could not identify any specific known invariances, we conjecture that combining the NT‑Xent loss with a unitary basis transformation inherently forces the encoder to preserve features. In our framework, positive pairs differ only by a basis change, so every inner product, and therefore every pairwise relationship, remains exactly the same.\n\n    In contrast, conventional SSL augmentations (for example, color vs. texture distortions) unevenly warp different subspaces. This uneven distortion lets the model minimize its loss by selectively ignoring or emphasizing certain features, which can introduce blind spots or unwanted invariances [2]. Because our transforms apply a uniform change across the entire feature space, there is no incentive for the encoder to drop or over‑emphasize any subset of features.\n\n\n3. We have explained the Manifold term on Appendix A.1.2 on **page 16** with the manifold hypothesis assumption. We will revise the script to refer the section with manifold hypothesis and related references for normalized embedding spaces [1].\n\n4. We thank the reviewer for the suggestion. We have included the suggested reference and are happy to cover any other work we have missed in Appendix G. Since we could not find any known invariances the model induce, we are also happy to try and see specific ones if the reviewer has any suggestions.\n\n5. Since the frame projections require special design of architectures that operate with complex numbers, we have left the vision domain as a future work.\n\n6. We agree with the reviewer and will revise it by indicating an average difference of ~15% with the second best method (TS2Vec according to the CD).\n\n\n**Preventing invariances**\n\n**Perfect alignment**\n\nFrom [3], at the NT‑Xent optimum, perfect alignment across positive pairs is expected;\n\n$$\nf^\\*_{ \\mathcal{F} }(\\mathcal{F} (x) ) = f^\\*(x)\n$$\n\n**Extra invariance**\n\nSuppose now that the encoder also becomes invariant to some unintended transformation $W$:\n\n$$\nf^\\*(W x ) = f^\\*(x)\n$$\n\nThen for anchor $x$ consider the contrastive loss (omitting the temperature term for simplicity):\n\n$$\n\\ell(x)= -\\log\\frac{\\exp(f^\\*(x) \\cdot f^\\ast_{\\mathcal{F}}(\\mathcal{F}(x)))}\n{\\exp(f^\\*(x) \\cdot f^\\ast_{\\mathcal{F}}(\\mathcal{F}(x))) + \\sum_{\\begin{array}{c} \\text{all} \\\\ \\text{negatives} \\end{array}}\n\\exp\\bigl(f^\\*(x) \\cdot z_{\\rm neg}\\bigr)}\n$$\n\nBecause of the invariance, there are at least $K\\ge1$ negatives\n$\\{z_i = f^\\*(W x_i)}_{i=1}^K$\nwith\n$\\langle f^*(x) \\,z_i\\rangle \\approx 1$.\nEven if all other negatives were perfectly dissimilar, the denominator contains at least $K+1$ terms $e^{1/\\tau}$, so\n\n$$\n\\ell(x)\\ge-\\log\\frac{e^{1/\\tau}}{(K+1)\\,e^{1/\\tau}}=\\log(K+1)>0.\n$$\n\nHence, whenever multiple negatives become near‐positives through any unintended invariance, NT‑Xent in our setup cannot be driven to zero. In contrast, prior contrastive learning setups encourage the encoder to learn augmentation-induced invariances (i.e., drop or over-emphasize certain features in order to reduce the loss for positive pairs).\n\nWhile the assumption of optimal encoders is strong, it is consistent with prior theoretical works in contrastive learning, which also adopt this viewpoint to analyze properties [1,3].\n\n[1] Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. ICML 2020.\n\n[2] Views Can Be Deceiving: Improved SSL Through Feature Space Augmentation. ICLR 2024.\n\n[3] Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap. ICLR 2022.\n\nOnce again, we sincerely thank the reviewer for their insightful comments and suggestions, as well as for highlighting the invariant features and statistical significance, which have contributed to strengthening the quality of our work."}]

Author rebuttal:

Meta-review: This paper proposes a novel framework for unsupervised time series representation learning that avoids augmentation by leveraging frame projections. The method provides an elegant mathematical formulation, grounded in signal processing, and demonstrates strong empirical results across multiple benchmarks.

Reviewers appreciated the clarity of presentation, the solid theoretical underpinnings, and the meaningful improvements over competitive baselines. Strengths noted include the elegant solution to a problem typically addressed with heuristics, and the consistent empirical gains. Concerns raised centered on the choice of baselines, missing comparisons with some recent methods, and clarity regarding certain transformations.

The authors provided a detailed and constructive rebuttal, adding new experimental comparisons, clarifying methodological details, and addressing reviewer questions on baselines, mappings, and related work. Reviewers acknowledged these efforts positively in the discussion, and their updated assessments reflected increased confidence.

Overall, this submission presents a clear, technically sound, and impactful contribution to self-supervised learning for time series. The reviewers’ feedback, the thorough author response, and the subsequent discussion collectively support acceptance.
Decision: Accept (poster)