SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we investigate the effect of data augmentations in three different unsupervised learning frameworks which are SimCLR [15], BYOL [59] and TS-TCC [16]. For BYOL, the hidden size of the projector is set to 128, the exponential moving average parameter is set to 0.996. For TS-TCC, the  $\lambda_{1}$  and  $\lambda_{2}$  coefficients of temporal and contextual contrasting losses are set to 1, the same as in the original implementation. In TS-TCC, the authors proposed to use a weak (jitter and scale) and strong (permutation and jitter) augmentation together, which is shown as TS-TCC + Traditional Augs in the tables. During our experiments, we followed the original implementation of TS-TCC with their proposed architecture and applied additional augmentations after the strong one without changing the original contrastive learning framework. We set the scaling ratio to 2 and 10 for permutation (splitting the signal into a random number of segments with a maximum of 10 and randomly shuffling them). These parameters for augmentation strengths are set to the same values as in the original implementation.
Table 16: Performance comparison of our method in different CL frameworks for Activity Recognition
Method	UCIHAR	HHAR	USC
ACC↑	MF1↑	ACC↑	MF1↑	ACC↑	MF1↑
SimCLR + Traditional Augs.	87.05 ± 1.07	86.13 ± 0.96	85.48 ± 1.16	84.31 ± 1.31	53.47 ± 1.10	52.09 ± 0.95
SimCLR + Aug. Bank	65.27 ± 1.12	71.16 ± 1.24	67.95 ± 1.45	75.13 ± 1.32	43.28 ± 4.37	47.31 ± 4.68
SimCLR + DACL	73.12 ± 1.23	66.28 ± 1.11	80.89 ± 0.91	81.31 ± 0.78	53.61 ± 2.60	51.76 ± 2.21
SimCLR + Ours	91.60 ± 0.65	90.46 ± 0.53	88.05 ± 1.05	87.95 ± 1.10	60.13 ± 0.75	59.13 ± 0.69
BYOL + Traditional Augs.	83.41 ± 0.95	82.13 ± 1.12	86.41 ± 0.97	86.31 ± 1.10	58.34 ± 1.15	55.04 ± 1.15
BYOL + Aug. Bank	73.71 ± 0.74	69.80 ± 1.10	84.60 ± 0.93	84.65 ± 1.03	52.00 ± 1.21	49.14 ± 1.18
BYOL + DACL	73.86 ± 1.12	70.46 ± 1.24	82.76 ± 1.04	84.89 ± 0.93	47.14 ± 2.08	45.34 ± 2.98
BYOL + Ours	87.01 ± 1.10	84.92 ± 1.13	90.31 ± 1.16	90.45 ± 1.31	56.87 ± 0.91	55.01 ± 0.95
TS-TCC + Traditional Augs.	90.95 ± 0.87	90.30 ± 0.64	35.57 ± 1.43	40.13 ± 1.67	39.76 ± 1.61	43.12 ± 1.10
TS-TCC + Aug. Bank	76.78 ± 0.95	76.52 ± 0.97	20.25 ± 1.54	19.25 ± 1.32	21.37 ± 1.78	20.15 ± 1.48
TS-TCC + DACL	73.86 ± 1.12	70.46 ± 1.24	33.89 ± 1.87	37.41 ± 1.39	36.74 ± 1.36	40.18 ± 1.45
TS-TCC + Ours	91.86 ± 0.97	91.92 ± 1.02	38.45 ± 1.12	43.52 ± 1.33	42.61 ± 1.92	45.06 ± 1.11
Table 17: Performance comparison of our method in different CL frameworks for Heart Rate Prediction
Method	IEEE SPC12	IEEE SPC22	DaLia
MAE↓	RMSE↓	MAE↓	RMSE↓	MAE↓	RMSE↓
SimCLR + Traditional Augs.	20.67 ± 1.13	26.35 ± 0.98	16.84 ± 1.10	22.23 ± 0.72	12.01 ± 0.65	21.09 ± 0.86
SimCLR + Aug. Bank	27.31 ± 2.17	37.93 ± 2.96	27.84 ± 2.03	36.41 ± 3.98	35.87 ± 4.18	40.61 ± 3.74
SimCLR + DACL	21.85 ± 1.63	28.17 ± 1.75	14.67 ± 1.10	20.06 ± 1.21	18.44 ± 1.32	25.61 ± 1.45
SimCLR + Ours	16.26 ± 0.72	22.48 ± 0.95	12.25 ± 0.47	18.20 ± 0.61	10.57 ± 0.55	20.37 ± 0.73
BYOL + Traditional Augs.	20.68 ± 0.98	27.11 ± 0.85	21.16 ± 1.10	26.83 ± 1.05	12.03 ± 0.75	20.77 ± 0.83
BYOL + Aug. Bank	26.08 ± 1.05	32.62 ± 0.93	21.87 ± 1.03	29.13 ± 1.03	18.63 ± 0.91	28.30 ± 0.87
BYOL + DACL	26.45 ± 1.23	33.50 ± 1.32	21.29 ± 1.13	27.34 ± 1.33	15.11 ± 0.93	23.21 ± 0.83
BYOL + Ours	19.85 ± 0.88	26.10 ± 0.94	22.08 ± 1.24	28.20 ± 1.13	11.45 ± 0.63	20.38 ± 0.80
TS-TCC + Traditional Augs.	11.08 ± 1.03	16.97 ± 0.92	16.10 ± 1.23	26.11 ± 1.11	16.18 ± 1.03	24.27 ± 0.95
TS-TCC + Aug. Bank	11.44 ± 1.01	17.06 ± 0.94	13.79 ± 1.21	22.41 ± 1.08	17.28 ± 1.12	25.41 ± 0.98
TS-TCC + DACL	11.60 ± 1.16	18.26 ± 1.20	15.25 ± 1.26	24.40 ± 1.10	16.27 ± 1.16	24.28 ± 0.97
TS-TCC + Ours	10.82 ± 0.65	16.93 ± 0.73	13.63 ± 1.02	21.80 ± 1.11	15.90 ± 0.57	23.81 ± 0.89
Tables 16 17 and 18 compares the performance of three data augmentation techniques, traditional time-series augmentations, DAGL and our proposed method, in contrastive learning frameworks of BYOL, SimCLR, and TS-TCC.
Table 18: Performance comparison of our method in different CL frameworks for CVD classification
Method	CPSC 2018	Chapman
AUC↑	AUC↑
SimCLR + Traditional Augs.	67.86 ± 3.41	74.69 ± 2.04
SimCLR + Aug. Bank	81.78 ± 1.24	94.75 ± 0.90
SimCLR + DACL	82.38 ± 0.84	92.28 ± 0.97
SimCLR + Ours	85.30 ± 0.45	95.90 ± 0.82
BYOL + Traditional Augs	75.41 ± 1.34	85.63 ± 1.43
BYOL + Aug. Bank	83.51 ± 1.12	91.03 ± 1.18
BYOL + DACL	77.61 ± 1.16	81.62 ± 1.24
BYOL + Ours	83.25 ± 1.03	91.23 ± 1.15
TS-TCC + Traditional Augs	87.07 ± 1.10	92.03 ± 1.17
TS-TCC + Aug. Bank	86.67 ± 1.04	92.15 ± 1.02
TS-TCC + DACL	87.63 ± 0.83	92.21 ± 0.86
TS-TCC + Ours	88.05 ± 0.37	92.11 ± 0.75
The results show that the BYOL is more robust to the choice of augmentations than SimCLR, which is also indicated in the original paper [59]. Also, another important outcome of this ablation experiment is that when the TS-TCC framework is used for datasets HHAR and USC, the performance decreases compared to other datasets. A possible explanation for this decrease in the TS-TCC might be the hyper-parameters of the augmentations that are used in the paper. The authors change the strength of the permutation window from dataset to dataset. In our experiments, we used the same hyperparameter for all activity recognition datasets, which can explain the outcome. This ablation experiment also shows that the degree of traditional augmentations is important for contrastive learning to learn class invariant representations.
E.3 Do we still need data augmentations?
In this section, we conduct experiments to observe the performance of methods without additional augmentations. During our experiments, we searched for the best traditional augmentation technique for each method in a given task. We searched over common time series augmentation methods in literature (Table 22), and applied them with baselines. Specifically, we apply Resample for Activity Recognition, Permutation with Noise for Heart rate Prediction and Noise with Scaling for CVD Classification. We have observed that these augmentations yield the best results for all baselines when applied prior to the proposed techniques. However, for GenRep, we found that applying the augmentations after generating instances results in better performance, similar to the original work [51]. We, therefore, apply these specified augmentations for each baseline and report the corresponding results.
Different from other baselines, we observed performance increases for a few datasets when GenRep is applied without any augmentations. This phenomenon can be attributed to the generation of low-quality and less realistic positive samples, where additional augmentations lead to alterations in semantic information, due to less number of samples during training VAE models. However, in the end, we observe that applying additional augmentations always increases the performance on average for all baselines in each task.
Table 19: Performance comparison of methods without Augs. in Activity Recognition datasets
Method	UCIHAR	HHAR	USC
ACC↑	MF1↑	ACC↑	MF1↑	ACC↑	MF1↑
IDAA [53]	82.23 ± 0.69	79.84 ± 0.89	88.98 ± 0.62	89.01 ± 0.55	59.23 ± 1.10	56.11 ± 1.54
w/o Aug.	64.42 (-17.81)	65.17 (-14.67)	86.44 (-2.54)	86.31 (-2.70)	35.22 (-24.01)	33.62 (-22.59)
GenRep [51]	87.22 ± 1.05	86.48 ± 0.95	87.05 ± 0.95	86.45 ± 0.90	50.13 ± 2.85	49.50 ± 2.73
w/o Aug.	88.01 (+0.79)	88.12 (+1.64)	86.51 (-0.54)	86.33 (-0.22)	48.31 (-1.82)	47.33 (-2.17)
DACL [22]	73.12 ± 1.23	66.28 ± 1.11	80.89 ± 0.91	81.31 ± 0.78	53.61 ± 2.60	51.76 ± 2.21
w/o Aug.	45.17 (-27.95)	44.84 (-21.44)	56.70 (-24.19)	56.55 (-25.76)	27.12 (-26.49)	26.99 (-24.77)
Ours	91.60 ± 0.65	90.46 ± 0.53	88.05 ± 1.05	87.95 ± 1.10	60.13 ± 0.75	59.13 ± 0.69
w/o Aug.	84.04 (-5.56)	83.34 (-7.12)	86.70 (-1.35)	86.72 (-1.23)	45.55 (-14.58)	44.94 (-14.19)
Table 20: Performance comparison of methods without Augs. in Heart Rate Prediction datasets
Method	IEEE SPC12	IEEE SPC22	DaLia
MAE↓	RMSE↓	MAE↓	RMSE↓	MAE↓	RMSE↓
IDAA [53]	19.02 ± 0.96	27.42 ± 1.11	15.37 ± 1.21	22.41 ± 1.42	11.12 ± 0.64	20.45 ± 0.69
w/o Aug.	20.19 (+1.17)	28.51 (+1.09)	16.34 (+0.97)	25.75 (+3.34)	16.01 (+4.89)	25.62 (+5.17)
GenRep [51]	21.02 ± 1.41	28.42 ± 1.65	15.67 ± 1.23	22.33 ± 1.43	25.41 ± 1.62	36.83 ± 1.87
w/o Aug.	20.51 (-0.51)	28.35 (-0.07)	23.07 (+7.40)	33.20 (+10.87)	20.03 (-5.38)	31.01 (-5.82)
DACL [22]	21.85 ± 1.63	28.17 ± 1.75	14.67 ± 1.10	20.06 ± 1.21	18.44 ± 1.32	25.61 ± 1.45
w/o Aug.	22.75 (+0.90)	29.90 (+1.73)	20.88 (+6.21)	29.51 (+2.70)	28.24 (+9.45)	37.33 (+11.72)
Ours	16.26 ± 0.72	22.48 ± 0.95	12.25 ± 0.47	18.20 ± 0.61	10.57 ± 0.55	20.37 ± 0.73
w/o Aug.	19.41 (+3.15)	26.23 (+3.75)	16.41 (+4.16)	25.71 (+7.51)	16.73 (+6.16)	27.43 (+7.06)
Table 21: Performance comparison of methods without Augs. in CVD classification datasets
Method	CPSC 2018	Chapman
AUC↑	AUC↑
IDAA [53]	80.90 ± 0.73	93.63 ± 0.91
w/o Aug.	79.00 (-1.90)	92.37 (-1.26)
GenRep [51]	52.49 ± 3.43	86.72 ± 1.13
w/o Aug.	45.17 (-7.32)	84.51 (-2.21)
DACL [22]	82.38 ± 0.84	92.28 ± 0.97
w/o Aug.	73.00 (-9.38)	75.10 (-17.18)
Ours	85.30 ± 0.45	95.90 ± 0.82
w/o Aug.	79.67 (-5.63)	93.48 (-2.42)
Table 22: Common time series augmentations [19]
Domain	Augmentation	Details
Time	Noise	Add Gaussian noise sampled from normal distribution, N(0, 0.4)
Scale	Amplify channels by a random distortion sampled from normal distribution N(2, 1.1)
Shuffle	Randomly permute the channels of the sample. (Not available for Heart rate Prediction)
Negate	Multiply the value of the signal by a factor of -1
Permute	Split signals into no more than 5 segments, then permute the segments and combine them into the original shape
Resample	Interpolate the time-series to 3 times its original sampling rate and randomly down-sample to its initial dimensions
Rotation	Rotate the 3-axial (x, y, and z) readings of each IMU sensor by a random degree, which follows a uniform around a random axis in the 3D space. (Only applied for Activity Recognition)
Time Flip	Flip the time series in time for all channels, i.e., xAug[n] = x[-n]
Random Zero Out	Randomly chose a section to zero out
Permutation + Noise	Combination of Permutation and Noise
Noise + Scale	Combination of Noise and Scaling
Frequency	Highpass	Apply a highpass filter in the frequency domain to reserve high-frequency components
Lowpass	Apply a lowpass filter in the frequency domain to reserve low-frequency components
Phase shift	Shift the phase of time-series data with a randomly generalized number
Noise in Frequency	Add Gaussian noise, sampled from normal distribution N(0, 0.5), to the frequency spectrum
E.4 The effect of interpolating phase components
Here, we investigate the effect of phase interpolation of two samples on the CL performance. In our proposed method, we bring the phase components of the two coherent signals together by adding a small value to the anchor's phase in the direction of the other sample. In this section, we apply the opposite case of our proposed method and increase the gap of phase difference between the anchor and randomly chosen sample. However, we mix their amplitudes according to our proposed method to only observe the phase effect. In other words, we perform the mixup as in Equation 46. Note that the phase mixing in Equation 46 differs from the proposed method only by the sign change.
$$
\mathbf {x} ^ {+} = \mathcal {F} ^ {- 1} \left(A \left(\mathbf {x} ^ {+}\right) \angle P \left(\mathbf {x} ^ {+}\right)\right) \quad \text {w h e r e}
$$
$$
A \left(\mathbf {x} ^ {+}\right) = \lambda_ {A} A (\mathbf {x}) + \left(1 - \lambda_ {A}\right) A (\tilde {\mathbf {x}}) \quad \text {a n d} \tag {46}
$$
$$
P (\mathbf {x} ^ {+}) = P (\mathbf {x}) + \Delta \Theta * (1 - \lambda_ {P})
$$
Also, It is important to note that we sample the mixing coefficients for both amplitude and phase from the same distributions in the proposed method to have a fair comparison. Tables 23 24 25.
Table 23: Performance comparison of our method and its ablation regarding the phase interpolation in SimCLR and BYOL frameworks for Activity Recognition
Method	UCIHAR	HHAR	USC
ACC↑	MF1↑	ACC↑	MF1↑	ACC↑	MF1↑
SimCLR + Traditional Augs.	87.05 ± 1.07	86.13 ± 0.96	85.48 ± 1.16	84.31 ± 1.31	53.47 ± 1.10	52.09 ± 0.95
SimCLR + Phase Gap	79.62 ± 1.10	80.57 ± 1.03	86.55 ± 0.83	86.68 ± 0.71	53.61 ± 2.60	51.76 ± 2.21
SimCLR + Ours	91.60 ± 0.65	90.46 ± 0.53	88.05 ± 1.05	87.95 ± 1.10	60.13 ± 0.75	59.13 ± 0.69
BYOL + Traditional Augs.	83.41 ± 0.95	82.13 ± 1.12	86.41 ± 0.97	86.31 ± 1.10	58.34 ± 1.15	55.04 ± 1.15
BYOL + Phase Gap	78.66 ± 0.63	75.45 ± 1.02	85.82 ± 0.91	85.16 ± 0.92	56.14 ± 0.67	56.20 ± 0.75
BYOL + Ours	87.01 ± 1.10	84.92 ± 1.13	90.31 ± 1.16	90.45 ± 1.31	56.87 ± 0.91	55.01 ± 0.95
Table 24: Performance comparison of our method and its ablation regarding the phase interpolation in SimCLR and BYOL frameworks for Heart Rate Prediction
Method	IEEE SPC12	IEEE SPC22	DaLia
MAE↓	RMSE↓	MAE↓	RMSE↓	MAE↓	RMSE↓
SimCLR + Traditional Augs.	20.67 ± 1.13	26.35 ± 0.98	16.84 ± 1.10	22.23 ± 0.72	12.01 ± 0.65	21.09 ± 0.86
SimCLR + Phase Gap	18.90 ± 1.43	25.29 ± 1.56	14.60 ± 1.03	19.84 ± 1.15	17.57 ± 1.13	27.72 ± 1.35
SimCLR + Ours	16.26 ± 0.72	22.48 ± 0.95	12.25 ± 0.47	18.20 ± 0.61	10.57 ± 0.55	20.37 ± 0.73
BYOL + Traditional Augs.	20.68 ± 0.98	27.11 ± 0.85	21.16 ± 1.10	26.83 ± 1.05	12.03 ± 0.75	20.77 ± 0.83
BYOL + Phase Gap	25.93 ± 0.96	32.68 ± 0.90	21.87 ± 1.03	29.13 ± 1.03	17.46 ± 0.83	27.24 ± 0.83
BYOL + Ours	19.85 ± 0.88	26.10 ± 0.94	22.08 ± 1.24	28.20 ± 1.13	11.45 ± 0.63	20.38 ± 0.80
Table 25: Performance comparison of our method and its ablation regarding the phase interpolation in SimCLR and BYOL frameworks for CVD classification
Method	CPSC 2018	Chapman
AUC↑	AUC↑
SimCLR + Traditional Augs.	67.86 ± 3.41	74.69 ± 2.04
SimCLR + Phase Gap	77.45 ± 1.10	91.95 ± 0.91
SimCLR + Ours	85.30 ± 0.45	95.90 ± 0.82
BYOL + Traditional Augs	75.41 ± 1.34	85.63 ± 1.43
BYOL + Phase Gap	83.11 ± 1.03	91.02 ± 1.11
BYOL + Ours	83.25 ± 1.03	91.23 ± 1.15
E.5 The comparison of Mixup methods
In this section, we give a detailed comparison of prior mixup methods with ours below tables, which are the explicit numbers for Figure 2. Our method demonstrates superior performance compared to previous mixup techniques in 11 out of 14 metrics, indicating its effectiveness. Additionally, the Amplitude Mixup technique, which yields comparable results in two datasets, further supports our claim regarding the destructive effect of simultaneously mixing phase and magnitude for time series. The relatively lower performance of Amplitude Mixup for some datasets can be explained by its limited diversity in generating positive samples since this technique has no solution for mixing the phase of samples in randomly chosen pairs. In other words, as the phase of the augmented instance is the same as the anchor in Amplitude Mix, the diversity of generated positive samples is less compared to other techniques.
Table 26: Performance comparison of ours with prior mixups in Activity Recognition datasets
Method	UCIHAR	HHAR	USC
ACC↑	MF1↑	ACC↑	MF1↑	ACC↑	MF1↑
Geo	36.31 ± 10.15	33.21 ± 12.25	33.16 ± 8.32	31.15 ± 9.25	24.85 ± 9.43	21.64 ± 8.94
Amp	81.76 ± 0.89	80.78 ± 0.78	87.85 ± 0.83	85.53 ± 1.10	41.29 ± 0.56	39.77 ± 1.03
Spec	40.14 ± 2.05	38.34 ± 1.95	56.73 ± 2.01	53.54 ± 1.98	23.45 ± 2.55	21.30 ± 2.41
Cut	50.21 ± 1.34	48.23 ± 1.23	57.71 ± 1.12	53.87 ± 1.09	25.63 ± 2.95	23.41 ± 3.11
Binary	74.13 ± 1.12	71.31 ± 1.10	77.12 ± 0.75	75.23 ± 0.95	42.21 ± 0.97	41.53 ± 1.10
Linear	82.23 ± 2.10	80.25 ± 1.93	80.11 ± 2.05	81.31 ± 1.73	40.15 ± 1.43	39.71 ± 1.14
Ours	84.30 ± 0.73	83.23 ± 0.58	84.51 ± 1.10	83.98 ± 1.03	45.36 ± 0.97	43.14 ± 0.81
Table 27: Performance comparison of ours with prior mixups in Heart Rate Prediction datasets
Method	IEEE SPC12	IEEE SPC 22	DaLia
MAE↓	RMSE↓	MAE↓	RMSE↓	MAE↓	RMSE↓
Geo	32.65 ± 7.25	48.90 ± 9.87	37.15 ± 6.74	36.32 ± 6.21	38.45 ± 7.31	41.32 ± 6.21
Amp	23.01 ± 0.95	30.10 ± 1.04	18.07 ± 1.13	23.13 ± 1.43	19.05 ± 1.63	30.41 ± 1.65
Spec	24.09 ± 4.10	38.41 ± 3.98	24.41 ± 4.10	29.93 ± 4.10	26.71 ± 4.34	35.31 ± 3.93
Cut	24.98 ± 3.93	35.67 ± 4.15	21.77 ± 4.45	28.43 ± 3.97	31.75 ± 4.10	43.56 ± 3.88
Binary	32.23 ± 1.67	40.21 ± 1.98	22.55 ± 1.87	28.78 ± 2.10	19.71 ± 2.15	28.83 ± 2.45
Linear	24.31 ± 1.54	31.29 ± 1.75	18.52 ± 1.43	22.54 ± 1.49	24.16 ± 1.89	32.46 ± 1.97
Ours	21.13 ± 0.89	28.21 ± 1.15	16.17 ± 0.85	21.13 ± 1.05	16.64 ± 1.20	28.43 ± 1.43
Table 28: Performance comparison of ours with prior mixups in CVD classification datasets
Method	CPSC 2018	Chapman
AUC↑	AUC↑
Geo	45.65 ± 6.43	61.32 ± 5.79
Amp	84.10 ± 1.05	89.83 ± 1.12
Spec	69.26 ± 3.10	70.48 ± 3.05
Cut	72.20 ± 2.98	79.23 ± 2.75
Binary	80.53 ± 1.62	82.56 ± 1.45
Linear	78.02 ± 1.43	90.21 ± 1.15
Ours	83.79 ± 1.10	93.85 ± 1.05
F Illustrative Examples
In this section, we show examples of the destructive behavior of linear mixup and how our proposed mixup technique solves this problem. In Figure 5 a), we show two PPG waveforms that are obtained from IEEE SPC15 with the same label i.e., the same heart rate value. Also, we give the corresponding frequency domain transformations of these two waveforms in Figure 5 b) where the frequency axis is converted to heart rate in beats-per-minute i.e.,  $1\mathrm{Hz}$  corresponds to  $60~\mathrm{bpm}$ .
Figure 5: a) The waveforms of anchor and random sample, b) The frequency domain  $(A(\mathbf{x}))$  representation of two samples.
When the linear mixup is applied as in Equation 47 with a  $\lambda$  of 0.9, the resulting waveform is anticipated to contain heart rate information to an extent similar to both the anchor and the sample.
$$
\mathbf {x} ^ {+} = \lambda \mathbf {x} + (1 - \lambda) \tilde {\mathbf {x}} \tag {47}
$$
However, when there is a phase difference greater than  $\pi /2$  between these two samples in the frequencies where the task-specific information is carried, the linear mixup destroys the information.
Figure 6: a) The waveform of anchor and augmented sample with linear mixup, b) The frequency domain  $(A(\mathbf{x}))$  representations of samples where the augmented waveform has lost all the information in the critical frequency band, i.e., the task-specific information is lost.
As our proposed mixup prevents this problem and interpolates between features of two samples, the information is not lost but rather enhanced as both samples have the same label, shown in Figure 7.
Figures 5 and 6 demonstrate the destructive behavior of linear mixup instead of feature interpolation. The linear mixup technique destroys the task-specific information even though the two samples have the same labels and the mixup ratio is relatively high.
Figure 7: a) The waveform of anchor and augmented sample with proposed mixup technique, b) The frequency domain  $(A(\mathbf{x}))$  representations of samples where the augmented waveform carries the information in the critical frequency band as an interpolation of two samples.
As can be seen From figures 6 and 7, our proposed mixup technique not only prevents information loss due to linear mixup but also generates an interpolated sample.
G Performance in Supervised Learning Paradigm
We also conduct experiments in the supervised learning paradigm with our proposed mixup method to see its effectiveness in different learning paradigms. We compare the performance of our method with prior mixup techniques. During the experiments, we follow the original implementation where the mixup is applied to the same minibatch after random shuffling. In the seminal work of mixup [31], the authors stated that interpolating only between inputs with equal labels does not lead to performance gains. Therefore, we only perform the tailored mixup without implementing any VAEs. We implement the tailored mixup for the supervised learning as in Equation 48. Although the phase equations can be simplified by omitting the absolute value, we leave it similar to Equation 4 for consistency.
$$
\begin{array}{l} \mathbf {x} ^ {+} = \mathcal {F} ^ {- 1} (A (\mathbf {x} ^ {+}) \angle P (\mathbf {x} ^ {+})) \quad \text {w h e r e} A (\mathbf {x} ^ {+}) = \lambda_ {A} A (\mathbf {x}) + (1 - \lambda_ {A}) A (\tilde {\mathbf {x}}) \text {a n d} \\ P \left(\mathbf {x} ^ {+}\right) = \left\{ \begin{array}{l l} P (\mathbf {x}) - \left| \Delta \Theta \right| * \left(1 - \lambda_ {P}\right), & \text {i f} \Delta \Theta > 0 \text {a n d} \lambda_ {A} \geq 0. 5 \\ P (\mathbf {x}) + \left| \Delta \Theta \right| * \left(1 - \lambda_ {P}\right), & \text {i f} \Delta \Theta \leq 0 \text {a n d} \lambda_ {A} \geq 0. 5 \\ P (\tilde {\mathbf {x}}) - \left| \Delta \Theta \right| * \left(1 - \lambda_ {P}\right), & \text {i f} \Delta \Theta > 0 \text {a n d} \lambda_ {A} <   0. 5 \\ P (\tilde {\mathbf {x}}) + \left| \Delta \Theta \right| * \left(1 - \lambda_ {P}\right), & \text {i f} \Delta \Theta \leq 0 \text {a n d} \lambda_ {A} <   0. 5 \end{array} \right. \tag {48} \\ y ^ {+} = \lambda_ {A} y _ {\mathbf {x}} + (1 - \lambda_ {A}) y _ {\bar {\mathbf {x}}}, \\ \end{array}
$$
where the coefficient for the  $\lambda_{A}$  is chosen from a beta distribution with  $\alpha \in [0.1, 0.4]$  within the same range of the original implementation [31]. The mixing for the phase is constrained to our original implementation with a uniform  $\lambda_{P} \sim U(0.9, 1)$ . We searched for the best  $\alpha$  value for each time-series task and augmentation method. Unlike linear mixup and our proposed approach, for cutmix, we followed the recommendation from the original paper and searched the  $\alpha$  value close to 1.
Table 29: Performance comparison in Activity Recognition within supervised learning scheme
Method	UCIHAR	HHAR	USC
ACC↑	MF1↑	ACC↑	MF1↑	ACC↑	MF1↑
W/o Augs.	65.66 ± 0.23	61.21 ± 0.15	91.58 ± 0.07	91.64 ± 0.11	71.93 ± 0.54	68.43 ± 0.78
Linear Mix	77.06 ± 0.18	73.21 ± 0.17	93.64 ± 0.17	93.67 ± 0.08	74.45 ± 0.28	71.93 ± 0.43
Amp Mix	70.96 ± 0.19	67.14 ± 0.33	92.50 ± 0.15	92.54 ± 0.10	74.02 ± 0.19	71.90 ± 0.26
Binary Mix	69.01 ± 0.36	71.63 ± 0.11	92.36 ± 0.19	92.42 ± 0.10	72.81 ± 0.15	70.98 ± 0.35
CutMix	67.14 ± 0.54	63.31 ± 0.48	90.37 ± 0.43	90.36 ± 0.76	57.89 ± 0.34	61.45 ± 0.57
Ours	81.60 ± 0.15	79.35 ± 0.13	94.02 ± 0.05	94.00 ± 0.06	74.85 ± 0.19	72.45 ± 0.34
Table 30: Performance comparison in Heart Rate Prediction within supervised learning scheme
Method	IEEE SPC12	IEEE SPC 22	DaLia
MAE↓	RMSE↓	MAE↓	RMSE↓	MAE↓	RMSE↓
W/o Augs.	20.01 ± 0.03	27.16 ± 0.05	20.29 ± 0.87	26.60 ± 1.13	6.58 ± 0.10	11.30 ± 0.58
Linear Mix	20.07 ± 0.09	26.93 ± 0.10	19.98 ± 0.12	24.90 ± 0.51	6.97 ± 0.14	12.07 ± 0.51
Amp Mix	20.14 ± 0.07	26.98 ± 0.07	19.61 ± 0.07	24.11 ± 0.21	11.20 ± 0.17	16.07 ± 0.43
Binary Mix	21.05 ± 0.13	27.02 ± 0.08	19.62 ± 0.10	25.23 ± 0.13	7.35 ± 0.16	12.17 ± 0.53
CutMix	20.12 ± 0.06	26.89 ± 0.11	19.64 ± 0.13	24.18 ± 0.20	10.78 ± 1.23	14.40 ± 1.43
Ours	19.97 ± 0.05	26.98 ± 0.10	19.45 ± 0.12	24.35 ± 0.18	6.49 ± 0.08	11.69 ± 0.10
Table 31: Performance comparison in CVD classification within supervised learning scheme
Method	CPSC 2018	Chapman
AUC↑	AUC↑
W/o Augs.	82.01 ± 0.51	92.27 ± 0.35
Linear Mix	80.29 ± 0.93	93.02 ± 0.33
Amp Mix	80.01 ± 0.36	89.11 ± 0.27
Binary Mix	78.10 ± 0.98	80.31 ± 0.36
CutMix	80.75 ± 0.78	89.17 ± 0.58
Ours	83.75 ± 0.32	95.26 ± 0.24