SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To demonstrate the effectiveness of unsupervised learning in our task, we tune the parameters of Affine Transformation layer by supervised learning (i.e., supervised domain adaptation) to obtain its "upper bound". Even trained in an unsupervised manner, our method achieves results comparable to its supervised version (Table 1). This difference of 1.5 degrees indicates that our unsupervised learning approach is effective.
Table 1: Results of comparative experiments.
Method	MAE (deg)
MMD [49]	28.73 ± 8.50
D-CORAL [16]	13.83 ± 8.45
DAAN [23]	16.80 ± 7.88
DSAN [50]	14.03 ± 5.88
AdvSKM [27]	13.05 ± 5.19
HoMM [18]	11.03 ± 4.42
Ours	7.32 ± 2.85
Supervised	5.63±1.95
Experiments on different participants To further demonstrate the generalization ability of our method, we conducted experiments with five additional participants of varying body types (see the supplementary materials for body profile details). Each of these five participants wore the devices in three supervised on-body positions and performed three distinct physical activities sequentially: ping-pong, basketball, and boxing. Each activity lasted for a minimum duration of one minute. In total, this comprehensive evaluation approach yielded 15 (5 participants  $\times$  3 on-body displacements) unique data segments across all participants, comprising 81,848 total frames. For each participant,
Table 2: Experimental results on five additional participants.
User ID	Averaged MAE (deg)
D-CORAL	DSAN	HoMM	AdvSKM	Ours
1	11.84 ± 1.49	11.86 ± 1.57	11.96 ± 1.73	11.84 ± 1.80	12.07 ± 1.04
2	16.59 ± 1.81	17.04 ± 1.80	16.53 ± 1.70	16.63 ± 0.95	16.69 ± 2.34
3	14.94 ± 5.89	13.68 ± 4.19	15.24 ± 6.93	15.71 ± 6.89	12.70 ± 2.79
4	11.76 ± 1.11	12.04 ± 1.16	12.06 ± 1.49	12.08 ± 1.73	9.30 ± 2.77
5	19.41 ± 4.37	19.32 ± 4.34	20.58 ± 5.88	19.44 ± 4.21	13.96 ± 2.88
Avg	14.91 ± 0.34	14.79 ± 0.22	15.27 ± 0.48	15.14 ± 0.18	12.94 ± 0.25
we conducted 5 times 3-fold cross-validation experiments and recorded the average result. As Table 2 shows, our method achieves optimal or near-optimal results for all five additional users, indicating the effectiveness of our method against user diversity.
To demonstrate the effectiveness of the three novel techniques proposed, we conduct an ablation study as follows:
- FFE: Remove the Fourier Feature Encoding (FFE) layer from the LSTM network.
- Affine: Remove the Affine Transformation layer and tune all the parameters of the network during adaptation after supervised pretraining. For the adaptation, we use a weighted sum of two loss terms: i) a supervised loss  $\mathcal{L}_{mse}$  applied on the training data; and ii) our sequence discrepancy loss  $\mathcal{L}$  applied on the test data.
- SD Loss: Replace our Sequence Discrepancy (SD) loss with a more naive and stricter version:
$$
\mathcal {L} ^ {\prime} = \sum_ {t} \sum_ {k = 0} ^ {n - 1} \left(\hat {\theta} _ {t} ^ {(k)} - \overline {{\Theta}} _ {t}\right) ^ {2} \tag {10}
$$
where  $\overline{\Theta}_t = \mathrm{mean}(\hat{\theta}_t^{(0)},\dots,\hat{\theta}_t^{(n - 1)})$
Table 3: Ablation study.
Case	FFE	Affine	SD Loss	MAE
1	-	-	-	13.40±5.58
2	+	-	-	11.81±6.65
3	-	+	-	12.58±8.62
4	-	-	+	14.36±5.96
5	-	+	+	10.27±4.20
6	+	-	+	12.36±6.99
7	+	+	-	9.38±8.78
Ours	+	+	+	7.32±2.85
We have enumerated all combinations of the three components proposed. As Table 3 shows, it can be observed that: i) Ours achieves the best performance, indicating the effectiveness of the three novel techniques proposed; ii) FFE and affine transformation improve the performance in all cases; iii) the effectiveness of SD loss relies on affine transformation. In both cases with and without FFE, the co-existence of affine transformation and SD loss show better performance than using SD loss alone (Ours over Case 6, Case 5 over Case 3).
5.4 Justification of Motivation
Affine Transformation Figure 4 demonstrates the effective alignment of the target and source domains with our method. As shown in Figure 4a and 4b, although light-weight, the proposed Affine Transformation layer effectively aligns the sensor reading distributions of test data (blue) to those in the training set (orange), thereby significantly improving the accuracy of joint angle estimation.
Sequence Discrepancy Loss The design rationale of this component is rooted in the insight that our sequence discrepancy loss  $\mathcal{L}$  shares a similar optimization landscape with the estimation error of joint angles during optimization (Figure 5). This consistency effectively offers the advantage of
(a) Before adaptation
(b) Ours
Figure 4: Original and adapted data distributions of our method, HoMM and AdvSKM. Note that due to method characteristics, our method is aligned directly using the sensor reading, while the latter two are aligned in the latent space. "dim 1" and "dim 2" are obtained using t-SNE.
(c) HoMM
(d) AdvSKM
leading the network optimization toward the optimal point. As shown in Figure 6, i) before adaptation, the outputs of different regressors significantly differ from each other, leading to a high sequence discrepancy; ii) after adaptation, the discrepancy among regressors reduced and the accuracy of joint angle estimates for all regressors was improved significantly at the same time.
Fourier Feature Encoding Existing literature shows the existence of spectral bias: the tendency of neural networks to prioritize learning low-frequency functions [48]. Figure 7a shows the frequency distribution difference between the prediction and ground truth. Without the component of FFE, the network output shows a large deviation in the region of low-frequency domain from the ground truth (on the left side of this plot). This over-emphasis on the low-frequency components inevitably dominates the prediction capability and weakens the reconstruction of the high-frequency signals.
Our experiment confirms that Fourier Feature Encoding (FFE) reduces the spectral bias of neural networks. Therefore, as Figure 7 shows, our method can approximate the mapping between sensor readings and joint angles in a more faithful way.
(a) Error w.r.t. Bias
Figure 5: Rationale of our Sequence Discrepancy (SD) loss ( $c = 5$ ,  $l = 0$ ). Our SD loss shares similar minima with the MAE errors. Error: errors of joint angle estimation. As defined in Eq. 2, Bias  $B = [b_{1}, b_{2}]^{T}$  and Scaling  $W_{s} = \text{diag}[s_{1}, s_{2}]$ .
(b) SD loss w.r.t. Bias
(c) Error w.r.t. Scaling
(d) SD loss w.r.t. Scaling
(a) Before adaptation
Figure 6: Justification of Sequence Discrepancy Loss  $(c = 5, l = 0)$ . The accuracy of joint angle estimation improves along with the reduction of discrepancy among regressors. A1, A5, A9: auxiliary regressors 1, 5, 9. Main: main regressor. GT: ground truth.
(b) After adaptation
(a) Frequency Domain
Figure 7: Justification of Fourier Feature Encoding (FFE)  $(c = 0, l = -1)$ . (a) Time series of network outputs with/without FFE and the ground truth in the time domain. (b) Differences of network outputs between with/without FFE and the ground truth in the frequency domain.
(b) Time Domain
Limitations The performance of the proposed method is dependent on the quality of the data in both the training and test sets. When the data quality is poor (e.g., noisy), the error in joint angle prediction may still be high even if the sensor readings are well aligned (see Figure 8a and 8b).
(a) Before adaptation
Figure 8: Failure cases (noisy data).  $x$ -axis and  $y$ -axis ([ $r_1, r_2$ ]): sensor readings.  $z$ -axis: joint angles. Blue and Orange: displacements in the training and test sets, respectively.
Specifically, according to the frequency characteristics of elbow movement, we define the sensor signal greater than  $5\mathrm{Hz}$  as noise, and calculated the signal-to-noise ratio (SNR) for the sensor signals within the 11 data sets collected from a single user. Furthermore, we compiled the mean absolute error (MAE) for each set after implementing our adaptive approach. It can be found that our proposed method exhibits significant effectiveness when the SNR exceeds  $10\mathrm{dB}$ . Conversely, maintaining satisfactory outcomes becomes challenging when the SNR falls below  $10\mathrm{dB}$  (see the supplementary materials for the visualization).
Future Work As abovementioned, we believe that extending our method to learning with noisy data would be interesting future work. In addition, our work opens up a number of research directions for future efforts. First, in most domain adaptation methods, measuring the similarity between the distribution of the target domain and the source domain is a key step. While in our work, we found that the output of the auxiliary regressors can be used as an indicator of data distribution shifts from the source domain, and verified the feasibility of domain adaptation using only target domain data in flexible sensor applications. In future work, we hope to implement this idea in more diverse applications. Second, although we have only used translation and scaling, this work proves affine-based domain adaptation to be a promising solution to mitigate the data distribution shifts of flexible sensors. The effectiveness of rotation and reflection remains to be studied in our future work.
Our work proposes a novel self-adaptive motion tracking network to address the challenging data distributional shifts caused by on-body displacements of flexible sensors. To mitigate the effects of such displacements, we propose three novel techniques. First, we propose an Affine Transformation layer that can remap shifted data distributions to those in the training set efficiently. Second, we propose a Fourier-encoded LSTM network that can learn richer frequency components in the input signals and thus improves the accuracy of joint angle estimation. Finally, we propose a Sequence Discrepancy loss equipped with auxiliary regressors that can adapt the parameters of Affine Transformation effectively in an unsupervised manner. Experimental results show that our method can effectively adapt to unknown displacements of flexible sensors worn at different positions.
This work is supported by National Natural Science Foundation of China (62072383), the Fundamental Research Funds for the Central Universities (20720210044), and partially supported by Royal Society (IEC \NSFC \211022).