# Domain adaptation for handwriting trajectory reconstruction from IMU sensors

Florent Imbert, Romain Tavenard, Yann Soullard, Eric Anquetil

<figure><img src="image_1.png" /></figure>

## To cite this version:

Florent Imbert, Romain Tavenard, Yann Soullard, Eric Anquetil. Domain adaptation for handwriting trajectory reconstruction from IMU sensors. ICDAR 2024 Workshops, ADAPDA, Aug 2024, Athènes, Greece. [hal-04605593](https://hal.science/hal-04605593)

HAL Id: hal-04605593

https://hal.science/hal-04605593

Submitted on 7 Jun 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Domain adaptation for handwriting trajectory reconstruction from IMU sensors

Florent Imbert, Romain Tavenard, Yann Soullard, Eric Anquetil

► To cite this version:

Florent Imbert, Romain Tavenard, Yann Soullard, Eric Anquetil. Domain adaptation for handwriting trajectory reconstruction from IMU sensors. ICDAR 2024 Workshops, ADAPDA, Aug 2024, Athènes, Greece. pp.3-11, 10.1007/978-3-031-70645-5_1. hal-04605593

HAL Id: hal-04605593

https://hal.science/hal-04605593v1

Submitted on 7 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Domain adaptation for handwriting trajectory reconstruction from IMU sensors

Florent Imbert<fnref n="1" />, Romain Tavenard<fnref n="2" />, Yann Soullard<fnref n="2" />, and Eric Anquetil<fnref n="1" />

**Abstract.** Digital pens are commonly used to write on digital devices, providing the handwriting trace and enhancing human-computer interaction. This study focuses on a digital pen equipped with kinematic sensors, allowing users to write on any surface while simultaneously preserving a digital trajectory of handwriting. This technology holds significant potential as a valuable educational tool, particularly in classrooms where it can facilitate the process of learning to write. A major issue is based on the difference in captured signals between adults and children. For similar handwriting trace, we have large differences in sensor signals due to differences in speed and confidence in the handwriting gesture of children. To address this, we investigate a domain adaptation approach to build a unified intermediate feature representation aimed at facilitating the trajectory reconstruction. We demonstrate the interest of domain adaptation methods in leveraging existing knowledge for application in different contexts. Specifically, we compare our domain adaptation approach with two other methods: training the model from scratch and fine-tuning the model.

**Keywords:** Domain Adaptation · Online Handwriting · Trajectory Reconstruction · Digital Pen · Inertial Measurement Units · Deep Neural Network.

## 1 Introduction

This work focuses on reconstructing digital handwriting trajectories using the Digipen stylus [3]. The Digipen is equipped with 2 accelerometers, 1 gyroscope and a force sensor, and it can be used to write on any surface. While Inertial Measurement Units (IMU) are commonly used in tracking systems due to their cost-effectiveness, their signals are often imprecise due to high noise levels, presenting challenges. This issue is particularly prominent in the context of handwriting trajectory reconstruction, where precision is crucial, especially for e-learning purposes that require accurate feedback.

Although the Digipen stylus can be used on any surface and by different users, state-of-the art works [9, 10] have experimented handwriting reconstruction approaches from data written by adults on tablets. Using the Digipen in

<footnote n="1">¹ IRISA, Université de Rennes, INSA Rennes, Rennes, France</footnote>

<footnote n="2">² IRISA, Université Rennes 2, Rennes, France</footnote>

<footnote>florent.imbert@irisa.fr</footnote>
# Domain adaptation for handwriting trajectory reconstruction from IMU sensors

Florent Imbert<fnref n="1" />, Romain Tavenard<fnref n="2" />, Yann Soullard<fnref n="2" />, and Eric Anquetil<fnref n="1" />

**Abstract.** Digital pens are commonly used to write on digital devices, providing the handwriting trace and enhancing human-computer interaction. This study focuses on a digital pen equipped with kinematic sensors, allowing users to write on any surface while simultaneously preserving a digital trajectory of handwriting. This technology holds significant potential as a valuable educational tool, particularly in classrooms where it can facilitate the process of learning to write. A major issue is based on the difference in captured signals between adults and children. For similar handwriting trace, we have large differences in sensor signals due to differences in speed and confidence in the handwriting gesture of children. To address this, we investigate a domain adaptation approach to build a unified intermediate feature representation aimed at facilitating the trajectory reconstruction. We demonstrate the interest of domain adaptation methods in leveraging existing knowledge for application in different contexts. Specifically, we compare our domain adaptation approach with two other methods: training the model from scratch and fine-tuning the model.

**Keywords:** Domain Adaptation · Online Handwriting · Trajectory Reconstruction · Digital Pen · Inertial Measurement Units · Deep Neural Network.

## 1 Introduction

This work focuses on reconstructing digital handwriting trajectories using the Digipen stylus [3]. The Digipen is equipped with 2 accelerometers, 1 gyroscope and a force sensor, and it can be used to write on any surface. While Inertial Measurement Units (IMU) are commonly used in tracking systems due to their cost-effectiveness, their signals are often imprecise due to high noise levels, presenting challenges. This issue is particularly prominent in the context of handwriting trajectory reconstruction, where precision is crucial, especially for e-learning purposes that require accurate feedback.

Although the Digipen stylus can be used on any surface and by different users, state-of-the art works [9, 10] have experimented handwriting reconstruction approaches from data written by adults on tablets. Using the Digipen in

<footnote n="1">¹ IRISA, Université de Rennes, INSA Rennes, Rennes, France</footnote>

<footnote n="2">² IRISA, Université Rennes 2, Rennes, France</footnote>

<footnote>florent.imbert@irisa.fr</footnote>
another experimental context, e.g. on data written by children, or on another surface, e.g. on paper, leads to very different input signals. On the one hand, children's handwriting are of variable speed depending of the assertiveness in the handwriting gesture. On the other hand, handwriting on paper produces noisier signals due to friction than when the user writes on a tablet. This leads us to consider a domain adaptation method for dealing with the different domains of data.

In this context, we present a domain adaptation approach designed to enhance the adaptability of our model to deal with the different data sources. Initially trained on data acquired from tablets by adults, our model aims to effectively handle a additional types of data: handwriting acquired from tablets by children. In fact, while collecting labelled adult data on a tablet is not a problem, it's more complicated to collect data from children (contact a school). This approach is expected to broaden the applicability of handwriting trajectory reconstruction, making it more versatile and robust across different user groups. To our knowledge, no domain adaptation method addresses handwriting trajectory reconstruction from different sources, e.g. from adults vs children.

## 2 Related works

**Handwriting Trajectory Reconstruction** Traditional approaches [5, 7], which do not rely on deep learning, often struggle with sensor noise and error accumulation. These methods typically involve a series of preprocessing steps, such as applying low-pass filters to remove high-frequency noise and using coordinate transformation matrices to adjust for sensor orientation. However, the cumulative effect of noise and small errors during these preprocessing steps can lead to significant inaccuracies in the reconstructed trajectories over time.

Deep learning methods offer a more advanced approach to handling IMU data by learning and adapting to noise patterns, potentially providing more accurate trajectory reconstructions. For instance, [6] applied multi-task learning for joint classification and trajectory reconstruction. [4] explored magnetic signals, while [10] addressed the Stabilo Digipen, using linear interpolation to align pen and tablet signals as preprocessing in training. This alignment is crucial for matching sequences of variable sizes and having a point-to-point matching for training the network. A Convolutional Neural Network is used for online handwriting trajectory reconstruction.

Recently, [9] improved on [10] by proposing a complete pipeline for online handwriting trajectory reconstruction (Fig 1). Specifically, they used a specific preprocessing based on the Dynamic Time Warping (DTW) to align the ground truth with the input signal, which has the advantage of preserving the dynamics of handwriting, unlike the linear interpolation used in [10] and use a TCN model to extract local context information.

**Domain adaptation methods** Domain adaptation (DA) adjusts a model trained on one domain (source) to perform well on a different but related domain
another experimental context, e.g. on data written by children, or on another surface, e.g. on paper, leads to very different input signals. On the one hand, children's handwriting are of variable speed depending of the assertiveness in the handwriting gesture. On the other hand, handwriting on paper produces noisier signals due to friction than when the user writes on a tablet. This leads us to consider a domain adaptation method for dealing with the different domains of data.

In this context, we present a domain adaptation approach designed to enhance the adaptability of our model to deal with the different data sources. Initially trained on data acquired from tablets by adults, our model aims to effectively handle a additional types of data: handwriting acquired from tablets by children. In fact, while collecting labelled adult data on a tablet is not a problem, it's more complicated to collect data from children (contact a school). This approach is expected to broaden the applicability of handwriting trajectory reconstruction, making it more versatile and robust across different user groups. To our knowledge, no domain adaptation method addresses handwriting trajectory reconstruction from different sources, e.g. from adults vs children.

## 2 Related works

**Handwriting Trajectory Reconstruction** Traditional approaches [5, 7], which do not rely on deep learning, often struggle with sensor noise and error accumulation. These methods typically involve a series of preprocessing steps, such as applying low-pass filters to remove high-frequency noise and using coordinate transformation matrices to adjust for sensor orientation. However, the cumulative effect of noise and small errors during these preprocessing steps can lead to significant inaccuracies in the reconstructed trajectories over time.

Deep learning methods offer a more advanced approach to handling IMU data by learning and adapting to noise patterns, potentially providing more accurate trajectory reconstructions. For instance, [6] applied multi-task learning for joint classification and trajectory reconstruction. [4] explored magnetic signals, while [10] addressed the Stabilo Digipen, using linear interpolation to align pen and tablet signals as preprocessing in training. This alignment is crucial for matching sequences of variable sizes and having a point-to-point matching for training the network. A Convolutional Neural Network is used for online handwriting trajectory reconstruction.

Recently, [9] improved on [10] by proposing a complete pipeline for online handwriting trajectory reconstruction (Fig 1). Specifically, they used a specific preprocessing based on the Dynamic Time Warping (DTW) to align the ground truth with the input signal, which has the advantage of preserving the dynamics of handwriting, unlike the linear interpolation used in [10] and use a TCN model to extract local context information.

**Domain adaptation methods** Domain adaptation (DA) adjusts a model trained on one domain (source) to perform well on a different but related domain
<figure><img src="image_3.png" /><figcaption>Fig. 1. Pipeline of handwriting reconstruction.</figcaption></figure>

(target), mainly when labeled data are rare or unavailable in the target domain. The goal is to minimize domain shift, or differences in data distribution, between the source and target domains. DA can be:

- Supervised: Labeled data from the target domain are available.

- Semi-Supervised: Both labeled and unlabeled data from the target domain are available.

- Unsupervised: Only unlabeled data from the target domain are available.

Domain adaptation approaches can be broadly categorized into divergence-based, adversarial-based, and reconstruction-based methods. Divergence-based DA minimizes statistical differences between source and target distributions using measures like Maximum Mean Discrepancy (MMD) or Kullback-Leibler (KL) divergence [8]. These methods focus on reducing the distribution gap to improve model performance on the target domain. Adversarial-based DA, inspired by Generative Adversarial Networks (GAN), employs a discriminator and feature extractor to create domain-indistinguishable features through adversarial training [1]. This approach leverages the power of adversarial learning to align features from different domains, making them indistinguishable to a domain classifier. Reconstruction-based DA, uses autoencoders to maintain reconstruction ability across domains, thereby creating robust, domain-agnostic feature representations [2]. By ensuring that features can be accurately reconstructed regardless of the domain, these methods enhance the model's robustness and adaptability. Together, these approaches help models generalize better to target domains by aligning data distributions, creating domain-invariant features, or ensuring robust feature representations, thereby improving performance and reliability in diverse settings.

## 3 DANN-based method for handwriting reconstruction

To get a shared feature space for different data sources, we explore an adversarial-based DA method, named Domain-Adversarial Training of Neural Networks (DANN).
<figure><img src="image_2.png" /><figcaption>Fig. 1. Pipeline of handwriting reconstruction.</figcaption></figure>

(target), mainly when labeled data are rare or unavailable in the target domain. The goal is to minimize domain shift, or differences in data distribution, between the source and target domains. DA can be:

- Supervised: Labeled data from the target domain are available.

- Semi-Supervised: Both labeled and unlabeled data from the target domain are available.

- Unsupervised: Only unlabeled data from the target domain are available.

Domain adaptation approaches can be broadly categorized into divergence-based, adversarial-based, and reconstruction-based methods. Divergence-based DA minimizes statistical differences between source and target distributions using measures like Maximum Mean Discrepancy (MMD) or Kullback-Leibler (KL) divergence [8]. These methods focus on reducing the distribution gap to improve model performance on the target domain. Adversarial-based DA, inspired by Generative Adversarial Networks (GAN), employs a discriminator and feature extractor to create domain-indistinguishable features through adversarial training [1]. This approach leverages the power of adversarial learning to align features from different domains, making them indistinguishable to a domain classifier. Reconstruction-based DA, uses autoencoders to maintain reconstruction ability across domains, thereby creating robust, domain-agnostic feature representations [2]. By ensuring that features can be accurately reconstructed regardless of the domain, these methods enhance the model's robustness and adaptability. Together, these approaches help models generalize better to target domains by aligning data distributions, creating domain-invariant features, or ensuring robust feature representations, thereby improving performance and reliability in diverse settings.

## 3 DANN-based method for handwriting reconstruction

To get a shared feature space for different data sources, we explore an adversarial-based DA method, named Domain-Adversarial Training of Neural Networks (DANN).
**Motivation** The variability of handwriting sources complicates the task of handwriting reconstruction. Children handwriting poses a unique challenge due to the ongoing development of graphomotor skills, resulting in dynamic and inconsistent handwriting patterns as children learn to write. This translates into longer signal sequences than adults and a wider range of possible values (Fig. 3). This difference makes it difficult the handwriting reconstruction using a model trained on tablet-acquired data.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Visualization of the x component of the Digipen's rear accelerometer over time in seconds, from left to the right: a F from adult on tablet, a F from children on tablet. We notice that the same pattern is written but not with the same intensity due to the different level of automation of handwriting (adult vs child)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Data visualization with the Multidimensional Scaling (MDS) method, we can see that children's data on tablets (in red) takes on a wider range of values due to their handwriting which is still being learned.</figcaption></figure>

**DANN as a solution** Domain-Adversarial Training of Neural Networks [1] are specialized neural network architectures designed to address the challenge of domain shift, where a model trained on one domain (source) is expected to perform well on a different but related domain (target). These networks operate by learning features that are domain-invariant, meaning they are useful and
**Motivation** The variability of handwriting sources complicates the task of handwriting reconstruction. Children handwriting poses a unique challenge due to the ongoing development of graphomotor skills, resulting in dynamic and inconsistent handwriting patterns as children learn to write. This translates into longer signal sequences than adults and a wider range of possible values (Fig. 3). This difference makes it difficult the handwriting reconstruction using a model trained on tablet-acquired data.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Visualization of the x component of the Digipen's rear accelerometer over time in seconds, from left to the right: a F from adult on tablet, a F from children on tablet. We notice that the same pattern is written but not with the same intensity due to the different level of automation of handwriting (adult vs child)</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Data visualization with the Multidimensional Scaling (MDS) method, we can see that children's data on tablets (in red) takes on a wider range of values due to their handwriting which is still being learned.</figcaption></figure>

**DANN as a solution** Domain-Adversarial Training of Neural Networks [1] are specialized neural network architectures designed to address the challenge of domain shift, where a model trained on one domain (source) is expected to perform well on a different but related domain (target). These networks operate by learning features that are domain-invariant, meaning they are useful and
generalizable across both domains. This is typically achieved through a shared feature extractor on which additional components are built: a domain classifier and a task-specific classifier. The domain classifier is trained to determine the domain of the input data, whereas the task-specific classifier focuses on predicting the label of the dedicated task. In our context, the task-specific classifier is trained to reconstruct the handwriting trajectory.

Training involves a twist (Fig. 4): the domain classifier's gradients are reversed during backpropagation, which encourages the feature extractor to generate features that are indistinguishable between domains, thus fooling the domain classifier. This technique, known as adversarial training, helps the network to minimize the representation gap between the source and target domains, leading to better performance on the target domain without requiring extensive labeled data from it.

**Application** In this work, the reconstruction part of the DANN is the TCN-based network from [9] which is currently the state-of-the art model for handwriting reconstruction from the Digipen sensors. We named it baseline model (BM) in the following. Then we have slice the baseline model as follows: the 4 blocks of the non-causal TCN is the feature extractor (in green in Fig. 4). Each TCN block is composed of 2 convolutions with dilation 1 and 2 respectively and a kernel size of 3. The next two dense layers refer to the label predictor (in blue in Fig. 4) which in our context corresponds to the trajectory reconstruction. The domain classifier (in pink in the Figure 4), is made up of a max polling layer followed by two dense layers.

In our case study, we aim to transition from adult handwriting on a tablet (source) to children handwriting on a tablet (target). During training, we provide mixed matches to two model branches: (1) the feature extractor and the label predictor (green + blue in Fig. 4) pre-trained on adult data, and (2) a feature extractor and domain classifier (green + pink in Fig. 4).

<figure><img src="image_6.png" /><figcaption>Fig. 4. DANN from [1].</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. TCN model from [9]. Also named baseline model (BM).</figcaption></figure>
generalizable across both domains. This is typically achieved through a shared feature extractor on which additional components are built: a domain classifier and a task-specific classifier. The domain classifier is trained to determine the domain of the input data, whereas the task-specific classifier focuses on predicting the label of the dedicated task. In our context, the task-specific classifier is trained to reconstruct the handwriting trajectory.

Training involves a twist (Fig. 4): the domain classifier's gradients are reversed during backpropagation, which encourages the feature extractor to generate features that are indistinguishable between domains, thus fooling the domain classifier. This technique, known as adversarial training, helps the network to minimize the representation gap between the source and target domains, leading to better performance on the target domain without requiring extensive labeled data from it.

**Application** In this work, the reconstruction part of the DANN is the TCN-based network from [9] which is currently the state-of-the art model for handwriting reconstruction from the Digipen sensors. We named it baseline model (BM) in the following. Then we have slice the baseline model as follows: the 4 blocks of the non-causal TCN is the feature extractor (in green in Fig. 4). Each TCN block is composed of 2 convolutions with dilation 1 and 2 respectively and a kernel size of 3. The next two dense layers refer to the label predictor (in blue in Fig. 4) which in our context corresponds to the trajectory reconstruction. The domain classifier (in pink in the Figure 4), is made up of a max polling layer followed by two dense layers.

In our case study, we aim to transition from adult handwriting on a tablet (source) to children handwriting on a tablet (target). During training, we provide mixed matches to two model branches: (1) the feature extractor and the label predictor (green + blue in Fig. 4) pre-trained on adult data, and (2) a feature extractor and domain classifier (green + pink in Fig. 4).

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** DANN from [1].</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** TCN model from [9]. Also named baseline model (BM).</figcaption></figure>
## 4 Experimental results

We experiment our approach on two datasets, one for adult tablet data (9629 samples), another for children tablet data (3910 samples). Each dataset contains characters, words, word groups, equations, and shapes. Following [9], we compute the Fréchet distance to evaluate the quality of reconstruction on children's characters. We trained a DANN with children and adult data, and compared it qualitatively (Fig. 6) and quantitatively (Table 1) to the following methods: baseline model trained on adult data and fine-tuned on children data and the baseline model trained from scratch on children data.

<table><caption>**Table 1.** Comparison of reconstruction methods between training from scratch, fine tuning and domain adaptation method on children test data with the Fréchet distance.</caption><thead><tr><th>Model</th><th colspan="3">BM</th><th>DANN</th></tr><tr><th>Training Data</th><th>Adult</th><th>children</th><th>Adult & Children</th><th>Adult & children</th></tr></thead><tbody><tr><th>Learning strategy</th><td>From scratch</td><td>From scratch</td><td>Fine tuning<br>on children data<br>from a pretrain model<br>on adult data</td><td>DA<br>from pretrain model<br>on adult data</td></tr><tr><th>Fréchet distance</th><td>0.470</td><td><strong>0.345</strong></td><td>0.348</td><td>0.349</td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Comparison of reconstruction methods on children test data, on the first line the ground truth, on the second the Baseline Model trained on adult data, on the third the Baseline Model trained on child data, then the Baseline Model trained on adult data and fine-tuned on child data, on the last line the DANN pretrain on adult data.</figcaption></figure>

Table 1 shows that methods integrating children data perform the best overall. Specifically, all the methods trained on children's data improve the Baseline
## 4 Experimental results

We experiment our approach on two datasets, one for adult tablet data (9629 samples), another for children tablet data (3910 samples). Each dataset contains characters, words, word groups, equations, and shapes. Following [9], we compute the Fréchet distance to evaluate the quality of reconstruction on children's characters. We trained a DANN with children and adult data, and compared it qualitatively (Fig. 6) and quantitatively (Table 1) to the following methods: baseline model trained on adult data and fine-tuned on children data and the baseline model trained from scratch on children data.

<table><caption>**Table 1.** Comparison of reconstruction methods between training from scratch, fine tuning and domain adaptation method on children test data with the Fréchet distance.</caption><thead><tr><th>Model</th><th colspan="3">BM</th><th>DANN</th></tr><tr><th>Training Data</th><th>Adult</th><th>children</th><th>Adult & Children</th><th>Adult & children</th></tr></thead><tbody><tr><th>Learning strategy</th><td>From scratch</td><td>From scratch</td><td>Fine tuning<br>on children data<br>from a pretrain model<br>on adult data</td><td>DA<br>from pretrain model<br>on adult data</td></tr><tr><th>Fréchet distance</th><td>0.470</td><td><strong>0.345</strong></td><td>0.348</td><td>0.349</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Comparison of reconstruction methods on children test data, on the first line the ground truth, on the second the Baseline Model trained on adult data, on the third the Baseline Model trained on child data, then the Baseline Model trained on adult data and fine-tuned on child data, on the last line the DANN pretrain on adult data.</figcaption></figure>

Table 1 shows that methods integrating children data perform the best overall. Specifically, all the methods trained on children's data improve the Baseline
<table><caption>**Table 2.** Comparison of reconstruction methods between training from scratch, fine tuning and domain adaptation method on adult test data with the Fréchet distance.</caption><thead><tr><th>Model</th><th colspan="3">BM</th><th>DANN</th></tr><tr><th>Training Data</th><th>Adult</th><th>children</th><th>Adult & Children</th><th>Adult & children</th></tr></thead><tbody><tr><th>Learning strategy</th><td>From scratch</td><td>From scratch</td><td>Fine tuning<br>on children data<br>from a pretrain model<br>on adult data</td><td>DA<br>from pretrain model<br>on adult data</td></tr><tr><th>Fréchet distance</th><td><strong>0.332</strong></td><td>0.378</td><td>0.386</td><td><u>0.364</u></td></tr></tbody></table>

Model trained only on adult data. The advantage of DANN is that it keeps a common representation of adult and children's features, and table 2 shows that it performs well in both areas, making it a 2-in-1 solution. Unlike from scratch and fine tuning models (Table 1), which lose out on performance in the domain where they are not learned (Table 2). Regarding the qualitative analysis (Fig. 6), we observe that the trajectory reconstruction using the DANN is quite satisfactory and it seems closest to the ground truth than using the other approaches on those examples. The unique representation shared by the two data sources seems help the model in the trajectory reconstruction, especially on the hovering part. Additionally, this work is still in progress and the DANN has potential for further improvement.

## 5 Conclusion

This paper shows the benefits of retaining knowledge from one domain (adults on tablet) and moving on to a second (children on tablet). Our areas for improvement include optimizing the lambda parameter to adaptively manage the weight of each branch over time, which could enhance DANN performance. Another area for improvement involves creating DANN batches and studying factors such as padding. We will also investigate domain adaptation from data acquired on tablet to data written on paper.

**Acknowledgments.** This project is financed by the KIHT French-German bilateral ANR-21-FAI2-0007-01 project and these four partners, IRISA, KIT, Learn & Go and Stabilo. This work was performed using HPC resources from GENCI-IDRIS (Grant 2021-AD011013148)

**Disclosure of Interests.** The authors have no competing interests to declare that are relevant to the content of this article.

## References

1. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks (2016)
<table><caption>**Table 2.** Comparison of reconstruction methods between training from scratch, fine tuning and domain adaptation method on adult test data with the Fréchet distance.</caption><thead><tr><th>Model</th><th colspan="3">BM</th><th>DANN</th></tr><tr><th>Training Data</th><th>Adult</th><th>children</th><th>Adult & Children</th><th>Adult & children</th></tr></thead><tbody><tr><th>Learning strategy</th><td>From scratch</td><td>From scratch</td><td>Fine tuning<br>on children data<br>from a pretrain model<br>on adult data</td><td>DA<br>from pretrain model<br>on adult data</td></tr><tr><th>Fréchet distance</th><td><strong>0.332</strong></td><td>0.378</td><td>0.386</td><td><u>0.364</u></td></tr></tbody></table>

Model trained only on adult data. The advantage of DANN is that it keeps a common representation of adult and children's features, and table 2 shows that it performs well in both areas, making it a 2-in-1 solution. Unlike from scratch and fine tuning models (Table 1), which lose out on performance in the domain where they are not learned (Table 2). Regarding the qualitative analysis (Fig. 6), we observe that the trajectory reconstruction using the DANN is quite satisfactory and it seems closest to the ground truth than using the other approaches on those examples. The unique representation shared by the two data sources seems help the model in the trajectory reconstruction, especially on the hovering part. Additionally, this work is still in progress and the DANN has potential for further improvement.

## 5 Conclusion

This paper shows the benefits of retaining knowledge from one domain (adults on tablet) and moving on to a second (children on tablet). Our areas for improvement include optimizing the lambda parameter to adaptively manage the weight of each branch over time, which could enhance DANN performance. Another area for improvement involves creating DANN batches and studying factors such as padding. We will also investigate domain adaptation from data acquired on tablet to data written on paper.

**Acknowledgments.** This project is financed by the KIHT French-German bilateral ANR-21-FAI2-0007-01 project and these four partners, IRISA, KIT, Learn & Go and Stabilo. This work was performed using HPC resources from GENCI-IDRIS (Grant 2021-AD011013148)

**Disclosure of Interests.** The authors have no competing interests to declare that are relevant to the content of this article.

## References

1. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks (2016)
2. Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation (2016)

3. Harbaum, T., Serdyuk, A., Kreß, F., Hamann, T., Barth, J., Kämpf, P., Imbert, F., Soullard, Y., Tavenard, R., Anquetil, E., Delahaie, J.: KIHT: Kaligo-based Intelligent Handwriting Teacher. In: DATE 2024. Valencia, Spain (Mar 2024)

4. Liu, Y., Huang, K., Song, X., et al.: Maghacker: Eavesdropping on stylus pen writing via magnetic sensing from commodity mobile devices. In: Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. MobiSys '20, Association for Computing Machinery (2020). https://doi.org/10.1145/3386901.3389030, https://doi.org/10.1145/3386901.3389030

5. Miyagawa, T., Yonezawa, Y., Itoh, K., Hashimoto, M.: Handwritten pattern reproduction using pen acceleration and angular velocity. IEICE Trans. (2000)

6. Ott, F., Rügamer, D., Heublein, L., et al.: Joint classification and trajectory regression of online handwriting using a multi-task learning approach. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 266–276 (2022)

7. Pan, T.Y., Kuo, C.H., Hu, M.C.: A noise reduction method for imu and its application on handwriting trajectory reconstruction. In: 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). pp. 1–6 (2016). https://doi.org/10.1109/ICMEW.2016.7574685

8. Shen, J., Qu, Y., Zhang, W., Yu, Y.: Wasserstein distance guided representation learning for domain adaptation (2018)

9. Swaileh, W., Imbert, F., Soullard, Y., Tavenard, R., Anquetil, É.: Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network. International Journal on Document Analysis and Recognition (IJDAR) pp. 1–14 (2023)

10. Wehbi, M., Luge, D., Hamann, T., et al.: Surface-free multi-stroke trajectory reconstruction and word recognition using an imu-enhanced digital pen. Sensors (2022)
2. Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation (2016)

3. Harbaum, T., Serdyuk, A., Kreß, F., Hamann, T., Barth, J., Kämpf, P., Imbert, F., Soullard, Y., Tavenard, R., Anquetil, E., Delahaie, J.: KIHT: Kaligo-based Intelligent Handwriting Teacher. In: DATE 2024. Valencia, Spain (Mar 2024)

4. Liu, Y., Huang, K., Song, X., et al.: Maghacker: Eavesdropping on stylus pen writing via magnetic sensing from commodity mobile devices. In: Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. MobiSys '20, Association for Computing Machinery (2020). https://doi.org/10.1145/3386901.3389030, https://doi.org/10.1145/3386901.3389030

5. Miyagawa, T., Yonezawa, Y., Itoh, K., Hashimoto, M.: Handwritten pattern reproduction using pen acceleration and angular velocity. IEICE Trans. (2000)

6. Ott, F., Rügamer, D., Heublein, L., et al.: Joint classification and trajectory regression of online handwriting using a multi-task learning approach. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 266–276 (2022)

7. Pan, T.Y., Kuo, C.H., Hu, M.C.: A noise reduction method for imu and its application on handwriting trajectory reconstruction. In: 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). pp. 1–6 (2016). https://doi.org/10.1109/ICMEW.2016.7574685

8. Shen, J., Qu, Y., Zhang, W., Yu, Y.: Wasserstein distance guided representation learning for domain adaptation (2018)

9. Swaileh, W., Imbert, F., Soullard, Y., Tavenard, R., Anquetil, É.: Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network. International Journal on Document Analysis and Recognition (IJDAR) pp. 1–14 (2023)

10. Wehbi, M., Luge, D., Hamann, T., et al.: Surface-free multi-stroke trajectory reconstruction and word recognition using an imu-enhanced digital pen. Sensors (2022)