# Does modifying visual feedback facilitate learning to write?

Jean-François Connan, Marianne Jover, Alexandrine Saint-Cast, Jérémy Danna

<figure><img src="image_1.png" /></figure>

## To cite this version:

Jean-François Connan, Marianne Jover, Alexandrine Saint-Cast, Jérémy Danna. Does modifying visual feedback facilitate learning to write?. 20th Conference of the International Graphonomics Society, Jun 2022, La Palmas de Gran Canaria, Spain. hal-03971708

HAL Id: hal-03971708

https://hal.science/hal-03971708

Submitted on 15 Feb 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Does modifying visual feedback facilitate learning to write?

Jean-François CONNAN<sup>a,b,c</sup>, Marianne JOVER<sup>a</sup>, Alexandrine SAINT-CAST<sup>b</sup> and Jérémy DANNA<sup>c</sup>

<sup>a</sup> Aix-Marseille Univ, PSYCLE,

Maison de la recherche, 29 avenue Robert Schuman, 13621 Aix-en-Provence, FRANCE

<sup>b</sup> Institut Supérieur de Rééducation Psychomotrice, ISRP,

19-25 rue Gallieni, CS 20178, 92641 Boulogne Billancourt, FRANCE

<sup>c</sup> Aix-Marseille Univ, LNC,

3 place Victor Hugo, 13003 Marseille, FRANCE

jfconnan@isrp.fr, marianne.jover@univ-amu.fr, asaintcast@isrp.fr, jeremy.danna@univ-amu.fr.

**Abstract.** The purpose of handwriting is to produce a legible trace quickly and fluently. Its motor control and learning therefore rely on an efficient integration of visual and proprioceptive feedback, with a transition from a control based on the written trace in beginner to a control based mainly on writing movements in expert writer. The aim of this study was to test the effect of a partial deletion of the written trace and an increase of visual information on the kinematics after the performance in a learning task. Twenty-four adults learned to write six new pseudoletters with a stylus controlled by non-dominant hand on a touch screen digital tablet. Half of the pseudoletters were trained in the modified visual FB conditions and the other half without modification (control condition). Results revealed that, in the short-term, the pseudoletters trained with the modified visual feedback were written faster and more fluently than those trained in the control condition, without spatial accuracy reduction. This method seems to be efficient, at least in proficient adults and is currently tested in children with dysgraphia.

## 1. Introduction

Digital tablets in school classrooms enable new methods to help students learn to write, thanks to real-time computer-assisted feedback (FB). For example, it is possible to change the digital trace on the screen in order to add other visual information. Danna & Velay (2015) have listed different strategies, such as changing the color of the trace to inform on the writing speed or its thickness depending on the pressure exerted by the pen on the tablet. Both strategies have been tested in adults and appear to be promising (Loup-Escande et al., 2017). However, care must be taken not to overload the visual information processing. Indeed, during the learning process, visual control is used significantly at the beginning, then gradually decreases making way for a more predictive control. A strategy for optimizing the learning or the rehabilitation of handwriting would be to facilitate this transition from a control based on the written trace to a control based on the handwriting movement. To this aim, it is possible to decrease the visual perception of the written trace, totally or partially, to lead the writer to focus on the ongoing movement. The total deletion of visual FB has already been tested in adults who were asked to write without seeing their hand, and the authors reported a positive effect on writing speed and fluency, but the effect on the legibility of the trace has not been evaluated (Portier & van Galen, 1992). This effect was tested more recently by Bara & Bonneton-Botté (2021) who asked young children to learn isolated cursive letters without seeing the written trace. They confirmed a positive effect on the kinematics of handwriting, but the total trace deletion impacted the quality of the written trace. This deterioration can be explained by the participants' lack of knowledge of the outcome after the trial.

To our knowledge, the effects of a partial deletion of the written trace has never been tested. This idea is inspired by the *Light Drawing* (or *Light Painting*, e.g., Hu et al., 2020), a photographic technique of moving a light source while taking a long exposure photograph, either to illuminate a subject or space, or to shine light at the camera to 'draw'. The writer sees only the point of light from the tip of a pen (equipped with a light emitting diode) during the trial and sees the product of his/her performance right after. During the movement, the light point becomes a luminous trail that follows the moving pen. This luminous trail results from the retinal persistence, and lasts several tens of milliseconds after a disappearance of a stimulus. As a result, the length of this luminous trail depends on the movement velocity: the longer the trail, the faster the movement. Therefore, this technique provides a real-time visual FB on both the position of the written trace and the movement velocity. We have transposed this technique on a digital tablet and tested this method of decreasing visual FB during action on learning to write. After a pilot study (Connan et al., 2021) showing a limited effect of this strategy, we added another condition, based on supplementary visual FB on movement fluency after the performance. In this latter case, the trial was produced without modification of the written trace. The present study aims at testing the effect of this mixed method on learning to write a pseudoletter in adults, with their nondominant hand.

## 2. Method
**2.1 Participant**

Twenty-four right-handed adults (mean age: 24.6 +− 6.32 SD years, 22 women) volunteered for the experiment. None of the participants presented any known neurological or attentional deficits, as determined by a detailed questionnaire prior to the experiment. They all had normal or corrected-to-normal vision. All participants signed a written informed consent before starting the experiment.

**2.2 Task and procedure**

The task consisted in learning to write six new pseudoletters with the non-dominant hand (see Fig. 1). Participants were asked to write the pseudoletters on a touch screen digital tablet (Windows surface pro® 12.3 inches; sampling rate at 60 Hz) with a stylus (surface pen, HB lead). They were required to reproduce it in a square (4x4 cm) under the model, which remained displayed during the trial.

<figure><img src="image_3.png" /><figcaption>**Figure 1.** Description of the experimental design (A) and pseudoletters (B). The pseudoletters were created from the combination of elementary strokes. The pseudoletters were divided into two sets of three. Each set included a 4-strokes pseudoletter, a 5-strokes pseudoletter, and a 6-strokes pseudoletter. The participants learned one set in the control condition and the other one in the modified visual FB conditions.</figcaption></figure>

The experiment began with a short familiarization with the tablet and the two modified visual FB. The learning task included a pre-test, a training session, and two post-tests. In the pre-test (PRE), the participants had to reproduce six pseudoletters twice without FB modification. During the training session, participants wrote 10 times one of the six pseudoletters, with or without modified visual FB according to the experimental condition. After each training session, the participant had to reproduce two times the trained pseudoletter without FB modification (short-term post-test -ST_PST). The following day, two post-tests were performed. First, the participants had to recognize the six learned pseudoletters from a set of twenty-four pseudoletters (recognition task). The second long-term post-test (LT_PST) was exactly the same the PRE and ST_PST. The presentation order of the six pseudoletters was the same between the PRE and LT_PST. In both training and test sessions, the model remained displayed during the trial.

**2.3 Visual FB Modification**

During the training session under the experimental condition, two visual FB modifications were applied, either a real-time visual modification during the trial (called “snake”) or a postponed additional visual FB after the trial (called “enriched postponed”).

**Snake** (trials 1, 3, 5, 7, 9): The black trace of the pen's movement remained displayed for a time window of 192 ms and then disappeared. This condition created a real-time animation of a *black snake* following the pen tip movement, whose size varied according to the velocity of the pen's movement and disappeared when the pen stopped moving. This time window has been chosen empirically from a pilot experiment (Connan et al., 2021), so that the writer can see the last stroke of the letter he is writing. At the end of the trial, the pseudoletter produced by the participant appeared in its entirety and the model remained displayed on the screen for visual comparison.

**Enriched postponed** (trials 2, 4, 6, 8, 10). During an enriched postponed trial, the writer wrote the pseudoletter with a usual visual FB of the written trace. At the end of the trial, the model and the pseudoletter produced by the participant remained displayed on the screen. Over the pseudoletter, red dots appeared wherever the movement was less fluent. These red dots corresponded to the abnormal velocity peaks determined by the Signal-to-Noise velocity peaks difference (SNvpd, Danna et al., 2013).

As can be seen in Fig. 1, three pseudoletters were trained with both “snake” and “enriched postponed” trials (modified visual FB condition) and three others were trained in control condition (without visual FB modification). The conditions and pseudoletters orders were counterbalanced between the participants.

**2.4 Data analysis**

From the position and the axial pressure of the stylus, we segmented the signal to distinguish the written segments from the lifts. Only the written trace was analyzed through five variables, namely a) the mean velocity (millimeters/second), b) the number of abnormal velocity peaks, c) the mean stylus pressure (arbitrary unit – A.U. – normalized between 0 and 1), d) the trace length (millimeters) that corresponds to the total distance traveled by
the stylus from the stylus was first in contact with the tablet until the pseudoletter was completed, and e) the spatial error between the produced pseudoletter and a reference pseudoletter.

A total of 2304 data were collected. Fifty trials (2.17%) were excluded for the statistical analysis because of recording problem or outliers. A comparison of performance during the tests with Omnibus tests based on a Linear Mixed Model (LMM) (LMM, GAMLj module, Galluci, 2019) were conducted with JAMOVI® (The jamovi project, 2020; R Core Team, 2019). We realized Omnibus tests with three "Test" conditions (PRE, ST_PST, and LT_PST) and two "FB" conditions (with or without modified visual FB) as fixed factors with repeated measures, and with the "participants" and "pseudoletters" as random factors. All significance levels were set at p = 0.05. Fisher LSD post hoc tests with Bonferroni correction were applied for multiple comparisons.

We also tested whether the “FB” condition had an impact on the recognition of learned pseudoletters in a recognition test. We observed no significant difference between the pseudoletters learned with or without the modified visual FB.

## 3. Results

The average evolution of velocity, movement dysfluency, and stylus pressure at the tests and during the training were presented on the figure 2.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Mean velocity, dysfluency, and stylus pressure during the test (PRE-TEST, ST PST and LT PST) and training phases. Error bars correspond to the standard error.</figcaption></figure>

**Velocity:** The analysis revealed a main effect of Test on velocity (F(2,806) = 82.72, p < 0.001) and a FB\*Test interaction (F(2,806) = 4.39, p < 0.05). The post hoc tests revealed, both between the PRE and the ST_PST and between the PRE and the LT_PST, a significant increase for pseudoletters learned in the control (p < 0.001 for both comparisons) and the modified (p < 0.001 for both comparisons) conditions. Interestingly, as can be seen in fig 2A, the velocity at ST_PST was higher for pseudoletters learned in the modified condition than in the control condition (p < 0.05).

**Movement dysfluency:** The analysis revealed a main effect of Test (F(2,806) = 44.79, p < 0.001) and a FB\*Test interaction (F(2,806) = 6.82, p < 0.001). The post hoc comparisons between PRE and ST_PST revealed a significant decrease of dysfluency for pseudoletters learned both in the control (p < 0.001) and the modified (p < 0.001) conditions. The comparisons between PRE and LT_PST revealed a significant decrease for the pseudoletters learned in the modified condition (p < 0.001) whereas this difference did not reach the significant threshold for the pseudoletters learned in the control condition (p = 0.06). Finally, as can be seen in fig 2B, the dysfluency at ST_PST was lower for the pseudoletters learned in the modified condition than those learned in the control condition (p < 0.001).

**Trace length:** The analysis revealed a main effect of Test (F(2,806) = 3.997, p < 0.05). The post-hoc tests revealed that the trace length tends to increase between the PRE and the ST_PST (p = 0.06) and increases significantly between the PRE and LT_PST (p < 0.05). No difference was observed between the two post-tests. Finally, the analysis did not reveal a significant effect of FB, nor any interaction between factors.

**Stylus pressure:** The analysis revealed main effects of Test (F(2,806) = 29.8, p < 0.001) and FB (F(1,806) = 16.5 p < 0.001), and a significant FB\*Test interaction (F(2,156) = 4.75, p < 0.01). As can be seen in fig 2C, the post hoc tests revealed a significant increase of pressure between the PRE and LT_PST (p < 0.001) for the pseudoletters learned both in the control and the modified conditions (p < 0.01 for both comparisons). Surprisingly the pseudoletters learned in the modified condition was produced with a lower pressure than those learned in the control condition in the ST_PST (p < 0.001).

**Spatial error:** The analysis revealed a main effect of Test (F(2,806) = 3.84, p < 0.05). The post-hoc tests did not reveal any differences between the PRE and the two post-tests. Only a decrease of spatial error between ST_PST
and LT_PST was observed (p < 0.05). The analysis did not reveal a significant effect of FB and interaction between factors.

## 4. Discussion

This study aimed to evaluate the effect of visual modification of the digital trace on a digital tablet in a pseudoletter learning task. The results revealed a short-term effect in favor of our mixed visual modification method. Writers wrote the pseudo-letters faster and more fluently, while remaining as accurate, when trained with the modified visual FB than those trained in a control condition.

In this experiment, we chose a mixed FB modification strategy. One in two trials, the partial deletion of the trace decreased spatial information but increased writing speed information (the faster the subject the longer the trace). Furthermore, we hypothesized that not seeing the shape of the letter during execution would increase the speed of the movement due to a decrease in online adjustments related to back and forth between the model and the production. This hypothesis is in line with Van Doorn and Keuss (1992) who observed an increase in latency in the case of writing without hand and stylus vision. This increase in latency would be related to the additional processing of information prior to the action, i.e. a pro-active strategy. In contrast to Bara & Bonneton (2021)'s study in children, the writer had access to the full shape of the pseudoletter after the performance to check if the shape is correct. On the next trial, we added visual information about the fluency of the movement once the pseudoletter was written in usual condition (without modification of the trace). The writer thus had different visual FB depending on the trial, but in all cases, the modification led the writer to focus on the writing movement. The results of our study suggested that the mixed strategy facilitated the learning process at the kinematic level (speed and dysfluency) without affecting spatial accuracy. We assume that such an alternation of visual FB modification limits the dependency phenomena, as Winstein & Schmidt (1990) already showed in the field of motor learning.

Surprisingly, we also observed that pseudoletters learned in the modified visual FB condition were produced with less pressure than those trained in the control condition. This finding is not in line with the literature. Indeed, many authors suggested that an increase in pen pressure would result from a strategy of maximizing proprioceptive information when visual FB are minimized (Van Doorn, 1992). In the present study, we deem that the partial vision of the trace allowed the participants to see the contact between the pen and the surface. This might have refrained them to press harder.

In the long term, the specific benefits of the visual modification disappeared, with a positive effect of the training whatever the condition. It is possible that the number of trials was not important enough to obtain long term effects. In conclusion, this method seems to be efficient, at least in proficient adults. Care must be taken to ensure that such suppression of FB does not come at the expense of trace quality in younger children for whom visual FB is crucial until the representation of letter shape is complete (Chartrel & Vinter, 2006). We are currently testing it in a rehabilitation protocol with children with dysgraphia.

## References

* Danna, J., & Velay, J.-L. (2015). Basic and supplementary sensory feedback in handwriting. *Frontiers in Psychology* 6:169.

* Loup-Escande, E., Frenoy, R., Poplimont, G., Thouvenin, I., Gapenne, O., & Megalakaki, O. (2017). Contributions of mixed reality in a calligraphy learning task: Effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance. *Computers in Human Behavior*, *75*, 42-49.

* Portier, S.J., & Van Galen, G.P. (1992). Immediate vs. postponed visual feedback in practicing a handwriting task. *Hum. Mov. Sci.*, *11*, 563-592.

* Bara, F., & Bonneton-Botté, N. (2021). Handwriting isolated cursive letters in young children: Effect of the visual trace deletion. *Learning & Instruction*, *74*, 101439.

* Hu, J., Lu, L., Zhu, M., Li, Z., Wang, C., Gu, S., Meng, L., Liu, H., Hu, Z., & Xu, Y. (2020). The cognitive and socioemotional effects of a short-term light painting intervention. *Social Behav. Personality*, *48(8)*, e9270.

* Connan, J.F., Jover, M., Saint-Cast, A., & Danna, J. (2021). Comment les nouvelles technologies peuvent-elles aider le scripteur à mieux écrire ? *Étude pilote sur la modification de la perception visuelle de la trace*. A.N.A.E., 170, 77-89.

* Danna, J., Paz-Villagrán, V., and Velay, J. L. (2013). Signal-to-Noise velocity peak difference: a new method for evaluating the handwriting movement fluency in children with dysgraphia. *Res. Dev. Disabil.* 34, 4375-4384.

* Galluci, M. (2019). *GAMLj*: General analyses for linear models. [jamovi module]. Retrieved from https://gamlj.github.io/.

* R Core Team (2019). R: A Language and environment for statistical computing. (Version 3.6) [Computer software]. Retrieved from https://cran.r-project.org/.

* The JAMOVI Project (2020). Jamovi. (Version 1.2) [Computer Software]. Retrieved from https://www.jamovi.org.

* Chartrel, E., & Vinter, A. (2006). Rôle des informations visuelles dans la production de lettres cursives chez l'enfant et l'adulte. *Annee. Psychol. 106*, 45-65. doi: 10.4074/S0003503306001047

* Van Doorn, RRA (1992). Axial pen pressure during handwriting movements in conditions with and without vision. *Journal of Human Movement Studies*, 23, 41-52.