Does Finger-Tracking Point to Child Reading Strategies?
Authors: Claudia Marzi, Anna Rodella, Andrea Nadalini, Loukia Taxitari, Vito Pirrelli (Università di Pisa, 2020)
Topic: Multimodal

Abstract
The movement of a child's index finger that points to a printed text while (s)he is reading may provide a proxy for the child's eye movements and attention focus. We validated this correlation by showing a quantitative analysis of patterns of “finger-tracking” of Italian early graders engaged in reading a text displayed on a tablet. A web application interfaced with the tablet monitors the reading behaviour by modelling the way the child points to the text while reading. The analysis found significant developmental trends in reading strategies, marking an interesting contrast between typically developing and atypically developing readers.

Introduction
Recent experimental evidence in visual perception analysis (Lio et al., 2019) shows that eye movements and finger movements strongly correlate during scene exploration, at both individual and group levels. In Lio et al.’s (2019) experiment, subjects are invited to explore a blurred image displayed on a touchscreen by moving their fingers on the display. Picture areas that are located immediately above the touch point of the subject’s finger on the screen are automatically shown in high resolution, thereby simulating the subject’s central (foveal) vision. The experiment proves that the subjects’ image-exploring patterns in the two modalities (optical and tactile) are highly congruent. The result is not surprising. A familiar context which exploits this synergistic behaviour is when children are learning to read. Despite the undoubtedly different dynamics of the two types of text exploration, finger-pointing to text helps children learn to look at print, and supports critical early reading behaviours: directional movement, attention focus, and voice-print match (Mesmer and Lake, 2010; Uhry, 2002). ReadLet (Ferro et al., 2018a; Ferro et al., 2018b) is a web application with a tablet front-end, designed to support online monitoring of silent and oral reading abilities through finger-tracking. Finger-tracking consists of recording the time series of touch events on the tablet screen where a child is reading a short story, while the child is pointing to the text with the index finger of her dominant hand.2 Preliminary analyses of our finger-tracking data (Pirrelli et al., 2020) highlighted a diminishing influence of word frequency and word length on reading time as readers get older and more proficient (from 3rd to 6th grades). With increasing exposure to written words, differences in tracking time between high and low frequency words gradually tend to decrease, suggesting a ceiling effect in the entrenchment of both high- and low-frequency lexical representations in long-term memory (Zoccolotti et al., 2009). Similarly, word length was found to significantly interact across grades. Younger readers show increasing difficulty with longer words, with a steeper time increment for word length > 6, while older readers are slowed down when words are longer than 8 letters. This integrates previous evidence (De Luca et al., 2008), confirming that not even the most experienced readers can avoid the slowing down effect of word length. The two-fold interaction of word frequency and word length with grade levels strongly suggests that Italian children use a lexical route to decoding even at early stages of their reading development, despite the transparency of Italian orthography (Bates et al., 2001; Davies et al., 2013). It also suggests that young readers can make use of sublexical information, whenever they are confronted with words that are not contained in their orthographic lexicon. This developmental dynamic would account for the stronger sensitivity of less skilled readers to both lexical frequency and word length. As lexical information increases with age, with rarer and longer words finding their way into the reader’s orthographic lexicon, the reader makes an increasingly prominent use of lexical information and an increasingly sparser use of sublexical information. In this paper, we provide a finer-grained quantitative analysis of the finger-tracking profile of typically developing readers, offering further evidence that their reading strategy results from an optimal, interactive combination of both lexical and sublexical information. The evidence is compared with the finger-tracking profile of difficult readers. To provide a more realistic developmental profile of these effects, we restrict our focus on nouns only, which are less likely to be skipped while finger-tracking, and present a narrower range of variability in both length and frequency.

Conclusion
We provided evidence that finger-tracking data of reading children can highlight congruent developmental patterns in the acquisition of literacy skills. We only replicated established benchmark effects reported in the psycholinguistic literature on decoding transparent orthographies. Nonetheless, to our knowledge, this is the first time that fingertracking patterns are shown to significantly correlate with more established reading data. Unsurprisingly, typically developing readers were shown to read at a faster rate than atypical readers. Our comparative analysis shows that both groups of readers are sensitive to the same lexical effects, but that atypical readers rely on an impoverished lexicon. We take this evidence to show that although the two groups adopt the same strategy, they differ in their global ability in serial information processing, which has a boosting influence on lexical development and reading speed. Despite our promising results, one could legitimately wonder why we propose using fingertracking as a proxy of a more established technology such as eye-tracking. Portability and task ecology are our strongest arguments. ReadLet can be used in almost any environment with no data-acquisition specialist or invasive, anxietyprovoking equipment. This has practical consequences for research in education, computer science, human cognition and medical sciences. Our architecture supports highly parallel and distributed processes of data acquisition, which can be delivered in real time to research, clinical and education centers as terminals for data harvesting and quantitative analysis. Large-scale studies can be conducted, paving the way to more generalizable results than ever in the past. In addition, the possibility to take single-subject measurements on more occasions and in different environments makes finger-tracking evidence usable not only in group studies but also for individual diagnostic purposes. Furthermore, the fine-grained, multimodal evidence of different signal streams which are aligned with time and with linguistically annotated texts provides invaluable training data for artificial neural networks and classification algorithms designed to solve engineering problems or simulate neurophysiological correlates of cognitive tasks. Last but not least, we know that reading probes are a commonly used for monitoring progress in reading fluency and text comprehension (Miura Wayman et al., 2007), but take huge time and effort to collect. The use of a tablet for extended reading enables deriving this information unobtrusively and continuously, wherever the child fancies reading, even at home.