# Eye Tracking of Occluded Self-Moved Targets: Role of Haptic Feedback and Hand-Target Dynamics

Frederic S Danion, James Mathew, J. Randall Randall Flanagan

<figure><img src="image_1.png" /></figure>

## To cite this version:

Frederic S Danion, James Mathew, J. Randall Randall Flanagan. Eye Tracking of Occluded Self-Moved Targets: Role of Haptic Feedback and Hand-Target Dynamics. eNeuro, 2017, 4 (3), 10.1523/ENEURO.0101-17.2017. hal-01794138

HAL Id: hal-01794138

https://amu.hal.science/hal-01794138

Submitted on 1 Jun 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Sensory and Motor Systems

# Eye Tracking of Occluded Self-Moved Targets: Role of Haptic Feedback and Hand-Target Dynamics

Frederic Danion,¹ James Mathew,¹ and J. Randall Flanagan²

DOI:http://dx.doi.org/10.1523/ENEURO.0101-17.2017

¹Aix Marseille Univ, CNRS, Institut de Neurosciences de la Timone, Marseille, France and ²Department of Psychology and Centre for Neurosciences Studies, Queen's University, Kingston, ON, Canada

## Abstract

Previous studies on smooth pursuit eye movements have shown that humans can continue to track the position of their hand, or a target controlled by the hand, after it is occluded, thereby demonstrating that arm motor commands contribute to the prediction of target motion driving pursuit eye movements. Here, we investigated this predictive mechanism by manipulating both the complexity of the hand-target mapping and the provision of haptic feedback. Two hand-target mappings were used, either a rigid (simple) one in which hand and target motion matched perfectly or a nonrigid (complex) one in which the target behaved as a mass attached to the hand by means of a spring. Target animation was obtained by asking participants to oscillate a lightweight robotic device that provided (or not) haptic feedback consistent with the target dynamics. Results showed that as long as 7 s after target occlusion, smooth pursuit continued to be the main contributor to total eye displacement (~60%). However, the accuracy of eye-tracking varied substantially across experimental conditions. In general, eye-tracking was less accurate under the nonrigid mapping, as reflected by higher positional and velocity errors. Interestingly, haptic feedback helped to reduce the detrimental effects of target occlusion when participants used the nonrigid mapping, but not when they used the rigid one. Overall, we conclude that the ability to maintain smooth pursuit in the absence of visual information can extend to complex hand-target mappings, but the provision of haptic feedback is critical for the maintenance of accurate eye-tracking performance.

**Key words:** Smooth pursuit; prediction; eye-hand coordination; internal models; hand-target mappings; target occlusion; haptic feedback

## Significance Statement

The ability to predict visual consequences arising from our actions is central in daily activities. Here, we tested this ability by means of a task that required participants to track with the eyes a target that was occluded and whose motion was driven by the hand using simple or complex hand-target mappings both with and without haptic feedback. Our results showed that, despite a general drop in performance after target occlusion, smooth pursuit activity persisted under all conditions. Although haptic feedback was not critical under the simple mapping, it clearly improved performance under the complex one. We conclude that haptic feedback is critical to supplement vision when predicting the behavior of objects with complex dynamics.

## Introduction

The ability to anticipate sensory consequences resulting from self-initiated movement is central for current theories of motor control (Wolpert and Flanagan, 2001;

Shadmehr et al., 2010; Wolpert et al., 2011). This ability can be demonstrated in various motor tasks ranging from object manipulation to eye–hand coordination. For instance, it is well established that smooth pursuit eye

Received March 23, 2017; accepted June 8, 2017; First published June 26, 2017.

Authors report no conflict of interest.

Author contributions: F.D. and J.R.F. designed research; F.D. and J.M.

performed research and analyzed data; F.D. prepared the figures; F.D. and J.R.F. interpreted the results and wrote the paper.

Support for this research was provided by the CNRS (PICS N° 191607) and a French National Grant (REM ANR-13-APPR-0008).
movements are substantially improved when the viewed
target is moved by the subject's hand in comparison to
when it is moved by an external agent. This improvement
is characterized by a higher gain in smooth pursuit
(Mather and Lackner, 1975; Gauthier et al., 1988; Vercher
et al., 1995), fewer saccades (Steinbach and Held, 1968;
Steinbach, 1969; Angel and Garland, 1972; Mather and
Lackner, 1975), and a shorter temporal lag between target
and eye position (Steinbach and Held, 1968; Gauthier and
Hofferer, 1976; Domann et al., 1989; Vercher et al., 1996).
To account for these observations, it is proposed that the
oculomotor system has access to an estimate of the
current hand position through the combination of sensory
feedback, arm efferent copy, and knowledge of hand-
target dynamics (Scarchilli et al., 1999; Ariff et al., 2002).
Overall, it is postulated that eye tracking profits from the
ability to both predict future states of the limb (Ariff et al.,
2002) and learn the mapping between hand actions and
their visual consequences (Sailer et al., 2005).

The advantage of self-generated versus externally gen-
erated target motion in pursuit eye tracking is also seen
when vision of the moving target is occluded. A large
number of studies have examined eye movement behav-
ior when vision of an externally driven moving target is
transiently occluded (Bennett and Barnes, 2003, 2006;
Madelain and Krauzlis, 2003; Orban de Vivry et al., 2008).
A typical observation is that ~200 ms after target blank-
ing, performance in eye-tracking starts to deteriorate as
indicated by a drop in smooth pursuit velocity and an
increase in the contribution of catch-up saccades. Al-
though only a few studies have investigated eye tracking
when a self-moved target is temporarily masked, they
indicate improved performance. In a seminal study, Gau-
thier and Hofferer (1976) explored the ability of partici-
pants to track a visual target that was moved by
oscillating either the finger or the elbow. At some point the
target was masked, and participants were asked to keep
oscillating their limb while tracking the target as if it was
still visible. Their results showed that participants were
able to maintain smooth pursuit (albeit with a lower gain)
over several successive cycles of movement lasting sev-
eral seconds, which contrasts with the rapid decay of
smooth pursuit when using an externally moved target.
More recently, Berryhill et al. (2006), who investigated the
ability to track a self-moved pendulum in the dark, also
showed that participants could maintain smooth pursuit
for several seconds (albeit with a low gain). Overall, the
benefit of self-moving a target on smooth pursuit perfor-

mance extends to situations in which the target is oc-
cluded.

The goal of the current study was to further investigate
the ability of humans to track an invisible self-moved
target, focusing on two key issues. First, we asked
whether this ability extends to situations in which partic-
ipants use a more complex mapping between hand and
target motion. Second, we asked how this ability depends
on receiving haptic feedback about the interaction be-
tween hand and target motion. To date, the contribution
of haptic feedback to eye tracking has been (indirectly)
investigated with deafferented patients under a simple
hand-target mapping (Vercher et al., 1996), but not under
a complex mapping and not when the target is occluded.
To achieve these goals, we designed an experiment in
which participants were asked to track a target on a
screen whose animation was obtained by oscillating hor-
izontally a grasped object attached to a lightweight ro-
botic device. Two visual hand-target mappings were
used, either a rigid one in which hand and target motion
matched perfectly (simple dynamics) or a nonrigid one in
which the target behaved as a mass attached to the hand
by means of a spring (complex dynamics). Using the
robotic device, haptic feedback congruent with the target
dynamics could be provided or removed. Although previ-
ous studies have shown that people can learn to control
nonrigid objects both when appropriate haptic feedback
is provided (Dingwell et al., 2002, 2004; Mehta and
Schaal, 2002; Nagengast et al., 2009) and when it is not
(Mehta and Schaal, 2002; Mah and Mussa-Ivaldi, 2003),
performance is typically improved by haptic feedback
(Danion et al., 2012; Farshchiansadegh et al., 2016).
Based on these results, we predicted that whereas eye
tracking performance after target occlusion would be re-
duced when moving the nonrigid target in comparison to
the rigid target, this deficit would be limited when haptic
feedback was provided.

Method

Participants

Fourteen self-proclaimed right-handed participants (age, 22.2 ± 1.8 years; nine female) participated in this study. None of the participants had neurologic or visual disorders. They were naive as to the experimental conditions and hypotheses and had no previous experience of ocular motor testing. All participants gave written informed consent before the study. Each participant received $10 for participation. The Queens University ethics committee approved the experimental paradigm (2014-12-3-04), which complied with the Declaration of Helsinki.

Apparatus

The experimental setup is illustrated in Figure 1. Participants were comfortably seated facing a screen positioned in a frontal plane 57 cm away from the participant's eye (Fig. 1A). Thus, at the center of the screen a target displacement of 1 cm corresponded to 1° in terms of visual angle. To minimize measurement errors, participants' head movements were restrained by a chin rest and a padded forehead rest so that the eyes in primary

Acknowledgments: We thank Martin York and Mohamed Albaghdadi for technical support and logistic support. We also thank the anonymous reviewers for helpful comments and suggestions.

Correspondence should be addressed to Frederic Danion, Aix Marseille
Univ, Institut de Neurosciences de la Timone, 27 Bd Jean Moulin, 13385
Marseille, France. E-mail: frederic.danion@univ-amu.fr.

DOI:http://dx.doi.org/10.1523/ENEURO.0101-17.2017

Copyright © 2017 Danion et al.

This is an open-access article distributed under the terms of the Creative
Commons Attribution 4.0 International license, which permits unrestricted use,
distribution and reproduction in any medium provided that the original work is
properly attributed.
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Photographs of the experimental setup. **A**, Overview of the experimental setup. Red dot indicates target position when the laser is on. **B**, The grasped object and the lightweight robotic device. **C**, The laser and optical scanner. See text for more details.</figcaption></figure>

position were directed toward the center of the screen. A bib was positioned under the participants' chin to block vision of their hands. Participants controlled the position of a target on the screen by moving a grasped object attached to a lightweight robotic arm (Phantom Haptic Interface 3.0L, SensAble Technologies) in a frontal plane (Fig. 1B). When the target was at the center of the screen, the elbow and shoulder were comfortably positioned so that both hand and target lay in the participant's midsagittal plane (Fig. 1A). Hand movements were recorded at a sampling rate of 1000 Hz with a resolution of 0.1 mm.

The target (filled red circle 0.5° in diameter) was projected on the screen using a laser beam (39080; Edmund Optics) moved by an optical scanner (MG350DT; General Scanning) servo-controlled by a PC (Fig. 1C). The delay in the servo-command was <1 ms. The optical scanner motion was restricted to one dimension so that the target moved only along the horizontal axis. An infrared video-based eye tracker (RK-726PCI pupil/corneal tracking system; ISCAN) was used to record the position of gaze of the left eye in the work plane at 400 samples/s. Before the experiment, we calibrated the output from the eye tracker by recording the raw eye positions as participants fixated on a grid composed of 25 known locations. The mean values during fixation intervals at each location were then used for converting offline raw eye tracker values to horizontal and vertical eye position in degrees of visual angle.

Two types of hand-target visual mapping were used. When participants had to move the rigid target, its motion was an exact replicate of the actual hand trajectory in the frontal plane: if the hand moved by 1 cm to the left, the target also moved by 1 cm to the left on the screen. When haptic feedback was implemented for the rigid target (Rigid-Hapt), interaction forces provided by the robotic device simulated the physical behavior of a 1-kg point mass. In the no haptic version of the rigid target (Rigid-NoHapt), the motors of the robotic device were simply turned off. When subjects had to move the nonrigid target (Spring), the visual target was simulated as a mass-spring object with the following properties: mass, 1 kg; stiffness, 40 N/m; damping, 1.66 N/m/s; resting length, 0 m. These values are about one-third of values used in previous studies investigating the manipulation of nonrigid objects (Dingwell et al., 2002, 2004; Nagengast et al., 2009; Danion et al., 2012; Landelle et al., 2016). The rational for decreasing object inertia was to prevent possible fatigue effects while keeping a 1-Hz resonance frequency as in other studies; the resonance frequency (F) of a mass-spring system depends on its mass (m) and its stiffness (k) such that

$$F = (1/2\pi) * \sqrt{(k/m)}$$

Depending on the experimental conditions, haptic feedback of the nonrigid target could be implemented in three
different ways. First, haptic feedback could be provided such that it was congruent with the visual dynamics of the object (Spring-Hapt), meaning that the same parameters were used to simulate physical and visual behavior. Second, haptic feedback could be removed in the sense that motors of the robotic device were turned off (Spring-NoHapt). Third, haptic feedback could be incongruent with the visual dynamics of the object (Spring-DissHapt). In this case, we introduced a dissociation between the visual and haptic stiffness of the mass-spring (visual stiffness, 48 N/m; haptic stiffness, 32 N/m) while keeping mass and damping similar to previous values. This dissociation led to distinct resonance frequencies for the visual and haptic dynamics (1.25 vs. 0.8 Hz).

## Procedure

In all trials, participants were instructed to track as accurately as possible the target moving on the screen. The target motion was always driven by the subject's hand. However, depending on the experimental condition, the visual mapping between hand movement and target motion could be either rigid or elastic, with either no haptic feedback, congruent haptic feedback, or incongruent haptic feedback. Overall, the following five experimental conditions were tested: Rigid-Hapt, Rigid-NoHapt, Spring-Hapt, Spring-NoHapt, and Spring-DissHapt. Each subject completed first a familiarization session in which they practiced the task with visual feedback followed by an experimental session in which, in each trial, visual feedback was initially present and then removed.

During the familiarization session, subjects were asked to perform random oscillatory movements to move the target (for a similar procedure, see Steinbach and Held, 1968; Angel and Garland, 1972). The underlying motivation was to favor the acquisition of knowledge about hand-target dynamics. Subjects were encouraged to use the whole extent of the screen (±20°) while making sure that the target did not fall outside the screen boundaries. To facilitate the production of random movements, a template was given before the training session. During demonstration trials, subjects did not move their hands and simply had to observe the replay of a trial performed by one of the experimenters under the same mapping condition. When subjects subsequently moved the target, we ensured that absolute target speed was comparable across conditions by encouraging subjects to maintain an average absolute target velocity close to 30°/s: this was possible by computing online mean absolute target velocity while the experimenters provided verbal feedback to the subject when necessary. This procedure ensured an overall mean target velocity of 30.2°/s with minimal changes across subjects (SD = 0.23°/s), experimental conditions (SD = 0.52°/s), and trials (SD = 0.80°/s). Each subject completed one block of 20 trials in each of the five experimental conditions. Each trial was 16 s long. Subjects were not explicitly informed about the nature of the mapping between their hand movement and the target motion before completing these experimental conditions. The order of blocks was randomized across subjects.

Overall, a total of 100 trials (5 × 20 trials) were collected in this familiarization session.

Trials in the continuation session were similar to those in the familiarization session except that subjects were asked to produce target motion that was sinusoidal and the target was blanked after 7 s. Sinusoidal target motion was encouraged to simplify data analysis and allow comparison with previous work (Gauthier and Hofferer, 1976). A template with a target moving sinusoidally (period = 1000 ms; peak-to-peak amplitude = 15°; resulting in 30°/s mean absolute target velocity) was given before the continuation session. Subjects were encouraged to reproduce this pattern in the subsequent trials (effective mean period = 1.067 ms; effective mean amplitude = 17.3°). Each trial was 16 s long. Approximately 7 s after trial initiation, the target was removed from the screen (i.e., blanked) until the end of the trial. During the blanking period, subjects were instructed to keep oscillating the target while tracking it with their gaze as if it was still displayed on the screen. Each subject completed a block of three trials in each experimental condition. The order of the blocks was randomized across subjects. A total of 15 trials (5 × 3 trials) was collected during this experimental session. The overall duration of the familiarization with the experimental session averaged 60 min. Participants could request additional breaks at any time, but most of them only took the break offered between blocks.

## Data analysis

Because the stimuli were moving exclusively along the horizontal meridian, we focused our analyses on the horizontal component of eye movements. We performed a sequence of analysis to separate periods of smooth pursuit, saccades, and blinks from the raw eye position signals. The identification of the blinks was performed by visual inspection. This procedure led to the removal of ~1% of eye recordings. Eye position time series were then low-pass filtered with a Butterworth (fourth order) using a cutoff frequency of 25 Hz. The resultant eye position signals were differentiated to obtain the velocity traces. The eye velocity signals were low-pass filtered with a cutoff frequency of 25 Hz to remove the noise from the numerical differentiation. The resultant eye velocity signals were then differentiated to provide the accelerations traces that we also low-pass filtered at 25 Hz to remove the noise. A dedicated Matlab script was run to identify saccades. This identification was based on the acceleration and deceleration peaks of the eye (>1500°/s²). Further visual inspection allowed to identify smaller saccades (<1°) that could not be identified automatically by our program. Based on these computations, periods of pursuit and of saccades were extracted.

During the experimental (continuation) trials, we distinguished regular pursuit from the periods of target occlusion (after ~7 s). The first part and last part of each trial were analyzed separately. Each continuation trial was segmented into 1-s bins. The bin segmentation was set with respect to the initiation of target blanking. The first bin that followed the target blanking was named +1, and
the bin just preceding the target blanking was named –1. We used the same policy to name the surrounding bins. A total of seven bins preceding the target blanking, and eight bins following target blanking could be reliably extracted from each trial. For each of these 1-s bins, we computed the same dependent variables. To assess baseline performance (i.e., in the presence of visual feedback), dependent variables were averaged across the five bins preceding target blanking (–5, –4, –3, –2, –1; bins –6 and –7 were discarded because stable performance was not reached yet). To assess performance during continuation (i.e., when target was invisible), dependent variables were averaged across the last five bins (4, 5, 6, 7, and 8).

To assess the participants' ability to predict the dynamics of the target, we extracted the following dependent variables. First, we computed the mean absolute position error (PE) by averaging the absolute difference in position between the target and the eye over the whole trial, including periods of both saccades and smooth pursuit (note that excluding saccades from PE evaluation did not change our findings). Second, we computed the mean absolute velocity error (VE), i.e., the average absolute difference between the eye and target velocity. Note that although PE was evaluated over the whole trial (including periods of both saccades and smooth pursuit), VE was computed only during smooth pursuit periods. Third, as a gross index of temporal coupling, we computed the coefficient of correlation between eye and target position. Fourth, to evaluate more closely the temporal relationship between eye and target movements, we computed the lag between the two using a cross-correlation technique based on the eye and target position signal (a positive lag indicating the eye is lagging behind the target).

Finally, to assess the relative contribution of saccades and smooth pursuit, we computed for each trial the total distance traveled by the eye with saccades (Orban de Xivry et al., 2006) and then expressed this as a percentage of the total distance traveled by the eye using both saccades and smooth pursuit. To better characterize smooth pursuit, we computed the smooth pursuit gain by averaging the ratio between instantaneous eye and target velocities during phases of smooth pursuit (to avoid numerical instabilities, only situations where absolute target velocity was >10°/s were considered).

## Statistical analysis

Two-way ANOVA was used to assess the effects of target mapping and haptic feedback, but data before and after target blanking were analyzed separately. To subsequently investigate the effect of dissociating visual and haptic feedback, the condition Spring-DissHapt was contrasted with Spring-NoHapt and Spring-Hapt by means of one-way ANOVA. To obtain a normal distribution, z-score transformation was used for coefficients of correlation. Newman-Keuls corrections were used for post hoc t tests to correct for multiple comparisons. A conventional 0.05 significance threshold was used for all analyses.

## Results

### Typical trials

Figure 2 plots five typical trials performed by the same subject in each experimental condition. As can be seen, when the target was visible (first half of each trial), accurate smooth pursuit was observed in all five conditions. After target occlusion, although the rate and the amplitude of catch-up saccades increased, episodes of smooth pursuit were still observable (albeit with a lower gain). We also noticed a temporal drift between eye and target motion such that the eye started to lead the target, especially under the two conditions with the Spring mapping.

### Kinematics of target motion

Before target blanking, average group data indicated a mean period of target oscillation of 1065 ms, which was fairly close to the intended value (1000 ms). Blanking the target did not alter the mean period of oscillation (1061 ms). Concerning overall target movement amplitude, its mean value was 17.8° before target occlusion, which was slightly above the intended value (15°). After target blanking, this amplitude decreased by 10% under Rigid-NoHapt ($t(26) = 2.11; p < 0.05$), whereas it increased by 13% under Spring-NoHapt ($t(26) = 3.37; p < 0.01$).

### Eye motion: smooth pursuit versus saccades

Figure 3A presents mean group data showing the relative contribution of smooth pursuit to eye tracking. This figure shows that before target occlusion, the percentage of total distance covered by the eye with smooth pursuit was high (~83%), thereby confirming that the task was primarily performed using smooth pursuit eye movements. After target occlusion, this contribution decreased substantially (down to 60%; see also Fig. 2), but smooth pursuit remained the main contributor for eye movement. For both regular and blanked periods, ANOVAs showed no significant main effects of mapping and haptic feedback, as well as no interaction ($F(1,13) < 3.75, p > 0.05$). Further analyses showed that the increased contribution of saccades after target blanking was associated with a 30% increase in saccade rate and an approximate doubling of saccade amplitude (Fig. 2). Overall, despite an increase in the contribution of saccades, the key observation is that smooth pursuit activity persisted several seconds after target occlusion.

Regarding the gain of smooth pursuit, mean group data are presented in Figure 3B. As expected from previous studies (Gauthier and Hofferer, 1976; Berryhill et al., 2006), target occlusion was followed by a drop in smooth pursuit gain, which was observed in all conditions. Averaged across conditions, the gain decreased from 0.74 to 0.61, representing an 18% drop. Further analyses showed that, both before and after target occlusion, the gain was not significantly affected by mapping, haptic feedback, or the interaction between these factors ($F(1,13) < 0.89, p > 0.05$).
<figure><img src="image_4.png" /><figcaption>**Figure 2.** Typical trials performed by the same subject in each experimental condition. Vertical dotted line indicates the moment of target occlusion. Note the progressive drift in the temporal coupling between eye and target, and the larger contribution of saccades.</figcaption></figure>

## Accuracy of eye tracking performance at the spatial level

Having shown that smooth pursuit persists after target occlusion, we assessed its accuracy with respect to target motion. Figure 4 shows key parameters accounting for the spatial accuracy of eye tracking. Figure 4A presents the time course of PE across the five experimental conditions. Consider first the period before target occlusion. As expected, we found an effect of mapping ($F(1,13) = 5.94$; $p = 0.03$), consistent with the view that our task was more difficult under Spring than Rigid. On average, PE was 37% greater under Spring than Rigid (2.37 vs. 1.73°). However, we found no effect of haptic feedback ($F(1,13) = 0.02$; $p = 0.88$) and no interaction ($F(1,13) = 2.56$; $p = 0.13$). After target blanking, PE increased substantially in all experimental conditions. This time, the effects of haptic feedback ($F(1,13) = 6.56$, $p = 0.02$), mapping ($F(1,13) = 86.02$, $p < 0.001$), and their interaction ($F(1,13) = 13.75$; $p = 0.003$) were all significant. *Post hoc* analysis showed that whereas haptic feedback did not influence PE under Rigid, PE was

25% smaller under Spring-Hapt in comparison to Spring-NoHapt (6.36 vs. 8.49°; $p < 0.001$). Overall, this analysis of PE shows that, although haptic feedback had little influence on eye tracking accuracy as long as the target was visible, it became very important after target blanking under the Spring mapping.

Most of these observations were further supported by the analysis of the velocity error (VE; Fig. 4B). Indeed, when the target was visible, we also found an effect of mapping ($F(1,13) = 7.15$; $p = 0.02$) consistent with the view that tracking error was greater under Spring than Rigid (16.4 vs. 20.3°/s; 24%). Again we found no effect of haptic feedback ($F(1,13) = 0.006$; $p = 0.94$) as well as no interaction ($F(1,13) = 0.16$; $p = 0.69$). Furthermore, after target blanking, VE increased substantially in all conditions. However, once again this alteration was limited by the provision of haptic feedback under Spring but not under Rigid, as indicated by an interaction between haptic feedback and mapping ($F(1,13) = 10.71$, $p = 0.006$). *Post hoc* analysis confirmed that whereas VE values were sim-
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Contribution of saccades and smooth pursuit as a function of experimental condition and time in the vicinity of target occlusion. **A**, Percentage of total distance covered by smooth pursuit. **B**, Gain of smooth pursuit velocity. Error bars represent SE; vertical dotted lines denote the initiation of target occlusion; numbers circled (in red or black) denote the numbering of each bin (following the rationale described in Materials and Methods).</figcaption></figure>

ilar in Rigid-Hapt and Rigid-NoHapt (28.4 vs. 25.9°/s; *p* = 0.17), VE was 13% smaller in Spring-Hapt compared with Spring-NoHapt (38.0 vs. 43.5°/s; *p* < 0.001).

## Accuracy of eye tracking performance at the temporal level

As for spatial accuracy, temporal accuracy of eye tracking decreased after target occlusion. To investigate the temporal coordination between eye and target, we first present the correlation coefficient between the two corresponding position signals (Fig. 5A). When the target was visible, we found no significant main effect of mapping and haptic feedback as well as no interaction ($F(1,13) < 1.39; p > 0.26$). In contrast, when the target was blanked, we found a mapping × haptic feedback interaction ($F(1,13) = 54.88; p < 0.001$) such that *R* values were similar under the two Rigid conditions (NoHapt 0.71 vs. Hapt 0.65; *p* = 0.12), but were greater under Spring-Hapt compared with Spring-NoHapt (0.30 vs. -0.01; *p* < 0.001). Overall, this analysis adds further evidence that haptic feedback was helpful when the target was occluded, but only under the Spring mapping.

To further examine this alteration in temporal accuracy, Figure 5B and C present the temporal lag between eye and target and between eye and hand, respectively. Importantly, one should keep in mind that, in contrast to Rigid, where hand and target motion are inherently synchronized, there is a temporal delay between hand and target motion under Spring. Because of the mass-spring dynamics, the motion of the target lagged hand motion by ~200 ms (bottom three rows in Fig. 2). Mean group lags—combining data from pre- and postocclusion—were 182, 171, and 245 ms, respectively, in Spring-NoHapt, Spring-Hapt, and Spring-DissHapt (in the latter condition, the delay was greater because of the lower stiffness of the visual spring).

Concerning the temporal relationship between eye and target (Fig. 5B), we found that when the target was visible, eye and target were rather well synchronized under Rigid (mean lag, -13 ms) whereas a clear eye lead was seen under Spring (mean lag, -53 ms). This difference in eye behavior was corroborated by a main effect of mapping ($F(1,13) = 11.06; p = 0.005$). Unexpectedly, when the target was blanked, the eye began to lead even more on
<figure><img src="image_6.png" /><figcaption>**Figure 4.** Spatial accuracy of eye tracking as a function of experimental condition and time in the vicinity of target occlusion. A, Position error. B, Velocity error between eye and target. Error bars represent SE; vertical dotted lines denote the initiation of target occlusion. Note how the detrimental effects of target blanking are reduced by the provision of haptic feedback under Spring. The numbers circled (in red or black) denote the numbering of each bin.</figcaption></figure>

the target, a phenomenon observed in all conditions albeit with different intensity. First, this effect was more pronounced under Spring than Rigid (–208 vs. –100 ms; *F*(1,13) = 73.94; *p* < 0.001). Second, we also found an interaction between mapping and haptic feedback (*F*(1,13) = 12.95; *p* = 0.003), such that a smaller phase lead was observed under Spring-Hapt than Spring-NoHapt (–186 vs. –230 ms; *p* < 0.05). In contrast, the phase lead was similar under both Rigid conditions (*p* = 0.14). Overall haptic feedback appeared helpful in limiting the temporal drift of the eye induced by the target blanking under Spring.

To provide a better understanding of eye–hand temporal coordination, we present the temporal lag between eye and hand (Fig. 5C). As expected when the target was visible, there was a clear effect of mapping (*F*(1,13) = 232.3; *p* < 0.001). Whereas under Spring the eye lagged behind the hand (mean lag, 117 ms), the eye and hand were synchronized under Rigid (mean lag, –12 ms). After target blanking, the timing between eye and hand changed substantially, with the eye leading the hand under all conditions. However, the magnitude of this eye

lead depended on both mapping and haptic feedback, as revealed by an interaction between the two factors (*F*(1,13) = 50.06; *p* < 0.001). *Post hoc* analysis indicated that in the absence of haptic feedback, the lead of the eye became similar under Rigid-NoHapt and Spring-NoHapt (–82 and –88 ms, respectively; *p* = 0.46), despite substantial differences before target blanking. In contrast, when haptic feedback was provided, the differences in eye–hand lag persisted between the two mappings (Rigid-Hapt, –118 ms; Spring-Hapt, –38 ms; *p* < 0.001). Overall, this analysis reinforces the view that the provision of haptic feedback is important for eye–hand coordination after target masking.

## Dissociation between haptic and visual feedback

Many dependent variables (PE, VE, eye–target correlation, and eye–target lag) showed that the provision of haptic feedback was helpful in reducing the detrimental effects of target blanking under the Spring mapping. For each of these variables, we ran an additional ANOVA that compared the three Spring conditions, namely Spring-DissHapt, Spring-NoHapt, and Spring-Hapt. Except for
<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /><figcaption>**Figure 5.** Temporal accuracy of eye tracking as a function of experimental condition and time in the vicinity of target occlusion. A, Coefficient of correlation between eye and target. B, Temporal lag between eye and target. A negative lag indicates that the eye precedes the target. C, Temporal lag between eye and hand. A negative lag indicates that the eye precedes the hand. Error bars represent SE; vertical dotted lines denote the initiation of target occlusion; numbers circled (in red or black) denote the numbering of each bin. Note how the detrimental effects of target blanking are reduced by the provision of haptic feedback under Spring.</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>**Figure 6.** Percentage of total distance covered by smooth pursuit as a function of experimental condition and time in the vicinity of target occlusion. This figure compares the performance of two groups of participants that tracked either a self-moved target or an externally moved one. See text for more details. Error bars represent SE; vertical dotted line denotes the initiation of target occlusion; numbers circled (in red or black) denote the numbering of each bin.</figcaption></figure>

VE, we found in all cases a significant difference across these three conditions ($F(2,26) > 4.15; p < 0.05$). *Post hoc* analyses consistently showed a lack of difference between Spring-DissHapt and Spring-NoHapt, while showing reliably a difference between Spring-DissHapt and Spring-Hapt ($p < 0.05$). Overall these additional analyses show that when haptic feedback was not congruent with visual dynamics of the target, the benefit provided by haptic feedback was lost.

## Comparison between a self-moved and an externally moved target

One major conclusion drawn from this study is that, after target occlusion, smooth pursuit remains the major contributor of eye motion, with ~60% of the total distance covered by the eye. An implicit assumption is that the contribution of smooth pursuit would have been smaller for an externally moved target. To test this assumption, we tested seven new participants using a similar paradigm, except this time the motion of the target was preprogrammed and the hand was immobile. After a familiarization session, each participant was tested successfully with pure sinusoidal trajectories (frequency, 1 Hz; amplitude, 15°) and target trajectories taken from a randomly selected participant of the previous experiment (with a different previous participant matched to each new participant). As in the previous conditions, the target was blanked after 7 s, and each participant performed a block of three trials in each condition. Figure 6 shows the results of this control experiment separately for the pure sinusoidal trajectories (Ext-PureSine) and the playback trajectories (Ext-Replay). For comparison purposes, we also present the mean performance of the seven participants whose trajectories were used for playback (Self-Mean). Although there was no significant difference between Ext-PureSine and Ext-Replay ($F(1,6) = 2.01; p = 0.21$), the contribution of smooth pursuit was always smaller in Ext compared with Self ($F(1,6) = 41.65; p < 0.001$). This

control experiment shows that for both visible and occluded targets, there is a greater contribution of smooth pursuit when tracking a self-moved target compared with an externally moved one. Those results extend earlier observations made in the context of nonoccluded targets (Steinbach and Held, 1968; Landelle et al., 2016) in the sense that they are also valid for occluded targets.

## Discussion

The goal of this study was to investigate the ability of humans to track with their eyes a self-moved target in the absence of visual feedback. Specifically, we asked whether haptic feedback could ameliorate the effects of removing visual feedback when the dynamics relating hand and target motion were either simple or complex. Our experiment resulted in four key findings. First, we found that participants were able to maintain smooth pursuit activity after target occlusion, even under a complex hand-target mapping. Second, although largely expected, target occlusion was detrimental for the accuracy of eye tracking under all conditions. Third, the detrimental effects of target occlusion on eye tracking depended to a large extent on both the target dynamics and the availability of haptic feedback. Specifically, although haptic feedback did not provide much benefit to eye tracking under the Rigid mapping, it limited the detrimental effects of target occlusion under the Spring mapping. Finally, when haptic and visual feedback were dissociated (i.e., incongruent), the benefit of haptic feedback (seen under Spring-Hapt) was no longer observed. We now discuss in more detail these findings and their implications.

## Maintenance of smooth pursuit after target occlusion

The current study showed that participants can maintain reliable smooth pursuit activity for several seconds after the masking of a self-moved target, which further reinforces the view that retinal slip is not the only input to
the smooth pursuit control system (Gauthier and Hofferer, 1976; Vercher et al., 1996; Berryhill et al., 2006). However, the contribution of smooth pursuit to tracking was reduced when the target was occluded (switching from 83% to 60%) and the smooth pursuit gain decreased (from 0.74 to 0.61). These detrimental effects are consistent, at least qualitatively, with two earlier seminal studies. In the study of Jordan (1970) smooth pursuit contribution dropped from 98% to 38% when the target was occluded. In the study of Gauthier and Hofferer (1976), the gain of smooth pursuit dropped from ~1 to 0.7, and their Fig. 4 also speaks for a decreased contribution of smooth pursuit after target occlusion. Because both studies only used conventional hand-target mappings (i.e., rigid), one novel contribution of our study is to extend the ability of participants to maintain smooth pursuit activity under more complex mappings. In a recent study by Landelle et al. (2016), target occlusions were also investigated under a Spring mapping, but their duration (400 ms) was less challenging than in the current experiment. Thanks to longer periods of target occlusion (7 s) in our study, we showed that at least 2 s were necessary to stabilize eye behavior (Figs. 3, 4, and 5).

## The role of hand-target mapping and haptic feedback

Smooth pursuit activity can be maintained consistently after target occlusion, but the accuracy of eye tracking was altered by this procedure. Although this was largely expected (Berryhill et al., 2006), this alteration depended both on the type of hand-target mapping and the availability of haptic feedback. More specifically, all our analyses of spatial and temporal accuracy (Figs. 4 and 5) indicated an interaction between mapping and haptic feedback. In all cases, haptic feedback was helpful to circumvent the detrimental effects of target occlusion under the Spring mapping but not under the Rigid one. For instance, when maneuvering the spring target, the provision of haptic feedback led to a reduction in PE and VE of 25% and 13%, respectively.

The temporal coordination between eye and target was also altered by the occlusion, but the largest changes were observed under the Spring mapping, with the eye starting to substantially lead the target (Fig. 5B) and even the hand, a phenomenon that was also observed under the Rigid mapping (Fig. 5C). In the absence of haptic feedback, this temporal drift was so large under Spring that ultimately the eye-hand lag became similar under Rigid, suggesting that participants failed to maintain a representation of the spring linking target and hand motion. In contrast, when haptic feedback was provided, the drift was smaller (-20%), allowing to maintain different eye-hand timing under Spring and Rigid. We conclude that the provision of haptic feedback under Spring was helpful in maintaining the initial coordination between eye, hand, and target.

More generally, the contribution of haptic feedback is very context specific. When manipulating the rigid target both with and without occlusion, haptic feedback did not influence eye-hand coordination. In that sense, our results are consistent with earlier observations made by Vercher et al. (1996), who reported that, under a simple (i.e., rigid) mapping, deafferented patients did not differ from control participants when tracking a nonoccluded target. Moreover, when manipulating the nonoccluded spring target, we did not find any obvious contribution of haptic feedback. However, as soon as the spring target was occluded, a contribution of haptic feedback emerged. Overall, the results suggest that when participants maneuver familiar objects, haptic feedback is unnecessary to drive their eye motion (visual information and hand efference copy being sufficient). In contrast, when conditions become more challenging, such as with an unfamiliar object (i.e., with complex dynamics) and in the absence of visual feedback, haptic feedback can provide a critical input for eye tracking.

## Dissociation between haptic and visual feedback

Results showed that when the haptic and visual dynamics were dissociated, the benefit of haptic feedback under target occlusion was lost; namely, eye tracking performance became as poor as with no haptic feedback. In principle, participants could have learned, before target occlusion, the rather arbitrary mapping between haptic and visual feedback and use dit to predict target motion after target occlusion. However, our results indicate that this did not occur. Alternatively, following target occlusion, participants could have interpreted that haptic feedback was congruent with visual target motion (e.g., as in the Spring-Hapt or Rigid-Hapt conditions) and used this mapping to predict target motion. However, this is unlikely, because eye tracking performance should have become worse than with no haptic feedback. Instead, it seems that participants simply ignored the incongruent haptic feedback, presumably after learning that it was not consistent with visual feedback during the preblanking phase. All in all, this suggests that participants can flexibly rely on haptic feedback when it is helpful for the task, but can also ignore it when it is potentially harmful for the task.

## Eye lead after target occlusion

Under all experimental conditions, eye motion was shifted forward in time after the target was occluded. Averaged across conditions, the mean lead of the eye over the hand was 76 ± 31 ms. Similarly, Gauthier and Hofferer (1976) reported that “the eye led the finger by an averaged 60-ms delay in all tests involving tracking of an imaginary target actively moved by the finger.” They suggested that this lead arises because hand and eye movements have different response times to motor commands, mainly because the eye has considerably lower inertia and is driven by relatively stronger muscles. It was proposed that in total darkness, since there is no more need to compensate for this asymmetry in motor systems, a phase lead of the eye would emerge. More recently, a rather similar phenomenon was observed when participants were asked to try to look at the perceived position of their hand during unseen reaching movements (Ariff et al., 2002). In that context, saccades provided an unbiased estimate of hand position at *t* + 200 ms. Namely, participants initiated saccades that landed close to where their hand would be 200 ms later. The authors interpreted
that finding as evidence that the brain uses a forward model allowing the eye to estimate the future hand position in real time as movement unfolds (Scarchilli et al., 1999). Although attractive, this scheme does not account for the eye lead when participants are explicitly required to track their hand. A possible reason for this behavior may stem from natural eye–hand coordination during object manipulation. Indeed, when participants are asked to transport an object, their gaze is typically leading the hand (Johansson et al., 2001; Sailer et al., 2005). Specifically, participants make so-called proactive saccades, meaning that their gaze is directed toward the location where they plan to bring the object. We propose that when the target was occluded, although our participants were explicitly required to track the current position of the target, they might experience difficulties refraining from this proactive gaze behavior.

## Conclusions

Overall, we conclude that the ability to maintain smooth pursuit in the absence of visual information extends to complex hand-target mappings, but the provision of haptic feedback is critical for the maintenance of accurate eye-tracking performance. More generally, this study extends the view that haptic feedback is critical not only for manipulating nonrigid objects efficiently (Sternad et al., 2001; Huang et al., 2006; Danion et al., 2012), but also to coordinate proficiently eye and hand actions.

## References

* Angel RW, Garland H (1972) Transfer of information from manual to oculomotor control system. J Exp Psychol 96:92-96. Medline

* Ariff G, Donchin O, Nanayakkara T, Shadmehr R (2002) A real-time state predictor in motor control: study of saccadic eye movements during unseen reaching movements. J Neurosci 22:7721-7729. Medline

* Bennett SJ, Barnes GR (2006) Smooth ocular pursuit during the transient disappearance of an accelerating visual target: the role of reflexive and voluntary control. Exp Brain Res 175:1-10.CrossRef Medline

* Bennett SJ, Barnes GR (2003) Human ocular pursuit during the transient disappearance of a visual target. J Neurophysiol 90: 2504-2520.CrossRef Medline

* Berryhill ME, Chiu T, Hughes HC (2006) Smooth pursuit of nonvisual motion. J Neurophysiol 96:461-465. CrossRef Medline

* Danion F, Diamond JS, Flanagan JR (2012) The role of haptic feedback when manipulating nonrigid objects. J Neurophysiol 107: 433-441. CrossRef Medline

* Dingwell JB, Mah CD, Mussa-Ivaldi FA (2004) Experimentally confirmed mathematical model for human control of a non-rigid object. J Neurophysiol 91:1158-1170. CrossRef Medline

* Dingwell JB, Mah CD, Mussa-Ivaldi FA (2002) Manipulating objects with internal degrees of freedom: evidence for model-based control. J Neurophysiol 88:222-235. Medline

* Domann R, Bock O, Eckmiller R (1989) Interaction of visual and non-visual signals in the initiation of smooth pursuit eye movements in primates. Behav Brain Res 32:95-99. Medline

* Farshchiansadegh A, Melendez-Calderon A, Ranganathan R, Murghey TD, Mussa-Ivaldi FA (2016) Sensory agreement guides kinetic energy optimization of arm movements during object manipulation. PLoS Comput Biol 12:e1004861. CrossRef Medline

* Gauthier GM, Hofferer JM (1976) Eye tracking of self-moved targets in the absence of vision. Exp Brain Res 26:121-139. Medline

* Gauthier GM, Vercher JL, Mussa Ivaldi F, Marchetti E (1988) Oculomotor tracking of visual targets: control learning, coordination control and coordination model. Exp Brain Res 73:127-137. Medline

* Huang FC, Gillespie RB, Kuo AD (2006) Human adaptation to interaction forces in visuo-motor coordination. IEEE Trans Neural Syst Rehabil Eng Publ IEEE Eng Med Biol Soc 14:390-397. CrossRef Medline

* Johansson RS, Westling G, Bäckström A, Flanagan JR (2001) Eye-hand coordination in object manipulation. J Neurosci 21:6917-6932. Medline

* Jordan S (1970) Ocular pursuit movement as a function of visual and proprioceptive stimulation. Vision Res 10:775-780. Medline

* Landelle C, Montagnini A, Madelain L, Danion F (2016) Eye tracking a self-moved target with complex hand-target dynamics. J Neurophysiol 116:1859-1870. CrossRef Medline

* Madelain L, Krauzlis RJ (2003) Effects of learning on smooth pursuit during transient disappearance of a visual target. J Neurophysiol 90:972-982. CrossRef Medline

* Mah CD, Mussa-Ivaldi FA (2003) Evidence for a specific internal representation of motion-force relationships during object manipulation. Biol Cybern 88:60-72. CrossRef Medline

* Mather J, Lackner J (1975) Adaptation to visual rearrangement elicited by tonic vibration reflexes. Exp Brain Res 24:103-105. Medline

* Mehta B, Schaal S (2002) Forward models in visuomotor control. J Neurophysiol 88:942-953. Medline

* Nagengast AJ, Braun DA, Wolpert DM (2009) Optimal control predicts human performance on objects with internal degrees of freedom. PLoS Comput Biol 5:e1000419. CrossRef Medline

* Orban de Xivry J-J, Bennett SJ, Lefèvre P, Barnes GR (2006) Evidence for synergy between saccades and smooth pursuit during transient target disappearance. J Neurophysiol 95:418-427. CrossRef Medline

* Orban de Xivry J-J, Missal M, Lefèvre P (2008) A dynamic representation of target motion drives predictive smooth pursuit during target blanking. J Vis 8:6:1-13. CrossRef

* Sailer U, Flanagan JR, Johansson RS (2005) Eye-hand coordination during learning of a novel visuomotor task. J Neurosci 25:8833-8842. CrossRef Medline

* Scarchilli K, Vercher JL, Gauthier GM, Cole J (1999) Does the oculomotor co-ordination control system use an internal model of the arm dynamics? Neurosci Lett 265:139-142. Medline

* Shadmehr R, Smith MA, Krakauer JW (2010) Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 33:89-108. CrossRef Medline

* Steinbach MJ (1969) Eye tracking of self-moved targets: the role of efference. J Exp Psychol 82:366-376. Medline

* Steinbach MJ, Held R (1968) Eye tracking of observer-generated target movements. Science 161:187-188. Medline

* Sternad D, Duarte M, Katsumata H, Schaal S (2001) Bouncing a ball: tuning into dynamic stability. J Exp Psychol Hum Percept Perform 27:1163-1184. Medline

* Vercher JL, Gauthier GM, Guédon O, Blouin J, Cole J, Lamarre Y (1996) Self-moved target eye tracking in control and deafferented subjects: roles of arm motor command and proprioception in arm-eye coordination. J Neurophysiol 76:1133-1144. Medline

* Vercher JL, Quaccia D, Gauthier GM (1995) Oculomotor coordination control: respective role of visual and non-visual information in ocular tracking of self-moved targets. Exp Brain Res 103:311-322. Medline

* Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nat Rev Neurosci 12:739-751. CrossRef Medline

* Wolpert DM, Flanagan JR (2001) Motor prediction. Curr Biol CB 11:R729-R732. Medline