# A Trajectory Model for Desktop-Scale Hand Redirection in Virtual Reality

Flavien Lebrun, Sinan Haliyo, Gilles Bailly

► To cite this version:

Flavien Lebrun, Sinan Haliyo, Gilles Bailly. A Trajectory Model for Desktop-Scale Hand Redirection in Virtual Reality. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.105-124, 10.1007/978-3-030-85607-6_8. hal-03330566

HAL Id: hal-03330566

https://hal.sorbonne-universite.fr/hal-03330566

Submitted on 1 Sep 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Trajectory Model for Desktop-scale Hand Redirection in Virtual Reality

Flavien Lebrun, Sinan Haliyo, and Gilles Bailly

Sorbonne Université, CNRS, ISIR, Paris, France

**Abstract.** In Virtual Reality, visuo-haptic illusions such as hand redirection introduce a discrepancy between the user's hand and its virtual avatar. This visual shift can be used, for instance, to provide multiple virtual haptic objects through a single physical proxy object. This low-cost approach improves the sense of presence, however, it is unclear how these illusions impact the hand trajectory and if there is a relationship between trajectory and the detection of illusion. In this paper, we present an empirical model predicting the hand trajectory as a function of the redirection. It relies on a cubic Bézier curve with 4 control points. We conduct a two alternative forced choice (2AFC) experiment to calibrate and validate our model. Results show that (1) our model predicts well the hand trajectory of each individual using a single parameter; (2) the hand trajectory better explains the detection of the illusion than the amplitude of the redirection alone; (3) a user specific calibration allows to predict per-user redirected trajectories and detection probabilities. Our findings provide a better understanding of visuo-haptic illusions and how they impact the user's movements. As such they may provide foundations to design novel interaction techniques, e.g. interacting in a scene with multiple physical obstacles.

**Keywords:** Visuo-haptic illusion · Hand redirection · Detection Threshold · Trajectory estimation.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Model Overview. Our hand trajectory model is a Bézier Curve. The resulting trajectory $B(l)$ is defined by 4 control points: $P_0$, $C_1$, $C_2$ and $P_t$. We show that the redirection can be accounted for by only adjusting the $x$ coordinate of point $C_2$. This adjustment depends on the redirection amplitude $A$ and a user-dependent factor $b(U)$.</figcaption></figure>

# 1 Introduction

The Central nervous system integrates inputs from all senses to construct a unified percept of reality. Virtual Reality (VR) through *Head Mounted Displays* gives means to subtly distort the sense of vision, which can be used to skew the integration with other modalities such as haptic to create various illusions. Among these visuo-haptic illusions, *hand redirection* (e.g. haptic retargeting [2]), affects the virtual location of the hand so that users reach for a certain physical target in a different position than its virtual counterpart. This is especially useful in *passive haptics*, where a single physical prop can be used for 2 or more virtual neighboring objects.

Several empirical studies have been conducted showing that these illusions are imperceptible when the amplitude of redirection is below a certain threshold [14, 37]. However, the effect of different amplitudes of illusion on motor control has not been investigated.

In this article, we investigate how the amplitude of the redirection alters the hand trajectory. We make the hypothesis that the detection of the illusion stems from users observing the distortion of their own movement. The detection of the illusion can then be predicted from a geometric description of the hand trajectory. In this prospect, we propose an analytic model of the hand trajectory using
cubic Bézier curves, constructed with four control points (see figure 1). In this model, under the approximation that the trajectory is planar, 2D coordinates of these 4 points are sufficient to describe the hand movement, regardless of the coordinate system.

We conduct a two-alternative-forced-choice (2AFC) experiment on a target reaching task under hand redirection illusion. Collected data are used to refine and validate the model as well as to explore the relationship between trajectory adjustment, and the ability of the users to detect the illusion.

Our main findings are (1) our model predicts well the adjusted hand tra-
jectory of each individual using a single parameter (see figure 1); (2) the hand
trajectory better explains the detection of the illusion than the redirection am-
plitude alone; (3) a user specific calibration allows to predict per-user redirected
trajectories and the probability to detect the illusion.

Our findings provide a better understanding of visuo-haptic illusions and
how they impact users' movements. They also provide foundations to design
novel interaction techniques, e.g. interacting in a scene with multiple physical
obstacles or to develop low-cost calibration tasks without exposing the users to
the illusions.

2 Related Work

2.1 Visuo-haptic illusions

The brain integrates information from different senses to construct a robust
percept [13, 12]. The sense with the best reliability will be favored in this in-
tegration. This model of integration has been verified for the combination of
vision and vestibular cues for the perception of displacement [28] and for the lo-
calization of body parts with vision and proprioception [4, 10, 5]. Vision is often
more reliable than other senses and thus generally favored by the brain. This
phenomenon is called *visual dominance* [17, 7].

Several VR interaction techniques leverage visual dominance to influence
users' experience in a virtual environment (VE). For example in *redirected walking* [30], users have the feeling to walk along a straight line while they are made
to follow a curved path. It's achieved through a non-strict mapping between
head rotation and orientation in the real and virtual world.

Manipulating virtual representations induce visio-haptic illusions. These illu-
sions trick the user to perceive different shapes [3] or mechanical characteristics
such as weight or stiffness [33, 11, 24], or to overrate a haptic device's perfor-
mance [1]. Hand redirection is one of the most popular techniques to induce
such illusions.
## 2.2 Hand redirection

Hand redirection refers to an alteration of the mapping between users' real hands and their virtual avatars. One of the main application of hand redirection is in passive haptics, where a single physical object – prop – is used as haptic proxy to several virtual objects in neighboring locations [26, 2, 23, 9]. During the movement toward a virtual object, users' virtual hands are gradually shifted from their real hands. Users' unwittingly correct this shift. This leads to the redirection of users' real hands toward the physical prop.

A critical aspect of this approach is the *amplitude of redirection*, ie. the degree of mismatch between the real hands and their avatars. Indeed with an appropriate amount of redirection users remain unaware of the mismatch. However, when this one is too large, the illusion is detected and affects negatively the user experience [21]. It is thus important to determine the maximum amplitude of redirection – detection threshold – beyond which users do notice the illusion.

## 2.3 Detection Threshold

Several methods have been proposed to determine the detection threshold of visuo-haptic illusions. One method consists of asking if participants perceived a manipulation of their visual feedback during the interaction [25, 31, 8, 1, 20]. However, participants might have difficulties to judge a barely perceptible phenomenon. This method thus requires a large safety margin.

Another method relies on a two-alternative forced-choice experiment (2AFC) where the participants are only asked in which direction they think the visual feedback was manipulated. This forced choice approach is more robust to participants' subjectivity and provides a lower bound threshold (by using a psychometric function). This method has been used for instance to study redirect walking [34, 22] and visuo-haptic conflicts [27].

Previous work exploiting these methods investigate hand redirection detection threshold at the *population* level [14, 37]. However, humans have different perceptual abilities and this detection threshold differs from user to user. In this paper, we study whether this detection threshold can be adapted to each *individual* with a simple calibration task.

## 2.4 Redirected hand velocity and trajectory

Some works took an interest on the effect of the redirection on the hand velocity and trajectory. Gonzalez et al. looked at tangential velocities and noted that the minimum jerk model (MJM) [16] doesn't hold for large amplitudes of redirection [18]. They did not however analyse the evolution of hand trajectory. The correction of the hand trajectory under redirection has been *qualitatively* mentioned by Azmandian et al [2]. They observed the general shape of the trajectory and pointed out that it generally exhibits a kink towards the end. In this paper, we propose a trajectory model linking the amplitude of redirection and the shape of the real hand trajectory.
## 3 Approach

Hand redirection has been shown to enrich the interaction in VR, especially when users explore different haptic features [1, 33, 11, 24] or manipulate several objects [2, 26]. However, it remains unclear how hand trajectory is affected by redirection, and if there is a relation between the hand trajectory and the detection of the illusion. Our approach consists of elaborating a mathematical model to explain and predict the hand trajectory as a function of the amplitude of redirection (i.e. the angle defined between the virtual hand, the real hand and the starting position; Figure 2). We then use this model to explore the relationship between the users’ detection of the illusion and his hand trajectory.

Beyond a better understanding of hand redirection, the model has several implications for design. For instance, it can serve to define a light calibration task to estimate the appropriate detection threshold for each participant. It can also provide theoretical foundations to design novel interaction techniques, e.g it would be possible to dynamically adapt the amplitude of redirection to control the hand trajectory and avoid physical obstacles while maintaining the illusions.

### 3.1 Research questions

We investigate the relationship between the trajectory (T) of the hand movement, the amplitude of redirection (A) and the Detection Threshold ($A_{DT}$).

- **RQ1:** What is the influence of the amplitude of redirection on hand trajectory? ($A \to T$)

- **RQ2:** Does the detection threshold depend on the features of the trajectory? ($T \to A_{DT}$). In particular, we aim to study whether the probability to detect the illusion is better explained by the deformation of the trajectory under redirection ($A + T$), instead of A alone.

### 3.2 Problem formulation

**Hand trajectory.** Let $T = P_0, P_1, \dots P_n$ the trajectory of the hand movement where $p_0$ is the starting point and $P_n = P_t$ the position of the target ($t$). We assume it exists a function $f$ such as:

$$T = f(u, P_t, P_0, A) \qquad (1)$$

where $T$ is the trajectory produced by the participant $u$, when reaching the target $t$ at the location $P_t$, from the position $P_0$, with an amplitude of redirection $A$. We aim to determine the function $f$ to answer the first research question (RQ1).

**Model of gesture trajectory.** We model hand trajectory as a Bézier curve. A Bézier curve is a polynomial parametric curve, and is defined as :

$$B(l) = \sum_{i=0}^{k} \binom{k}{i} (1-l)^{k-i} l^i C_i \qquad (2)$$
where $k$ is the degree of the Bezier curve and $C_0, ..., C_k$ the control points. This parametric model is widely used in mechanical design and computer graphics and has also been used to model hand trajectories [15]. This formulation has several advantages. First, it considerably reduces the complexity of the description of a trajectory as the number of control points is small in comparison with the number of points of the trajectory $k << n$. Moreover, the description is smooth, continuous and invariant to the user speed and sampling rate. Finally, the position of the control points provides a convenient and geometrical interpretation of the trajectory (see figure 1. In particular the first and last control points are the start and end point of the trajectory, so these are fixed in our modeling.

$$C_0 = P_0; C_K = P_t \qquad (3)$$

Figure 4 shows the impact of moving the control point $C_2$ (while keeping the three other controls fixed) on the Bézier curve.

**Number of control points** A choice must be made regarding the degree $k$ of the Bézier curve, i.e. the number of control points. A higher $k$ better describes a given trajectory, but it increases the complexity of the model and reduces its interpretability. Moreover, it is important to use the same $k$ for different trajectories in the purpose to compare them. Based on pilot studies, we found that cubic Bézier curves with 4 controls points ($k = 4$) was sufficient to well describe, compare and explain the different trajectories.

Based on the equations 1 and 3, we can then reformulate our problem as estimating the function $g$ such as (see 1):

$$C_{1x}, C_{1y}, C_{2x}, C_{2y} = g(u, P_t, P_0, A) \qquad (4)$$

where $C_{ix}$ and $C_{iy}$ are the x and y coordinates of the control point $C_i$. To achieve this, we conducted a user experiment to collect data and to identify the model parameters (see section 4). Validation of the model will allow us to predict the trajectory for a given user $u$, under any redirection $A$.

**Detection threshold** Based on the literature [37, 34], the population model predicting the probability $P$ of detecting the illusion given the amplitude of redirection ($A$) is a psychometric function:

$$P(A) = \frac{1}{1 + \exp - \frac{A-PSE_G}{\sigma}} \qquad (5)$$

where $PSE_G$ is the global point of subjective equality, i.e. the amplitude such as $P(PSE_G) = 50\%$. It can be seen as the amplitude where the participants estimate that no redirection is applied and is usually different than 0. $\sigma$ is the spread (inverse slope). $A_{LT}$ and $A_{RT}$ are two other specific values such as $P(A_{LT}) = 25\%$ and $P(A_{RT}) = 75\%$. The detection threshold $A_{DT}$ is then defined as:

$$A_{DT} = A_{RT} - A_{LT} \qquad (6)$$
The smaller $\sigma$ the steeper the slope of the psychometric curve is and, in our case, the smaller $A_{DT}$ is. The same methodology can be applied for the whole population (population model) or each individual (individual model).

So, the second research question (RQ2) has the objective to estimate $PSE_G$ and $\sigma$ as a function of the trajectory T:

$$P(A, T) = \frac{1}{1 + \exp -\frac{A-PSE_G(T)}{\sigma(T)}} \quad (7)$$

# 4 Data Collection

Our user experiment is similar to Zenner et al. [37] where participants perform a pointing task in VR while experiencing visuo-haptic illusions. Their primary objective was to investigate the probability to detect the illusion depending on the amplitude of redirection. In this experiment, we also investigate hand trajectory to refine, calibrate and test our model. This experiment has been approved by the IRB 2020-CER-2020-61.

## 4.1 Participants

10 participants (8 male and 2 female, between 25 and 30 years old) took part in the study. 2 participants were left-handed, 8 were right-handed. 4 participants wore glasses and one wore contact lenses. They did not report any other visual impairment or neuromuscular disorder. All the participants except 2 had experienced a VR headset before, but all less than three times. None were familiar with VR.

## 4.2 Apparatus

**Physical setup.** Participants were seated in front of a standard table and with a fixed position seat. They wore an HTC Vive head-mounted-display (HMD), a white-noise headphones and a right-hand glove with a cluster of optical markers on the index finger. Both positions and orientations of the HMD and the glove were tracked by a motion capture system (Optitrack) providing submillimeter precision. We thus saved 3D positions of the real hand and projected them on a horizontal plane. We used a sampling rate of 0.03s resulting in around 40 points per trajectory.

On the table, there was an haptic marker for "starting position", a joystick placed on the left of the starting position and 6 cylindrical targets located along a semi-circular arc illustrated Figure 2-left. Only the 4 central targets were actually touched by participants, the targets at the two extremities of the arc were just used as lures. These targets are located at 30 cm distance from the starting position. The 30 cm distance was chosen to be easily reachable by participants while they are seated on the chair. The angular distance between targets is 15°. It is large enough to avoid accidental physical collisions and to study the influence of target orientation on hand trajectory and illusion detection.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Left: The experimental table with 6 physical targets. Right: The virtual scene as seen by participants during the experiment. The phantom of the real hand (left) is displayed for illustration purpose but was not visible during the experiment. The dashed lines represent the straightest reach toward the target for the real hand (orange) and the virtual hand (purple). The curved lines is a representation of the actual trajectory of the two hands.</figcaption></figure>

**Virtual scene.** The virtual scene is illustrated on Fig. 2-right. It mimics the real set-up. It is implemented with the *Unity3D* game engine and shows the table, the starting position, one virtual target as well as an avatar of the participant's right hand. The virtual target has the same color and shape as the real one.

**Hand redirection implementation.** The virtual target's position was computed from the position of the chosen real target and the amplitude of redirection (see Figure 2). The shift between the virtual and the real hands was implemented such as it increases linearly during the reaching motion and the real hand reaches the real target simultaneously as the virtual hand reaches the virtual target. As such, users effectively touch a real object which provides a *haptic confirmation*. Participants compensate the shift while reaching the target, resulting in a curved trajectory (see Figure 2). Note that in our implementation, the hand is considered as a single point, the forefinger tip.

## 4.3 Experimental design

**Stimulus and Task.** The experiment is a *two-alternative forced choice* (2AFC): Once the right arm of the participants is in the starting position, the trial starts and the virtual target is displayed. The participants are asked to touch the target with their right hand and then to come back to the starting position (we made this choice to minimize task difference among participants, thus we restrain even left handed participants in using their right arm). The participants are asked to move naturally toward the target as we do not know the effect of speed on the illusion detection. If the participants are too fast or too slow the experimenter
asks them to slow down or to accelerate. After they came back to the starting
position, they move the joystick in the corresponding direction with their left
hand to indicate whether their real hand was positioned on the left or the right
of its virtual avatar. No feedback is provided.

**Conditions.** In this experiment, we controlled two factors. The primary factor is AMPLITUDE of redirection with 15 levels from -13° to 13° (-13°, -10°, -8°, -6°, -5°, -4°, -2°, 0°, 2°, 4°, 5°, 6°, 8°, 10°, 13°). The second factor is TARGET Orientation from -22.5° to 22.5° (step of 15°).

**Procedure.** The participants were first instructed about the goal of the exper-
iment and the task to perform. In particular, the concept of hand redirection
was explained in these terms: "*A virtual hand that follows the position of your
hand is displayed in the VE. During the reaching task an offset will be gradually
introduced between this virtual hand and your real hand. The virtual hand will
be located either on the left or the right of your real hand.*"

Participants put on the glove, HMD and headphone. They then performed
a training phase. It consists of 2 blocks of 10 trials where they experience hand
redirection with an amplitude of either -13° or +13° (corresponding to the high-
est amplitudes in the main experiment). During the training phase, participants
received feedback at the end of the trial regarding the direction (left or right)
of the redirection.They were also informed about the trial time as they have to
calibrate their speed so that the trial time is between 1s and 2s. Finally, during
the first block, the position of the real hand was displayed in the virtual scene
in addition to the hand avatar to understand the concept of hand redirection.

**Design.** We used a within-subject design. Each participant completed 4 blocks. In each block, the participants tested the 60 combinations of AMPLITUDE and TARGET in a randomized order. In summary, the experimental design is : 10 participants × 4 blocks × 15 AMPLITUDES × 4 TARGETS = 2400 trials.

**Dependent variables.** The two dependent variables are CHOICE (left or right) and Hand TRAJECTORY, i.e. the sequence of points to reach the target.

# 5 Analysis 1: Trajectory and Amplitude of redirection

In this section, we analyze how the amplitude of redirection influences the posi-
tion of the two control points C₁ and C₂. We first describe our empirical findings.
We then refine our model and compare four model variants.

**5.1 Empirical results**

**Method.** We first removed 60 (2.5%) outliers trajectories. We calculated for each amplitude of redirection the distance (MSE) between a given trajectory
and the mean trajectory. Each trajectory with a MSE larger than a threshold was plotted for a verification of their wrong shape. For the resulting 2340 trajectories, we estimated the four parameters $C_{1x}$, $C_{1y}$, $C_{2x}$, $C_{2y}$ of the Bézier curve that minimize the Dynamic Time Warping (DTW) distance [32, 35]. DTW is appropriate as it is independent of the user speed. We use the python package DTAIDistance for calculating the DTW distance and the function "minimize" of the "scipy.
s-optimize" package with the Nelder-Mead algorithm for the optimization method. The resolution of the Bézier curve is 250 points per trajectory.

**Result.** The Figure 3 shows the mean value of the four parameters with 95% confidence Interval (CI) as a function of the amplitude of redirection. We were expecting that the four parameters vary with the amplitude, but the results show that three parameters can be approximated by a constant: $C_{1x} = 0.35cm$ ($ci = [-0.0, 0.8]$), $C_{1y} = 15.3cm$ ($ci = [15.2, 15.4]$), $C_{2y} = 43.4cm$ ($ci = [42.6, 44.2]$). However, $C_{2x}$ linearly increases with the amplitude ($R^2 = 0.99$, MSE = 1.9).

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** The value of the four parameters $C_{1x}$, $C_{1y}$, $C_{2x}$, $C_{2y}$ as a function of the amplitude of redirection. Error bars show 95% confidence interval.</figcaption></figure>

**Discussion.** We learned three things. First, as $C_1$ is fixed and aligned with $\vec{P}_0\vec{P}_T$, it confirms that the initial direction of the user's movement is towards the virtual target. Second, as only one parameter varies, these results suggest that using a Cubic Bézier curve model is appropriate. Third, it exists a simple and elegant linear relationship between $C_{2x}$ and the amplitude of redirection. We can thus revisit our model, reduce its complexity and improve its explainability.
## 5.2 Refining and evaluating the model

Based on our findings, we revisit our model (equations 3 and 4) and propose four model variants summarized in Table 1. Three parameters ($C_{1x}$, $C_{1y}$, $C_{2y}$) are fixed. We introduce two novel parameters a and b to approximate the x coordinate of $C_2$:

$$C_{2x} = aA + b \qquad (8)$$

where A is the amplitude of redirection, a is the slope, i.e the sensitivity to the amplitude and b, the intercept, reflects the natural human bias at doing curved trajectories even when no hand redirection is applied [36]. To study whether these two parameters are the same for all participants (population parameter) or participant dependent (individual parameter), we defined four model variants (Table 1) reflecting the four configurations.

<table><caption>**Table 1.** Comparisons of four model variants in terms of fixed and free (population and individual) parameters, number of free parameters (k), distance (DTW), Likelihood and BIC. The model with b as user-dependent parameter has the lowest BIC score.</caption><thead><tr><th>Model</th><th>Fixed parameters</th><th>Population parameters</th><th>Individual parameters</th><th>k</th><th>DTW-LL</th><th>BIC</th></tr></thead><tbody><tr><td><i>M</i></td><td><i>C</i><sub>1x</sub> <i>C</i><sub>1y</sub>, <i>C</i><sub>2x</sub></td><td>a, b</td><td>-</td><td>2</td><td>670</td><td>501</td><td>1012</td></tr><tr><td><i>M</i><sub>b</sub></td><td><b><i>C</i></b><sub>1x</sub> <b><i>C</i></b><sub>1y</sub>, <b><i>C</i></b><sub>2x</sub></td><td><b>a</b></td><td><b>b</b></td><td><b>11</b></td><td><b>576</b></td><td><b>401</b></td><td><b>857</b></td></tr><tr><td><i>M</i><sub>a</sub></td><td><i>C</i><sub>1x</sub> <i>C</i><sub>1y</sub>, <i>C</i><sub>2x</sub></td><td>b</td><td>a</td><td>11</td><td>661</td><td>457</td><td>969</td></tr><tr><td><i>M</i><sub>a,b</sub></td><td><i>C</i><sub>1x</sub> <i>C</i><sub>1y</sub>, <i>C</i><sub>2x</sub></td><td>-</td><td>a, b</td><td>20</td><td>559</td><td>396</td><td>896</td></tr></tbody></table>

## 5.3 Model comparison

We compare the capacity of the model variants to accurately predict the trajectories of each class $C_A^u$ where A is the Amplitude of redirection and u a user (participant). To achieve this, we first define $d(A, u, m)$ the average DTW distance between the predicted trajectory $T_{pred}(A, u, m)$ and all observed trajectories $T_{obs}^0(A, u)...T_{obs}^N(A, u)$ of the class $C_A^u$ for a given model m:

$$d(A, u, m) = \frac{1}{N} \sum_{i=0}^{N} DTW(T_{pred}(A, u, m), T_{obs}^{i}(A, u)) \qquad (9)$$

We can then use a Boltzmann soft-max function to transform the distance $d(A, u, m)$ into probability $P(C_A^u | A, u, m)$:

$$P(C_A^u | A, u, m) = \frac{e^{-\beta d(A, u, m)}}{\sum_a e^{-\beta d(A, u, m)}} \qquad (10)$$

where the parameter $\beta$ indicates how much the probability distribution is concentrated around the positions of the smallest distance. We chose $\beta = 1$.
**Model likelihood.** Based on the equation 10, we now compare the result of models fitness function. In Bayesian terms, we compare the likelihood of the data given the model, that is the maximum probability $P(C_A^u)$ that the model chooses the correct class of trajectories $C_A^u$. Formally, we estimate:

$$LL(m) = \sum_{A,u} \log P(C_A^u|A, u, m, \theta_m^p) \qquad (11)$$

where $\theta_m^u$ is the set of parameters of the model $m$ for the participant $u$.

**BIC score.** In the process of model selection, it is common to include a penalty term for model complexity, i.e. for the number of parameters [29]. The Bayesian Information Criterion (BIC score) is commonly used. It is estimated as $BIC = -2LL + k \times \log(N)$ where $LL$ is the likelihood (equation 11), $k$, the number of parameters (i.e. individual parameters + population parameters), and $N = 150$ ( 10 participants × 15 amplitudes ), the number of points to predict.

**Result.** The Table 1 indicates that the model $M_{a,b}$ better fits the data. It is not surprising as it has much more parameters than the other model variants. When penalizing for the number of parameters, the BIC score suggests that $M_b$ better explains the data. This result indicates that $a$ is not sensitive to the user id. In contrast, model prediction benefits the estimation of $b$ for each user.

## 5.4 Discussion

**RQ1:** *What is the influence of the amplitude of redirection on hand trajectory?*
Our analysis showed a clear impact of amplitude of redirection on hand trajectory. More precisely, we learned that 1) if we model hand trajectory as a simple

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Visualisation of all the trajectories for three different amplitudes of redirection and for the same participant. We display the Bezier curve resulting from the model $M_b$. All the control point are fixed except $C_2$ that have a linear relation with the amplitude.</figcaption></figure>
cubic Bézier curve, the amplitude of redirection **only affects** the x coordinate of a single control point ($C_2$). This makes the model highly interpretable (Figure 4). Moreover, 2) this x coordinate increases linearly with the amplitude of the redirection; 3) the slope is independent of the participant, i.e increasing the amplitude by 1° moves $C_{2x}$ of 1.13cm on the right; 4) the intercept is user dependent (mean = 2.1 cm, std = 3.9). This result is inline with [36] indicating that even when no redirection is applied humans perform curved trajectories (due to visual perceptual distortion). The degree and direction of curvature depends on the participant.

Interestingly, the calibration of the model is easy to perform. Indeed, the individual parameter *b* is the intercept, i.e. it is the value of $C_{2x}$ when $A = 0$ (equation 8). It is thus possible to estimate *b* for each participant **without** experiencing hand redirection. *b* can be estimated by simply performing a reaching task without illusion.

# 6 Analysis 2: Detection threshold

In this section we study the second research question RQ2: *Does the detection threshold depend on the features of the trajectory?*. We first evaluate the probability to detect the illusion as a function of the amplitude of redirection and then as a function of the deformation of the hand trajectory.

## 6.1 Amplitude of Redirection and Detection Threshold

Figure 5-Left illustrates the probability $P(A)$ to detect the illusion as a function of the amplitude of redirection $A$ (psychometric function corresponding to the equation 5) for the whole population. We found $PSE_G = -1.13°$, $A_{LT} = -4.52°$, $A_{RT} = 2.27°$. We compute the detection threshold $A_{DT} = A_{RT} - A_{LT} = 6.79°$. Our results are in the same order of magnitude than Zenner et al. study [37] ($PSE_G = -0.28°$, $R = 8.19°$). The difference can be due to the experimental setup and/or the absence of haptic confirmation at the end of the movement.

## 6.2 Hand Trajectory and Detection Threshold

We now investigate the link between the hand trajectory and the probability to detect the illusion. As $C_{2x}$ is sufficient to describe the hand trajectory, we analyze the probability to detect the illusion $P(C_{2x})$ as a function of $C_{2x}$ with the same methodology of section 6.1. We discriminated $C_{2x}$ into 15 groups based on its magnitude. We made this choice to compare the two psychometric functions, $P(A)$ and $P(C_{2x})$ of the figure 5.

Figure 5-right shows the result of the psychometric fit for $P(C_{2x})$ ($MSE = 64.0$) which is better than the one of $P(A)$ ($MSE = 68.3$), Figure 5-Left. It shows that the more the hand trajectory is curved the easier the illusion is detected. This result was expected given the linear relation between $A$ and $C_{2x}$ outlined in section 5. However, the fact that both the amplitude of redirection and the trajectory explain the illusion detection requires further explanations.
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Psychometric fit of the answer of the 2AFC experiment. Left : detection according to the amplitude of redirection. Right : detection of the illusion according to the $C_{2x}$ coordinate.</figcaption></figure>

## 6.3 Further explanations

To better understand the role of $C_{2x}$ on illusion detection, we analysed its magnitude as a function of the absolute amplitude of redirection ($|A|$) and whether the participants answer correctly (detection of the illusion) or not (the illusion works) to the 2AFC task. We removed extreme amplitudes $0^{\circ}$ as there was no error and $\pm 13^{\circ}$ as all participants detected the illusion for these amplitudes.

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** $x$ coordinate of $C_2$ as a function of the redirection amplitude and whether the answer of participants is correct or false.</figcaption></figure>

Our results are illustrated Figure 6. A two-way ANOVA confirmed the effect of AMPLITUDE on the amplitude of $C_{2x}$ ($F_{5,45} = 109.4, p < .0001$). ANOVA also revealed an effect of ANSWER (Correct vs., False) on the amplitude of $C_{2x}$ ($F_{1,9} = 7.6, p < .05$). A post Tukey-test indicates that at a given amplitude, $C_{2x}$
is larger when the illusion is detected (mean= 3.4cm) than when the illusion is not detected (mean= 7.4 cm). ANOVA does not reveal AMPLITUDE × ANSWER interaction effect.

**Discussion** The amplitude of redirection is the primary factor to explain and predict whether the participants will detect or not the illusion. However, given an amplitude of redirection, we demonstrate that the curvature of the hand trajectory, i.e. the magnitude of $C_{2x}$, refines the prediction. Indeed, participants performing low curved trajectories (i.e. small $C_{2x}$) are less likely to detect the illusion. The user dependent parameter $b$ reflects this natural tendency to perform curved trajectories (to the right ($b > 0$) or to the left ($b < 0$)) under no redirection. We thus decided to study more precisely the influence of $b$ on the detection threshold.

<figure><img src="image_8.png" /><figcaption>Fig. 7. Evaluation of $PSE_I$ (in blue) and $\sigma_I$(in orange) for each participant (1 - 10) according to parameter $b$.</figcaption></figure>

## 6.4 User-dependent detection threshold

We now evaluate the relation between the user-dependent parameter $b$ and the probability to detect the illusion expressed in equations 5 and 7:

$$P(A, b) = \frac{1}{1 + \exp -\frac{A-PSE_I(b)}{\sigma_I(b)}} \quad (12)$$

where $PSE_I$ is the Point of Subjective Equality for each Individual ($PSE_G$ was the Point of Subjective Equality global, i.e. for the whole population). Figure 7 illustrates $PSE_I$ and $\sigma_I$ as a function of $b$. While no clear relationship is revealed
regarding $\sigma_I$ ($R^2 = 0.25$; MSE = 14.39), there is a weak relationship between b and $PSE_I$ ($R^2 = 0.63$; MSE = 12.32):

$$PSE_I = -0.39 \times b - 0.19 \qquad (13)$$

Thus there is a relation between the value of $C_{2x}$ when no redirection is applied $C_{2x} = b$ and $PSE_I$. In other words, a user performing naturally a curved trajectory to the right ($b > 0$) will be more sensitive to a redirection to the right ($PSE_I < 0$). This result is important, because after estimating $b$, a designer can estimate $PSE_I$ of a user and know in which direction (left or right) the user is less likely to detect the illusion. The designer can also measure $b$ of each individual of a given population and estimate the unique range of amplitude of redirection that best fit this population.

This is what is illustrated Figure 8. On the left, we see the range of amplitude of redirection (blue) which is not detected for each participant. The intersection is small: The vertical surface inficates the maximal ranges for which the illusion is not detected for respectively 70%, 80% and 90% of the population. It results that only the amplitudes of redirection in [-0.7°;0.7°] is not detected for 70% of the population.

However, when $b$ is known for each participant, it is possible to choose a unique range of amplitude of redirection and to adapt it to each participant (based only on the parameter $b$). This is what is illustrated on Figure 8-right where each range of amplitude is virtually re-centered based on $b$, offering a range of [-1.7°,1.7°] which is almost four times larger.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Left: The horizontal blue rectangles depict the range of redirection amplitudes for which the illusion is not detected for each participant. The vertical surfaces indicate the maximal ranges for which the illusion is not detected for respectively 70% (green), 80% (red) and 90% (yellow) of the population. The wider are these ranges, the more flexibility is offered to the designers. However, increasing the number of participants decreases this population range. Right: When taking individual differences, i.e. $b$ into account, we can artificially recenter their range, increasing the range of amplitudes for the whole population.</figcaption></figure>
# 7 Discussion

## 7.1 Main findings

We revisit our two research questions from section 3.

**RQ1: What is the influence of the amplitude of redirection on hand trajectory?**
We have shown that a cubic Bézier curve constructed using four control points $C_i, i \in [0; 3]$ approximates the hand trajectory quite faithfully. Moreover, the coordinates of 3 out of 4 control points do not depend on the redirection for a given user. The control point $C_2$ is the only one which is modified by the amplitude of redirection: $C_{2x}$ linearly increases with the amplitude such as $C_{2x}(A) = 1.13A + 1.97$ at the population level. At the individual level, further analysis revealed that the intercept is user dependent, $C_{2x}(A, u) = 1.13A + b(u)$. These results demonstrate that 1) the amplitude of redirection influences only a single feature of the trajectory when described with a Bézier curve; 2) the trajectory can be better approximated when considering the parameter $b$ which 3) can be easily estimated for each user. Indeed, $b$ indicates the curvature of the trajectory when users do not experience an illusion.

**RQ2: Does the detection threshold depend on the features of the trajectory?**
It was well known that the detection threshold depends on the amplitude of redirection. We demonstrated that it also depends on the trajectory. In particular, we demonstrated that once the amplitude of redirection is given, users performing trajectories with low curvature (i.e. smaller $C_{2x}$) are less likely to detect the illusion. Our hypothesis is that users detect the illusion through observing the distortion of their hand motion.

To determine individual detection thresholds, we used a psychometric function with two parameters $PSE_I$ and $\sigma_I$. Our analysis shows a relationship between $PSE_I$ and $b$ but not with $\sigma_I$. In other words, estimating $b$ for each participant can increase the range of amplitudes of redirection of 258% without risking the illusion to be detected by at least 70% of our participants.

## 7.2 Implications for design

Our findings on **RQ1** suggest a method for designers to anticipate user hand trajectory during a redirected reaching task. In particular, designers can *easily* elaborate a calibration task to estimate $b$ and refine the trajectory model as it does not require to expose the users to the illusion ($A = 0$). This trajectory model can for instance be exploited advantageously to make users unwittingly circumvent obstacles or encountered-type haptic devices [6, 19]. Our findings on **RQ2** indicate a simple way of adapting the range of amplitude of redirection to each user. The knowledge of the parameter $b$ allows the computation of individual $PSE_I$. The designer can then center the population range of amplitudes around the $PSE_I$ to minimize the risk of detecting the illusion. Again, this only requires to estimate $b$ without exposing the user to the illusion.
## 7.3 Limitation and Future work

Future studies should be conducted to validate the robustness of the model. First, left-handed participants had to reach for targets with their right hand. This choice was made to facilitate the comparison of data between participants. Even though we interact with the environment with both hands, we favor our dominant hand for reaching tasks. Thus this could have impacted the hand trajectory and their illusion detection threshold. Moreover, the speed of the hand can have an impact on the detection of the illusion and was not constant among participants. Then, participants were informed about the illusion at the beginning of the experiment, thus the calculated detection threshold is likely bigger.

Furthermore, concerning the trajectory model, we assumed that 3 of the 4 coordinates of the Bézier control points are fixed. However the Figure 3 shows that $C_{1x}$ and $C_{2y}$ are slightly affected by the amplitude of redirection. As future work, we plan to refine our model to investigate whether it significantly improves the prediction of the beginning and the end of the trajectory. The distance to the target is fixed in our task and different distances should be tested as well.

Finally, the use of a 2AFC experiment for the determination of detection threshold is debatable. On one hand, users can sense that something is odd in their movement, without being able to pinpoint the relative position of their real and virtual hand. On the other hand, if users do not focus on the position of their hands we would certainly find a larger range of non-detection. We hypothesize that there is not a clear breaking point of the illusion. In particular, we plan to investigate whether user involvement might have an impact on *sigma* and thus on the ability to detect the illusion.

## 8 Acknowledgments

We would like to thank Benoit Geslain and Hugues Lebrun for their valuable feedback, as well as the participants of experiments.

## References

1. Abtahi, P., Follmer, S.: Visuo-haptic illusions for improving the perceived performance of shape displays. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 150:1–150:13. CHI ’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3173574.3173724

2. Azmandian, M., Hancock, M., Benko, H., Ofek, E., Wilson, A.D.: Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 1968–1979. CHI ’16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2858036.2858226

3. Ban, Y., Kajinami, T., Narumi, T., Tanikawa, T., Hirose, M.: Modifying an identified curved surface shape using pseudo-haptic effect. In: 2012 IEEE Haptics Symposium (HAPTICS). pp. 211–216 (March 2012). https://doi.org/10.1109/HAPTIC.2012.6183793
4. van Beers, R.J., Wolpert, D.M., Haggard, P.: When feeling is more important than seeing in sensorimotor adaptation. Current Biology **12**(10), 834–837 (2002).
https://doi.org/https://doi.org/10.1016/S0960-9822(02)00836-9

5. Block, H.J., Sexton, B.M.: Visuo-proprioceptive control of the hand in older adults. bioRxiv (2020). https://doi.org/10.1101/2020.01.18.911354

6. Bouzbib, E., Bailly, G., Haliyo, S., Frey, P.: Covr: A large-scale force-feedback robotic interface for non-deterministic scenarios in VR. CoRR abs/2009.07149 (2020), https://arxiv.org/abs/2009.07149

7. Burns, E., Razzaque, S., Panter, A., Whitton, M., McCallus, M., Brooks, F.: The hand is slower than the eye: a quantitative exploration of visual dominance over proprioception. In: IEEE Proceedings. VR 2005. Virtual Reality, 2005. pp. 3–10 (2005). https://doi.org/10.1109/VR.2005.1492747

8. Burns, E., Razzaque, S., Panter, A.T., Whitton, M.C., McCallus, M.R., Brooks, F.P.: The hand is more easily fooled than the eye: Users are more sensitive to visual interpenetration than to visual-proprioceptive discrepancy. Presence: Teleoperators and Virtual Environments **15**(1), 1–15 (2006).
https://doi.org/10.1162/pres.2006.15.1.1

9. Cheng, L.P., Ofek, E., Holz, C., Benko, H., Wilson, A.D.: Sparse haptic proxy: Touch feedback in virtual environments using a general passive prop. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. p. 3718–3728. CHI ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025753

10. van Dam, L.C.J., Ernst, M.O.: Knowing each random error of our ways, but hardly correcting for it: An instance of optimal performance. PLOS ONE **8**(10), 1–9 (10 2013). https://doi.org/10.1371/journal.pone.0078757

11. Dominjon, L., Lecuyer, A., Burkhardt, J., Richard, P., Richir, S.: Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments. In: IEEE Proceedings. VR 2005. Virtual Reality, 2005. pp. 19–25 (March 2005). https://doi.org/10.1109/VR.2005.1492749

12. Ernst, M.O., Banks, M.S.: Humans integrate visual and haptic information in a statistically optimal fashion. Nature **415**(6870), 429–433 (Jan 2002).
https://doi.org/10.1038/415429a

13. Ernst, M.O., Bülthoff, H.H.: Merging the senses into a robust percept. Trends in Cognitive Sciences **8**(4), 162–169 (Apr 2004).
https://doi.org/10.1016/j.tics.2004.02.002

14. Esmaeili, S., Benda, B., Ragan, E.D.: Detection of scaled hand interactions in virtual reality: The effects of motion direction and task complexity. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 453–462 (2020).
https://doi.org/10.1109/VR46266.2020.00066

15. Faraway, J.J., Reed, M.P., Wang, J.: Modelling three-dimensional trajectories by using bézier curves with application to hand motion. Journal of the Royal Statistical Society: Series C (Applied Statistics) **56**(5), 571–585 (2007).
https://doi.org/https://doi.org/10.1111/j.1467-9876.2007.00592.x
https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9876.2007.00592.x

16. Flash, T., Hogan, N.: The coordination of arm movements: an experimentally confirmed mathematical model. Journal of Neuroscience **5**(7), 1688–1703 (1985).
https://doi.org/10.1523/JNEUROSCI.05-07-01688.1985

17. Gibson, J.J.: Adaptation, after-effect and contrast in the perception of curved lines. Journal of experimental psychology **16**(1), 1 (1933)
18. Gonzalez, E.J., Abtahi, P., Follmer, S.: Evaluating the minimum jerk motion model for redirected reach in virtual reality. In: The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology. p. 4–6. UIST ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3332167.3357096

19. Gonzalez, E.J., Abtahi, P., Follmer, S.: REACH+: Extending the Reachability of Encountered-Type Haptics Devices through Dynamic Redirection in VR, p. 236–248. Association for Computing Machinery, New York, NY, USA (2020)

20. Gonzalez, E.J., Follmer, S.: Investigating the detection of bimanual haptic retargeting in virtual reality. In: 25th ACM Symposium on Virtual Reality Software and Technology. pp. 1–5 (2019)

21. Gonzalez-Franco, M., Lanier, J.: Model of illusions and virtual reality. Frontiers in Psychology 8, 1125 (2017). https://doi.org/10.3389/fpsyg.2017.01125, https://www.frontiersin.org/article/10.3389/fpsyg.2017.01125

22. Grechkin, T., Thomas, J., Azmandian, M., Bolas, M., Suma, E.: Revisiting detection thresholds for redirected walking: Combining translation and curvature gains. In: Proceedings of the ACM Symposium on Applied Perception. pp. 113–120. SAP ’16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2931002.2931018

23. Han, D.T., Suhail, M., Ragan, E.D.: Evaluating remapped physical reach for hand interactions with passive haptics in virtual reality. IEEE Transaction+s on Visualization and Computer Graphics 24(4), 1467–1476 (April 2018). https://doi.org/10.1109/TVCG.2018.2794659

24. Jauregui, D.A.G., Argelaguet, F., Olivier, A., Marchal, M., Multon, F., Lecuyer, A.: Toward "pseudo-haptic avatars": Modifying the visual animation of self- avatar can simulate the perception of weight lifting. IEEE Transactions on Visualization and Computer Graphics 20(4), 654–661 (April 2014). https://doi.org/10.1109/TVCG.2014.45

25. Kasahara, S., Konno, K., Owaki, R., Nishi, T., Takeshita, A., Ito, T., Kasuga, S., Ushiba, J.: Malleable embodiment: Changing sense of embodiment by spatial-temporal deformation of virtual human body. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 6438–6448. CHI ’17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025962

26. Kohli, L.: Redirected touching: Warping space to remap passive haptics. In: 2010 IEEE Symposium on 3D User Interfaces (3DUI). pp. 129–130 (March 2010). https://doi.org/10.1109/3DUI.2010.5444703

27. Lee, Y., Jang, I., Lee, D.: Enlarging just noticeable differences of visual-proprioceptive conflict in vr using haptic feedback. In: 2015 IEEE World Haptics Conference (WHC). pp. 19–24 (June 2015). https://doi.org/10.1109/WHC.2015.7177685

28. MacNeilage, P.R., Banks, M.S., Berger, D.R., Bülthoff, H.H.: A bayesian model of the disambiguation of gravitoinertial force by visual cues. Experimental Brain Research 179(2), 263–290 (2007)

29. Raftery, A.E.: Bayesian model selection in social research. Sociological methodology pp. 111–163 (1995)

30. Razzaque, S., Kohn, Z., Whitton, M.C.: Redirected Walking. In: Eurographics 2001 - Short Presentations. Eurographics Association (2001). https://doi.org/10.2312/egs.20011036

31. Rietzler, M., Geiselhart, F., Gugenheimer, J., Rukzio, E.: Breaking the tracking: Enabling weight perception using perceivable tracking offsets. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing
Systems. pp. 128:1–128:12. CHI ’18, ACM, New York, NY, USA (2018).
https://doi.org/10.1145/3173574.3173702

32. Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing **26**(1), 43–49 (1978). https://doi.org/10.1109/TASSP.1978.1163055

33. Samad, M., Gatti, E., Hermes, A., Benko, H., Parise, C.: Pseudo-haptic weight: Changing the perceived weight of virtual objects by manipulating control-display ratio. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. pp. 320:1–320:13. CHI ’19, ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300550

34. Steinicke, F., Bruder, G., Jerald, J., Frenz, H., Lappe, M.: Estimation of detection thresholds for redirected walking techniques. IEEE Transactions on Visualization and Computer Graphics **16**(1), 17–27 (Jan 2010). https://doi.org/10.1109/TVCG.2009.62

35. Velichko, V., Zagoruyko, N.: Automatic recognition of 200 words. International Journal of Man-Machine Studies **2**(3), 223–234 (1970).
https://doi.org/https://doi.org/10.1016/S0020-7373(70)80008-6

36. Wolpert, D.M., Ghahramani, Z., Jordan, M.I.: Perceptual distortion contributes to the curvature of human reaching movements. Experimental brain research **98**(1), 153–156 (1994)

37. Zenner, A., Krüger, A.: Estimating detection thresholds for desktop-scale hand redirection in virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 47–55 (2019). https://doi.org/10.1109/VR.2019.8798143