# Electrotactile Feedback For Enhancing Contact Information in Virtual Reality

Sebastian Vizcay, Panagiotis Kourtesis, Ferran Argelaguet Sanz, Claudio Pacchierotti, Maud Marchal

<figure><img src="image_1.png" /></figure>

## To cite this version:

Sebastian Vizcay, Panagiotis Kourtesis, Ferran Argelaguet Sanz, Claudio Pacchierotti, Maud Marchal. Electrotactile Feedback For Enhancing Contact Information in Virtual Reality. ICAT-EGVE 2021 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments, Sep 2021, Sankt Augustin, Germany. 10.2312/egve.20211331. hal-03350039

HAL Id: hal-03350039

https://inria.hal.science/hal-03350039

Submitted on 21 Sep 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Electrotactile Feedback For Enhancing Contact Information in Virtual Reality

S. Vizcay<fnref n="1" /> ID, P. Kourtesis<fnref n="1" /> ID, F. Argelaguet<fnref n="1" /> ID, C. Pacchierotti<fnref n="2" /> ID and M. Marchal<fnref n="3" /> ID

## Abstract

This paper presents a wearable electrotactile feedback system to enhance contact information for mid-air interactions with virtual objects. In particular, we propose the use of electrotactile feedback to render the interpenetration distance between the user's finger and the virtual content is touched. Our approach consists of modulating the perceived intensity (frequency and pulse width modulation) of the electrotactile stimuli according to the registered interpenetration distance. In a user study (N=21), we assessed the performance of four different interpenetration feedback approaches: electrotactile-only, visual-only, electrotactile and visual, and no interpenetration feedback. First, the results showed that contact precision and accuracy were significantly improved when using interpenetration feedback. Second, and more interestingly, there were no significant differences between visual and electrotactile feedback when the calibration was optimized and the user was familiarized with electrotactile feedback. Taken together, these results suggest that electrotactile feedback could be an efficient replacement of visual feedback for enhancing contact information in virtual reality avoiding the need of active visual focus and the rendering of additional visual artefacts.

## CCS Concepts

* Human-centered computing → Haptic devices; Virtual reality; User studies;

**Keywords:** electrotactile feedback; virtual reality; human computer interaction; contact precision; contact accuracy; contact rendering; surface; interpenetration; haptic interface

## 1. Introduction

Immersive virtual reality (VR) is becoming a common platform for education and training, where precision and accuracy are important aspects [Car17,JK18]. Visual feedback is important for contact accuracy and precision [TMGL20]. However, when interacting in VR, if no other measures are adopted, the user's avatar might interpenetrate the virtual content, which results in significantly reducing the accuracy, precision, and the realism of the interaction. To avoid such an undesired behavior and facilitate mid-air interactions in VR a popular technique is the *God-object Method*, where the virtual hand is constrained by the collision with the virtual object(s) [ZS94,SH16]. Using the *God-object Method*, there is a decoupling of the visual representation of the user's avatar (e.g., the virtual hand) from its actual position by, e.g., constraining the avatar on the virtual object's surface even when the actual tracked position would be inside the object [PB16].

However, using this popular technique, while the virtual hand is rendered on the collided object's surface, the user's physical hand is

still able to move in the real environment unconstrained (i.e., penetrating the virtual object's coordinates), which results in failing to perceive the inaccuracy and imprecision of the contact [ZS94,SH16]. Considering the dominance of visual information, a visual interpenetration feedback could provide the required information to facilitate contact precision and accuracy. Nevertheless, providing additional visual cues, would occlude other virtual surfaces and render visual artefacts in the virtual environment. Visual artefacts negatively affect contact precision and accuracy [QBY*15]. To cope with this limitation, a haptic feedback proportional to the interpenetration may provide the user with information regarding the contact precision and accuracy. This haptic information hence indicates the user's interpenetration in the virtual content [MPS*18], which would enable to somehow mimic the behavior of real contacts, where interpenetrating deeper in a virtual object elicits stronger tactile sensations.

Haptic VR interfaces provide vibrotactile, pressure, or skin stretch feedback sensations [PSS*17]. However, these displays need to embodied one or more mechanical actuators that move an end-effector in contact with the user's skin, limiting their wearability, portability, and comfort [PSS*17]. In this respect, electrotactile haptic stimulation has been recently recognized as a promising technique to provide rich tactile sensations across the hand or arm through a single wearable actuator [CMH08]. Applying electrical stimulation

<footnote n="1">¹Inria, Univ. Rennes, IRISA, CNRS – 35042 Rennes, France.</footnote>

<footnote n="2">²CNRS, Univ. Rennes, IRISA, Inria – Rennes 35042, France.</footnote>

<footnote n="3">³Univ. Rennes, INSA, IRISA, Inria, CNRS – 35042 Rennes, France and Institut Universitaire de France.</footnote>
to the skin elicits tactile sensations by directly activating the sensory nerve endings, de facto bypassing the skin mechanoreceptors. The electrical current is produced by a local polarization near the skin, generated by two or more electrotactile interfaces (electrodes). With respect to other tactile solutions, electrotactile stimulation only requires very thin electrodes placed at the sensation point (e.g., the fingertip), providing a wearable and comfortable solution to tactile haptics.

In a VR pilot study (N=3), electrotactile feedback provided haptic information pertinent to heat, texture, jolt, and texture [PW16]. In a small size user study (N=10), electrotactile feedback was able to provide the VR user with tactile sensations such as roughness, friction, and hardness [YK17]. In a VR user study (N=19), electrotactile feedback was successful in improving grasping [HDC*16]. Despite its promising features, the benefits of electrotactile feedback in VR interaction are still under-investigated. This paper presents the design and evaluation of a wearable electrotactile feedback system to enhance contact information for mid-air interactions with virtual objects.

The main contributions of our work can be summarized as follows:

* Design of an electrotactile feedback technique for enhancing contact information in VR. A technique where the intensity of the electrotactile feedback is proportional to the interpenetration (i.e., distance between real and virtual hand).

* A user study (N=21) for evaluating electrotactile interpenetration feedback's performance against no interpenetration feedback, a visual interpenetration feedback, and a visual-electrotactile interpenetration feedback.

## 2. Related work

**Wearable haptics in VR.** Popular techniques to provide rich wearable haptic feedback are through moving platforms, that can orient and/or translate on the skin, pin-arrays, shearing belts and tactors, pneumatic jets, and balloon-based systems [PSS*17]. Frisoli et al. [FSSB08] were among the first to present a fingertip haptic display for improving curvature discrimination using a moving platform. It was composed of a parallel platform and a serial wrist. Gabardi et al. [GSLF16] improved this device and applied it in VR interaction. The pose of the user's finger was tracked through external optical sensors and collisions between the finger avatar and the virtual surface were rendered by the device. Minamizawa et al. [MKKT07] presented a wearable fingertip device for weight rendering in VR. It consisted of two DC motors moving a belt in contact with the user's fingertip. A similar device was also used in [PSH*16] for multi-finger manipulation of objects in VR and in [SPdT*20, DTPML18] for augmenting tangible objects in VR. More recently, Girard et al. [GCWG17] developed a wearable fingertip device capable of rendering 2-DoF skin stretch stimuli in VR. The considered use cases comprised tapping on a virtual bottle and feeling the texture and weight of a virtual object.

**Electrotactile haptics.** Due to the difficulty in designing effective interaction paradigms, electrotactile interfaces have rarely been used in VR scenarios. Jorgovanovic et al. [JDD*14] evaluated the capabilities of human subjects to control grasping force in closed loop

using electrotactile feedback. It was tested in an experiment of virtual robotic grasping. More recently, a pilot study by Pamungkas and Ward [PW16] used electrotactile stimulation on the hand to convey the sensation of contacts in VR. Use cases comprised playing with a virtual bouncing ball, setting up a camp fire, feeling different surface textures, and shooting a gun. Hummel et al. [HDC*16] developed an electrotactile delivery system composed of a small tactor with eight electrodes for each finger. They tested it in three VR assembly tasks for remote satellite servicing missions, which included pressing a button, switching a lever switch, and pulling a module from its slot. Finally, Yem et al. [YVKK18] employed electrotactile stimulation to elicit flexion/extension illusory movements to the user's fingertip. Doing so, the system was able to induce haptic feedback sensations, providing information about virtual objects' stiffness and stickiness properties.

**Contact information in VR.** In the last years, researchers focused on the sensation of making and breaking contact [dTHP*20, CPMP17, CMPP19, YGKP20], as it is proven to be an important factor for immersion in VR. Kuchenbecker et al. [KFK*08] endowed a desktop kinesthetic interface with a passive tactile end-effector. The kinesthetic feedback provided by the desktop device bends the internal springs of the tactile end-effector and brings a shell in contact with the user's finger, providing the sensation of making/breaking contact with the virtual object. In a similar fashion, Gonzalez et al. [GBG15] attached a ring of optical sensor to the end-effector, being able to measure the distance of the fingertip from the ring and adjust the transition between free space and contact. Finally, Prachyabrued and Borst [PB16] evaluated the impact of visual feedback in pinch grasping a virtual object, where best performance was obtained when showing both hand models (tracked and constrained) and when coloring the fingers of the object being grasped.

To the best of our knowledge, the effects of using electrotactile feedback for enhancing contact information in VR mid-air interactions has scarcely been studied.

## 3. Electrotactile Interpenetration Feedback

The tactile sensation elicited by electrotactile stimulation can be controlled by both, the parameters chosen for the design of the electrodes and the parameters of the delivered electrical signal [KTOH17, FGV08, KKMT99, LKS09]. Individual pads in an electrode can be disabled or enabled (enabled as either cathode or anode) meanwhile signal's intensity, pulse width and frequency can be increased/decreased to apply some modulation. Pad selection determines the location where the stimulus is going to be applied.

Changes in the current intensity is the most common way to increase/decrease the strength of the sensation, but the feeling is highly variable across subjects and relatively small increases can quickly elicit uncomfortable sensations. Changes in the pulse width and frequency can also elicit changes in the perceived strength of the sensation, although less significantly than when altering the current intensity [Aie98]. Changes in the frequency are also employed to increase/decrease how fast the tickles are delivered.

Given the disadvantages of intensity modulation, we fixed the intensity value for each participant, chosen after a brief calibration so as to avoid any undesired uncomfortable/painful sensation. We
chose pulse width modulation to control the strength of the sensation and frequency modulation to control its speed. We chose to place the electrodes at the fingertip, as it is the area of the body most often used for pointing, grasping, and manipulating objects [PSS*17]. To produce the stimulation at the center of the finger pad, we select an anode-cathode configuration with the widest possible dynamic range for the given electrode. As mentioned before, we modulate the electrotactile interpenetration feedback proportionally to the distance between the constrained user's fingertip avatar (i.e., the object surface) and its actual position as tracked by the system. In other words, both the perceived intensity and speed of the electrotactile sensation increase as the user's physical hand interpenetrates the virtual object.

To achieve this effect, the pulse width is modulated linearly from 200 µs to 500 µs. On the other hand, we considered the perception of frequency to be logarithmic [VS13] and we apply a gamma correction so as to linearize how fast the tingles are perceived. The proposed frequency modulation allows us to explore a range of frequencies from 30 Hz to 200 Hz, which is the maximum frequency of the device. Fig. 1 shows the two inputs with respect to the normalized interpenetration (1: maximum, 0: zero, see subsection 3.2 for the details). Choices regarding the range of stimulation and the active pads have been done after a short preliminary study, so as to provide the most comfortable tactile sensations to the finger pad.

<figure><img src="image_3.png" /><figcaption>**Figure 1:** electrotactile interpenetration feedback modulation via pulse width (left) and frequency (right).</figcaption></figure>

## 3.1. Electrical Stimulator and Electrodes

Our custom electrotactile stimulator has 32 channels, to which up to 5 electrodes can be connected simultaneously. Each channel may be defined either as a cathode or as an anode. The stimulator produces a biphasic cathodic stimulation through square waves. Electrical intensity and pulse width can be set per each channel in the ranges of [0.1, 9] mA and [30, 500] µs, respectively. The stimulus frequency can be set in the range [1, 200] Hz.

We used electrodes whose cathodes are distributed in a matrix array of 2 rows by 3 columns, surrounded by two larger lateral anodes that are interconnected by a segment going through both rows of cathodes (see the insets in the top-left corner of Figure 2). This particular layout has being designed to widen the dynamic range of sensations.

<figure><img src="image_4.png" /><figcaption>**Figure 2:** *Experimental apparatus.* The electrical stimulator is attached to the forearm and the electrode is placed in contact with the finger pad. The two insets represent detailed views of the electrode. The user's hand is tracked by a HTC Vive Tracker.</figcaption></figure>

## 3.2. System Design

The interpenetration of the finger avatar into the virtual content drives our pulse width and frequency modulation. After a series of pilot user studies, we determined that, for a cube of edge of 15 cm, 3 cm represents the maximum interpenetration achieved in most interactions. For this reason, we normalized the interpenetration *d* by this value, i.e., a normalized interpenetration $\hat{d}$ of 1 means a true interpenetration into the virtual object of 3 cm.

The position of the participant's hand is tracked using a HTC Vive Tracker attached to the back of the hand. The *physics-based* hand avatar (physics-based as its interactions are handled by the physics engine) collides with the other objects in the virtual environment while trying to follow the position indicated by the tracker. As described at the beginning of section 3, this avatar resolves all virtual collisions and avoids any undesired virtual interpenetration. Whenever it comes into contact with a virtual object, its motion is constrained to stay on the object's surface, regardless of how much the user's hand moves into the object. As detailed above, this discrepancy between the user's real hand and its avatar drives our electrotactile interpenetration feedback.

## 4. Evaluation

This section describes a user study evaluating the proposed electrotactile interpenetration feedback system. Our main objective was to assess the potential advantages and disadvantages of using electrotactile interpenetration feedback while performing a task requiring a high level of accuracy and precision. In particular, we focused on a task in which the user is asked to contact a virtual surface with his/her index finger while minimizing the amount of interpenetration between the virtual surface and the real index finger. In such a task, when the virtual hand avatar cannot interpenetrate virtual surfaces, it is difficult for the user to precisely assess the amount of interpenetration between his/her finger and the virtual surface without any additional feedback.

Our main hypothesis is that additional interpenetration feedback, either visual and/or electroactile, would increase the awareness of
the user, effectively decreasing the amount of interpenetration. Yet, electrotactile interpenetration feedback will provide a more coherent experience. In addition to consider the electrotactile interpenetration feedback method presented in the previous section, for comparison purposes, we also modulated the availability of visual cues. First, we considered a visual interpenetration feedback technique. Second, we considered the interaction with two different virtual objects, a fully-opaque virtual cube and a wireframe-only cube. We also considered the latter so as to limit the visual cues which could anticipate the contact with the virtual object.

Hence, the independent variable (IV) *Interpenetration Feedback* has four levels: 1) No Interpenetration Feedback; 2) Visual Interpenetration Feedback; 3) Electrotactile Interpenetration Feedback, 4) Visuo-electrotactile Interpenetration Feedback (i.e., interpenetration information is provided both visually and haptically). The four levels of the IV are examined in two object rendering methods: 1) fully-opaque and 2) wireframe-only. In the following, we detail how these techniques are implemented in our experimental study.

## 4.1. Experimental Task

The task required participants to touch a virtual surface (a cube of size 15 × 15 × 15 cm) which was located on a virtual table (see Figure 3). Participants started the experience in a seated position in front of said virtual table. The cube was placed on the user's left side, while a resting area was located on the user's right side. For each trial, participants were asked to start with their right hand in the resting area, to move to touch with their index finger the top side of the virtual cube precisely while keeping a steady contact for three seconds, and then they were notified that should return their hand again to the resting area. Participants were instructed to minimize the interpenetration between the virtual surface and their real index finger while ensuring a continuous contact with the cube. To ensure that all participants kept a consistent timing, auditory feedback (a short beep sound) was used to notify the users when they had to start the trial and when the three seconds of contact were achieved. During the whole interaction, participants were asked to keep their hand in a closed fist, with the index finger fully extended for facilitating a similar posture and an approximate collocation between the physical and the virtual hand. The experimenter was prompting each participant to have this posture when a new block was starting, while during the breaks between the blocks, the participant could have a rest and move her/his hand and fingers freely or just relax, which assisted with avoiding a postural fatigue.

## 4.2. Electrotactile Interpenetration Feedback Calibration

The calibration of the electrotactile interpenetration feedback is a key step to ensure that all participants perceived a consistent tactile feedback. As the calibration cannot be done automatically, participants were asked to calibrate the intensity of the electrotactile interpenetration feedback multiple times throughout the experiment. If the calibration is not done per each user, parameters such as the skin's conductivity and/or a tolerance could result in stimuli that are not perceived (i.e., too low intensity) or uncomfortable or even painful (i.e., too high intensity).

The right range of current intensity was determined using the

method of limits. Starting from a low intensity value, the intensity was increased considering steps of 0.1 mA. This incremental procedure continued, step by step, until the participant confirmed that the electrotactile feedback was perceived, defining his/her sensation detection threshold. Then, the incremental approach continued until the participant confirmed that the electrotactile feedback caused discomfort, defining his/her pain/discomfort threshold. Similarly, starting from a high intensity value, the intensity was decreased considering steps of 0.1 mA. This starting high intensity value was defined as the 3rd quartile of the range between the detection and pain/discomfort thresholds. This decremental procedure continued, step by step, until the participant confirmed that the electrotactile feedback was not perceived anymore.

During the entire calibration process, the signal frequency was set to 200 Hz and the pulse width to 500 µs. These values were chosen as they represent the strongest stimulation that could be achieved through the interpenetration feedback, as detailed in the previous Section. The final intensity value to be used throughout the experiment was calculated from a linear interpolation between the detection and discomfort thresholds. We chose an intermediate value between these thresholds, as it would result in a perceivable yet comfortable stimulus

## 4.3. Participants and Apparatus

A total of 21 individuals [age: mean(SD) = 25.67(2.74); range = 23 – 33 years old; gender: 14 males, 6 females, and 1 non-binary] were recruited through an in-campus advertisement and a call on institutional email lists. 21 participants were familiar with using VR head-mounted displays (VR-HMDs), 10 participants had a previous experience with receiving tactile feedback in VR (predominantly vibrotactile), and only 2 had experienced electrotactile feedback in the past.

An HTC Vive Pro HMD was used for immersion. The PC had an NVIDIA RTX 2070. An HTC Vive Tracker attached to the back of the palm (see Figure 2) and 2 Lighthouse base stations were used for tracking. See Figure 2 for the placement of electrotactile stimulator, which weighted 300 g. A Velcro strap was used to hold the electrode, its tightness was adjusted to achieve a comfortable yet firm placement of the electrode.

The VR application was built using Unity in conjunction with the SteamVR Unity plugin (see Figure 3). The collisions were directly handled by the SteamVR plugin and the Unity physics simulation. A blackboard was used for providing any critical information/instructions within the participant's field of view.

## 4.4. User Representation and Visual Interpenetration feedback

The user could only see a visual representation of their right hand (proxy.Avatar). As stated earlier, the hand avatar cannot go through the virtual object so as to better mimic a real interaction. The hand is always kept in a closed fist, with the index finger fully extended, which is the same pose we asked participants to keep while interacting with the cube (see Figure 4).

An outline effect was used as visual interpenetration feedback
<figure><img src="image_5.png" /><figcaption>**Figure 3:** Depiction of the subjective view of the user during the experiment. The user could touch the virtual cube placed on the top of the table of the left. The semitransparent cube on the right side defines the resting area. The blackboard in front of the user provided general information during the experiment.</figcaption></figure>

(see Figure 4. Outline effects are commonly used to denote interactivity or selection [ZCH*19, SCZ*20]. However, the implemented outline was dynamically rendered proportional to interpenetration's depth. The outline was parameterized using a scale factor and the size of the border. The scale factor went from ×1 (edge of 15 cm³) to ×1.2 (edge of 18 cm³). The border varied between 1 and 5 pixels.

When both electrotactile and visual interpenetration feedbacks were provided, a mismatch between the two modalities may result in a haptics' uncanny valley effect, which compromises the performance [BGFOH18]. For this reason, the modification of border size and width of the outline was determined through a pilot study. The choice of this particular parameters' range aimed to ensure that, comparably to the electrotactile interpenetration feedback, the perception of the visual interpenetration feedback's intensity is proportional to the size of the interpenetration.

## 4.5. Experimental Protocol

This study received the approval of the authors' institution ethical committee. As it was carried out during the COVID-19 pandemic, a preventive hygiene and disinfection protocol was implemented. Participants signed an informed consent form prior to their participation. The set up consisted of a VR area, a table, and a chair. The table was co-located with the virtual table. A pre-exposure questionnaire was filled to acquire demographic data. Participants went through a tutorial for performing the experimental task.

The experiment followed a latin square design, 4 types of interpenetration feedback × 2 types of cube shading × 2 parts of the experiment (i.e., 4 × 2 × 2). Each experiment's part had 4 blocks, where each block consisted of 12 repetitions. In total, each participant performed 96 repetitions (i.e., 48 per experiment's part). In total, 3 calibrations were performed a) Before Part 1; b) After Part 1; c) After Part 2. Finally, a post-exposure questionnaire was filled to acquire the participant's subjective ratings on the types of feedback, with a focus on electrotactile interpenetration feedback.

<figure><img src="image_6.png" /><figcaption>**Figure 4:** Visual shading conditions (left & right) and visual interpenetration feedback (right).</figcaption></figure>

## 4.6. Dependent Variables and Hypotheses

In order to answer the hypothesis of our experiment and considering the experimental task we measured the following metrics: average (avg_d), standard deviation (std_d) and maximum (max_d) interpenetration during the three seconds in which the user was asked to remain in contact with the virtual cube surface. As we did not track the user's finger but the back of the hand, the interpenetration was computed by measuring the distance between the position of the real fingertip in the virtual environment and the position of the avatar fingertip constrained on the surface of the cube. The interpenetration measures provide information regarding the awareness of the real interpenetration (accuracy) and their ability to keep a constant and not oscillatory contact with the virtual surface (precision). In this respect our hypotheses were:

* [H1] The addition of interpenetration feedback (either visual or electrotactile) will increase the precision and the accuracy of the task.

* [H2] The combination of the visual and electrotactile interpenetration feedback will result on the best precision and accuracy.

* [H3] Precision and accuracy will be higher for the fully shaded condition.

* [H4] Precision and accuracy will improve along the experiment due to learning effects.

The sensation and discomfort thresholds (in mA) were obtained during the three calibration processes. We expected that the thresholds will vary due to changes in skin conductivity or due to habituation. In this respect our hypothesis was:

* [H5] Sensation and discomfort thresholds will monotonically increase during the experiment.

Finally, the participants' subjective report were collected through a post-exposure questionnaire. The questionnaire is included as supplementary material. The questionnaire encompassed questions pertaining to the perceived usefulness, coherence, and resemblance with a real interaction of the electrotactile interpenetration feedback, as well as some comparison questions between the interpenetration feedback modalities. In this respect our hypotheses were:

* [H6] Electrotactile interpenetration feedback will be reported as useful and coherent.

* [H7] There will be no significant differences between visual and electrotactile interpenetration feedback in terms of usefulness, coherence, and/or resemblance.
## 5. Results

The homogeneity and normality assumptions were examined. The performance variables violated the normality assumption. The values were converted into logarithms since the variables' distributions were substantially positively skewed. No violation was then detected. The same amendments were made for the intensity values, which also violated the normality assumption. Finally, as the shading condition showed no significant results, which suggests the rejection of **H3**. In sake of clarity, the shading condition will not be considered in the results report. Thus, for the following analyses the conditions were the type of interpenetration feedback and the part of experiment.

A two-way repeated measures ANOVA was performed for each performance variable, as well as a post-hoc pairwise comparisons test respectively. When the sphericity assumption was violated, then the Greenhouse Geisser's correction was considered. Furthermore, a pairwise comparison was conducted to explore potential differences in the intensity values amongst the three calibrations (i.e., initial, middle, and final). Also, a Pearson's correlational analysis was performed to investigate the presence of relationships between the intensity values and the performance variables. Finally, the post-exposure self-reports (Likert scale) by the participants were explored. A non-parametric pairwise comparison was implemented to examine the presence of differences on the questionnaire's items. The Bonferroni correction was implemented accordingly to the number of comparison's pairs in each analysis to amend p-values' inflation due to multiple comparisons. The descriptive statistics for all performance variables and the intensities, as well as the results for the maximum interpenetration are all included in the supplementary material for completeness.

### 5.1. Index Finger Interpenetration

**Average Index Interpenetration - Accuracy** The ANOVA's outcomes indicated that all effects were statistically significant. The type of feedback produced a main effect of *F*(1.97, 39.31) = 25.89, *p* < .001 postulating a substantial difference amongst the feedback types. In line with **H1**, an *ω*² = 0.54, 95% CI[0.35, 0.66] suggesting a very large effect of the type of feedback on average interpenetration (i.e., accuracy). In support of **H4**, the part of the experiment yielded an effect of *F*(1, 20) = 31.74, *p* < .001, suggesting a significant difference between the two parts of the experiment with a very large effect size of *ω*² = 0.58, 95% CI[0.26, 0.75]. The interaction between the parts of the experiment and the types of feedback produced an effect of *F*(2.11, 42.21) = 5.72, *p* = .006 indicating significant differences among the pairs between the types of feedback and the parts of experiment. An *ω*² = 0.18, 95% CI[-0.01, 0.34] was yielded for these interactions indicating that they have a large effect on the accuracy.

The post-hoc pairwise comparisons are displayed in Figure 5. The results support both **H1** and **H2**. In the 1st part, the visual *g* = 0.90, *p* < .001 and combined feedback *g* = 1.10, *p* < .001 were substantially improved compared to the no interpenetration feedback condition, with a large effect in both comparisons. Also, the visual *g* = 0.52, *p* = .026 and combined feedback *g* = 0.67, *p* = .002 were substantially better than the electrotactile condition, with a moderate effect size in both comparisons. In the 2nd part, the visual

<figure><img src="image_7.png" /><figcaption>**Figure 5:** *Average Interpenetration in cm. Comparisons per Type of Feedback and Part of the Experiment. IF = Interpenetration Feedback*</figcaption></figure>

*g* = 1.10, *p* < .001, combined *g* = 1.69, *p* < .001, and electrotactile interpenetration feedback *g* = 1.10, *p* < .001, were significantly better compared to the no interpenetration feedback condition, with a very large effect in every comparison. No further significant differences were detected in both parts. Furthermore, the electrotactile *g* = 0.86, *p* < .001 and combined feedback *g* = 0.69, *p* < .001 were significantly improved in the 2nd part compared to the 1st part, with a large and a moderate effect correspondingly. In contrast, the equivalent comparisons for the no interpenetration feedback and visual interpenetration feedback were not statistically significant. Notably, only the size of change for electrotactile interpenetration feedback was significantly greater than the size of change for the no interpenetration feedback *g* = 0.64, *p* = .005 and visual interpenetration feedback *g* = 0.60, *p* = .010, with a moderate effect size for both.

**Standard Deviation Index Interpenetration - Precision** The results of the ANOVA suggested statistically significant effects at .05 significance level. Supporting **H1**, the main effect of the type of feedback on index oscillations (i.e., precision) yielded an effect of *F*(1.88, 37.69) = 28.92, *p* < .001, and of *ω*² = 0.57, 95% CI[0.39, 0.68] postulating significant differences between the diverse types of feedback with a very large effect size. Alligned with **H4**, the main effect for the part of experiment on the precision produced an *F*(1, 20) = 9.32, *p* = .006, and of *ω*² = 0.27, 95% CI[-0.01, 0.54] suggesting a significant difference between the two part of the experiment with a large effect. The interaction among the diverse pairs derived from the parts of the experiment and the types of feedback yielded an interaction effect of *F*(2.38, 47.66) = 4.30, *p* = .014, and of *ω*² = 0.13, 95% CI[-0.03, 0.29] postulating significant difference between the diverse pairs with a large effect size.

The post-hoc pairwise comparisons are illustrated in Figure 6. The results appear to support both **H1** and **H2**. In the 1st part of experiment, the visual *g* = 1.06, *p* < .001 and combined feedback *g* = 1.109, *p* < .001 were founded substantially greater than the no interpenetration feedback condition, with a very large effect in both comparisons. Also, the visual *g* = 0.79, *p* < .001 and combined feedback *g* = 0.90, *p* < .001 were found to be substantially better
<figure><img src="image_8.png" /><figcaption>**Figure 6:** Index Oscillation in cm. Comparisons per Type of Feed-back and Part of the Experiment. IF = Interpenetration Feedback</figcaption></figure>

than the electrotactile interpenetration feedback, with a large effect size in both comparisons. In the 2^nd^ part of experiment, the visual g = 1.32, p < .001, combined g = 1.71, p < .001, and electrotactile interpenetration feedback g = 1.07, p < .001, were found to be significantly better than the no interpenetration feedback condition, with a very large effect in every comparison. No further significant differences were detected in both parts. Furthermore, only the electrotactile g = 0.74, p < .001 was found to be substantially improved in the 2^nd^ part compared to the 1^st^ part, with a large effect. The equivalent comparisons for the no interpenetration feedback, visual interpenetration, and combined interpenetration feedback were not statistically significant. Importantly, the size of change for electrotactile interpenetration feedback from the 1^st^ to the 2^nd^ part was significantly larger than the size of change of the no interpenetration feedback g = 0.54, p = .027 and visual interpenetration feedback g = 0.59, p = .012, with a moderate effect size in both comparisons.

## 5.2. Calibration of Electrotactile Feedback

The pairwise comparisons are displayed in Figure 7. The sensation threshold (i.e., the intensity that the participant commences perceiving the electrotactile feedback), the discomfort threshold (i.e., intensity that the participant commences feeling discomfort due to electrotactile feedback), and the actual value (i.e., the weighted intensity that was implemented to provide electrotactile interpenetration feedback) of the middle (i.e., after the 1^st^ and before the 2^nd^ part of experiment) and final calibration (i.e., after the 2^nd^ part of experiment) were statistically greater than the equivalent intensities of initial calibration (i.e., before the 1^st^ part of experiment). Specifically, the sensation threshold g = 1.45, p < .001, the discomfort threshold g = 0.91, p < .001, and the actual value g = 1.08, p < .001 of middle calibration were substantially larger than the equivalent intensities of initial calibration, while the effect size was large to very large in each comparison. Similarly, the sensation threshold g = 0.70, p = .005, the discomfort threshold g = 0.83, p < .001, and the actual value g = 0.88, p < .001 of final calibration were significantly larger than the equivalent intensities of initial calibration, while the effect size was large in every comparison. However,

<figure><img src="image_9.png" /><figcaption>**Figure 7:** Intensity values in mA. Comparisons for each calibration.</figcaption></figure>

no significant differences were detected between final and middle calibration, which postulates the rejection of **H5**. Finally, none of the intensity values was significantly correlated either positively or negatively with any of the performance variables.

## 5.3. Subjective Results

The descriptive statistics of the post-exposure self-reports are displayed in Table 1, which support **H6**. In support of **H7**, the Wilcoxon signed rank test yielded no significant differences in the comparisons between the visual and electrotactile interpenetration feedback pertaining to usefulness (i.e., how useful was the feedback for the task's completion), coherence (i.e., how much coherent was the feedback with touching the surface), and resemblance (i.e., how much the feedback resembles touching a real surface). The Wilcoxon rank sum test, for the question pertaining to the combined feedback condition, indicated a significant difference *p* = .024 between electrotactile and combined feedback, postulating that the participants reported that they relied significantly more on the electrotactile interpenetration feedback rather than both interpenetration feedbacks (i.e., visual and electrotactile). However, in the same question, there was no significant difference between visual and electrotactile interpenetration feedback suggesting a balanced preference.

<table><caption>**Table 1:** Self-reports on Electrotactile Interpenetration Feedback</caption><thead><tr><td>Criterion</td><td>Median</td><td>Mode</td><td>Range</td></tr></thead><tbody><tr><td>Usefulness</td><td>6</td><td>5</td><td>2 - 7</td></tr><tr><td>Coherence</td><td>5</td><td>5</td><td>4 - 7</td></tr><tr><td>Resemblance</td><td>2.5</td><td>1</td><td>1 - 6</td></tr><tr><td>Pleasantness</td><td>4</td><td>4</td><td>3 - 6</td></tr><tr><td>Perceptual Update</td><td>5.5</td><td>6</td><td>2 - 7</td></tr><tr><td>Combined Synchronicity</td><td>6</td><td>7</td><td>2 - 7</td></tr></tbody></table>

*N = 21; Max = 7; ≥ 4 indicate positive response;

Median = middle number of the provided responses;

Mode = most frequent response

Moreover, the participants reported that the visual interpenetra-
tion feedback was useful and coherent, albeit that provided a feeling that is different from touching a real surface. Similarly, it was reported that the electrotactile interpenetration feedback was useful to very useful and coherent, albeit that provided a feeling that is different to very different from touching a real surface. Also, the self-reports suggested that the electrotactile interpenetration feedback provided neither a pleasant nor unpleasant sensation, as well as that it provided a frequent to very frequent perceptual update of pressing down the surface. Finally, the self-reports indicated that the combination of electrotactile and visual interpenetration feedback had a very coherent to completely coherent synchronization, and it was perceived as moderately similar as to touching a real surface.

## 6. Discussion

Our findings agree with previous studies, where bimodal feedback increased the accuracy and precision in contact tasks [TMGL20]. Also, our results are in accordance with a meta-analysis of user interaction studies, where the bimodal (i.e., visuohaptic) feedback was found to improve performance [BPG*06]. However, a haptic uncanny effect may substantially compromise the benefits of bimodal feedback on performance [BGFOH18]. In our study, the dynamic visual interpenetration feedback was matched to the varying intensities of electrotactile interpenetration feedback, which aligns with the suggestions to mitigate or evade a haptic uncanny effect [BGFOH18]. Hence, our results further support the importance of having matched feedback modalities, when implementing bimodal feedback.

Furthermore, our quantitative and qualitative outcomes align with the qualitative results of Pamungkas, et al. [PW13], where the participants reported that the electrotactile interpenetration feedback assisted them with performing a VR robot-teleoperation task faster and more accurately. However, in our quantitative user study the benefits of providing electrotactile interpenetration feedback became apparent only in the 2^nd^ part of experiment, suggesting the presence of a learning effect. Also, the intensity values for providing electrotactile interpenetration feedback were substantially increased in the middle calibration, which were used in the 2^nd^ part of experiment. In conjunction, our findings postulate that the advantages of electrotactile interpenetration feedback may surface, when the user has been familiarized with it, and/or the intensity of feedback is well-adjusted to skin's conductivity changes and/or user's increased tolerance to the electrotactile interpenetration feedback. However, the intensity values were not correlated with neither accuracy nor precision. Thus, the increased familiarity with the electrotactile interpenetration feedback that was inferred due to the observed learning effect, appears to better explain why the advantageous effects of electrotactile interpenetration feedback on performance were surfaced only during the 2^nd^ part of experiment.

It should be noted that the learning effect was not generally present. The learning effect was identified only for the bimodal visuotactile feedback regarding accuracy, as well as for the electrotactile interpenetration feedback regarding both accuracy and precision. These findings indicate that the learning effect was important only in conditions where the electrotactile interpenetration feedback was used. Also, the size of the improvement in the accuracy and precision from the 1^st^ to the 2^nd^ part of experiment was significantly greater for the electrotactile interpenetration feedback

condition compared to the size of improvement for no interpenetration feedback and visual interpenetration feedback conditions respectively. This finding also postulates that the learning effect was observed due to the novelty of the electrotactile interpenetration feedback, and not due a practice effect related to the task. Nevertheless, the effects of the familiarity with electrotactile interpenetration feedback and the refined calibration on user's performance should be further scrutinized in prospective studies.

Finally, the results postulate the effectiveness of electrotactile and/or visual interpenetration feedback. However, the implemented visual interpenetration feedback requires constant visual focus, would occlude other visual information, and render visual artefacts in a fully interactive virtual environment, which may reduce substantially the contact precision and accuracy [QBY*15]. In contrast, the suggested electrotactile interpenetration feedback appears to facilitate multiple contact interactions, although this should be further examined. Thus, the presented electrotactile feedback system seems to be the most appropriate and effective choice for improving precision and accuracy of mid-air interactions in immersive VR

## 7. Conclusion

In this paper, we proposed an electrotactile rendering algorithm to enhance contact information in a virtual environment based on the modulation of the pulse width and frequency of the electrical signal. In order to explore the benefits and limitations of the proposed method, we conducted a user study evaluating the electrotactile interpenetration feedback by examining the performance in touching accurately and precisely a virtual rigid object.

The results showed that participants achieved significantly better performance in terms of accuracy and precision when visual and/or electrotactile interpenetration feedback was provided. More importantly, there were no significant differences between visual and electrotactile interpenetration feedback when the calibration was optimized and the user was familiarized with electrotactile interpenetration feedback. Considering that visual discrimination is dominant in such scenarios and a visual interpenetration feedback renders visual artefacts, the findings indicate a promising future for electrotactile-based systems for improving contact precision and accuracy in interactive VR systems.

Further works could explore other alternatives for the modulation and location of the electrotactile feedback or consider other tactile actuators. Also, future work should attempt to replicate the experimental findings with finger tracking capabilities. Further experiments could explore interactions requiring more fingers (e.g. grasp operation) or the interaction with dynamic objects (e.g. pushing a virtual object). Finally, further iterations of the system should strive to improve its wearability.

## Acknowledgments

This work was supported by the European Union's Horizon 2020 research and innovation program under grant agreement No. 856718 (TACTILITY).
## References

[Aie98] AIELLO G. L.: Multidimensional electrocutaneous stimulation. *IEEE Trans. Rehabilitation Engineering* 6, 1 (mar 1998), 95–101. 2

[BGFOH18] BERGER C. C., GONZALEZ-FRANCO M., OFEK E., HINCKLEY K.: The uncanny valley of haptics. *Science Robotics* 3, 17 (2018), 5, 8

[BPG*06] BURKE J. L., PREWETT M. S., GRAY A. A., YANG L., STILSON F. R., COOVERT M. D., ELLIOT L. R., REDDEN E.: Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In *Proc. 8th international Conf. Multimodal interfaces* (2006), pp. 108–117. 8

[Car17] CARRUTH D. W.: Virtual reality for education and workforce training. In *2017 15th International Conf. Emerging eLearning Technologies and Applications (ICETA)* (2017), pp. 1–6. 1

[CMH08] CHOUVARDAS V., MILIOU A., HATALIS M.: Tactile displays: Overview and recent advances. *Displays* 29, 3 (2008), 185–194. doi: https://doi.org/10.1016/j.displa.2007.07.003. 1

[CMPP19] CHINELLO F., MALVEZZI M., PRATTICHIZZO D., PACCHIEROTTI C.: A modular wearable finger interface for cutaneous and kinesthetic interaction: control and evaluation. *IEEE Trans. Industrial Electronics* 67 (2019), 706–716. 2

[CPMP17] CHINELLO F., PACCHIEROTTI C., MALVEZZI M., PRATTICHIZZO D.: A three revolute-revolute-spherical wearable fingertip cutaneous device for stiffness rendering. *IEEE Trans. Haptics* 11, 1 (2017), 39–50. 2

[dTHP*20] DE TINGUY X., HOWARD T., PACCHIEROTTI C., MARCHAL M., LÉCUYER A.: Weatavix: Wearable actuated tangibles for virtual reality experiences. In *Haptics: Science, Technology, Applications: 12th International Conference, EuroHaptics 2020, Leiden, The Netherlands, September 6–9, 2020, Proceedings* (2020), vol. 12272, Springer Nature, p. 262. 2

[DTPML18] DE TINGUY X., PACCHIEROTTI C., MARCHAL M., LÉCUYER A.: Enhancing the stiffness perception of tangible objects in mixed reality using wearable haptics. In *Proc. of IEEE Conf. Virtual Reality and 3D User Interfaces* (2018), pp. 81–90. 2

[FGV08] FOLGHERAITER M., GINI G., VERCESI D.: A multi-modal haptic interface for virtual reality and robotics. *Journal of Intelligent and Robotic Systems* 52, 3-4 (2008), 465–488. 2

[FSSB08] FRISOLI A., SOLAZZI M., SALSEDO F., BERGAMASCO M.: A fingertip haptic display for improving curvature discrimination. *Presence: Teleoperators and Virtual Environments* 17, 6 (2008), 550–561. 2

[GBG15] GONZALEZ F., BACHTA W., GOSSELIN F.: Smooth transition-based control of encounter-type haptic devices. In *Proc. of IEEE International Conf. Robotics and Automation* (2015), pp. 291–297. 2

[GCWG17] GOSSELIN F., CHABRIER A., WEISTROFFER V., GIRARD A.: Design and development of miniature wearable haptic interfaces for the fingertips. In *Proc. of IEEE World Haptics Conference* (2017), pp. 611–616. 2

[GSLF16] GABARDI M., SOLAZZI M., LEONARDIS D., FRISOLI A.: A new wearable fingertip haptic interface for the rendering of virtual shapes and surface features. In *Proc. IEEE Haptics Symposium* (2016), pp. 140–146. 2

[HDC*16] HUMMEL J., DODIYA J., CENTER G. A., ECKARDT L., WOLFF R., GERNDT A., KUHLEN T. W.: A lightweight electrotactile feedback device for grasp improvement in immersive virtual environments. In *Proc. of IEEE Conf. Virtual Reality* (2016), pp. 39–48. 2

[JDD*14] JORGOVANOVIC N., DOSEN S., DJOZIC D. J., KRAJOSKI G., FARINA D.: Virtual grasping: closed-loop force control using electro-tactile feedback. *Computational and mathematical methods in medicine* 2014 (2014). 2

[JK18] JENSEN L., KONRADSEN F.: A review of the use of virtual reality head-mounted displays in education and training. *Education and Information Technologies* 23, 4 (2018), 1515–1529. 1

[KFK*08] KUCHENBECKER K., FERGUSON D., KUTZER M., MOSES M., OKAMURA A.: The touch thimble: Providing fingertip contact feedback during point-force haptic interaction. In *Proc. Symposium on Haptic interfaces for virtual environment and teleoperator systems* (2008), pp. 239–246. 2

[KKMT99] KAJIMOTO H., KAWAKAMI N., MAEDA T., TACHI S.: Tactile feeling display using functional electrical stimulation. In *Proc. of ICAT* (1999), p. 133. 2

[KTOH17] KACZMAREK K. A., TYLER M. E., OKPARA U. O., HAASE S. J.: Interaction of Perceived Frequency and Intensity in Fingertip Electrotactile Stimulation: Dissimilarity Ratings and Multidimensional Scaling. *IEEE Transactions on Neural Systems and Rehabilitation Engineering* 25, 11 (nov 2017), 2067–2074. doi:10.1109/TNSRE.2017.2702628. 2

[LKS09] LOZANO C. A., KACZMAREK K. A., SANTELLO M.: Electrotactile stimulation on the tongue: Intensity perception, discrimination, and cross-modality estimation. *Somatosensory & motor research* 26, 2-3 (2009), 50–63. 2

[MKKT07] MINAMIZAWA K., KAJIMOTO H., KAWAKAMI N., TACHI S.: A wearable haptic display to present the gravity sensation-preliminary observations and device design. In *Proc. World Haptics* (2007), pp. 133–138. 2

[MPS*18] MELI L., PACCHIEROTTI C., SALVIETTI G., CHINELLO F., MAISTO M., DE LUCA A., PRATTICHIZZO D.: Combining wearable finger haptics and augmented reality: User evaluation using an external camera and the microsoft hololens. *IEEE Robotics and Automation Letters* 3, 4 (2018), 4297–4304. 1

[PB16] PRACHYABRUED M., BORST C. W.: Design and Evaluation of Virtual Interpenetration Cues in Virtual Grasping. *IEEE Trans. Visualization and Computer Graphics* 22, 6 (jun 2016), 1718–1731. 1, 2

[PSH*16] PACCHIEROTTI C., SALVIETTI G., HUSSAIN I., MELI L., PRATTICHIZZO D.: The hRing: a wearable haptic device to avoid occlusions in hand tracking. In *Proc. IEEE Haptics Symposium* (2016). 2

[PSS*17] PACCHIEROTTI C., SINCLAIR S., SOLAZZI M., FRISOLI A., HAYWARD V., PRATTICHIZZO D.: Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. *IEEE Trans. Haptics* 10, 4 (2017), 580–600. 1, 2, 3

[PW13] PAMUNGKAS D. S., WARD K.: Tele-operation of a robot arm with electro tactile feedback. In *IEEE/ASME International Conf. Advanced Intelligent Mechatronics: Mechatronics for Human Wellbeing* (2013), pp. 704–709. doi:10.1109/AIM.2013.6584175. 8

[PW16] PAMUNGKAS D. S., WARD K.: Electro-tactile feedback system to enhance virtual reality experience. *International Journal of Computer Theory and Engineering* 8, 6 (2016), 465–470. 2

[QBY*15] QIAN K., BAI J., YANG X., PAN J., ZHANG J.: Virtual reality based laparoscopic surgery simulation. In *Proc. 21st ACM Symposium on Virtual Reality Software and Technology* (2015), pp. 69–78. 1, 8

[SCZ*20] SIDENMARK L., CLARKE C., ZHANG X., PHU J., GELLERSEN H.: Outline Pursuits: Gaze-assisted Selection of Occluded Objects in Virtual Reality. In *Proc. of ACM CHI Conf. Human Factors in Computing Systems* (apr 2020). 5

[SH16] SAGARDIA M., HULIN T.: A fast and robust six-dof god object heuristic for haptic rendering of complex models with friction. In *Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology* (New York, NY, USA, 2016), VRST '16, Association for Computing Machinery, p. 163–172. doi:10.1145/2993369.2993374. 1

[SpdT*20] SALAZAR S. V., PACCHIEROTTI C., DE TINGUY X., MACIEL A., MARCHAL M.: Altering the stiffness, friction, and shape perception of tangible objects in virtual reality using wearable haptics. *IEEE Trans. Haptics* 13, 1 (2020), 167–174. 2

[TMGL20] TRIANTAFYLLIDIS E., MCGREAVY C., GU J., LI Z.: Study of multimodal interfaces and the improvements on teleoperation. *IEEE
Access 8 (2020), 78213-78227. doi:10.1109/ACCESS.2020.2990080.1, 8

[VS13] VARSHNEY L. R., SUN J. Z.: Why do we perceive logarithmically? *Significance* 10, 1 (feb 2013), 28–31. 3

[YGKP20] YOUNG E. M., GUEORGUIEV D., KUCHENBECKER K. J., PACCHIEROTTI C.: Compensating for fingertip size to render tactile cues more accurately. *IEEE Trans. Haptics* 13, 1 (2020), 144–151. 2

[YK17] YEM V., KAJIMOTO H.: Wearable tactile device using mechanical and electrical stimulation for fingertip interaction with virtual world. In *IEEE Virtual Reality (VR) (2017)*, pp. 99–104. doi:10.1109/VR.2017.7892236.2

[YVKK18] YEM V., VU K., KON Y., KAJIMOTO H.: Effect of Electrical Stimulation Haptic Feedback on Perceptions of Softness-Hardness and Stickiness while Touching a Virtual Object. In *Proc. of IEEE Conf. Virtual Reality and 3D User Interfaces* (2018), pp. 89–96. 2

[ZCH*19] ZHAO Y., CUTRELL E., HOLZ C., MORRIS M. R., OFEK E., WILSON A. D.: SeeingVR: A set of tools to make virtual reality more accessible to people with low vision. In *Proc. of ACM CHI Conf. Human Factors in Computing Systems* (2019), vol. 14, pp. 1–14. 5

[ZS94] ZILLES C., SALISBURY J. K.: A constraint-based god-object method for haptic display. In *ASME Haptic Interfaces for Virtual Environment and Teleoperator Systems* (1994), vol. 1, pp. 149–150. 1