# MultiVibes: What if your VR Controller had 10 Times more Vibrotactile Actuators?

Grégoire Richard, Thomas Pietrzak, Ferran Argelaguet, Anatole Lécuyer, Géry Casiez

► To cite this version:

Grégoire Richard, Thomas Pietrzak, Ferran Argelaguet, Anatole Lécuyer, Géry Casiez. MultiVibes: What if your VR Controller had 10 Times more Vibrotactile Actuators?. Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2023), Oct 2023, Sidney, Australia. pp.1-10. [https://doi.org/10.1109/ISMAR.2023.1026667](https://doi.org/10.1109/ISMAR.2023.1026667)

HAL Id: hal-04216306

https://inria.hal.science/hal-04216306v1

Submitted on 24 Sep 2023

**HAL** is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# MultiVibes: What if your VR Controller had 10 Times more Vibrotactile Actuators?

Grégoire Richard¹* Thomas Pietrzak¹ Ferran Argelaguet² Anatole Lécuyer² Géry Casiez¹,³

¹Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRISTAL, Lille, France

²Inria Rennes - Bretagne Atlantique, Rennes, France ³Institut Universitaire de France, Paris, France

<figure><img src="image_2.png" /><figcaption>Figure 1: (a) *MultiVibes* prototype comprising 10 actuators in contact with the skin that can be controlled individually in amplitude and frequency. (b) Interaction with the edge of a virtual cube. (c) As the user slides the controller along the edge of the cube, they feels a phantom vibration point moving inside their hand according to the point of contact. The vibration point location is determined by the three closest actuators, controlled using our funneling model.</figcaption></figure>

## ABSTRACT

Consumer-grade virtual reality (VR) controllers are typically equipped with one vibrotactile actuator, allowing to create simple and non-spatialized tactile sensations through the vibration of the entire controller. Leveraging the funneling effect, an illusion in which multiple vibrations are perceived as a single one, we propose *MultiVibes*, a VR controller capable of rendering spatialized sensations at different locations on the user's hand and fingers. The designed prototype includes ten vibrotactile actuators, directly in contact with the skin of the hand, limiting the propagation of vibrations through the controller. We evaluated *MultiVibes* through two controlled experiments. The first one focused on the ability of users to recognize spatio-temporal patterns, while the second one focused on the impact of *MultiVibes* on the users' haptic experience when interacting with virtual objects they can feel. Taken together, the results show that *MultiVibes* is capable of providing accurate spatialized feedback and that users prefer *MultiVibes* over recent VR controllers.

**Index Terms:** Human-centered computing—Human computer interaction (HCI)—Interaction paradigms—Virtual reality; —Interaction devices—Haptic devices

## 1 INTRODUCTION

Haptic integration is a crucial aspect of immersive experiences in VR [4, 27, 36, 47], as haptic feedback can improve performance [9], presence [15, 26] and embodiment [16, 44]. There are many ways to integrate haptic feedback for improved touch and interactions in immersive virtual environments (IVEs), especially with handheld haptic devices. Compared to grounded devices, handheld devices offer a much larger workspace to interact with the environment. They can also be operated with the hand as opposed to non-handheld wearable devices [17, 30].

Several research projects introduce rich haptic feedback, either kinesthetic [14, 21, 56] or tactile [6, 54], for specific interactions that may not easily transfer to consumer-grade VR devices. However, consumer-grade VR controllers usually integrate haptic feedback, in the form of vibrotactile feedback, mostly rendered with one actuator making the frame resonate, allowing for basic effects. Yet, tactile feedback can be enhanced using multiple actuators to provide spatial feedback and leverage haptic illusions, such as saltation or funneling effects. Several works have started to explore such solutions [10, 11, 13, 25, 31]. Some have kept the shape of controllers with cylindrical prototypes [13, 25, 31] but none implemented spatialized models that allow active interaction with VR environments. As Cabaret et al. [11] put it, current VR controllers offer “monolithic vibration of the entire hand-held device”, while some research prototypes can be technologically complex for commercial purposes in the short term. Thus, there is room for improving existing VR controllers by enriching simple technologies, like vibrotactile feedback, using multiple actuators and making use of haptic illusions.

In this work, we present the design of *MultiVibes*, a haptic VR controller, featuring multiple actuators that allow spatialized rendering of vibrotactile feedback on the controller surface. With a simple design and leveraging the funneling effect, *MultiVibes* can render moving vibration patterns and enrich interactions in VR. We first discuss the related work and theory behind spatialized vibrations before presenting the design of the controller. Finally, we introduce two subsequent experiments that evaluate the capacities of *MultiVibes* in terms of perception and user experience. Our results show that spatialized vibrations enrich interactions in IVEs, and that the design of *MultiVibes* is convincing enough to not show significant differences when compared with recent consumer-grade VR controllers such as the *Valve Index* controller. With *MultiVibes*, unlike other consumer-grade VR controllers, it is possible to render information regarding contacts with the environment, either a timely contact like hitting the ground with a pickaxe or prolonged contacts such as petting an animal, or even more symbolic information related to direction and orientation, or moving patterns to signify a progress bar.

<footnote>*e-mail: gregoire.richard@inria.fr</footnote>
## 2 RELATED WORK

An interaction technique can be considered as the coupling of a physical device – the haptic device – and an interaction language – designing and computing haptic stimuli [38]. In this section, we consider haptic interactions through this prism and review the literature regarding haptic solutions, particularly graspable and handheld devices, and the scope of spatialized tactile experiences. We limit the scope of our review to ungrounded devices and focus on handheld haptic devices. We refer to more extensive reviews on haptic devices: haptic gloves [43], and wearable haptic systems [40].

### 2.1 Handheld haptic devices

Most consumer-grade VR gears already include one or two handheld controllers to interact with the IVEs. These controllers usually include some vibrotactile feedback, always limited to one actuator inside the frame to make it resonate [51]. This configuration, although cost-effective, provides a limited range of haptic sensations.

Notable work has been conducted to extend the haptic sensations that such devices can deliver. We discuss a few such studies with approaches relevant to our own, whether in terms of shape, rendering methods, and actuators’ position. For example, some works explored how the shape of the device can be altered [21, 55]. Devices for grasping interaction also make up a significant number of haptic peripherals proposed by research studies, especially for wearable devices [8, 34], but also handheld ones [1, 14].

Researchers have proposed specific handheld devices that incorporate cylindrical peripherals, integrating vibrotactile feedback [11, 31], electrotactile feedback [25] or cutaneous feedback [13]. Kajimoto [25] proposed a cylindrical handheld device that integrates both capacitive sensing and tactile feedback, through electrodes, covering the entire surface of the cylinder. To our knowledge, it is the first controller to combine tactile feedback and capacitive sensing before it was used in consumer grade gear [51]. Their experiment showed that circumferential patterns on the device allowed participants to recognize the direction of the stimuli. Although this prototype included a high density of electrodes, it was limited to electrotactile stimulations, which can be less accepted than vibrotactile feedback [48], and it has not been used in interactive context.

Chen et al. [13] proposed two controllers that embed tactile pins inside the frame, creating pin arrays made up of 15 solenoids in each controller. The pin arrays were arranged in eight columns facing the eight cardinal directions. Using both hands, their prototypes allowed the recognition of impacts through cardinal directions and were shown to enhance immersive properties of virtual environments in two use cases. Chen et al. opted for pressure vectors instead of vibrotactile feedback as they hypothesize it is not possible to create spatialized vibrations on a controller without making the whole frame resonate. Cabaret et al. [11] used vibrotactile feedback to show that two actuators, located at the top and bottom of a handle-shaped prototype, could provide good sense of direction and approximate position, using phantom vibrations (see Section 2.2.2). Although mostly successful in perceiving direction, the participants of this study had trouble correctly identifying the stimuli’s position and judged most vibration models as moderately realistic. The authors hypothesize that a more spatialized vibrotactile feedback, using more than 2 actuators, in 3D spatialization, might be an interesting approach for VR. They proposed another approach [10] with a spherical prototype for VR interaction, that includes five actuators, one for each fingertip, fitted inside the frame of the prototype. They focused on the design of different haptic sensations like impacts and textures on fingertips, and also implemented a 1D funneling effect based on a linear model (see Section 2.2.2) which is different from our approach of rendering haptic feedback on the palm and fingers and using a 2-dimensional funneling effect. Finally, proposing a device relatively similar to Kajimoto in terms of shape, Lacote et al. [31] showed that it was possible to provide spatial feedback with 5 actuators fitted inside a 3D printed handle and that vibrations were easier to follow than taps. They based the vibrations on Stimulus Onset Asynchrony (SOA) (see Section 2.2.1) and did not explore their handle as a mean of interaction.

Although there has been a lot of research proposing novel handheld haptic devices, especially for VR uses, the gap between devices made for research purposes and consumer-grade devices does not seem to narrow. Consumer-grade VR controllers offer limited haptic feedback, while some research prototypes offer novel haptic interactions that can be too complex for home use. Some research prototypes explored shapes close to actual VR controllers [13, 25, 31], but they did not design the model of the feedback to allow for active interaction with a VR environment. The prototype and models by Cabaret et al. [10] differ from the usual controller shape and they do not provide a funneling model to spatialize the feedback over the surface of their prototype. In the second subsection of our review, we cover the different modalities of vibrotactile feedback.

### 2.2 Spatialized vibrotactile feedback

Vibrations are the most common haptic feedback available for handheld devices such as controllers. They can be used to convey information about contact [30], texture [2, 39], or other properties such as directions [11, 13] in a virtual environment. The accuracy of the spatial perception of the vibrations is important to simulate properties such as directions and position. It can be improved using more actuators or leveraging the limitations of human perception. The number of actuators fitted in a controller is often limited by the size of the actuators and their possible resonance with the frame. Leveraging the limitations of human perception with vibrotactile feedback can be achieved using the saltation effect [50], the apparent tactile motions [24] or the funneling effect [3, 5].

#### 2.2.1 Saltation and apparent tactile motions

The saltation effect, also known as the cutaneous rabbit illusion, is a tactile illusion evoked by tapping two or more separate skin regions in rapid succession [18, 50]. In this illusion, the different taps are felt as a train of discontinuous and discrete taps that move from one spot of tapping to the other. The saltation effect was shown to also work in 2 dimensions [29]. Apparent tactile motions are another kind of tactile illusion that relies on overlapping actuation times of close actuators to evoke the feeling of movement [23]. This illusion depends on what is called Stimulus Onset Asynchrony (SOA), which is the time between two consecutive actuations. Both saltation and apparent tactile motions can recreate the illusion of movement for a vibrotactile feedback with a spatiotemporal mapping, such as pre-recorded patterns. However, these effects are not suitable for active real-time VR exploration, as users’ movements are free and unconstrained.

#### 2.2.2 Funneling

When using multiple actuators, it becomes possible to create phantom vibrations that are felt between actual vibrations [3, 5]. These phantom sensations are referred to as the funneling effect. The funneling effect has been used in a wide range of works. The funneling effect has been shown to work for different parts of the body [5, 28], including the hand. For further reading on the funneling effect, we refer to the taxonomy of phantom sensations by Park et al. [41].

**Dimensionality in funneling** The funneling effect corresponds to the phantom vibration perceived between two actuators when they vibrate at the same frequency. Although the funneling effect was historically characterized in one dimension, Lipari [32] generalized it to two dimensions, using 3 or 4 actuators to simulate phantom vibrations at any 2D point within the convex hull defined by all actuators. In the triangular cell model, with actuators located at each vertex of a triangle, the 2D model becomes a combination of two 1D funneling models by creating an intermediate phantom vibration,
shown in green in Figure 2. The square cell model proposed by Lipari relies on the same theory, but requires a regular matrix of actuators [32].

<figure><img src="image_3.png" /><figcaption>Figure 2: Funneling model in 2 dimensions with 3 actuators.</figcaption></figure>

Controlling actuators' amplitude The Energy Summation Model (ESM) in the Pacinian channels [24, 35], defines the amplitude of the phantom vibration as the square root of the sum of the squared intensities of the two actuators:

$$I_{V_1}^2 = I_{A_1}^2 + I_{A_2}^2 \quad (1)$$

where $I_{A_1}$ and $I_{A_2}$ are the intensities of the physical actuators, and $I_{V_1}$ is the amplitude of the resulting phantom vibration (Figure 2). This model allows better control of both the amplitude and perceived location of the phantom vibration than the linear or logarithmic models proposed by previous work [3, 24, 42]. Israr et al. [23] initially proposed the ESM for 1D funneling, coupled with SOA to render a second dimension. Lipari et al. [33] proposed their own funneling model, and compared it to the ESM for one dimensional funneling, but they did not find significant differences between the two models. The ESM defines the following relationship between the intensities of the vibrations:

$$\beta I_{A_1}^2 = (1 - \beta) I_{A_2}^2 \quad (2)$$

where $\beta$ corresponds to the ratio of the distance between $A_1$ and $V_1$, and the distance between $A_1$ and $A_2$. The intensities of $A_1$ and $A_2$ in relation to the one of $V_1$ and $\beta$ is defined as:

$$\begin{cases} I_{A_1} = I_{V_1} \sqrt{1 - \beta} \\ I_{A_2} = I_{V_1} \sqrt{\beta} \end{cases} \quad (3)$$

To our knowledge, no study implemented the energy summation model in two dimensions with 3 actuators.

## 2.3 Summary

There are many ways to enrich vibrotactile feedback to convey more comprehensive information about directions and guidance, or contacts. Using multiple actuators, it is possible to create effects that are helpful for users in immersive settings. Although a few studies propose devices that explore those possibilities, none, to our knowledge, used multiple actuators to render spatial vibrations on the surface of a VR controller. In this work, we propose the use of multiple voice coils to spatialize vibrations in 2D for VR interactions.

# 3 DESIGN AND IMPLEMENTATION

The overall aim of the design of MultiVibes is to enhance current consumer-grade VR controllers by increasing the spatial resolution of vibrotactile feedback, by increasing the number of actuators and leveraging the funneling effect. We provide here the link to the project webpage, including the different models and a demo of the 2D funneling effect.

<figure><img src="image_4.png" /><figcaption>Figure 3: Views of the prototype. On the left, the frame of the controller, made of 3 modules, the header, and the Vive Tracker, fitted with the 10 actuators. In the center right, the actual prototype and the electronic board.</figcaption></figure>

## 3.1 Design Goals

This work aims to design a prototype, including a matrix of actuators, that could be easily replicated and used as a means of interaction to serve as a VR controller. The actuator matrix, where each actuator can controlled in amplitude and frequency, can be used to generate funneling-based vibrations on the surface covered by the matrix. Inspired by Chen's prototype [13], we wanted to reproduce a similar actuator matrix scheme, with two actuators per finger, for a total of 10 actuators. The design of such a controller with spatialized vibrotactile feedback raises several research questions, mainly in terms of user perception and user experience. The first question is related to the level of spatialized information that users can perceive, and the second question is to what extent spatialized feedback coupled with the controller will enhance the user experience within IVEs. Before describing the different experiments set up to answer those questions, we describe the hardware of the prototype.

## 3.2 Hardware and Electronics

We aimed to keep the dimensions of our prototype close to those of existing controllers. This is a strong constraint in terms of volume/surface available. The actuators used were HiWave Haptic Exciter (HIHX9C005-8 Audio & Haptic Exciter), 26 mm long and 12 mm wide. The actuators, ten in total, were fitted directly on the surface of the frame, inside holes designed so that the mobile part of each actuator was not in contact directly with the frame, to prevent it from resonating with the actuators, in a way that the moving part would also be in contact with the users' hand. The resonance of the frame was a key design constraint as having the frame resonating would prevent any funneling effect. The controller frame was 3D printed, having a cylindrical shape, with a circular section with a diameter of 40 mm (see Figure 3). The actuators are connected to an Teensy 4.1 board through a custom-made electronic board equipped with a CMOS AND gate to modulate the frequency and amplitude signals to generate the tactile signal. The tactile feedback consisted of a square shape signal with a controllable frequency up to 1000 Hz [20, 22], modulated with a 31 kHz square signal with a variable duty cycle to control the vibration amplitude. 31 kHz is the fastest clock-type signal with a 16 MHz microcontroller and 256 levels of precision duty cycle. The tactile information was transmitted to the Teensy 4.1 board using raw HID at 1000 Hz to minimize communication delays [12]. The 10 actuators were placed on the frame as described in the following subsection.

## 3.3 Actuators' position

We positioned the actuators in the frame similarly to HapticVec [13], with a matrix of actuators, that can be seen on Figure 3 and as illustrated in Figure 4. The size of the actuators significantly influenced the overall design of the frame and the positioning of the actuators.
<figure><img src="image_5.png" /><figcaption>Figure 4: Actuators’ matrix projected on a hand with approximative positions (positions may vary with hand morphology).</figcaption></figure>

We first designed smaller modular cylinders that could fit five actuators each. These modular cylinders have a 40 mm diameter, and a 30 mm height. Thus, the actuators are approximately 25 mm apart from each other. The modules were glued to align the position of the actuators. The frame of the controller consists of three of those modular cylinders with a 3D-printed cap on top to attach the Vive tracker. Our design could integrate up to fifteen actuators (see Figure 3). However, actuators were positioned considering how users grasped the device, and the size constraint of the actuators. The size of the actuators did not allow each fingertip to be in contact with exactly one actuator. Thus, we decided to position actuators in three columns, one on the fingertips and one on the metacarpal bones, as both are among the parts of the hands most sensitive to vibrations [46, 52, 53]. The third column was placed at the bottom of the palm and the tenth actuator was used for the thumb.

## 3.4 Vibration Model

We based our vibration model on the funneling effect, and more precisely, the Lipari’s triangular cell model [32], as the matrix we proposed is not completely regular (see Fig. 3), and an extension of the energy summation model [23, 42] to 2D. If we go back to Figure 2, the amplitude of $A_1$ and $A_2$ is based on the position of the phantom vibration $V_1$, which amplitude is also based on the position of the resulting phantom vibration $V_2$. The amplitude of $V_1$ and $A_3$ are based on a 1D funneling problem, varying with the value of $\alpha$, in the same way as Equation 3 from Sec. 2.2.2. Finally, by expressing all amplitudes in terms of the resulting phantom vibration $V_2$, we obtain:

$$ \left\{ \begin{array}{l} I_{A_1} = I_{V_2} \sqrt{\alpha (1 - \beta)} \\ I_{A_2} = I_{V_2} \sqrt{\alpha \beta} \\ I_{A_3} = I_{V_2} \sqrt{1 - \alpha} \end{array} \right. \qquad (4) $$

This model creates a bijective relationship between the surface covered by the actuators and the amplitude of each actuator for each point belonging to the surface.

The frequency of the vibration for all actuators was kept identical, at the resonance frequency of the HiWave Haptic Exciter, which is 250 Hz.

## 3.5 Assessing the prototype

We designed two experiments to validate the design of our controller and spatialized feedback and evaluate its ability to enhance VR interaction and user experience. The first research question deals with vibrotactile rendering and the perception of feedback by participants, while the second research question deals more directly with the contribution of spatialized feedback to haptic experience.

The first experiment evaluates the ability of the controller to spatialize vibrotactile information at the level of a user’s hand, when the user perceives vibration patterns which move over time passively, without active movement. The second experiment looks at the contribution of spatialized feedback to the participants’ haptic experience

in an interactive context with active movement. Finally, we briefly introduce different scenarios that showcase the implementation of the spatialized feedback with different VR interactions.

# 4 SPATIO-TEMPORAL DISCRIMINATION EXPERIMENT

This section presents a perceptual validation experiment that assesses the user’s ability to recognize spatio-temporal vibratory patterns. The work of Kajimoto et al. [25] partially inspired the experiment. In their paper, they proposed an experimental protocol in which participants had to recognize spatio-temporal patterns rendered with an electrotactile feedback device. They had to recognize the direction and general movement of the rendered patterns. Using our prototype with the funneling effect, we propose a revisited version of this experimental protocol in a VR context.

This experiment was approved by the local ethics committee.

## 4.1 Objectives

We want to assess users’ ability to perceive and keep track of a vibrating point moving in simple patterns, such as circles, lines, or more complex patterns, provided only through haptic means (i.e. without associated visual feedback). This experiment is related to our first research question concerning haptic rendering and vibration perception.

## 4.2 Participants

Twenty-one participants participated in the experiment ($M = 32.2, SD = 7.6$) and were recruited by word of mouth. We recruited right-handed ($n = 16$) and left-handed ($n = 5$) participants, assuming that laterality would not affect performance. The only inclusion criterion for participants was based on their ability to hold the frame while in contact with all actuators. Therefore, they were selected on the basis of the size of their hand. This led to all participants being male.

## 4.3 Apparatus

Participants were immersed in the IVE wearing a Valve Index headset, which provided visual feedback and a 6-DOF head tracking. Participants held a Valve Index controller in their left hand and the prototype controller in their right hand. The virtual environment was implemented using Unity3D (2019.2.12f1) and the Unity Steam VR plugin. The experiment ran on an Intel (R) Core (TM) i7-9750H 2.60 GHz CPU with 32 GB of RAM and an 8 GB 2070 RTX GeForce Nvidia GPU. There was no virtual representation of the participants inside the IVE besides the left and right controllers. Furthermore, since vibrations are noisy, participants wore a Peltor 33 dB noise reduction headset to avoid bias when answering.

## 4.4 Experimental protocol

**Generic instructions** As the controller targets VR contexts, we opted to implement the experiment in VR, to collect data from the controller and feedback in the intended context of use, and also to avoid participants using physical-world information to locate the position of the vibrations. Each participant was first informed about the experiment, its general objective, and the tasks they would have to perform. They were told they would be feeling vibration patterns that they needed to identify, with no further details about the patterns. Then they were asked to fill out a consent form and we collected general information (age, gender, and hand dominance). The participants then sat on a chair and were equipped with the experimental apparatus.

**Grasping instructions and calibration** Specific instructions were given to the participants on how to hold the prototype so that each of the actuators was in contact with the skin. The experimenter guided the participant to correctly position their hand on the prototype. The experiment started with a calibration phase, where all actuators were triggered individually for 2 s each. This calibration phase had two objectives: first, to ensure that the prototype and each
actuator were working properly; second, to check whether their grip on the prototype allowed them to feel the vibrations of each actuator and adjust it if necessary. Participants were instructed to report if they stopped feeling vibrations at any point during this phase.

**Main task** The main experimental phase was divided into several trials. Each trial proceeded in the same way: participants started a trial by pressing the trigger on the left controller, and half a second later, a vibration pattern was played on the prototype, while an interface displayed 12 choices of patterns (see Figure 5). The participants then selected with their left hand the pattern they thought was the one that was played. They had up to 30 s for each trial to choose a pattern while the pattern was played in a loop. If they did not answer within the 30 s, the trial were considered as *missed*. The trial ended when the participant chose the pattern by pressing the left trigger. We proposed 12 different vibration patterns for this experiment, which were repeated in 5 blocks, giving a total of 60 trials. These patterns are described in Figure 5.

<figure><img src="image_6.png" /><figcaption>Figure 5: Matrix of the vibration patterns. On top, the circle patterns (C1 to C4), in the middle, the line patterns (L1 to L4), and at the bottom, the spiral patterns (S1 to S4).</figcaption></figure>

### 4.4.1 Vibration patterns

As the aim of the experiment was to evaluate the ability of the controller to produce dynamic vibratory effects that users could understand, we wanted to propose a range of patterns from simple ones (circles around the circumference of the controller, or vertical lines) to more complex ones (spirals, thus combining the simple patterns to varying degrees). In total, 4 patterns of circles, 4 lines, and 4 spirals were proposed.

The vibratory patterns were played at a constant speed in the experiment. The speed and intensity used in the experiment were calibrated with a pilot experiment (n = 5). Playing the pattern at 8 cm/s and setting the vibration intensity of 80 % of the maximum amplitude allowed for a continuous and good feeling of the patterns. Frequency was kept constant at 250 Hz. The participants involved in the pilot experiment did not participate in the main experiment.

## 4.5 Measures

For each trial, we recorded the response times and the selected pattern. In addition, at the end of the experiment, participants had to answer two questions: one on the continuity of the vibrotactile patterns presented during the experiment and another one on the comfort of the vibrations ("How continuous/comfortable did the vibrations feel during the experiment?"). To not overburden participants, these questions were kept general (covering all patterns), but

participants were allowed to elaborate on their responses using a text. Both questions were assessed using a 7-point Likert scale, with choices ranging from 1 - Not at all continuous/comfortable to 7 - Perfectly continuous/comfortable.

## 4.6 Hypotheses

The first hypothesis for this experiment concerns the ability of participants to keep track of a dynamic vibratory pattern. We hypothesize that the participants are able to recognize different patterns (H1). This first hypothesis can be broken down into two sub-hypotheses, the first concerning the participants' ability to recognize the general direction of the vibratory pattern (H1.1), and the second concerning their ability to position the pattern precisely in relation to their hand, and in relation to the controller (H1.2).

The second hypothesis is related to the quality of the vibratory effects and the perception of the continuity of the vibrations by the participants. Theoretically, the vibratory model described above (3.4) is bijective: every point in the actuator's matrix corresponds to a unique combination of actuators' at a certain amplitude. There should be no perceivable discontinuity in the intensity of the phantom vibration, either at the vertices or the edges of the triangular cells. Therefore, we hypothesized that the participants would perceive the proposed vibration patterns as continuous (H2).

## 4.7 Results

Regarding the recognition rates, Table 1 reports the recognition rates through the global confusion matrix. In this matrix, we can see the success percentages for each trial, and the confusion between trials. It is also possible to calculate the average percentage of success of the participants during the experiment. The results show an average success percentage of 58 % with a standard deviation of 13 %.

In addition, we computed the means and standard deviations for the time taken to answer each trial. On average, spirals took 16.6 s, lines took 14.0 s and circles took 12.1 s. The results are presented in Table 2. Participants reported their perception of the vibrations through two 7-Likert scale questions, one addressing the comfort (Mdn = 5, SD = 1.3) and the other the continuity (Mdn = 5, SD = 1.1) of the vibrations.

<table><caption>Table 1: Confusion matrix. Rows correspond to the tested patterns and columns to the recognized ones. Values are reported as percent of total number for each trials. Color transparency was linearly scaled from 0 to 100 %. Miss column represents the percentage of trials where participants did not answer before time ran out.</caption><thead><tr><th rowspan="2"></th><th colspan="4">Circles</th><th colspan="4">Lines</th><th colspan="4">Spirals</th><th rowspan="2">Miss</th></tr><tr><th>C1</th><th>C2</th><th>C3</th><th>C4</th><th>L1</th><th>L2</th><th>L3</th><th>L4</th><th>S1</th><th>S2</th><th>S3</th><th>S4</th></tr></thead><tbody><tr><th rowspan="4">Circles</th><th>C1</th><td>69</td><td>27</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>3</td><td>0</td></tr><tr><th>C2</th><td>6</td><td>20</td><td>29</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>2</td><td>2</td><td>4</td><td>29</td><td>7</td></tr><tr><th>C3</th><td>0</td><td>9</td><td>73</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td><td>0</td><td>11</td><td>3</td><td>1</td></tr><tr><th>C4</th><td>0</td><td>0</td><td>0</td><td>87</td><td>1</td><td>0</td><td>1</td><td>1</td><td>0</td><td>1</td><td>9</td><td>0</td><td>0</td></tr><tr><th rowspan="4">Lines</th><th>L1</th><td>4</td><td>3</td><td>0</td><td>2</td><td>23</td><td>1</td><td>8</td><td>3</td><td>27</td><td>3</td><td>7</td><td>4</td><td>9</td></tr><tr><th>L2</th><td>0</td><td>0</td><td>1</td><td>0</td><td>9</td><td>81</td><td>0</td><td>2</td><td>1</td><td>3</td><td>2</td><td>0</td><td>1</td></tr><tr><th>L3</th><td>0</td><td>3</td><td>2</td><td>3</td><td>14</td><td>1</td><td>55</td><td>4</td><td>4</td><td>1</td><td>9</td><td>0</td><td>5</td></tr><tr><th>L4</th><td>0</td><td>1</td><td>3</td><td>2</td><td>2</td><td>2</td><td>14</td><td>53</td><td>4</td><td>5</td><td>7</td><td>3</td><td>4</td></tr><tr><th rowspan="4">Spirals</th><th>S1</th><td>1</td><td>0</td><td>3</td><td>1</td><td>3</td><td>0</td><td>1</td><td>6</td><td>40</td><td>20</td><td>6</td><td>18</td><td>3</td></tr><tr><th>S2</th><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>76</td><td>5</td><td>6</td><td>0</td></tr><tr><th>S3</th><td>0</td><td>0</td><td>3</td><td>13</td><td>0</td><td>5</td><td>0</td><td>1</td><td>5</td><td>4</td><td>69</td><td>0</td><td>2</td></tr><tr><th>S4</th><td>2</td><td>4</td><td>3</td><td>0</td><td>1</td><td>0</td><td>1</td><td>1</td><td>13</td><td>9</td><td>6</td><td>54</td><td>7</td></tr></tbody></table>

## 4.8 Discussion

There were two main hypotheses for this experiment: H1 related to the recognition of patterns, in terms of patterns' groups (H1.1) and position (H1.2), and H2 related to the quality and continuity of the perceived vibration.
<table><caption>Table 2: Means and standard deviations for the time taken to answer each pattern.</caption><thead><tr><th rowspan="2"></th><th colspan="5">Circles</th><th colspan="3">Lines</th><th colspan="4">Spirals</th></tr><tr><th>C1</th><th>C2</th><th>C3</th><th>C4</th><th>L1</th><th>L2</th><th>L3</th><th>L4</th><th>S1</th><th>S2</th><th>S3</th><th>S4</th></tr></thead><tbody><tr><td>Mean time (s)</td><td>9.1</td><td>18.4</td><td>12.2</td><td>8.8</td><td>18.6</td><td>10.6</td><td>14.1</td><td>12.9</td><td>17.0</td><td>14.7</td><td>17.0</td><td>16.1</td></tr><tr><td>SD (s)</td><td>6.2</td><td>8.1</td><td>6.9</td><td>5.7</td><td>8.8</td><td>6.7</td><td>7.6</td><td>8.2</td><td>7.6</td><td>7.4</td><td>7.7</td><td>8.3</td></tr></tbody></table>

We first consider the results regarding comfort and continuity. On average, participants rated both questions 5 - Rather continuous/comfortable. These assessments are positive but limited to a general evaluation of all patterns and sensations. However, only one participant verbally reported not perceiving the entirety of a pattern. In general, these results suggest that the continuity of the patterns was overall good and lead us to partially validate **H2**.

### 4.8.1 Patterns' recognition

On average, the participants correctly identified the pattern played among the 12 patterns 58% of the time. It is important to note that the participants perceived the pattern essentially as haptic (tactile, through the vibratory stimulation of the controller, and proprioceptive, through the position of their hand and fingers) without any visual indication, although sight remains the main sense in humans in the elaboration of a sensory judgment [7]. Despite the lack of visual information, the participants were successful 58% of the time compared to 8% for random choices in identifying the correct pattern. These results show that the participants perceived and partially identified patterns solely based on tactile information.

**Recognition between groups** Here we discuss the recognition of patterns as pattern groups, i.e. if participants recognized lines as lines, spirals as spirals, or if there was confusion between groups. In addition to patterns C2 and L1, participants correctly identified the pattern group at least 70% of the time, with a mean success rate of 78%. This tends to validate **H1.1**. We discuss the results of specific patterns with respect to their position in the following paragraph. If we look closer at the average time to answer trials, we see differences between the groups. We suspected that participants might be using different strategies to identify different patterns. Participant 4003's comment was particularly revealing: "If it turned at a constant level, it was a circle; if it involved all the fingers or the whole palm without turning, it was a line; otherwise it was a spiral". We directed informal questions in post-experimental exchanges to determine the strategy employed by participants, and it turns out that 13 of them had similar reasoning.

**Patterns' position** Patterns vary greatly in terms of success, with C2 and L1 having the lowest recognition rate. They also have the highest average response time among patterns, which shows that participants had trouble identifying them. Pattern L1 started from the thumb, with the actuator located on A10 (see Figure 4), and propagated downwards between A8 and A2 then A7 and A1. The phantom vibration was located in a zone not covered by the user's hand under the thumb and between the bottom of the palm and the fingertips. As such, the low recognition rate for L1 can be explained by its position, in a zone that is difficult for participants to interpret. C2 on the other hand, was mistaken with patterns that were closer and more consistent than L1. C2 activated all actuators from the 2 top rows, which were common with other patterns such as C1, C3, and S4. More generally, we observe that patterns located close to the fingertips and the palm side show a high success rate, but with the exception of C2 and L1, the patterns show a success rate of at least 50%. In addition to the specific zones of the frame, it seems that the participants successfully located the different patterns and vibrations, validating **H1.2**. Taken together, the results lead us to validate **H1** partially.

Since active movement and exploratory procedures are crucial for interaction and touch [19], we present in the next section a second experiment that focuses on interaction with active movements.

## 5 EXPERIMENT ON HAPTIC EXPERIENCE

In the first experiment, participants remained passive while feeling the vibrations. With this experiment, we explored the effect of spatialized vibrotactile feedback on user experience during active exploration. We compared the haptic experience of the participants according to 3 distinct conditions: control condition with a *Valve Index controller* and its integrated haptic feedback, our *prototype controller with non-spatialized haptic feedback*, and our *prototype controller with spatialization*.

The experiment addresses the research question related to the contribution of spatialized haptic feedback to user experience. To what extent does the spatialization of vibrotactile feedback improve the user experience, particularly the haptic experience, with respect to realism, immersion, and diversity? Our goal is to be able to compare our haptic solution (the prototype and the spatialized vibration model) with a consumer controller in an interactive VR context. To this end, we propose an experiment in an interactive virtual reality context in which we compare the haptic experience of the participants after interacting with the environment through different conditions, including a Valve Index controller alongside with our controller with and without the spatialized feedback. The local ethics committee approved this experiment.

### 5.1 Participants

18 participants took part in this experiment (M = 29.8, SD = 7.9), among whom 4 were left-handed. Participants were recruited based on their size of hand, as in the previous experiment, and again all participants were male. The participants were recruited by word of mouth.

### 5.2 Experimental conditions

We wanted to compare our prototype with recent consumer-grade VR controllers. Therefore, we proposed a baseline condition with a *Valve Index* controller, with its built-in vibrotactile feedback, *CONTROL*. We compared this baseline condition with our prototype, under two different conditions. In one condition the prototype is used with spatialized vibrations (*VIBSPATIAL*), and the second one the prototype is used with non-spatialized vibrotactile feedback (*VIBUNIFORM*). *CONTROL* was based on the existing haptic feedback proposed by the controller. It was implemented through the *SteamVR* plugin [49], with vibrations at 50 Hz, and a 100% amplitude, which are the default values for vibrations signifying contacts. For *VIBUNIFORM*, the feedback was not based on a spatialized model. Instead, all actuators vibrated simultaneously at the same frequency and amplitude. The amplitude was set to 50% of the maximum output for each of the ten actuators, and the frequency was set at 250 Hz. The vibrotactile feedback for *VIBSPATIAL* is implemented with the vibration model described in section 3.4 with the same amplitude and frequency as the first experiment (80%). In all conditions, the vibrations were only activated when the controller was in contact with the cube.

### 5.3 Experimental protocol

The experiment consisted of a simple task of active movement, where participants had to stay in contact with the edges of a virtual cube, and felt the movement of their hand/of the controller with vibrations co-localized with the point of contact of the controller with the virtual cube(see Figure 1). The task, which required active movements from users, was designed in such a way as to elicit congruent visuo-haptic stimulation and leverage the sensorimotor loop, as it is a crucial aspect when studying perception [19].

Participants were first introduced to the experiment and then asked to read and sign a consent form and answer basic demographic questions (age and gender), before starting the experiment. The
experiment followed a within-subjects protocol with Latin-square counterbalancing. Within the IVE, participants had an avatar limited to the controllers (see Video in the Supplementary Material). The right hand and controller were represented by a cylinder of the same size as the frame of the prototype.

Each condition went the same way as described below. After being introduced to the modalities of the experiment, the experimenter explained to the participants how to properly hold the controller, with the same instructions as in experiment 1, and helped them get the apparatus on. The apparatus was the same as in experiment 1. After gearing up, the participants were immersed in the IVE. A virtual hand-animated demonstration in the IVE showed participants how to complete the different movements. There were four different kinds of movement: a downward motion, an upward one, a circular motion, and a lateral motion. Each motion was repeated ten times, resulting in forty movements. After completing the task, the participants were asked to remove the gear (HMD and controllers) and answer the questionnaire. The protocol was repeated for each of the three conditions. For CONTROL and VIBUNIFORM, the vibrations triggered by contact with the cube were always the same. For VIBSPATIAL, the vibration point moved with the contact point between the controller and the cube in the IVE, creating a congruence between what the participants would see and feel.

## 5.4 Measures

The focus of this experiment was the added value of spatialized vibrotactile feedback on the user experience. As such, we decided to use the Haptic Experience Questionnaire, proposed by Sathiyamurthi *et al.* [45], to assess the subjective touch experience for each condition (see 3). We implemented all 20 items and asked participants to rate each item on a 7-point Likert scale. We also asked participants to rate each condition on a 1-10 scale (1 being "terrible controller overall and 10 "great controller overall"), which we call the global likability of the condition, and explain what motivated their score on a free text. This was done to determine what factors made them like or dislike each condition.

<table><caption>Table 3: Haptic experience questionnaire as proposed by Sathiyamurthy *et al.* [45]. Order of items is as proposed by the original authors.</caption><thead><tr><th>ID</th><th>Questions</th></tr></thead><tbody><tr><td><strong>Autotelism</strong></td><td></td></tr><tr><td>A-1</td><td>I like having the haptic feedback as part of the experience.</td></tr><tr><td>A-2</td><td>I like how the haptic feedback itself feels, regardless of its role in the system.</td></tr><tr><td>A-3</td><td>I disliked the haptic feedback.</td></tr><tr><td>A-4</td><td>I would prefer the system without the haptic feedback.</td></tr><tr><td>A-1</td><td>The haptic feedback felt satisfying.</td></tr><tr><td><strong>Expressivity</strong></td><td></td></tr><tr><td>E-2</td><td>I felt adequate variations in the haptic feedback.</td></tr><tr><td>E-4</td><td>The haptic feedback changes depending on how things change in the system.</td></tr><tr><td>E-5</td><td>The haptic feedback reflects varying inputs and events.</td></tr><tr><td>E-1</td><td>The haptic feedback all felt the same.</td></tr><tr><td><strong>Immersion</strong></td><td></td></tr><tr><td>I-4</td><td>The haptic feedback increased my involvement in the task.</td></tr><tr><td>I-3</td><td>The haptic feedback helped me focus on the task.</td></tr><tr><td>E-3</td><td>The haptic feedback helped me distinguish what was going on.</td></tr><tr><td>I-2</td><td>I felt engaged with the system due to the haptic feedback.</td></tr><tr><td><strong>Realism</strong></td><td></td></tr><tr><td>R-1</td><td>The haptic feedback was realistic.</td></tr><tr><td>R-2</td><td>The haptic feedback was believable.</td></tr><tr><td>R-3</td><td>The haptic feedback was convincing.</td></tr><tr><td><strong>Harmony</strong></td><td></td></tr><tr><td>H-3</td><td>The haptic feedback felt disconnected from the rest of the experience.</td></tr><tr><td>H-5</td><td>The haptic feedback felt out of place.</td></tr><tr><td>I-1</td><td>The haptic feedback distracted me from the task.</td></tr><tr><td>H-4</td><td>The haptic feedback felt appropriate when and where I felt it.</td></tr></tbody></table>

## 5.5 Hypotheses

We aim to assess the benefits of spatialized vibrotactile feedback for the user's haptic experience. This spatialization should allow to increase the richness of the haptic feedback during interactions with the environment. We therefore hypothesize that the VIBSPATIAL condition is significantly superior to the CONTROL and VIBUNIFORM conditions in terms of overall haptic experience (H3), and

particularly in terms of expressiveness (H3.1), immersion (H3.2) and realism (H3.3).

VIBUNIFORM proposes limited haptic feedback, without variations. We hypothesize that this feedback would be considered unrealistic, but also judged as not harmonious by the participants. Thus, we make the hypothesis that VIBUNIFORM is significantly less harmonious and realistic than VIBSPATIAL and CONTROL (H4).

## 5.6 Results

Because the data collected through the questionnaires were ordinal and did not follow normal distributions, each item was analyzed with nonparametric tests using Friedman analysis and Wilcoxon post hoc pairwise tests with Bonferroni correction.

### 5.6.1 Global likability

A Friedman analysis on global likability scores revealed significant differences between conditions ($\chi^2(3) = 17.1, p < 0.0001$). Post-hoc Wilcoxon analysis revealed significant differences between VIBSPATIAL ($Mdn = 8$) and both CONTROL ($Mdn = 6.5, p < 0.003$) and VIBUNIFORM ($Mdn = 6.5, p < 0.0001$) (see Figure 6).

<figure><img src="image_7.png" /><figcaption>Figure 6: Aggregated scores for global likeability for each condition, ranging from 1 to 10.</figcaption></figure>

### 5.6.2 Haptic experience

The questionnaire recently proposed by Sathiyamurthy *et al.* [45] has not yet been validated by other experiments. To provide additional data, we reproduced the dimension reduction approach to verify that we obtained a distribution of the different items close to that proposed by the authors of the questionnaire. These results are available in the supplementary material section of the article. As we were unable to reproduce all the initial groups found in Sathiyamurthy *et al.* study, we decided to test the items individually.

We performed Friedman analyzes on all items of the haptic experience questionnaire. For items with significant tests, we performed Wilcoxon post hoc analyzes. These analyzes revealed significant differences for items A3, A4, A1, E2, E4, E5, E1, I4, E3, R1, R2, R3, H3, H5 and H4 ($p < 0.05$). The set of responses to the haptic experience questionnaire is summarized in Figure 7.

## 5.7 Discussion

In this experiment, we aimed to evaluate the contribution of our prototype to haptic experience. Taken together, our results show that spatialized vibrotactile feedback is superior to the baseline conditions (CONTROL, VIBUNIFORM). To be precise, the user's haptic experience and the global appreciation of the participants were significantly higher.

With H3, we assumed that the condition VIBSPATIAL would be superior to the other two conditions in terms of haptic experience. Of the 20 questions that made up the questionnaire, 8 of them showed that VIBSPATIAL obtained significantly higher ratings than the other two conditions. Thus, we can validate H3. We also hypothesized that VIBSPATIAL would be superior in terms of expressiveness (H3.1), immersion (H3.2) and realism (H3.3). Regarding expressiveness, composed of items E2, E4, E5 and E1, we can confirm the hypothesis: while the VIBSPATIAL ratings for E5 were only significantly higher compared to CONTROL, for the others, the VIBSPATIAL ratings were significantly higher than for the other conditions. For realism, made up of items R1, R2, R3, the results were similar: R1
<figure><img src="image_8.png" /><figcaption>Figure 7: Boxplot of haptic experience scores, for each items. Red markers above the boxes represent significant differences.</figcaption></figure>

and R2 items showed the significant superiority of VIBSPATIAL, while R3, which assessed the convincing aspect of the feedback, differed slightly. To our surprise, CONTROL was significantly lower than the other two conditions. We interpret this result as an expression of a higher expectation of the haptic feedback proposed by the controller *Valve Index*, which seemed, following the feedback of several participants, “more ergonomic”. With these results, we can validate **H3.3**. The results are more nuanced with respect to immersion (Items I4, I3, E3, I2). Items I4 and E3 showed a significant superiority of VIBSPATIAL, which is encouraging. However, statistical analyzes did not reveal any differences for items I3 and I2. I3 dealt with helping the participants concentrate on completing the task. We consider that, given the experimental protocol and the task at hand, this was not surprising: the task did not have any particular goal as participants only had to focus on the generated haptic sensations. Thus, it seems reasonable to us that the different haptic modalities did not contribute differently to concentration during the task. We can also note that the scores on this question were high for all conditions. For item I2, which deals with engagement, we believe the task and interactions were not varied enough to show differences. It would be interesting to propose a richer interactive context to test this new hypothesis. However, we can, with two items among four, partially validate **H3.2**.

Finally, we hypothesized that the realism and harmony of CONTROL would be superior to VIBUNIFORM (H4). Our results do not allow us to confirm these assumptions. These results are positive: the design of the controller seems to be good enough for participants to rate it as a suitable controller, even when compared to the Valve Index controller. The overall rating scores are consistent with this, since VIBUNIFORM and CONTROL have the same median. The scores for VIBUNIFORM still have a larger variance.

This second experiment evaluated the use of *MultiVibes* in the context of active movements and interaction with virtual objects. The results show that the users preferred *MultiVibes* and spatialized feedback over consumer-grade controllers. In the following section, we discuss the limitations of the prototype and our findings.

# 6 DISCUSSION

We presented two experiments assessing *MultiVibes*. Results of the first experiment show that participants can recognize the vibration patterns ~60% of the time among 12 different patterns. Compared to Kajimoto *et al.* [25], who only presented one pattern to participants, and Lacote *et al.* [31], who presented 4 patterns (2 lines and 2 circles) with a recognition rate at ~75%, our results with *MultiVibes*, with 3 times the number of patterns, and more complex patterns, are encouraging. Results from the second experiment show that *MultiVibes* is better in terms of haptic experience compared to current commercial VR controllers. We used a 2-dimensional funneling effect as vibration model, which allows for active interaction with environments as we did in the second experiment, which is not possible with vibration models like the one used by Lacote *et al.*, based on SOA. The haptic experience data from the second experiment can

also be used as an assessment of the validity of the questionnaire introduced by Sathiyamurthy *et al.* [45]. We provide the PCA analysis of the questionnaire items in the supplementary material.

**Limitations** The experiments described in the previous sections highlighted certain prototype limitations and the perception of vibrations. The first limitations are related to the design of the controller itself. Due to the radius of the frame, people with smaller hands could not hold the controller properly while touching all actuators. As such, the pools of participants for both experiments were biased towards one gender. Although there is no consensus on gender differences in vibration perception [37], we believe that making the prototype usable by any hand size is important. It could be possible to either reduce the radius of the frame or implement some capacitive sensing to determine hand and finger position. We develop this idea below. Another limitation is that for the funneling effect to work properly, vibrations from actuators must be localized. To do that, the actuators were in direct contact with the skin. Yet, if gripped too tightly, the actuators' moving part cannot vibrate freely, thus changing the perception of the intensity of the phantom vibration. Moreover, if users change their grip during immersion, the hand might lose contact with one or multiple actuators, changing the perception of the resulting vibration. For the two experiments previously described, the experimenter made sure participants' hand was correctly positioned, but it could be an issue for non-controlled setups.

**Future work** The results of the two experiments show that the design of *MultiVibes* can serve as a basis for controllers with enriched interactions in VR. Still, there are several considerations for future work to showcase the prototype and enhance its design.

We designed several scenarios that showcase multiple interactions based on the same 2D funneling model. These scenarios include tool-mediated interaction, with vibrations symbolizing hits between the tool and the environment, interactions with animals with prolonged contacts, and other abstract renderings of directions, positions, or even progress bars. Those scenarios showcased in the accompanying video (see Supplementary material) have not yet been tested in a controlled setup. We also remarked informally that the perception of textures, through hardness/softness, during contact, is different depending on the steepness of the signal at the start of the vibration.

We have also considered improving the prototype by adding capacitive sensing in the frame, such as the Valve Index controller or Kajimoto *et al.* [25] prototype. Capacitive sensing would allow for better control of the actuators: there is no need to activate any actuator not in direct contact with the fingers or palm. It would also allow for a regular actuator matrix that could fit all sizes of the hands. Capacitive sensing has already been implemented in consumer-grade controllers such as the Valve Index. Including multiple actuators, as we proposed with *MultiVibes* would further enhance haptic feedback inexpensively. Considering that adding actuators is a low-tech solution, unlike many other research haptic devices, we firmly believe it could be added to a new commercial
product in the short to midterms.

# 7 CONCLUSION

In this paper, we presented a prototype of a VR controller that can spatialize vibrotactile feedback, *MultiVibes*. *MultiVibes* is based on the funneling effect, using multiple actuators to enrich haptic feedback for interactions in IVEs. The design of the prototype frame, based on previous designs, can be easily replicated by 3D printing. To validate *MultiVibes*, we designed two experiments to assess it: one based on the perception of vibrations and the ability of users to recognize dynamic vibration patterns, and the other based on the positive effect of spatialized feedback on the haptic experience. Taken together, our results show that the design of the controller is convincing as a VR controller, even when compared with consumer-grade VR controller like the *Valve Index*. Moreover, the spatialized vibrations show great promise in terms of transmission of information and increased haptic and user experience.

# ACKNOWLEDGMENTS

This work was funded by the Inria Avatar project. The authors wish to thank Danny Kieken and Suliac Lavenant for their help with the demonstrators during the project.

# REFERENCES

[1] M. Achibet, M. Marchal, F. Argelaguet, and A. Lécuyer. The virtual mitten: A novel interaction paradigm for visuo-haptic manipulation of objects using grip force. In *2014 IEEE Symposium on 3D User Interfaces (3DUI)*, pp. 59–66, 2014. doi: 10.1109/3DUI.2014.6798843

[2] T. Ahmaniemi, J. Marila, and V. Lantz. Design of dynamic vibrotactile textures. *IEEE Transactions on Haptics*, 3(4):245–256, 2010.

[3] D. S. Alles. Information transmission by phantom sensations. *IEEE transactions on man-machine systems*, 11(1):85–91, 1970.

[4] J. E. Aspell, B. Lenggenhager, and O. Blanke. Keeping in touch with one's self: multisensory mechanisms of self-consciousness. *PloS one*, 4(8):e6488, 2009.

[5] A. Barghout, J. Cha, A. El Saddik, J. Kammerl, and E. Steinbach. Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion. In *2009 ieee international workshop on haptic audio visual environments and games*, pp. 19–23. IEEE, 2009.

[6] H. Benko, C. Holz, M. Sinclair, and E. Ofek. Normaltouch and texturetouch: High-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In *Proceedings of the 29th Annual Symposium on User Interface Software and Technology*, UIST '16, p. 717–728. Association for Computing Machinery, New York, NY, USA, 2016. doi: 10.1145/2984511.2984526

[7] L. Boban, D. Pittet, B. Herbelin, and R. Boulic. Changing finger movement perception: Influence of active haptics on visual dominance. *Frontiers in Virtual Reality*, p. 38, 2022.

[8] M. Bouzit, G. Burdea, G. Popescu, and R. Boian. The rutgers master ii-new design force-feedback glove. *IEEE/ASME Transactions on Mechatronics*, 7(2):256–263, 2002. doi: 10.1109/TMECH.2002.1011262

[9] P. W. Brasen, M. Christoffersen, and M. Kraus. Effects of vibrotactile feedback in commercial virtual reality systems. In *Interactivity, Game Creation, Design, Learning, and Innovation*, pp. 219–224. Springer, 2018.

[10] P.-A. Cabaret, T. Howard, G. Gicquel, C. Pacchierotti, M. Babel, and M. Marchal. Does multi-actuator vibrotactile feedback within tangible objects enrich vr manipulation? *IEEE Transactions on Visualization and Computer Graphics*, 2023.

[11] P.-A. Cabaret, T. Howard, C. Pacchierotti, M. Babel, and M. Marchal. Perception of spatialized vibrotactile impacts in a hand-held tangible for virtual reality. In *International Conference on Human Haptic Sensing and Touch Enabled Computer Applications*, pp. 264–273. Springer, 2022.

[12] G. Casiez, T. Pietrzak, D. Marchal, S. Poulmane, M. Falce, and N. Roussel. Characterizing latency in touch and button-equipped interactive systems. In *Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology*, UIST '17, pp. 29–39. ACM, New York, NY, USA, 2017. doi: 10.1145/3126594.3126606

[13] D. K. Chen, J.-B. Chossat, and P. B. Shull. Hapticvec: Presenting haptic feedback vectors in handheld controllers using embedded tactile pin arrays. In *Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems*, pp. 1–11, 2019.

[14] I. Choi, E. Ofek, H. Benko, M. Sinclair, and C. Holz. Claw: A multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems*, CHI '18, p. 1–13. Association for Computing Machinery, New York, NY, USA, 2018. doi: 10.1145/3173574.3174228

[15] H. Q. Dinh, N. Walker, L. F. Hodges, C. Song, and A. Kobayashi. Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In *Proceedings IEEE Virtual Reality (Cat. No. 99CB36316)*, pp. 222–228. IEEE, 1999.

[16] J. Fröhner, G. Salvietti, P. Beckerle, and D. Prattichizzo. Can wearable haptic devices foster the embodiment of virtual limbs? *IEEE Transactions on Haptics*, 12(3):339–349, 2018.

[17] G. García-Valle, M. Ferre, J. Breñosa, and D. Vargas. Evaluation of presence in virtual environments: Haptic vest and user's haptic skills. *IEEE Access*, 6:7224–7233, 2017.

[18] F. A. Geldard. *Sensory saltation: Metastability in the perceptual world*. Lawrence Erlbaum, 1975.

[19] J. J. Gibson. Observations on active touch. *Psychological review*, 69(6):477, 1962.

[20] G. D. Goff. Differential discrimination of frequency of cutaneous mechanical vibration. *Journal of experimental psychology*, 74(2p1):294, 1967.

[21] E. J. Gonzalez, E. Ofek, M. Gonzalez-Franco, and M. Sinclair. X-rings: A hand-mounted 360 shape display for grasping in virtual reality. In *The 34th Annual ACM Symposium on User Interface Software and Technology*, pp. 732–742, 2021.

[22] A. Gupta, T. Pietrzak, N. Roussel, and R. Balakrishnan. Direct manipulation in tactile displays. In *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*, pp. 3683–3693. San Jose, USA, 2016.

[23] A. Israr and I. Poupyrev. Control space of apparent haptic motion. In *2011 IEEE World Haptics Conference*, pp. 457–462. IEEE, 2011.

[24] A. Israr and I. Poupyrev. Tactile brush: drawing on skin with a tactile grid display. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, pp. 2019–2028, 2011.

[25] H. Kajimoto. Design of cylindrical whole-hand haptic interface using electrocutaneous display. In *International Conference on Human Haptic Sensing and Touch Enabled Computer Applications*, pp. 67–72. Springer, 2012.

[26] O. B. Kaul, K. Meier, and M. Rohs. Increasing presence in virtual reality with a vibrotactile grid around the head. In *IFIP Conference on Human-Computer Interaction*, pp. 289–298. Springer, 2017.

[27] K. Kilteni, R. Groten, and M. Slater. The sense of embodiment in virtual reality. *Presence: Teleoperators and Virtual Environments*, 21(4):373–387, 2012.

[28] J. Kim, S. Oh, C. Park, and S. Choi. Body-penetrating tactile phantom sensations. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems*, pp. 1–13, 2020.

[29] Y. Kim, J. Lee, and G. Kim. Extending "out of the body" saltation to 2d mobile tactile interaction. In *Proceedings of the 10th asia pacific conference on Computer human interaction*, pp. 67–74, 2012.

[30] C. Krogmeier, C. Mousas, and D. Whittinghill. Human–virtual character interaction: Toward understanding the influence of haptic feedback. *Computer Animation and Virtual Worlds*, p. e1883, 2019.

[31] I. Lacôte, C. Pacchierotti, M. Babel, D. Gueorguiev, and M. Marchal. Investigating the haptic perception of directional information within a handle. *IEEE Transactions on Haptics*, 2023.

[32] N. G. Lipari. *Improved 2D Interpolation for Intensity and Position Control in 2D Vibrotactile Meshes*. PhD thesis, University of Louisiana at Lafayette, 2020.

[33] N. G. Lipari and C. W. Borst. Study of 2d vibration summing for improved intensity control in vibrotactile array rendering. In *Advances in Visual Computing: 10th International Symposium, ISVC 2014*, Las
Vegas, NV, USA, December 8-10, 2014, Proceedings, Part II 10, pp. 325-334. Springer, 2014.

[34] Z. MA and P. Ben-Tzvi. Rml glove—an exoskeleton glove mechanism with haptics feedback. *IEEE/ASME Transactions on Mechatronics*, 20(2):641–652, 2015. doi: 10.1109/TMECH.2014.2305842

[35] J. C. Makous, R. M. Friedman, and C. J. Vierck. A critical band filter in touch. *Journal of Neuroscience*, 15(4):2808–2818, 1995.

[36] M. Marucci, G. Di Flumeri, G. Borghini, N. Sciaraffa, M. Scandola, E. F. Pavone, F. Babiloni, V. Betti, and P. Aricò. The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence. *Scientific Reports*, 11(1):1–15, 2021.

[37] G. Neely and L. Burström. Gender differences in subjective responses to hand-arm vibration. *International journal of industrial ergonomics*, 36(2):135–140, 2006.

[38] L. Nigay and J. Coutaz. A generic platform for addressing the multi-modal challenge. In *Proceedings of the SIGCHI conference on Human factors in computing systems*, pp. 98–105, 1995.

[39] A. M. Okamura, J. T. Dennerlein, and R. D. Howe. Vibration feedback models for virtual environments. In *Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146)*, vol. 1, pp. 674–679. IEEE, 1998.

[40] C. Pacchierotti, S. Sinclair, M. Solazzi, A. Frisoli, V. Hayward, and D. Prattichizzo. Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. *IEEE transactions on haptics*, 10(4):580–600, 2017.

[41] G. Park and S. Choi. Tactile information transmission by 2d stationary phantom sensations. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems*, pp. 1–12, 2018.

[42] J. Park, J. Kim, Y. Oh, and H. Z. Tan. Rendering moving tactile stroke on the palm using a sparse 2d array. In *International Conference on Human Haptic Sensing and Touch Enabled Computer Applications*, pp. 47–56. Springer, 2016.

[43] J. Perret and E. Vander Poorten. Touching virtual reality: A review of haptic gloves. In *ACTUATOR 2018; 16th International Conference on New Actuators*, pp. 1–5. VDE, Bremen, Germany, 2018.

[44] G. Richard, T. Pietrzak, F. Argelaguet, A. Lécuyer, and G. Casiez. Studying the role of haptic feedback on virtual embodiment in a drawing task. *Frontiers in Virtual Reality*, 1:573167, 2021.

[45] S. Sathiyamurthy, M. Lui, E. Kim, and O. Schneider. Measuring haptic experience: Elaborating the hx model with scale development. In *2021 IEEE World Haptics Conference (WHC)*, pp. 979–984. IEEE, 2021.

[46] L. Schacher, S. Bensaid, S. E.-G. Jeguirim, and D. Adolphe. Sensory and physiological issues. In *Advances in Modern Woven Fabrics Technology*. IntechOpen, 2011.

[47] M. Slater, V. Linakis, M. Usoh, and R. Kooper. Immersion, presence and performance in virtual environments: An experiment with tri-dimensional chess. In *Proceedings of the ACM symposium on virtual reality software and technology*, pp. 163–172. ACM, 1996.

[48] D. Stanke, T. Duente, and M. Rohs. Tactilewear: A comparison of electrotactile and vibrotactile feedback on the wrist and ring finger. In *Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society*, pp. 1–13, 2020.

[49] SteamVR. Interface isteamvr_action_vibration. Online: accessed 12-June-2023.

[50] H. Z. Tan and A. Pentland. Tactual displays for wearable computing. In *Digest of Papers. First International Symposium on Wearable Computers*, pp. 84–89. IEEE, 1997.

[51] ValveSoftware. Valve index controllers, 2019. last checked November 5, 2021.

[52] R. T. Verrillo. Vibrotactile sensitivity and the frequency response of the pacinian corpuscle. *Psychonomic Science*, 4(1):135–136, 1966.

[53] R. T. Verrillo. Vibration sensation in humans. *Music Perception*, 9(3):281–302, 1992.

[54] E. Whitmire, H. Benko, C. Holz, E. Ofek, and M. Sinclair. Haptic revolver: Touch, shear, texture, and shape rendering on a reconfigurable virtual reality controller. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems*, CHI '18, p. 1–12. Association for Computing Machinery, New York, NY, USA, 2018. doi: 10.1145/3173574.3173660

[55] S. Yoshida, Y. Sun, and H. Kuzuoka. Pocopo: Handheld pin-based shape display for haptic rendering in virtual reality. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems*, CHI '20, p. 1–13. Association for Computing Machinery, New York, NY, USA, 2020. doi: 10.1145/3313831.3376358

[56] A. Zenner and A. Krüger. Drag:on: A virtual reality controller providing haptic feedback based on drag and weight shift. In *Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems*, CHI '19, p. 1–12. Association for Computing Machinery, New York, NY, USA, 2019. doi: 10.1145/3290605.3300441