# Effect of Attention Saturating and Cognitive Load on Tactile Texture Recognition for Mobile Surface

Adnane Guettaf, Yosra Rekik, Laurent Grisoni

► To cite this version:

Adnane Guettaf, Yosra Rekik, Laurent Grisoni. Effect of Attention Saturating and Cognitive Load on Tactile Texture Recognition for Mobile Surface. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.557-579, 10.1007/978-3-030-85610-6_31. hal-03379590

HAL Id: hal-03379590

https://uphf.hal.science/hal-03379590

Submitted on 25 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Effect of Attention Saturating and
Cognitive Load on Tactile Texture
Recognition for Mobile Surface

Adnane Guettaf<fnref n="1" />, Yosra Rekik<fnref n="1" />, and Laurent Grisoni<fnref n="2" />

**Abstract.** We investigate users ability to recognize tactile textures on mobile surface when performing a primary task that either saturates the attention or is cognitively demanding. Our findings indicate that the attention saturating task decreases performance by 6.98% and increases frustration, mental demand and physical effort compared to a control condition. The recognition task can be done in an eyes-free style while continuing to perform the primary task. While cognitively demanding task demands more time to switch to the texture recognition task but decreases the time needed to recognize the texture without compromising accuracy compared to a control condition. The two tasks are handled sequentially with gaze attention directed to the current performed task. For both primary tasks, the recognition rate stays higher than 82% and the total time does not decrease, suggesting that tactile texture could be effectively recognized and used by users when performing a primary task. Finally, we discuss the implications of our work for tactile feedback based interaction.It is our hope that our findings will contribute toward a better understanding of tactile feedback perception on touchscreen when performing another primary task.

**Keywords:** Tactile texture · Haptic · Primary task · Secondary task · Attention saturating task · Cognitively demanding task.

# 1 Introduction

Mobile touchscreens are becoming increasingly important in everyday life, providing different information and services (e.g., communication, social media, gps, travel, education, banking, entertainment, etc...). The growth of mobile applications used in all aspects of life and the easiness of interaction with them through tactile input allow users to interact with touchscreen in many different ways and in different contexts, including situations where interaction complements another, a *primary* task, that either needs full attention (e.g., driving [10]) or is cognitively demanding(e.g., receiving a notification when typing a text [9]). Indeed, touchscreen applications most often use visual cues in order to provide feedback, e.g., confirm/verify/response a notification or to guide user finger

<footnote n="1">¹ Univ. Polytechnique Hauts-de-France, LAMIH, CNRS, UMR 8201, F-59313 Valenciennes, France {adnane.guettaf,yosra.rekik}@uphf.fr</footnote>

<footnote n="2">² university of lille, Villeneuve d'Ascq, France laurent.grisoni@univ-lille.fr</footnote>
movements to the correct target item [10], which may in practice induce some
perceptual distractions when a user is performing another primary task.

In the same time, touchscreens have been enhanced with tactile feedback that
provides users with stimulation when touching the surface [3, 1, 7, 24, 33, 22, 30].
In particular, a rich variety of tactile textures have been proposed that have
been shown to be identifiable by users [24, 27]. Tactile textures only require user
finger exploration on the screen, and can possibly free users from the constraint
to get visual feedback; which presents an interesting way to reduce the demand
for visual attention to the touchscreen, in the case of secondary interaction task.

There is little existing research concerning the use of tactile textures for
secondary tasks. Previous research demonstrated, for standard interaction sit-
uations in which no secondary task is involved, the ability for users to accu-
rately recognize tactile textures on touchscreens even when using different finger
speeds [27]; or when perceiving simultaneous but different textures [26]; or when
the size of the texture is small [19]. However, in all these studies participants
have to focus on only the tactile feedback based task. Therefore, it is unclear how
the user perception of tactile texture could be impacted by a real-world scenario
where participants have to interact with a primary task that can for example,
saturates the attention or is cognitively demanding. Among other fundamen-
tal questions, one can for instance ask the following: does users perception of
textures remain effective when performing another primary task? Does relative
position of the tactile surface matter in such setting? How do users handle the
two tasks? Do users continue interacting with the primary task when recognizing
textures? How do they distribute eye-gaze attention when recognizing textures?

To investigate these specificities, we conducted two experiments to examine
users' perception of tactile textures rendered on a mobile touchscreen when per-
forming another primary task on a laptop. In the first experiment, we study
the effect of an attention saturating task on textures recognition: the primary
task in this experiment is a highly visually demanding task which feature the
control of a pseudo-randomly perturbed moving ball, in a manner similar to
the approach used in [11]. In the second experiment, we study the effect of a
cognitively demanding task on textures recognition: as primary task we consider
a text typing task [9]. We also explore the effect of the touchscreen position
(on the left, right and forward) relatively to the primary display on the user
interaction. For both experiments we refer to a control situation, in which no
primary task is involved. Our findings indicate that the attention saturating task
decreases performance by 6.98% and increases frustration, mental demand and
physical effort, by comparison to the control situation. Different hand postures
and strategies to handle and perform the two tasks have been used. In partic-
ular, our findings confirm that the recognition task can be done eyes-free while
continuing performing the primary task. By contrast, the second experiment
shows that cognitively demanding task impacts texture recognition by increas-
ing time to switch to the texture recognition task but also decreases time needed
to recognize the texture without compromising accuracy. The primary and sec-
ondary tasks are handled sequentially with gaze attention directed to the current
performed task (i.e., either the primary display or the touchscreen). For both primary tasks, the recognition rate stays higher than 82% and the total time does not decrease, suggesting that tactile texture could be effectively recognized and used by users when performing a primary task. We also do not find an effect of the touchscreen position on the performances, but indeed on user behaviour.

This paper contributions constitute, to the best of our knowledge, the first empirical investigation of the effectiveness of tactile textures on touchscreen when performing another primary task that either saturates the attention or is cognitively demanding.

# 2 Related Work

In this section, we review previous work on tactile feedback in terms of devices, rendering techniques, textures perception studies and the effect of an attention saturating or cognitively demanding task on tactile feedback interaction.

## 2.1 Tactile feedback based devices

Touch interaction is the primary input modality of many modern mobile devices [24], when enhanced with tactile feedback, it allows to deliver information about touched elements. Vibrotactile stimuli [8, 6, 5] is the most used tactile feedback. It consists in vibrating a part or the entire touch screen display to induce a physical sensation of vibration. This kind of tactile feedback are more informative than descriptive. Consequently, many researchers investigated the use of physical augmentations on-top of the screen [20, 28, 10], and of course designed new devices with richer capabilities for tactile feedback [1, 7, 31, 24, 30, 27]. In particular, two main technologies have emerged to support mobile device-based tactile feedback: (1) electrovibration [3, 25, 32] which enhance the friction between the finger and the interaction surface and (2) ultrasonic technologies which reduce the friction through the “squeeze film effect” [1, 7, 24, 30]). In the remainder of this paper, we leverage the latter tactile feedback device [30].

## 2.2 Tactile textures rendering techniques

Three main rendering techniques have been proposed in the literature: SHO, SHT and LHT. Surface Haptic Object, or SHO is based on mapping a given texture with a discrete sampling of position and have been used by most existing surface devices (e.g., [3, 24, 1, 21, 25]). SHT (Surface Haptic Texture) have been introduced, recently, by Vezzoli et al [30,27] and relies on real time finger speed. Rekik et al. [27] compared the SHO and SHT techniques. Their findings indicate that SHT leads to the highest level of quality of tactile rendering for dense textures with either fast or moderate velocity; whereas SHO is still more accurate for sparser textures with moderate velocity due to positional shift. Considering these results, Rekik et al. [27] introduced the LHT (Localized Haptic Texture), a new rendering technique [27]. LHT separates the tactile rendering
into two different processes: first, the finger position is retrieved from the hardware, and the corresponding texture is selected through a search in a grid of taxels. The taxel texture is then rendered locally by defining only one period of the texture and then repeated in a loop at a rate that depends on the finger's speed. LHT was shown to provide a high-fidelity between the texture and its visual representation. In our work we used the LHT technique.

## 2.3 Tactile textures perception

Previous work have examined users perception of tactile textures on touchscreens devices. In [27], the authors investigated the user ability to perceive textures when using different finger speeds. In [19], the authors determined the smallest texture size that user can accurately perceive. Researchers have investigated the users ability to perceive simultaneous but different textures [26], provided the semantic perceptual space of textures [12] and studied the effect of different physically challenging contexts on textures perception [14]. Researchers have also looked at the benefits of tactile feedback to enhance physicality [23], improve pointing techniques [7], help visually impaired people to interact with objects [17] and enhance musical interaction [18]. However, these findings are likely to differ when users are making another primary task that either saturates the attention or is cognitively demanding.

## 2.4 Effect of attention saturating and cognitively demanding task on tactile feedback interaction

Harrison et al. [15] investigated the relevance of dynamic buttons displays based on pneumatic actuation when the user is performing simultaneously an attention saturating primary task. They employed the same attention saturating dual task framework than in [11, 4] in which users performed an attention-saturating task while simultaneously performing additional tasks on the pneumatic display. As in [11, 4] the goal of the attention saturating task was to keep a moving circle centered on a fixed crosshair. The attention needed to perform actions in the secondary task was measured as a drop in performance in the primary one. Results showed that pneumatic displays performs as well as physical buttons with fewer glances towards the surface when performing the primary task. Cockburn et al. [10] investigated users' performances when interacting with a touchscreen covered with a static stencil overlay while driving in a 2D emulator. Their results showed that with tactile feedback, users selected a target quickly and that stencil significantly reduced the visual attention demands on normal touchscreens with shorter eye-glances directed away from the primary task. Rydström et al. [29] used a driving simulator as a primary task while asking participants to interact with a secondary one through a haptic ridges rotary device. Their goal was to investigate whether haptic ridges can facilitate the interaction with the rotary device while driving. Their findings showed that driving performance did not significantly vary between haptic-only and haptic/visual conditions, and that adding haptic ridges to the visual information did not necessarily reduce the
<figure><img src="image_3.png" /><figcaption>Fig. 1: The visual representations of the four textures used in the experiment.</figcaption></figure>

gaze-away time from the road. Chen et al. [9] studied the effect of a cognitively demanding primary task, through a typing text, on recognizing spatiotemporal vibrotactile patterns which constitute the secondary task. They found a strong effect of the primary task on recognition rate. However, although the findings there-in are of valuable contribution, users’ perception of tactile textures on touchscreen devices and users’ mental model and behaviour were not captured. Additionally, we can not blindly apply those results to ultrasonic devices which provide different sensations of tactile feedback [30, 26].

# 3 Experiments

We conducted two experiments to investigate the effect of a primary task on tactile textures recognition on a touchscreen. We consider two primary tasks, the first saturates the attention while the second is cognitively demanding. The two primary tasks are described in the corresponding experiment section. The two experiments share many similarities that we describe in this section.

## 3.1 Method

We used *E-vita* [26] as the main device holding our secondary task. *E-vita* is a tactile feedback tablet that support friction reduction using ultrasonic vibrations. It uses the squeeze film effect technique and this by creating an over pressure between the user’s finger and the vibrating surface at an ultrasonic frequency [13], this air bearing alter the device’s coefficient of friction so that a user can feel different textures thanks to this variation in friction. The *E-vita* [26] tablet supports a sampling frequency of 50 Hz thanks to the capacitive sensor included in it’s 5-inch LCD display, which guaranties capabilities similar to commercial mobile devices.

We consider four different textures following previous studies [27, 26, 19, 30]. We encode the different textures with respect to different texture densities by considering the following spatial periodicity: *densest* – 1.2 mm; *dense* – 5.1 mm; *medium* – 25.5 mm and *sparse* – 51 mm. In Figure 1, the tactile textures are shown by alternating black and white bars; high friction is associated with the black color and low friction with the white color. Given that we use the Evita, which, when vibrating, reduces friction, this maps black to off and white to on to create tactile patterns. To render a given texture we used the LHT technique [27].

The primary task was implemented in JavaScript framework using the Node.js runtime and ran on a Dell laptop machine with a 13-inch LCD display screen with a desktop resolution of 1920×1080. Participants’ faces were also videotaped. In addition, one author observed each session and took detailed notes, particularly concerning think-aloud data, hand postures, and mental model.
## 3.2 Design

The experiment used a 2 × 3 × 4 × 3 within-subject design for the factors: *activity*, *position*, *texture* and *block*. Activity presents if the participant is doing a primary task or just waiting for the notification that we named here the *control condition*. Consequently, in the first experiment, the activity is either *centering the ball* (i.e., the attention saturating task) on the laptop or the *control condition* and in the second experiment, activity is either the *text-typing* (i.e., cognitively demanding task) or the *control condition*. Position is the position of the surface compared to the laptop and covers three values: right, left and froward. The tactile surface was oriented horizontally (i.e., parallel to the ground). For the right position, the surface was placed 5cm away diagonally from the bottom-right of the laptop with an orientation of 45°. For the left position, the surface was placed 5cm away diagonally from the bottom left of the laptop with an orientation of -45°. And for the forward position, the surface was centered and placed 5cm below the laptop keyboard which in turn was centered in front of participants. *Texture* covers the four presented textures in the previous section. *Block* covers 3 value (1-3), with the first block serving as a training phase.

## 3.3 Procedure

After asking our participants to seat comfortably on a desk in face to the laptop, participants had to answer a demographic questionnaire, after which, the experiment task was explained along with the additional requirements for both the primary and the secondary tasks. Participants then began the experiment.

The experimental trials were administered as blocks of 12 trials (4 textures × 3 repetitions), each block sharing a primary activity and a tablet position. For each activity, and for each tablet position, participants had to perform three blocks of textures identification. Blocks sharing a tablet position were administered consecutively to minimize physical device displacement; then grouped by primary activity to allow questionnaire assessment. The two primary *activities* were counterbalanced. Inside each *activity*, the three tablet *positions* were also counterbalanced. Inside each *block*, the four *textures* × three repetitions were randomly presented to the participants – a total of 216 (=2 activities × 3 positions × 3 blocks × 4 textures × 3 repetitions) trials per participant. After each block of trials, participants were encouraged to take a break. After each activity, participants completed a NASA-TLX worksheets.

## 3.4 Task

Both experiments required from participants to interact with a primary task on the laptop and to recognize textures on the tablet (secondary task) each time they receive a notification. Participants were asked to prioritize the primary task over the secondary one, and were told that their performance was being measured for both tasks.

In the experiment 1, participants had to perform an attention saturating task and a control condition, and in experiment 2, participants performed a cognitively demanding task and the same control condition. In the control condition,
<figure><img src="image_4.png" /><figcaption>Fig. 2: Experiment setup according to the three positions of the haptic table.</figcaption></figure>

participants had only to react to the notifications displayed on the laptop screen to perform the texture recognition task. The rational of adding the control condition, is to better understand the effect of the attention saturating/cognitively demanding primary task on user perception of tactile textures.

For the secondary task, participants were asked to move their index finger on the surface from right to left and inversely to perceive the texture, without a predefined hand or a starting finger position or time restrictions or limited number of clutches or swipes. Participants had the total liberty in choosing how to proceed to explore the whole texture. No visual feedback of the rendered texture was shown on the surface, only tactile feedback was sent to the participants while sensing the texture. In addition, as the Evita device makes noise when alternating high and low frictions, the participants were equipped with noise reduction headphones to avoid any bias. Once the participant recognized the perceived texture, he pressed on the “confirm” button that is positioned on the top of the tactile surface. This location was chosen to be sure to not disturb the texture perception task. A new screen is then displayed, and participants had to select the visual representation of the perceived texture from the four visual representations of the four evaluated textures on the tablet surface and then confirm their choice by pressing again the “confirm” button.

At the experiment phase, participants started interacting with the primary task. And after a random period of time between 2 and 14 seconds, a notification was shown up in the computer screen and a texture was rendered in the tablet surface. This indicate to the participants that they can start recognizing the rendered texture on the tablet surface. Participants were free to choose the appropriate strategy to handle the primary task and the secondary one while keeping in mind that the primary task should be prioritized. Participants had the total liberty to interact with both tasks at the same time or sequentially by switching from one to another etc. After ending a trial by selecting the visual representation of the perceived texture, our software presented the next notification after a random period of time between 2 and 14 seconds. Each experiment took around 90 minutes to complete. To reduce fatigue, sufficient resting periods were given between conditions and as required by the participants.

# 4 Experiment 1: Effect of an attention saturating task

In this experiment, we investigated the effect of an attention saturating primary task on the recognition of textures. We followed [15, 4, 11] and used the same primary task that saturates the attention. Our primary attention saturating task
<figure><img src="image_5.png" /><figcaption>(a) Accuracy</figcaption></figure>

<caption>(b) Reaction time</caption>

<caption>(c) Movement time (d) Number of swipes</caption>

<caption>Fig. 3: User performances in the first experiment.</caption>

Fig. 3: User performances in the first experiment.

featured a circle moving randomly according to a two-dimensional Perlin noise function. Participants were asked to keep the circle centered over a cross-hairs displayed in the center of the square as the best they could, and this by contracting its movement using the arrow keys of the laptop keyboard. Participants were asked to use one hand to perform the primary task while having the total liberty in choosing which hand to use. Participants were free to use the same hand to interact with the secondary task or to use their second hand. Participants were told that keeping the circle centered on the crosshair was the most important task and that their performance was being measured for both the primary and the secondary task. The procedure and task as well as common apparatus are presented in the Experiments section.

Thirteen participants (5 females) volunteered (not paid) to take part into the experiment. Participants ages were between 24 and 36 years (mean=30.23 years, sd=4.45years). All participants were right handed.

**4.1 Results**

Results for each of the dependent variables (*reaction time*, *movement time*, *total time*, *accuracy*, *number of swipes* and *number of clutches*) are presented below. All analyses are using multi-way ANOVA considering the following independent variable: activity, tablet position, texture and block.. Tukey tests are used post-hoc when significant effects are found. We also analyzed subjective responses.

## 4.2 Time Performance

We measured *reaction time*, *movement time* and *total time*. *Reaction time* was the interval time between the appearance of a notification on the laptop screen and the first touch on the surface device. It represents the time taken by the participant to react and switch to the secondary task. *Movement time* represents the time taken by the participant to sense and recognize the texture, from the first touch, until pressing the “confirm” button for the first time. *Total time* represents the time taken by a participant all along a trial, from the moment the notification is shown up, until pressing the “confirm” button. It is the sum of reaction and movement times. For time measures, we only considered timing data from correct trials to better account for user performance.

**Total time.** Repeated-measures ANOVA revealed a significant main effect of *block* (*F*<sub>2,24</sub> = 13.70, *p* < .0001) on *total time*. Post-hoc tests showed a significant decrease in the time between the first block and the two remaining (*p*<.05; block1: mean=6924 ms, s.d=445 ms, block2: mean=6174 ms, s.d=402 ms and
block3: 5744 ms, s.d=383 ms) due to a learning during the first block. As we are concerned with user performance after familiarization, the remaining analysis discards the first block.

There was a significant main effect of *texture* ($F_{3,36} = 15.37, p < .0001$) on *total time* with a significant *activity × position × texture* ($F_{6,71.83} = 2.23, p = .0498$) interaction. Post-hoc tests revealed that when centering the ball, and having the tablet on the right (respectively, forward), the densest texture is significantly better recognized than both the dense and the medium (respectively, medium) textures ($p < .05$).

**Reaction time.** There were no significant effects of activity ($p=.58$), position ($p=.36$) and texture ($p=.82$) on reaction time nor interaction ($p=.13$), with similar means of 1644 ms (sd= 115 ms) with the control condition, and 1803 ms (sd= 203 ms) when centering the ball (Figure 3b).

**Movement time.** There was a significant effect of textures ($F_{3,36.23} = 18.26, p < .0001$) on movement time. Post-hoc tests revealed that the densest texture (mean= 2619 ms, sd=376 ms) is recognized faster than the remainder textures (dense: mean=4513 ms, sd=445 ms, medium: 5117 ms, sd=472 ms, sparse: mean=4684 ms, sd=411 ms).

## 4.3 Accuracy

Accuracy is defined as the proportion of correct identifications of textures.

There were significant main effects of *activity* ($F_{1,12} = 3.86, p = .008$) and *texture* ($F_{3,36} = 3.17, p = .0357$) on *accuracy*. Post-hoc tests revealed that the control condition (mean=88.78%, sd=2.25%) is significantly more accurate than centering the ball (mean= 82.58%, sd= 2.50%) (Figure 3a). We also, found that the densest texture (mean= 94.44%, sd= 1.42%) is significantly better recognized than the remainder textures. There was no significant interaction ($p > .1951$), suggesting that the drawback of centering the ball activity are consistent across textures and surface positions and the benefits of the densest texture are consistent across different activities and tablet positions.

## 4.4 Number of swipes and clutches

**Number of directional swipes** is defined as the number of times the user moves his finger on the surface from left to right or right to left during the movement time. **Number of clutches** is defined as the number of times the user released his finger from the surface and than put it again on the surface from the first touch. We excluded error trials from analyses.

**Number of Swipes.** Analysis of number of swipes shows no significant effect of activity ($F_{1,11.98} = .99, p = .33$), with similar means of 6.35 (sd=.53) for the control situation and 5.26 (sd=.52) for centering the ball (Figure 3d). There was, however, a significant effect of *texture* ($F_{3,36.12} = 10.67, p < .0001$) on *number of swipes*. Post-hoc tests revealed that the densest texture (mean= 3.42, sd=.37) produced significantly less swipe gestures than the remainder textures (dense: mean=6.16, sd=.70, medium: mean=7.16, sd=.94 and sparse: mean= 6.52, sd=.76).

**Number of Clutches.** Similar to number of swipes, when analyzing number of clutches, we found no significant effect of activity ($F_{1,11.98} = 3.27, p = .09$)
<table><thead><tr><th rowspan="3"></th><th colspan="4">Experiment 1</th><th colspan="4">Experiment 2</th></tr><tr><th colspan="2">Control</th><th colspan="2">Centering</th><th colspan="2">Wilcoxon</th><th colspan="2">Typing</th><th colspan="2">Wilcoxon</th></tr><tr><th>mean</th><th>s.d</th><th>mean</th><th>s.d</th><th>Z</th><th></th><th>mean</th><th>s.d</th><th>mean</th><th>s.d</th><th>Z</th></tr></thead><tbody><tr><td>Mental demand</td><td>2.38</td><td>.41</td><td>3.84</td><td>.43</td><td>-3.20</td><td></td><td>2.75</td><td>.68</td><td>3.41</td><td>.65</td><td>-1.90</td></tr><tr><td>Physical demand</td><td>1.84</td><td>.66</td><td>3.30</td><td>.56</td><td>-2.75</td><td></td><td>1.75</td><td>.42</td><td>2.75</td><td>.64</td><td>-2.89</td></tr><tr><td>Temporal demand</td><td>2.76</td><td>.63</td><td>3.76</td><td>.32</td><td>-1.96</td><td></td><td>2.25</td><td>.42</td><td>3.08</td><td>.61</td><td>-2.77</td></tr><tr><td>Performance</td><td>4.07</td><td>.34</td><td>3.92</td><td>.34</td><td>.70</td><td></td><td>3.91</td><td>.44</td><td>3.58</td><td>.56</td><td>1.63</td></tr><tr><td>Frustration</td><td>1.84</td><td>.53</td><td>3.23</td><td>.55</td><td>-2.91</td><td></td><td>1.83</td><td>.58</td><td>2</td><td>.53</td><td>-1.15</td></tr><tr><td>Effort</td><td>2.15</td><td>.53</td><td>4</td><td>.44</td><td>-3.23</td><td></td><td>2.33</td><td>.60</td><td>2.91</td><td>.61</td><td>-1.86</td></tr></tbody></table>

Note: Wilcoxon-Signed-Rank tests are reported at *p*=.005 (*) significance levels. Table 1: Mean and s.d questionnaire responses, with 1=very low, and 5 = very high for experiment 1 and experiment 2. The significant tests are <u>highlighted</u>

with similar means of .55 (sd=.09) for the control condition and .94 (sd=.14) for centering the ball (F<sub>1,11.98</sub> = 3.27, *p* = .09). There was a significant effect of texture (F<sub>3,35.86</sub> = 6.83, *p* = .0009) on number of clutches. Post-hoc tests revealed that the densest texture (mean=.35, sd=.12) produced significantly less clutches than the remainder textures (dense: : mean=.78, sd=.17, medium: mean=.95, sd=.20 and sparse: mean=.92, sd=.17).

## 4.5 Subjective results and observations

Our quantitative data were accompanied by considerable qualitative data that capture users' mental models as they handle and perform the primary task and the secondary one.

### 4.5.1 Nasa TLX results
We recall that our participants were asked to rate the overall task after each activity condition. Overall, questionnaire responses (Table 1) showed that recognizing textures when centering the ball was significantly more demanding mentally and physically while having significantly higher perceived effort and being more frustrated than in the control condition.

We correlate these findings with comments from participants that felt that managing the ball while performing a texture recognition is more difficult than when they had just to identify the texture after receiving a notification. Some quotes: "for me, doing both tasks simultaneously was difficult", "it is stressful to center the ball and recognize the texture at the same time". In addition, one participant felt that "the overall task demands a lot of effort and is highly frustrating ... I try to quickly identify the texture to not lose time or getting the ball not centred but I felt confident in recognizing the textures".

### 4.5.2 Hands input posture
We instructed participants to prioritize the primary task (centering the ball) over the secondary one (recognizing the textures) while giving them the total liberty on the number of hands to use and which hand to use to handle and perform the primary and the secondary task. Interestingly, for a given surface position, we observed that once the participant
starts the task with a given hands posture, he continues with that posture until
finishing all the trials in that position. In the following, we present the different
hands postures used to perform the primary and the secondary task once the
notification has shown-up:

- **One-handed (dominant hand)** – **1H**. Two participants used their dominant hand (here right hand) to perform sequentially the primary and the secondary task during all the experiment and independently on the position of the touch surface (see Figure 4a).

- **Two-handed directional posture – 2HD.** This hands posture is strongly correlated to the position of the tactile surface and consists of using the hand closed to the tactile surface to perform the secondary task and the other hand to perform the primary one (see Figure 4b). For the forward position, as to interact with the primary task, participants have to press on the arrow keys which are localized at the extreme right of the keyboard and so the laptop, we then considered the primary task as being more on the right than the secondary one. For 2HD posture, when performing the secondary task, the hand used for the primary task remains on the keyboard arrow keys. While when the secondary task finished (i.e., participants have to only perform the primary task), we were able to observe three postures performed by the hand used for the secondary task: (1) **fingers-above**: the hand fingers are kept above the surface by placing the wrist in a stable position just below the surface (see Figure 4c), (2) **fingers-closed**: the participants' wrist was placed just below the surface, but the hand was a little bit moved back with a closed fingers (see Figure 4d), and (3) **hand-moves**: the hand used for the secondary task was putted on the office and maintained in a perimeter around the surface (see Figure 4e). In addition, we observed two participants using the *finger-above* posture often touching the screen of the surface before a notification shows up to anticipate the appearance of this latter. Overall, this hand posture is used by nine participants for the right, eleven for the left and nine for the forward surface position.

- **Two Arms Crossed – 2AC.** This hand posture is strongly correlated to the task priority and consists of using always the dominant (here right) hand for the primary task and the non-dominant (here left) hand for the secondary task despite the surface is on the right position. Consequently, the participants arms were crossed (see Figure 4f). Participants kept their dominant hand on the keyboard array keys when performing the secondary task, and when this latter is finished, they used either the *fingers-above* (see Figure 4g) or the *hand-moves* (see Figure 4h) for the hand used to interact with the secondary task while keeping the hands crossed. Overall, two participants used this posture for the right surface position.

- **Two Arms Semi-Crossed – 2ASC.** This posture occurred essentially in the *forward* position, when users used their non-dominant (left) hand to interact with the primary task, and their dominant (right) hand for the secondary task crossing a little their arms to perform both tasks (see Figure 4i). Here, also participants kept their dominant hand on the keyboard array keys
<figure><img src="image_6.png" /><figcaption>Fig. 4: The different hand postures used during the experiment.</figcaption></figure>

when performing the secondary task, and when this latter is finished, they used the *fingers-above* for the hand used for the secondary task while keeping the hands semi-crossed (see Figure 4j). Two participants used this posture.

### 4.5.3 Strategies used to handle the primary and the secondary task

We noticed different methods or behaviours adopted by our participants to prioritize the primary task (centring the ball) over the secondary task (recognizing the textures), which we grouped into three main strategies that we highlight here-after. Interestingly, we observed that once the participant starts the task with a given strategy, he continues with that strategy until finishing all the trials of the activity independently of the surface position or the hand posture used. We highlight the different strategies and the hand postures used hereafter:

- **Competitive interaction with Exclusive attention to Primary task (CEP).** Five of the thirteen participants kept interacting strongly with the primary task when they were interacting with the secondary one while their gaze attention was mostly conducted toward the laptop screen (i.e., the primary task) with a nearly eyes-free interaction with the tactile surface (i.e., the secondary task). In addition, all of them used to put more visual attention for the tactile surface only when they have to select the texture after perception, as it needs three button selections (confirm the end of perception, selecting a texture and then confirming the selection) with generally glances toward the primary task screen between those three actions. They also, rarely look at the tactile surface before starting perceiving the texture or for locating this latter during the experiment. Two participants using this strategy found that the activity is similar to “driving” while “checking the GPS on the phone” or “manipulating their multimedia car radio”.

Four of the five participants interacted with the primary task in the same way and with almost the same rhythm during all the experiment *i.e.*, even when they have additionally to identify a texture. While we observed a different behavior for the remaining participant which used to move sequentially the
ball from left to right, and from right to left by continuously alternating pressing on the left and the right arrow buttons to “insure keeping the ball centered” while performing the texture identification.

Three hands posture were associated to this strategy. 2-HD was used by respectively three, five and four participants for the right, left and forward surface positions. Two participants used 2AC for the right position and one participant with a 2ASC for the forward position.

- **Reactive Interaction with Shared Attention to Both tasks (RSB)**
Three of the thirteen participants interacted principally with the secondary task and reacted to the primary task only when necessary i.e., the ball moves away from the center, while keeping their hand over the arrow keys. Their gaze attention was shared almost equally between the primary and the secondary task when identifying textures, with certain cases where the primary task got more gaze attention then the secondary one. This strategy was used with a 2HD by three participants for both the right and left surface positions. Two of them continue using this hand posture for the forward position while the latter used a 2ASC.

- **Divided interaction with Exclusive attention to Secondary task (DES)**
Five of the thirteen participants stopped interacting with the primary task and switched to the secondary one until they select a texture. In addition, to insure that the ball will not deviate from the box center, participants perform quick identifications of the rendered textures. And sometimes, when the notification showed up, participants start by making sure to center well the ball before switching to the secondary task. Most of the gaze attention was conducted toward the secondary task with only few glances toward the primary one. Two participants used a 1H and the three latter used a 2HD for the three surface positions.

#### 4.5.4 Methodologies for identifying textures
To better understand how participants were performing, we report hereafter the four different strategies elaborated by participants in order to identify the *textures*; which is the by-product of the discussions that followed each *activity* condition. Four main strategies were identified:

- Visualizing the texture image in the user mental when perceiving the texture.

- Counting the number of all tactile feedback and then match its position with the visual texture.

- Using the densest and the sparsest textures as a reference.

- Comparing the distance between two successive feedback and compare it to the user' finger width to determine if the texture is sparse or medium. One participant said: "for the medium and sparse textures... I had to look to my finger to make the correspondence between the distance between two successive signals and my finger width to determine which texture I am feeling.".

These findings are correlated to the findings of Rekik et al., [27, 26, 14]. Interestingly, many participants were able to know that they had made a mistake
during the task for a particular texture when they were feeling the next one. With full practice, one may conjecture that the accuracy and the speed should eventually increase. Importantly, our participants used these four main strategies independently on the activity condition.

## 4.6 Discussion

Our key finding is that user's primary task had an impact on recognition rate without compromising the speed. We observed an average decrease of performance of 6.98% (from 88.78% to 82.58%) with an additional mental and physical demands, frustration and effort. These findings are consistent across different surface positions. We also observed different hands postures (one one-handed and three two-handed) and strategies to handle the primary and the secondary task with some participants making the secondary task without the need to see the surface device making the interaction with it eyes-free. Additionally, for the two-handed postures, the hand used for the primary task remains on the keyboard even when performing the secondary task to be ready to react to the primary task. However, interestingly, the hand used for the secondary stays in the surrounding area of the tablet even when the two hands are crossed or semi-crossed.

# 5 Experiment 2: Effect of a cognitively demanding task

In this experiment, we followed [9] and used the same text-typing<fnref n="3" /> exercise as our cognitively demanding primary task. We asked our participants to prioritize typing over texture recognition, and told them that their performance was being measured for both the primary and the secondary task. The procedure and task as well as common apparatus are presented in the Experiments section. Twelve new participants (3 females) volunteered (not paid) to take part into this experiment. Participant ages ranged between 22 and 41 years (mean=30.41 years, sd=6.05 years). All participants were right handed.

## 5.1 Results

We consider the same dependent variables than in experiment 1.

5.1.1 **Total time** As in experiment 1, we found a significant main effect of *block* ($F_{2,22} = 9.43, p = .0011$) on *total time* with the first block slower than the two remainder ones (block1: mean=8122 ms, s.d=872 ms, block2: mean=8017 ms, s.d=640 ms and block3: 5744 ms, s.d=667 ms). Post-hoc comparison confirms these differences ($p < .05$). As we are concerned with user performance after familiarization, the remaining analysis discards the first block.

Analysis of total time shows no significant effect of activity ($F_{1,11.03} = 0.43, p < .5251$), with similar means of total time between the control condition (mean=7761.085 ms, sd=608.148 ms), and typing (mean=8421 ms, sd=726 ms). There was a significant effect of *texture* ($F_{3,33.47} = 9.72, p < .0001$) with the densest texture (mean=6237 ms, sd=785 ms) being recognized significantly faster than

<footnote n="3">³ https://www.goodtyping.com/test.php</footnote>
all remainder textures (dense: mean=8821 ms, sd=1080 ms, medium: mean=9163 ms, sd=1012 ms, sparse: mean=8210 ms, sd=814 ms). Importantly, there was no significant interaction (*p* > .30) suggesting that these results are consistent.

**5.1.2 Reaction time** There was a significant effect of activity ($F_{1,11.02} = 7.95, p = .0166$) on *reaction time*, with the control condition faster (mean=2065 ms, sd=208 ms) than typing (mean=5124.839 ms, sd=661 ms) (see Figure 5b). Post-hoc comparison confirms differences between the control condition and typing (*p* < .05). We correlate this result with user behavior. For instance, contrarily to the control condition where participants started the texture recognition when receiving the notification, in the typing activity, participants continued writing their word/sentence before switching to the recognition task. There was no significant interaction (*p* > .35), suggesting that the drawbacks of typing are consistent across different textures and surface positions.

**5.1.3 Movement time** There was a significant effect of activity ($F_{1,11.02} = 7.47, p = .0194$) and *texture* ($F_{3,33.47} = 9.86, p < .0001$) on *movement time*. Posthoc tests revealed that typing (mean=3296 ms, sd=300 ms) is significantly faster than the control condition (mean= 5695 ms, sd=524 ms) (*p* < .05) (Figure 5c). We also found that the densest texture (mean=2719 ms, sd=415 ms) is recognized significantly faster than all remainder textures (dense: mean=4968 ms, sd=732 ms, medium: mean=5458 ms, sd=689 ms, sparse: mean=5014 ms, sd=595 ms). There were no significant interactions (*p* > .05), suggesting that the benefits of typing are consistent across textures and surface position.

**5.1.4 Accuracy** Analysis of count of trials containing an error shows no significant effect of activity ($F_{1,11} = 1.66, p = .2236$) on accuracy, with similar means of 87.15% with the control condition, and 84.95% for typing. There was a significant effect of texture ($F_{3,33.47} = 5.13, p = .0051$) with the densest texture (mean= 96.2963%, sd= 1.716964%) being more accurate than the remainder textures (dense: mean= 96.29%, sd= 1.71%, medium: mean= 77.54%, sd= 4.98%, sparse: mean= 85.41%, sd= 4.15%).

**5.1.5 Number of swipes** There was a significant main effect of activity ($F_{1,11.02} = 4.57, p = .05$) on *number of swipes*. Post-hoc tests revealed that the control condition (mean= 8.33, sd= .91) produced significantly more swipe gestures than when typing (mean= 6.15, sd= .85) (*p* < .05) (Figure 5d). This result is correlated to the movement time that decreases with the typing activity. There was no significant interaction (*p* > .12), suggesting that this finding is consistent across different tablet positions and textures.

**5.1.6 Number of Clutches** We found no significant effect on number of clutch nor interaction (*p* > .05) (mean= .81, sd=.15).

**5.1.7 Subjective Results and Observations** Nasa-TLX responses (Table 1) showed that *typing* was significantly more demanding physically and temporally than the control condition. However, contrarily to experiment 1, in this experiment, only the DES strategy is used to prioritise the handle of both
<figure><img src="image_7.png" /><figcaption>(a) Accuracy</figcaption></figure>

<caption>(b) Reaction time</caption>

<caption>(c) Movement time</caption>

<caption>(d) Number of swipes</caption>

<caption>Fig. 5: User Performances in the second experiment.</caption>

tasks. When participants received a notification, they first finish the word or the sentence they were writing then switch completely to the secondary task. Then-after, they make quick textures identification. The gaze attention was exclusively conducted toward the secondary task i.e., identifying the textures on the tactile surface when interacting with the secondary task, with some rarely glances toward the primary task for the *forward position*.

The switch from the primary task to the secondary one was accompanied with a switch in the number of used hands: from two hands to one hand. For instance, ten participants used exclusively their dominant hand when interacting with the secondary task, while the two remainder used the closest hand to the tactile surface to identify the texture (i.e., their right hand for *right* and *forward* positions and their left hand for the *left* position). Finally, all our participants used the same methodologies used in experiment 1 for identifying textures.

## 5.2 Discussion

Our key findings is that user's primary task (typing text) had an impact on recognition time and number of swipes and was mentally more demanding than the control condition. We observe an increase in reaction time of 148% (from 2065ms to 5124ms) and a decrease in movement time and number of swipes by respectively 42% (from 5695ms to 3296ms) and 26.17% without compromising total time and accuracy. These findings are consistent across different surface positions. These findings are also correlated to user strategy to handle the two simultaneous tasks while prioritizing the typing task over the texture recognition task. Our participants, first, finished writing the current word or sentence before moving to the textures recognition task. And then-after, our participants made quick interactions with the secondary task, before resuming the primary one.

# 6 Discussion & Implications

In this section, we discuss the implications of our results for attention saturating and cognitively demanding primary task, posture, tactile texture design and eyes-free interaction.

## 6.1 Attention saturating primary task

Our first experiment highlights an effect of the attention saturating primary task on recognition rate: we observed an accuracy drop from 88.78% to 82.58% when participants have to keep the ball centred. However, while at first glance, it may look like using tactile texture with such a primary task is unusable, the recognition accuracy did achieve good rate of 82.58%. Consequently, this
suggests that tactile textures could be effectively recognized and used by users on
touchscreens when making such primary task. However, researchers interested in
validating lab results in more realistic conditions may want to include tasks that
saturate the attention as a factor for their experiment. We believe that more
attention saturating tasks, in particular when considering a realistic scenario,
e.g., driving, would see a higher drop of accuracy.

## 6.2 Cognitively demanding primary task

Our second experiment highlights an effect of the cognitively demanding task (typing) on recognition time: we observed an increase in reaction time from 2065ms to 5124ms and a decrease in movement time from 5695ms to 3296ms without compromising the recognition rate or the total recognition time. However, in the real world, what matters is the total time to convey information, which is why we included the total time to present the stimulus. The differences between the different activities is not significant. Consequently, this suggests that tactile texture could be effectively recognized and used by users when typing-text on laptop. Additionally, we believe that with practice users can cognitively chunk the simultaneous two tasks and greatly reduce their reaction time. The typing activity was as accurate as the control condition, but with shorter movement time to recognize the texture, which requires more concentration. Thus, one may conjecture that the accuracy should eventually increase with practice.

## 6.3 Eyes-free interaction design

Our findings indicate that the perception of tactile textures can be made in an
eyes-free interaction when performing at the same time an attention saturating
primary task. This suggests that interacting with the tactile surface through
textures can permit a user to sense the regions without diverting the eyes from
the primary display during visually demanding tasks. Thus, designers should
consider tactile texture to create an eyes-free dialog between the surface and the
user especially when the user have to interact with another primary task.

## 6.4 Hands posture during two simultaneous tasks

Different hand postures have been used to handle and perform the primary and
the secondary task. In particular, For an attention saturating primary task, our
findings indicate that the 2HD posture is the most used posture. This posture is
strongly correlated to the position of the surface device (used for the secondary
task): the closed hand to the secondary task will perform it. This finding may
help designers to choose the appropriate position for the secondary task device
dependently on their preferred hand for the primary task.

## 6.5 Tactile texture design

Our results showed that the *densest* texture was the easiest and quickest one to identify among the four evaluated textures, and as most participants reported: *“it's easy to guess the densest texture”*. It also required less effort then the other textures with significantly less swipes and clutches when users had to perform an
attention saturating task simultaneously. Those findings suggest that the *densest* texture may be a good choice when designing tactile texture based interactions. Designers can also consider combining the densest texture with the sparse texture to create a large set of textures as our participants felt that "*it's easy to determine the difference*" between those two textures.

**6.6 Limitations & next steps**

Like any study, our study presents limitations. For example, in our studies participants were younger than the population average, were right-handed and all are students at the university. Undoubtedly, elder people, children or left-handed would behave differently. These issues are worthy of investigation, but are beyond the scope of the current work.

We observed different strategies to handle and perform the attention saturating task and the texture recognition one, accompanied with either exclusive attention to one task or shared attention to both tasks. However, the current study, do not allow us to determine which strategy is better.Consequently, our upcoming work will compare these different strategies while fixing the gaze attention to examine the effectiveness of tactile textures on touchscreen surfaces in each scenario and to determine its effectiveness when being able to see the tactile surface or not.

Finally, we do not found the same results nor the same behaviour (only one shared strategy to handle both tasks) when changing the primary task in our experiments. We believe that the overall message of this findings is simply that different interaction context produce different performances and behaviour on the texture recognition secondary task for the end user. These differences limit the overall generalization of our findings for other different primary tasks. As other scenarios exist where user can make a primary task while checking his mobile (e.g., a person using his smartphone while being in a meeting or speaking with another person [28] or making a reading comprehension task or a word search [2] or when the attention is fragmented [16]), additional work will be required to explore how best to recognize textures on touchscreen while performing other primary tasks.

**7 Conclusion**

In this paper, we conducted the first investigation on the effect of an attention saturating primary task and a cognitively demanding primary task on tactile textures recognition on ultrasonic haptic tablet. Our findings indicate that for both primary tasks, the recognition rates for tactile textures stays higher than 82% without compromising the total time. However, contrarily to the cognitively demanding task, the attention saturating task increases frustration and mental demand compared to the control condition. We have also gained insight into the mental models of users when handling two simultaneous tasks and have discussed their implications for tactile feedback based interaction. We hope that our findings will prove useful to tactile feedback designers assisting them toward designing novel tactile feedback based techniques that would help users making the secondary tasks without being distracted from the primary task.
References

1. Amberg, M., Giraud, F., Semail, B., Olivo, P., Casiez, G., Roussel, N.: Stimtac: A tactile input device with programmable friction. In: Proceedings of the 24th Annual ACM Symposium Adjunct on User Interface Software and Technology. pp. 7–8. UIST ’11 Adjunct, ACM, New York, NY, USA (2011). https://doi.org/10.1145/2046396.2046401, https://doi.org/10.1145/2046396.2046401

2. Barnard, L., Yi, J.S., Jacko, J.A., Sears, A.: Capturing the effects of context on human performance in mobile computing systems 11(2), 81–96 (Jan 2007). https://doi.org/10.1007/s00779-006-0063-x, https://doi.org/10.1007/s00779-006-0063-x

3. Bau, O., Poupyrev, I., Israr, A., Harrison, C.: Teslatouch: Electrovibration for touch surfaces. In: Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology. pp. 283–292. UIST’ 10, ACM, New York, NY, USA (2010). https://doi.org/10.1145/1866029.1866074, https://doi.org/10.1145/1866029.1866074

4. Bragdon, A., Nelson, E., Li, Y., Hinckley, K.: Experimental analysis of touch-screen gesture designs in mobile environments. In: Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11. p. 403. ACM Press. https://doi.org/10.1145/1978942.1979000, http://dl.acm.org/citation.cfm?doid=1978942.1979000

5. Brewster, S., Chohan, F., Brown, L.: Tactile feedback for mobile interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’07. pp. 159–162. ACM Press. https://doi.org/10.1145/1240624.1240649, http://dl.acm.org/citation.cfm?doid=1240624.1240649

6. Brown, L.M., Brewster, S.A., Purchase, H.C.: Multidimensional tactons for non-visual information presentation in mobile devices. In: Proceedings of the 8th conference on Human-computer interaction with mobile devices and services - MobileHCI ’06. p. 231. ACM Press. https://doi.org/10.1145/1152215.1152265, http://portal.acm.org/citation.cfm?doid=1152215.1152265

7. Casiez, G., Roussel, N., Vanbelleghem, R., Giraud, F.: Surfpad: Riding towards targets on a squeeze film effect. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 2491–2500. CHI ’11, ACM, New York, NY, USA (2011). https://doi.org/10.1145/1978942.1979307, https://doi.org/10.1145/1978942.1979307

8. Cauchard, J.R., Cheng, J.L., Pietrzak, T., Landay, J.A.: ActiVibe: Design and evaluation of vibrations for progress monitoring. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 3261–3271. ACM. https://doi.org/10.1145/2858036.2858046, https://dl.acm.org/doi/10.1145/2858036.2858046

9. Chen, Q., Perrault, S.T., Roy, Q., Wyse, L.: Effect of temporality, physical activity and cognitive load on spatiotemporal vibrotactile pattern recognition. In: Proceedings of the 2018 International Conference on Advanced Visual Interfaces - AVI ’18. pp. 1–9. ACM Press. https://doi.org/10.1145/3206505.3206511, http://dl.acm.org/citation.cfm?doid=3206505.3206511

10. Cockburn, A., Woolley, D., Thai, K.T.P., Clucas, D., Hoermann, S., Gutwin, C.: Reducing the attentional demands of in-vehicle touchscreens with sten-cil overlays. p. 33–42. AutomotiveUI ’18, Association for Computing Ma-chinery, New York, NY, USA (2018). https://doi.org/10.1145/3239060.3239061, https://doi.org/10.1145/3239060.3239061
11. Cuqlock-Knopp, V.G.: Engineering psychology and human performance by christopher wickens and justin hollands: 1999, 573 pages, $84.00 up-per saddle river, nj: Prentice-hall isbn 0-321-04711-7. Ergonomics in Design 8(4), 37-38 (2000). https://doi.org/10.1177/106480460000800411, https://doi.org/10.1177/106480460000800411

12. Dariosecq, M., Plénacoste, P., Berthaut, F., Kaci, A., Giraud, F.: Investigating the semantic perceptual space of synthetic textures on an ultrasonic based hap-tic tablet. In: HUCAPP 2020. Valletta, Malta (Feb 2020), https://hal.archives-ouvertes.fr/hal-02434298

13. Giraud, F., Amberg, M., Lemaire-Semail, B., Giraud-Audine, C.: Us-ing an ultrasonic transducer to produce tactile rendering on a touch-screen. In: 2014 Joint IEEE International Symposium on the Applica-tions of Ferroelectric, International Workshop on Acoustic Transduc-tion Materials and Devices & Workshop on Piezoresponse Force Mi-croscopy. pp. 1-4. IEEE. https://doi.org/10.1109/ISAF.2014.6922972, http://ieeexplore.ieee.org/document/6922972/

14. Guettaf, A., Rekik, Y., Grisoni, L.: Effect of physical challenging activity on tactile texture recognition for mobile surface. Proc. ACM Hum.-Comput. Interact. 4(ISS) (Nov 2020). https://doi.org/10.1145/3427318, https://doi.org/10.1145/3427318

15. Harrison, C., Hudson, S.E.: Providing dynamically changeable phys-ical buttons on a visual display. In: Proceedings of the 27th inter-national conference on Human factors in computing systems - CHI 09. p. 299. ACM Press. https://doi.org/10.1145/1518701.1518749, http://dl.acm.org/citation.cfm?doid=1518701.1518749

16. Harvey, M., Pointon, M.: Searching on the go: The effects of fragmented at-tention on mobile web search tasks. In: Proceedings of the 40th Interna-tional ACM SIGIR Conference on Research and Development in Informa-tion Retrieval. p. 155-164. SIGIR '17, Association for Computing Machin-ery, New York, NY, USA (2017). https://doi.org/10.1145/3077136.3080770, https://doi.org/10.1145/3077136.3080770

17. Israr, A., Bau, O., Kim, S.C., Poupyrev, I.: Tactile feedback on flat surfaces for the visually impaired. In: CHI '12 Extended Abstracts on Human Factors in Computing Systems. pp. 1571-1576. CHI EA '12, ACM, New York, NY, USA (2012). https://doi.org/10.1145/2212776.2223674, https://doi.org/10.1145/2212776.2223674

18. Kalantari, F., Berthaut, F., Grisoni, L.: Enriching musical interaction on tactile feedback surfaces with programmable friction. In: Aramaki, M., Davies, M.E.P., Kronland-Martinet, R., Ystad, S. (eds.) Music Technology with Swing, vol. 11265, pp. 387-401. Springer International Publishing. https://doi.org/10.1007/978-3-030-01692-0_25, http://link.springer.com/10.1007/978-3-030-01692-0_25, series Ti-tle: Lecture Notes in Computer Science

19. Kalantari, F., Grisoni, L., Giraud, F., Rekik, Y.: Finding the mini-mum perceivable size of a tactile element on an ultrasonic based hap-tic tablet. In: Proceedings of the 2016 ACM International Confer-ence on Interactive Surfaces and Spaces. pp. 379-384. ISS '16, ACM, New York, NY, USA (2016). https://doi.org/10.1145/2992154.2996785, https://doi.org/10.1145/2992154.2996785

20. Kane, S.K., Morris, M.R., Wobbrock, J.O.: Touchplates: low-cost tactile over-lays for visually impaired touch screen users. In: Proceedings of the 15th In-ternational ACM SIGACCESS Conference on Computers and Accessibility -
ASSETS '13. pp. 1-8. ACM Press. https://doi.org/10.1145/2513383.2513442, http://dl.acm.org/citation.cfm?doid=2513383.2513442

21. Kim, S.C., Israr, A., Poupyrev, I.: Tactile rendering of 3d features on touch surfaces. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology. pp. 531-538. UIST '13, ACM, New York, NY, USA (2013). https://doi.org/10.1145/2501988.2502020, http://doi.acm.org/10.1145/2501988.2502020

22. Leigh, D., Forlines, C., Jota, R., Sanders, S., Wigdor, D.: High rate, low-latency multi-touch sensing with simultaneous orthogonal multiplexing. In: Proc. of UIST. pp. 355-364. ACM (2014). https://doi.org/10.1145/2642918.2647353, http://doi.acm.org/10.1145/2642918.2647353

23. Levesque, V., Oram, L., MacLean, K.: Exploring the design space of programmable friction for scrolling interactions. In: 2012 IEEE Haptics Symposium (HAPTICS). pp. 23-30. IEEE. https://doi.org/10.1109/HAPTIC.2012.6183765, http://ieeexplore.ieee.org/document/6183765/

24. Levesque, V., Oram, L., MacLean, K., Cockburn, A., Marchuk, N.D., Johnson, D., Colgate, J.E., Peshkin, M.A.: Enhancing physicality in touch interaction with programmable friction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2481-2490. CHI '11, ACM, New York, NY, USA (2011). https://doi.org/10.1145/1978942.1979306, https://doi.org/10.1145/1978942.1979306

25. Oy, S.: Senseg feelscreen development kit. online available. http://www.senseg.com (2016)

26. Rekik, Y., Vezzoli, E., Grisoni, L.: Understanding users' perception of simultaneous tactile textures. In: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. pp. 5:1-5:6. MobileHCI '17, ACM, New York, NY, USA (2017). https://doi.org/10.1145/3098279.3098528, http://doi.acm.org/10.1145/3098279.3098528

27. Rekik, Y., Vezzoli, E., Grisoni, L., Giraud, F.: Localized haptic texture: A rendering technique based on taxels for high density tactile feedback. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 5006-5015. ACM. https://doi.org/10.1145/3025453.3026010, https://dl.acm.org/doi/10.1145/3025453.3026010

28. Roudaut, A., Rau, A., Sterz, C., Plauth, M., Lopes, P., Baudisch, P.: Gesture output: Eyes-free output using a force feedback touch surface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2547-2556. CHI '13, ACM, New York, NY, USA (2013). https://doi.org/10.1145/2470654.2481352, http://doi.acm.org/10.1145/2470654.2481352

29. Rydström, A., Grane, C., Bengtsson, P.: Driver behaviour during haptic and visual secondary tasks. In: Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications. p. 121-127. AutomotiveUI '09, Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1620509.1620533, https://doi.org/10.1145/1620509.1620533

30. Vezzoli, E., Sednaoui, T., Amberg, M., Giraud, F., Lemaire-Semail, B.: Texture rendering strategies with a high fidelity - capacitive visual-haptic friction control device. In: Proceedings, Part I, of the 10th International Conference on Haptics: Perception, Devices, Control, and Applications - Volume 9774. pp. 251-260. EuroHaptics 2016, Springer-Verlag, Berlin, Heidelberg (2016). https://doi.org/10.1007/978-3-319-42321-0_23, https://doi.org/10.1007/978-3-319-42321-0_23
31. Winfield, L., Glassmire, J., Colgate, J.E., Peshkin, M.: T-PaD: Tactile pattern display through variable friction reduction. In: Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07). pp. 421–426. IEEE. https://doi.org/10.1109/WHC.2007.105, http://ieeexplore.ieee.org/document/4145211/

32. Xu, C., Israr, A., Poupyrev, I., Bau, O., Harrison, C.: Tactile display for the visually impaired using TeslaTouch. In: Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems - CHI EA '11. p. 317. ACM Press. https://doi.org/10.1145/1979742.1979705, http://portal.acm.org/citation.cfm?doid=1979742.1979705

33. Zhang, Y., Harrison, C.: Quantifying the targeting performance benefit of electrostatic haptic feedback on touchscreens. In: Proc. of ITS. pp. 43–46. ACM (2015). https://doi.org/10.1145/2817721.2817730, http://doi.acm.org/10.1145/2817721.2817730