# Vibro-Tactile Implicit Interactions: So What?

Yulia Zhiglova, David Lamas, Ilja Smorgun, Paul Seitlinger

► To cite this version:

Yulia Zhiglova, David Lamas, Ilja Smorgun, Paul Seitlinger. Vibro-Tactile Implicit Interactions: So What?. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.685-688, 10.1007/978-3-030-29390-1_63. hal-02878634

HAL Id: hal-02878634

https://inria.hal.science/hal-02878634

Submitted on 23 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Vibro-Tactile Implicit Interactions: So What?

Yulia Zhiglova[0000-0001-7114-7972], David Lamas[0000-0003-0295-453X], Ilja Smorgun[0000-0002-5564-8416], and Paul Seitlinger[0000-0003-1817-4626]

Tallinn University, Tallinn, Estonia yzhigl@tlu.ee

**Abstract.** Tactile feedback is a powerful modality for designing human-computer interfaces. Here we explore new ways of communication through tactile senses that can be perceived implicitly. This paper outlines the preliminary phase of research the goal of which is to investigate to what extent a vibro-tactile stimulus can be meaningfully communicated on an unconscious/implicit level of perception. Ability to perceive information implicitly provides an alternative channel for communication and may therefore reduce our cognitive load. In this poster, we formulate our research problem, suggest a research framework and outline future experiments enabled by the prototype we have designed.

**Keywords:** peripheral interaction · vibro-tactile · body-centric.

## 1 Introduction

Touch is essential for humans. It elicits comfort and attachment and provides rich information about the world around us [5]. Tactile feedback can be a powerful modality for developing human-computer interfaces. A lot of HCI research concerns the mediated-touch technology and how various vibro-tactile stimuli may simulate sense of presence. Most of these studies work in the plane of conscious or explicit perception [2, 3]. Very few touch upon the implicit or unconscious [6, 1]. Unconscious perception can be defined as an automatic, effortless process which does not require bringing information into the attention focus [9]. Unconscious perception, evoked by tactile stimulation, may provide higher communication bandwidth. It can also reduce cognitive load that comes with using multiple devices simultaneously. Understanding the extent to which a mediated-touch technology may be used on a peripheral level can lay a foundation for numerous applications in healthcare, well-being and personal relationships. Further, the terms peripheral, implicit and unconscious will be used interchangeably.

## 2 Research Problem Formulation and Context

The traditional interaction with technology mostly happens through visual interfaces and keyboard. Such interactions require full attention from a person, increasing cognitive load. Therefore, HCI community sees a need in developing new ways of communication that would allow integrating technology seamlessly
in everyday tasks. For instance, already in 1997 Weiser talked about 'calm technology' which 'engages both the center and the periphery of our attention', allowing to concentrate on the important task [8]. Haptic feedback can be a powerful tool in designing 'calm' technology. From previous studies we know that specific combinations of parameters of vibro-tactile displays may simulate not only a variety of touch types but potentially be perceived implicitly, given an appropriate configuration of parameters [3, 6]. As there are few studies concerning unconscious perception of vibro-tactile stimuli, we see a need for further research. Our broader research goal is to explore how range of vibro-tactile displays can be designed to convey information on an unconscious level of perception in body-centric configurations. To achieve the goal we plan to answer the following question: What configurations of vibro-tactile parameters are more effective and efficient in communicating information on an unconscious level? The key parameters include frequency, intensity, and location on the body. The study will be based on controlled experiments, involving university students without tactile impairments as participants. We have designed a vibro-tactile prototype to enable first experiments. The purpose of the experiments is to investigate how parameters of the vibro-tactile display need to be configured. We assume that by changing one or a combination of the parameters the stimuli perception will vary with respect to the extent by which conscious vs. unconscious processes contribute to the person's somatosensory experience.

We focus our research in the context of remote relationships. Imagine a situation where a daughter and a mother are living in different cities. Daughter is about to have an exam at school and is busy preparing for it. Mother, knowing that, wants to support her daughter without distracting her by a phone call. Instead, she sends a caressing which daughter can feel on her skin through a wearable vibro-tactile display. The daughter perceives the virtual caressing, and since the stimuli is communicated on a periphery of her attention, she is not distracted from her main activity.

## 3 Related Work

Our overall research and methodology is greatly informed by the psychological studies of perception, specifically, by the dual process extension of the Signal Detection Theory [9]. We were inspired by research papers that applied these psychological theories in the domain of tactile and auditory perceptions. The focus of the papers was the ability to detect and identify physical changes evoked by the vibro-tactile device (e.g., [7]) and auditory stimuli (e.g., [4]). According to this prior work, two cognitive processes can be expected to be in play. In the event of a signal that enters the focus of attention, an explicit process takes effect that draws on working memory resources and leads to parsing the haptic (or auditory) scene and encoding the instances that have caused the change in perception. Consequently, the person will be able to not only detect a change but also identify the specific source of change. If, by contrast, the signal of change does not enter the attention focus, an implicit and effortless signal detection
process can nevertheless become active that will not consume the individuals working memory resources. The stimuli will only operate on global qualities and low-level features. However, while being less resource consuming, this implicit process will also be more noisy and will only allow for sensing a global change but not for identifying the specific source of change. Thus, it is the latter, namely implicit process, that underlies the phenomenon that is typically referred to as peripheral perception in HCI research. Some HCI researchers already used dual-process and divided attention theories to inform their research [1].

# 4 Research Framework

Our research framework is based on the ongoing scoping study (not published yet) of the state of the art of the vibro-tactile body-centric displays, psychological studies of perception and principles of psychophysics. From the scoping study we identified existing vibro-tactile patterns that simulate various types of touch and the existing algorithms that enable these simulations [3]. We also identified key vibro-tactile parameters that influence the perception of the stimuli. In psychophysics domain we explored the principles of tactile senses and how they can be manipulated by vibro-tactile stimuli. We worked in close cooperation with a psychology expert to understand the complex interplay between physical stimuli and the psychological space. We formulated how the physical space (vibro-tactile stimulation) maps onto the psychological space (level of processing), and represented each of them as a multi-dimensional space. The dimensions of the physical space include frequency and intensity of vibration and the location on a human body. The physical dimension represents the design space for our vibro-tactile displays. The psychological space includes such dimensions as affective states (arousal and valence) and level of processing (implicit and explicit). Currently, we only focus on the latter dimension and examine the possibility to evoke and measure the perception both on a conscious and unconscious level of information processing. Summarizing, our goal is to define a psychophysical function g that maps states in the physical space F (vibrotactile) onto states in the psychological space E. If there are three physical dimensions of frequency, intensity, and location, then a certain point in F represents a corresponding parameter configuration and can be notated as a vector $f_t$ at some time t in the course of the experiment. $f_t$, in any case, will evoke a particular level of processing, and can be represented as the vector $e_t$. The research question is then where exactly this psychological point will be localized in E. The goal will thus be to identify a function g in the form of an equation, by which we can quantitatively predict the level of processing of the physical event triggered by the device.

$$e_t = g(f_t)$$

# 5 Conclusion

The future work will entail controlled experiments based on the dual-process measurement model mentioned earlier [4]. The model will explore explicit and
implicit change detection of vibro-tactile stimuli as a discrete and continuous pro-
cess, respectively. The experimental design will be a change detection paradigm
that has participants distinguish signal from noise (presence vs. non-presence
of stimuli) and indicate their response confidence. This will result in a Re-
ceiver Operating Characteristic (ROC) curve, against which the dual-process
measurement model [4] can be fitted to estimate contributions of both explicit
and implicit processes of change detection. That way, we aim to address our main
methodological challenge of measuring unconscious states without affecting them
substantially. Also we plan to extend the psychophysical function by parameters
representing additional modalities such as sound and temporal dynamics within
the psychological space.

Thus far, we provided an overview of the initial stage of research with hope
to probe our initial research ideas and gather feedback. We hope this work can
spark curiosity for the topic and result in research collaborations.

References

1. Bakker, S., van den Hoven, E., Eggen, B.: Peripheral interaction: characteristics and considerations. Personal and Ubiquitous Computing **19**(1), 239–254 (Jan 2015). https://doi.org/10.1007/s00779-014-0775-2, https://doi.org/10.1007/s00779-014-0775-2

2. Eid, M.A., Al Osman, H.: Affective Haptics: Current Research and Future Directions. IEEE Access **4**, 26–40 (2016). https://doi.org/10.1109/ACCESS.2015.2497316

3. Israr, A., Poupyrev, I.: Tactile brush: Drawing on skin with a tactile grid display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2019–2028. CHI ’11, ACM, New York, NY, USA (2011). https://doi.org/10.1145/1978942.1979235, http://doi.acm.org/10.1145/1978942.1979235

4. McAnally, K.I., Martin, R.L., Eramudugolla, R., Stuart, G.W., Irvine, D.R., Matingley, J.B.: A dual-process account of auditory change detection. Journal of Experimental Psychology: Human Perception and Performance **36**(4), 994–1004 (2010). https://doi.org/10.1037/a0016895

5. Montagu, A.: Touching: The Human Significance of the Skin. Perennial library, HarperCollins (1986), https://books.google.ee/books?id=XU7Z_aqCYggC

6. Riener, A., Ferscha, A., Frech, P., Hackl, M., Kaltenberger, M.: Subliminal vibrotactile based notification of co2 economy while driving. In: AutomotiveUI (2010)

7. Spence, C., Gallace, A., Tan, H.Z.: The Failure to Detect Tactile Change: A Tactile Analogue of Visual Change Blindness. Psychonomic Bulletin and Review **13**(2), 300 – 303 (2006)

8. Weiser, M., Brown, J.S.: The Coming Age of Calm Technology, pp. 75–85. Springer New York, New York, NY (1997). https://doi.org/10.1007/978-1-4612-0685-9_6, https://doi.org/10.1007/978-1-4612-0685-9

9. Wixted, J.T.: Dual-process theory and signal-detection theory of recognition memory. Psychological Review **114**(1), 152–176 (2007). https://doi.org/10.1037/0033-295X.114.1.152