# TactCube: Designing Mobile Interactions with Ambient Intelligence

Pietro Battistoni, Marianna Di Gregorio, Marco Romano, Monica Sebillo, Giuliana Vitiello

## ► To cite this version:

Pietro Battistoni, Marianna Di Gregorio, Marco Romano, Monica Sebillo, Giuliana Vitiello. TactCube: Designing Mobile Interactions with Ambient Intelligence. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.599-609, 10.1007/978-3-030-85610-6_34. hal-04215511

HAL Id: hal-04215511

https://inria.hal.science/hal-04215511v1

Submitted on 22 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.

Correction to: Chapter “TactCube: Designing Mobile Interactions with Ambient Intelligence” in: C. Ardito et al. (Eds.): *Human-Computer Interaction – INTERACT 2021*, LNCS 12935, https://doi.org/10.1007/978-3-030-85610-6_34

In the original version of this book the name of the author "Marianna di Gregorio" was incorrect. This has now been corrected.
# TactCube: designing mobile interactions with Ambient Intelligence*

Pietro Battistoni¹[0000-0002-7477-2439], Marianna Di Gregorio¹[0000-0002-2743-9051],

Marco Romano¹[0000-0002-8581-3160],

Monica Sebillo¹[0000-0003-3731-6415], Giuliana Vitiello¹[0000-0001-7130-996X]

¹ Università di Salerno, Fisciano (SA), 84084, Italy

pbattistoni@unisa.it, madigregorio@unisa.it, marromano@unisa.it, msebillo@unisa.it, gvitiello@unisa.it

**Abstract.** This paper presents TactCube, a novel tangible user interface based on a cube-shaped device. It is designed to interact with Ambient Intelligence and allows users to give commands and receive rich feedback using only one hand. A guessability user study was conducted involving 60 participants and results were used to map the device interactions onto commands invoked on an Ambient Intelligence system. The study considered three smart devices, properly a smart speaker, a smart bulb, and a smart curtain. For each of the devices, users were asked to propose his/her favorite interaction modality through TactCube, for 12 different commands and a kind of admissible feedback for 8 different states of the ambient intelligence system. The analysis of the responses led us to derive a set of interaction design implications which could be taken into account for any tangible interface inside an intelligent ambient system.

**Keywords:** tangible interaction, user study, interactive environment.

## 1 Introduction

The Internet of Things has the vision to pervade our daily environment. One of the main characteristics of such environments is the growing amount of smart devices and their complexity. Indeed, due to the weaknesses of existing user interfaces, when interacting with intelligent environments, users are overloaded by the multitude of networked devices [3]. Identifying and activating the right device or service from a huge amount of existing devices to perform a specific task is a challenge for users in these environments. Moreover, devices can disappear in the background and can be invisible to users requiring new forms of interaction [1]. Researchers are dealing with the design of interfaces enabling the interaction through different devices and in different forms depending on the characteristics of the ambient environment [13]. In [1], a systematic review on meta-interfaces to interact with the environment points out that the most relevant unexplored issues are related to the lack of spontaneous discoverability [6] of such interfaces and the lack of adequate

* Partially supported by MIUR PRIN 2017 grant number 2017JMHK4F004, and MIUR 2019 grant number AIM1872991-2
feedback [5,11]. When Artificial Intelligence (AI) takes the orchestration of Smart Environment devices, this results into Ambient Intelligence (AmI). AmI should satisfy human requests by piloting the devices in a context-aware manner [4]. The context, in this case, refers to both the environment and user circumstances. For example, a context could depend on smart devices availability, environmental conditions, and user position. Thus, the need to let AmI understand the user context and requirements demands for new interactive Human-Computer Interfaces (HCIs). The idea and architecture of an utterly tactile interface device that interacts with an AmI were already presented in [2]. The proposed device, named TactCube, promotes a tactile-only interface that could allow human interaction with AmI without visual or audio means. Furthermore, an effort was made to design a small wireless cube-shaped hardware device that could be used with one hand only and to be kept in a pocket. This work aims to investigate the modalities of admissible interactions by users with AmI through a tangible mobile interface. The TactCube device was initially introduced to users, and three smart devices (smart bulb, speaker and curtain) were selected to conduct a guessability study. Guessability studies have been widely used to define sets of touch or motion gestures on various types of devices equipped with multitouch screens, both large-screened devices and common smartphones [7,9]. For each device, the study allowed us to investigate users' preferences among the set of admissible TactCube interactions to be mapped onto a set of device behaviors governed by an AmI system. Finally, the analysis of the responses led us to define a set of design implications that designers should consider when developing tangible mobile interfaces for ambient intelligence systems.

## 2 Interactive Cube

The idea behind TactCube was to design a fully tactile interface, helping users to communicate with AmI avoiding video and audio media. For the sake of brevity, we do not report here all the details on the TactCube prototype implementation, for which we address the reader to [2], and instead focus on what is relevant for the present study. TactCube design aims to offer a non-intrusive tangible interface, which let user communicate with an AmI system his/her contextual needs through simple gestures, just holding the small device in one hand. A users may start the interaction holding the cube with the fingers, as in Figure 1, and may perform the following 10 gestures: 1. a single tap of index finger on one of cube faces; 2. a double tap of index finger on one of cube face; 3. the forward rotation followed by a single tap; 4. the forward rotation followed by a double tap; 5. the backward rotation followed by a single tap; 6. the backward rotation followed by a double tap; 7. the right rotation (clockwise) followed by a single tap; 8. the right rotation followed by a double tap; 9. the left rotation (anticlockwise) followed by a single tap; 10. left rotation followed by a double tap.

Besides these ten gestures, five types of haptic feedback are also admissible, namely: 1. one short vibration; 2. two short vibrations; 3. a combination of short and long vibrations 4. a face of the device warming up; 5. a face of the device cooling down.
TactCube deploys WiFi connectivity with AmI and a Bluetooth Low Energy (BLE) solution that unveils the smart devices in proximity, helping the AmI system to recognize the user position inside the environment. TactCube communicates to the AmI contextual information about the users to pilot the smart devices and support their needs. As an example, if the user is turning the cube forward in a dark room, and AmI is adequately trained, by the context and the position of the user, it may infer that more light is needed and then switch the lamps on. If the same action was made in the proximity of a curtain and the light was already on, given the different context and user's position, the AmI system may infer that the request was related to the desire to open the curtain, instead.

<figure><img src="image_3.png" /><figcaption>Fig. 1. The TactCube prototype and a user interacting with it.</figcaption></figure>

# 3 The user study

The idea behind TactCube was to design a fully tactile interface, helping users to communicate with AmI avoiding TactCube provides users with multisensorial feedback involving touch, vibrations and temperature changing and a large number of motion and touch interactions to be mapped onto the commands. In our study, we investigate on the one hand which are users' favorite TactCube interactions for precise commands and feedback in a smart environment and, on the other, if such preferences may depend on the kind of smart device under control. To do so, we set up a mixed-design experiment performed with 60 participants equally distributed into 3 different groups. We considered 2 factors that are respectively: Factor 1 the 12 Tasks as a within-subject variable, and Factor 2 the 3 devices as a between-subject variable. Participants were selected from a group of 130 volunteers IT students who had passed one of the computer science exams dealing with the discipline of human computer interaction, at the authors' university. Based on a short questionnaire, we were able to select the 60 students, who best grasped the goal of the study, all of them being technologically skilled enough to interact with a tangible device like TactCube without distorting the experimental results. The chosen number of subjects was suitable to get preliminary insights and in line with similar experiments present in the scientific literature [10,9,12]. We divided our subject participants into three subgroups, properly Gr1, Gr2 and Gr3. To each subgroup we assigned a different device to interact with using TactCube - precisely a smart bulb, a smart speaker and a smart curtain, which are common everyday devices. Table 1 shows 20 guessability tasks for each device, including 12 navigation/action items and 8 feedback items. The tasks are split into three categories, specifically, *Action*, *Navigation* and *Feedback*. The items of each category are distributed into two sub-categories: AmI Environmental system and Smart Device. Then, we assign a label
to each item depending on its category and subcategory, precisely EA and DA for Action, EN and DN for Navigation, EF and DF for Feedback. The categories are adapted from the work presented in [10] which considers the generic tasks in a smartphone system environment. Apart from the categories Action and Navigation, we also added the category Feedback originally missing in [10] but necessary, as recommended also in [10], considering that also the feedback must be mapped correctly in the system to be interpreted by users. For each task, the participants were required to propose a specific gesture or feedback to associate to the prototype chosen from the 10 basic available gestures and the 5 kinds of feedback. The experiment took place in a controlled environment, that is the usability research lab of the University of XXXX, the participants were scheduled all along three days and only one participant at a time could access the laboratory, also due to Covid-19 restrictions. A workstation was set up for the participants, then, they were asked to sit and hold the TactCube device and interact with it. When the experiment session started, an application running in a computer desktop showed for each task the specific device behavior and asked for an interaction preference. Figure 1 shows a user during the experiment. To better evaluate the potential and rationale behind the specific interaction choices, a “think-aloud” technique was used. Moreover, users were photographed and recorded, in order to allow the researchers a systematic analysis of the experiment. At the end of the experiment, we asked participants to discuss any possible suggestion or issue. Each session lasted on average 30 minutes. During it, the authors took notes to discuss at the end of the session and of the whole experiment.

<table><caption>Table 1. The guessability tasks for the smart bulb, speaker and curtain.</caption><thead><tr><th>ID</th><th>Interacting with a smart bulb</th><th>Interacting with smart speaker</th><th>Interacting with a smart Curtain</th></tr></thead><tbody><tr><td colspan="4"><strong>Action</strong></td></tr><tr><td>EA1</td><td>Access to the environment</td><td>Access to the environment</td><td>Access to the environment</td></tr><tr><td>DA1</td><td>Turn on the light</td><td>turn on the speaker</td><td>Turn on the curtain</td></tr><tr><td>DA2</td><td>turn off the light</td><td>turn off the speaker</td><td>turn off the curtain</td></tr><tr><td>DA3</td><td>Stop/play</td><td>Stop/play</td><td>Stop/play</td></tr><tr><td colspan="4"><strong>Navigation</strong></td></tr><tr><td>EN1</td><td>select the smart bulb</td><td>select the smart speaker</td><td>select the curtain</td></tr><tr><td>EN2</td><td>deselect the smart bulb</td><td>deselect the smart speaker</td><td>deselect the curtain</td></tr><tr><td>DN1</td><td>increase the intensity of the light</td><td>increase the speaker volume</td><td>increases the amount of light that can filter</td></tr><tr><td>DN2</td><td>decrease the intensity of the light</td><td>decrease the speaker volume</td><td>decrease the amount of light that can filter</td></tr><tr><td>DN3</td><td>change the color of the light</td><td>change audio preset</td><td>change fissure preset</td></tr><tr><td>DN4</td><td>select a behavioural program<br>(the bulb simulates the sunrise, sunset...)</td><td>select playlist type<br>(the speaker plays rock, pop, classic ...)</td><td>select a behavioural program<br>(the curtain follows the movement of the sunrise, sunset...)</td></tr><tr><td>DN5</td><td>select the switch-on time</td><td>select the switch-on time</td><td>select the switch-on time</td></tr><tr><td>DN6</td><td>select the shutdown time</td><td>select the shutdown time</td><td>select the shutdown time</td></tr><tr><td colspan="4"><strong>Feedback</strong></td></tr><tr><td>EF1</td><td>Presence of intelligent environment</td><td>Presence of intelligent environment</td><td>Presence of intelligent environment</td></tr><tr><td>DF1</td><td>Device announcement</td><td>Device announcement</td><td>Device announcement</td></tr><tr><td>DF2</td><td>Device is not ready</td><td>Device is not ready</td><td>Device is not ready</td></tr><tr><td>DF3</td><td>Device is on</td><td>Device is on</td><td>Device is on</td></tr><tr><td>DF4</td><td>Device is off</td><td>Device is off</td><td>Device is off</td></tr><tr><td>DF5</td><td>Device is loading</td><td>Device is loading</td><td>Device is loading</td></tr><tr><td>DF6</td><td>action successfully accomplished</td><td>action successfully accomplished</td><td>action successfully accomplished</td></tr><tr><td>DF7</td><td>Device generated an error</td><td>Device generated an error</td><td>Device generated an error</td></tr></tbody></table>
## 4 Analysing the results

In the experiment, we involved 60 participants divided into three distinct groups. We asked each user to propose one interaction on the prototype for 12 tasks and select one kind of feedback from those eligible for further 8 tasks. Each group was also assigned to a distinct scenario in which to execute the tasks with a different smart device, specifically a bulb, a speaker, and a curtain. To evaluate the degree of consensus among participants we adopted the process of calculating an agreement score for each task [12,13]. An agreement score, $A_t$, reflects in a single number the degree of consensus among participants. Wobbrock [12] provides a mathematical calculation for agreement, where: $A_t = \sum_{i} \left( \frac{P_i}{P_t} \right)^2$. In equation, $t$ is a task in the set of all tasks $T$, $P_t$ is the set of proposed gestures for $t$, and $P_i$ is a subset of identical gestures from $P_t$. The range for $A$ is [0, 1]. Table 2 shows the agreement score reached by groups for each action and in relation to the specific smart device.

<table><caption>**Table 2.** Agreement scores for each item and group</caption><thead><tr><th></th><th>EA1</th><th>DA1</th><th>DA2</th><th>DA3</th><th>EN1</th><th>EN2</th><th>DN1</th><th>DN2</th><th>DN3</th><th>DN4</th><th>DN5</th><th>DN6</th><th>EF1</th><th>DF1</th><th>DF2</th><th>DF3</th><th>DF4</th><th>DF5</th><th>DF6</th><th>DF7</th></tr></thead><tbody><tr><th>Smart Bulb</th><td>0,55</td><td>0,49</td><td>0,50</td><td>0,40</td><td>0,63</td><td>0,31</td><td>0,22</td><td>0,26</td><td>0,11</td><td>0,14</td><td>0,15</td><td>0,60</td><td>0,52</td><td>0,52</td><td>0,38</td><td>0,46</td><td>0,75</td><td>0,66</td><td>0,50</td><td>0,91</td></tr><tr><th>Smart Speaker</th><td>0,41</td><td>0,42</td><td>0,46</td><td>0,47</td><td>0,44</td><td>0,30</td><td>0,29</td><td>0,33</td><td>0,19</td><td>0,26</td><td>0,14</td><td>0,68</td><td>0,58</td><td>0,50</td><td>0,46</td><td>0,51</td><td>0,66</td><td>0,61</td><td>0,51</td><td>0,74</td></tr><tr><th>Smart Curtain</th><td>0,49</td><td>0,36</td><td>0,36</td><td>0,43</td><td>0,42</td><td>0,34</td><td>0,20</td><td>0,19</td><td>0,12</td><td>0,16</td><td>0,19</td><td>0,40</td><td>0,55</td><td>0,55</td><td>0,28</td><td>0,46</td><td>0,74</td><td>0,59</td><td>0,50</td><td>0,82</td></tr></tbody></table>

A consideration that can already be made is that in all the proposed tasks a minimum consensus has always been found by all the groups, indeed the agreement score is always greater than 0.1. Anyway, five tasks related to *Actions* and *Navigation* achieved the highest degrees of consensus, which are EA1, DA1, DA2, DA3, EN1 and DN6 in Table 2. Indeed, their degrees of consensus are higher than 0.4. This means that in a group, more than 40% of the proposals were for a given interaction on average. On the other side, the agreement scores for the 8 feedback items are generally higher. In particular, EF1, DF1, DF3, DF4, DF5, DF6 and DF7 are higher or close to 0.5 as shown in Table 2. Figure 2 shows a graphical comparison of the choices made by the three groups for each task in the *Action* and *Navigation* categories. Regarding the first category, EA1 is about taking action to access an intelligent environment. In this case, the proposals by users of all groups focused almost exclusively on two actions, single tap and double tap with a small predominance for the latter. In particular, double tap was selected by all the users 37 times and single tap was selected 20 times. This indicates an almost total consensus among the three groups for actions that involve only the tap, excluding actions that involve rotations of the cube. DA1 concerns the switching on of a smart device. Gr1 and Gr2 had a similar behavior proposing mostly double tap and single tap. On the other hand, although Gr3 participants proposed a certain number of both double and single tap, they slightly preferred the right rotation followed by a single tap. Similarly, for DA2, which is about turning off a smart device, double and single tap are generally predominant preferences, however, Gr3 slightly preferred left rotation followed by a single tap. Some of the users across all groups explained their interaction preference commenting it. Many justified the tap or double tap because
they suggest the action of turning an electronic device on using a conventional but-
ton, however it should be noted that double tap has often been the predominant
choice. A participant from Gr1 explained: “I chose the double tap because the tap
reminds me of the click on the light button, but the double tap gives me the certainty
of not pressing it by mistake”. Some participants from Gr3 chose right or left rota-
tion because this recalls the gesture of turning a crank with which to open or close
a shutter or a Venetian blind. This suggests that in order to turn on and off a smart
device both tap, and double tap are interactions generally understandable by all the
users regardless of the type of device, but certain devices can also suggest a more
specific interaction connected to their particular affordance, as in the case of the
smart curtain. DA3 is related to the command stop, play or act on a program of a
smart device. As in the previous cases, there is a predominance in the use of tap,
although this time among the three groups the most common choice was the single
tap. However, the predominant choice relative to Gr1 is the forward rotation fol-
lowed by a single tap. In this case, one participant explained that the rotation sug-
gests the action of starting the car rotating the key, and another that it is linked to
the idea of a two-way switch. Regarding the Navigation category, only EN1 and
DN6 showed significant agreement where recurrent interaction patterns can be ap-
preciated. As regards to the EN1 item, for all the groups, the two most proposed
actions are right rotation to the right followed by a single tap, and double tap. The
rotation was proposed 34 times and the double tap 21 times. Then, both the rotation
and the double tap are interactions considered understandable by the three groups
regardless of the target smart device. In choosing this gesture, some users imagined
moving through a wheel control on a menu to select the desired device. Participants

<figure><img src="image_4.png" /><figcaption>Fig. 2. Comparison of preferences in the Action and Navigation categories.</figcaption></figure>

who used the double tap or the tap stated that they were imagining choosing the
device with a "click" being close to it. Finally, in DN6, related to the programmed
shutdown, the combination of backward rotation and double tap is the main prefer-
ence, chosen 41 times by all.

Concerning the tasks involving the preference of the type of feedback, the degree
of consensus among the participants of the different groups is particularly high and
homogeneous among the 3 groups. Figure 3 shows a comparison of the users’ pref-
erences for each item in the Feedback category. More specifically, DF7, which is
related to the feedback of a device error, received a very high level of consensus
among all groups on a combination of short and long vibrations. Indeed, 54 users indicated a sequence of vibrations as it suggests "*a shock*", "*an impact*" or "*a movement that cannot be completed*". Regarding the feedback of a successful action (DF6) the participants of the three groups, equally distributed, chose 31 times a short vibration and 29 times two short vibrations. Users explained their choice by associating it with the behavior of their phone. When a task ends, the phone shortly vibrates one or more times or makes a short beep. DF5 is the feedback status "loading". In this case, the choices of the participants were polarized on the face of the cube warming up. One participant said: "*In an electronic device the heating is often a clue that it is working*". Another added: "*an object that is working may heat up slightly*". On the other hand, regarding DF4, which is related to a device switching off, a majority of 48 participants proposed the cube face cooling down. Then, a smart device cooling down is perceived by the users as a device that stops working. Similar considerations can be made for DF3, which is the feedback related to the device switching on. The participants' preferences are distributed almost halfway between a short vibration and the face of the cube warming up. Also, in this case the warming suggests to users the idea of a device that gives off heat because it

<figure><img src="image_5.png" /><figcaption>Fig. 3. Comparison of preferences in the Feedback category.</figcaption></figure>

works, while the vibration suggests the correctness of an action or a state, due to previous experiences with mobile phones. Finally, DF1 and EF1, which refer to the feedback connected to the spontaneous discoverability of the smart device and the intelligent ambient, polarize the preferences of all groups around one short vibration or two short vibrations. Moreover, to understand whether the user preferences for each task is directly linked to the device chosen for the scenario, we ran the Kruskal-Wallis one-way ANOVA by ranks [8] on the participants' preferences expressed on each task. This analysis is a non-parametric test used to compare three or more sets of data and can be considered when the samples are independent. The association between two variables is statistically significant if the resulting P-value is less than 0.05. In our experiment, the test was executed in the IBM SPSS Statistics software. Except for DN4 (≈ 0.00), all the P-values calculated by the statistical method are higher than 0.05. Moreover, the DN4 item can be questionable, indeed the users' preferences are variegated. This means that, even if many users' comments suggest a link between the smart device form and the interaction preference,
statistically no evidence proves it, and more investigation is needed to understand
such potential links.

**5 Design implications and final remarks**

The guessability study presented here leads to implications that can be useful when
designing tangible interfaces for devices interacting with the AmI.

a) **Design the interactions to be meaningful and consistent with everyday life experience of your users.** Even if some preferences are suggested to users by the affordance of the controlled smart device, mainly the preferences are led by other factors such as previous experiences in other contexts such as mobile phones, or common buttons as light switches and the affordance of the device controller. An example is given by the results in the category *Action*: the interactions preferences for tasks such as turning on or off, and play can be related to the form factor of the smart device that suggests specific manipulations, although generally single or double taps are commonly understood by all users for this kind of tasks.

b) **Use the double tap without combination or the single tap if you are combining the tap with other gestures.** In combination with rotation, the single tap is the gesture chosen more frequently, nevertheless, when the gesture is not combined, double-tapping can be preferred as it gives users the certainty of not performing it accidentally.

c) **Use rotations to navigate the device parameters and settings.** In the *Navigation* category the main preferences are given to rotation even if in different directions and in different combinations with the tap, then it is not easy to find a recurrent pattern. Anyway, right rotation appears as a consistent preference for selecting a smart device in the environment, forward rotation can be indicated when increasing a device parameter, and backward rotation for programming shutdown time.

d) **Consider tactile feedback as strongly related to the real world.** In our experiment, the types of possible feedback are transversal to the various smart devices and are linked to behaviors related to the real world. **d1.** As in mobile phones, a short vibration or two short vibrations express something positive, such as the correctness of an action, a functioning state of the device, or also the discovery of a close smart device. **d2.** Heat perception is connected to something running on the device, and therefore to the switching on of the device or to a loading state. **d3.** Cold suggests that the device is not working and therefore can be used to indicate its shutdown. **d4.** Combinations of long vibrations are interpreted by users as something problematic and thus adapt to warn users.

The above implications should be used to design interactions that are truly meaningful to users. However, this study has some limitations to consider. It was developed within a static controlled environment and involving a limited number of smart devices and users. To refine and confirm our outcomes, it will be necessary to introduce the TactCube in the wild through a field study involving a larger number of devices and different environments. Finally, in the present work we explored only the 10 basic gestures provided by the TactCube prototype, while we plan in the future to consider a memorability study to show if a larger set of gestures can benefit or not the user interaction experience with such device.
References

1. Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y., Kankanhalli, M.: Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 1–18. CHI ’18, Association for Computing Machinery, New York, NY, USA (2018)

2. Battistoni P. and Sebillo M., "A Tactile User Device to Interact with Smart Environments," in Artificial Intelligence in HCI, Virtual, vol. 39. Available: https://doi.org/10.1007/978-3-030-77772-2_30.

3. Bilgin, A., Hagras, H., Malibari, A., Alghazzawi, D., Mohammed, J.: A computing with words framework for ambient intelligence. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics. pp. 2887–2892 (2013). https://doi.org/10.1109/SMC.2013.492

4. Cook, D.J., Augusto, J.C., Jakkula, V.R.: Ambient intelligence: Technologies, applications, and opportunities. Pervasive and Mobile Computing 5(4), 277 – 298 (2009)

5. Djajadiningrat, T., Wensveen, S., Frens, J., Overbeeke, K.: Tangible products: Re-dressing the balance between appearance and action. Personal Ubiquitous Comput. 8(5), 294–309 (Sep 2004)

6. Gellersen, H., Fischer, C., Guinard, D., Gostner, R., Kortuem, G., Kray, C., Rukzio, E., Streng, S.: Supporting device discovery and spontaneous interaction with spatial references. Personal and Ubiquitous Computing 13(4), 255–264 (2009)

7. Landua, S., Wells, L.: The merging of tactile sensory input and audio data by means of the talking tactile tablet. In: Understanding Touch and Motion Gestures for Blind People on Mobile Devices. vol. 60, pp. 291–297. EuroHaptics (2003)

8. Lazar, J., Feng, J.H., Hochheiser, H.: Research methods in human-computer interaction. Morgan Kaufmann (2017)

9. Romano, M., Bellucci, A., Aedo, I.: Understanding touch and motion gestures for blind people on mobile devices. In: Abascal, J., Barbosa, S., Fetter, M., Gross, T., Palanque, P., Winckler, M. (eds.) Human-Computer Interaction – INTERACT 2015. pp. 38–46. Springer International Publishing, Cham (2015)

10. Ruiz, J., Li, Y., Lank, E.: User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 197–206. CHI ’11, Association for Computing Machinery, New York, NY, USA (2011)

11. Wensveen, S., Djajadiningrat, J., Overbeeke, C.: Interaction frogger : a design framework to couple action and function through feedback and feedforward. In: Benyon, D. (ed.) Across the spectrum : designing interactive systems ; DIS 2004, Cambridge, Massachusetts, 1-4 August 2004. pp. 177–184. Association for Computing Machinery, Inc, United States (2004)

12. Wobbrock, J.O., Aung, H.H., Rothrock, B., Myers, B.A.: Maximizing the guessability of symbolic input. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems. pp. 1869–1872. CHI EA ’05, Association for Computing Machinery, New York, NY, USA (2005)

13. Wobbrock, J.O., Morris, M.R., Wilson, A.D.: User-defined gestures for surface computing. In: CHI ’09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1083–1092. CHI ’09, Association for Computing Machinery, New York, NY, USA (2009)