<figure><img src="image_1.png" /></figure>

HAL
open science

# Tilt Space: A Systematic Exploration of Mobile Tilt for Design Purpose

Chuanyi Liu, Ningning Wu, Jiali Zhang, Wei Su

► To cite this version:

Chuanyi Liu, Ningning Wu, Jiali Zhang, Wei Su. Tilt Space: A Systematic Exploration of Mobile Tilt for Design Purpose. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.497-517, 10.1007/978-3-030-29387-1_29. hal-02553860

HAL Id: hal-02553860

https://inria.hal.science/hal-02553860

Submitted on 24 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Tilt Space: A Systematic Exploration of Mobile Tilt for Design Purpose

Chuanyi Liu[0000-0001-8237-8979], Ningning Wu, Jiali Zhang, and Wei Su

School of Information Science & Engineering, Lanzhou University, Lanzhou, China
liuchuanyi96@hotmail.com, {wunn16, suwei}@lzu.edu.cn, jennyfocus@163.com

**Abstract.** Various application scenarios of a smartphone sometimes require one-handed and/or eyes-free interaction. Tilt-based interfaces have the potential to meet these requirements. Taking multiple application scenarios into account, we conducted an experiment to systematically investigate human ability in controlling tilt input of a mobile phone. Three visual feedback levels, i.e., fully visual feedback (FV), partially visual feedback (PV), and no visual feedback (NV), were investigated. Under the *NV* condition, the participants performed a task using an eyes-free method. The results revealed that trials were performed the fastest but were the most error-prone under the *NV* condition. The participants could easily distinguish 4 tilt orientation levels (*TOLs*) and 2 tilt magnitude levels (*TMLs*) or 8 *TOLs* and 2 *TMLs* under the *NV* condition with tolerance of an error rate 10% or 15%, respectively. We also found out that the participants' abilities to control tilt input were related to tilt *orientation* directions. The results have some implications for non-visual interface designs using tilt as primitive input.

**Keywords:** Mobile · Tilt · Design space.
# 1 Introduction

Smartphones are essential devices for us. We use them in our work, study, and life every day. A touch screen is its basic interactive component of a smartphone. Touch screens provide us simple and intuitive interactive methods, however, coupled with some limitations, e.g., “fat finger” [31] and hand occlusion [36]. Besides these limitations, other issues can be caused by various application scenarios of smartphones. When walking on the road, we may risk a traffic accident watching the screen of a smartphone. An eyes-free interaction method is more suitable under these conditions. Sometimes, not both of one’s hands are available to manipulate a mobile phone, e.g., when s/he is taking a bus or holding a bag in one hand. But one-handed thumb manipulation of a touchscreen-based mobile device is typically awkward, since one-handed thumb interaction limits the accessible area on the screen [16].

It is worth resolving these issues of a smartphone, especially for some conditions, e.g., on a bus or the road. Built-in tilt sensors provide an independent input channel for portable devices. Tilt gestures map 3D angles of a portable device as primitive input with interaction commands. Tilt gestures have the potential to resolve these issues, since we can feel a mobile phone’s posture by our proprioception without watching the screen, and operate the mobile phone with an eyes-free method and unimually.

It has long been interests to many researchers exploring tilt as primitive input for interfaces. Since Rekimoto [27] used the tilt of a small screen device as input, researchers have developed a variety of tilt-based interaction techniques. Early research on tilt focused on using tilt as an additional input channel for mobile devices to implement specific interactive techniques, such as menu navigation [27], scrolling [10, 22], panning and zooming of maps [17, 35], document and photo browsing [4, 25], and text entry [23, 38, 29]. However, these studies mainly focused on what can be done with the tilt input channel. Few studies considered human ability on controlling mobile tilt input and the design space of tilt-based interaction, especially with an eyes-free method.

In the study, we systematically investigated human ability in controlling tilt input of a mobile phone. The major objective of our research is to determine effects of different visual feedback levels on human ability in controlling tilt input and the upper bound of the human ability, especially under the NV condition.

# 2 Related Work

Tilt-based interaction has long been of interests to researchers of HCI. These literature can be divided roughly into two groups: studies on general human performance of tilt control and on specific implementation examples using tilt input.
## 2.1 General Human Performance of Tilt Control

Teather and Mackenzie [20] proved tilt conformed to Fitts' law [7] through ISO 9241-9 [14] 2D pointing task. Later, the authors extended their work to compare position- and velocity-control for tilt-based interaction with the similar 2D pointing task [34]. They found out that position-control performed approximately 2 times faster than velocity-control and had higher pointing throughput. Wang et al. [37] also studied tilt control from viewpoint of Fitts' law; they didn't use an accelerometer for tilt input but vision-based motion tracking. Sad and Poirier [28] confirmed tilt-based scrolling and pointing tasks conformed to Fitts' law without reporting pointing throughput.

Rahman et al. [26] systematically investigated human ability of tilt control according to the human wrist movement. They found out the subjects could control comfortably at least 16 levels on the supination/pronation axis. Baglioni et al. [1] explored human control ability on tilt input with a jerk gesture. Guo and Paek [9] studied human control ability of tilt on smartwatches, and found out that *ObjectPoint* performed better. Shima et al. [30] evaluated the performance of tilting operations on wrist-worn devices.

All these literature offered valuable cues for tilt-based interaction design, however, none of them had taken eyes-free method into account (to our knowledge). In our study, we explored tilt control according to different visual feedback levels, especially the eyes-free operation method was considered.

## 2.2 Specific Implementation Examples

Some researchers studied tilt-based text entry [5, 15, 23, 29, 38, 39]. These studies typically divided letters into groups and disambiguated letter selection through device tilt. Various UI tasks, such as scrolling [2, 10, 22, 32], document browsing [6], menu navigation [27], and display orientation switch [11], using tilt control were investigated. Pietroszek et al. [24] utilized a mobile device to perform 3D interaction on large displays. Geronimo et al. [8] also focused their study on controlling separated displays using mobile devices. They designed a tilting interaction framework for rapid developing web-based applications. Homaeian et al. [12] explored tilt input for data browsing techniques in cross-device environments, and found that tilt facilitated access to out-of-reach data. Kurosawa et al. [18] presented combination of a tilt operation and electromyography on smart watches using tilt to set the cursor's motion direction. Sun et al. [33] proposed a wrist-to-finger input approach that enabled one-handed and touch-free target selection on smart watches.

Tilt-based mobile games [3, 13, 21, 40] have been popular for years. Typically, using tilt could not improve interaction performance in these games, but tilt interaction gave users more challenges and fun.

# 3 Experiment

We base our experiment on an understanding of the mobile device application contexts. Figure 1 shows the three axes of rotation of a smartphone and the tilt
motion space. The built-in tilt sensor can be used to detect the pitch and roll orientations and magnitude of a mobile device by analyzing the projection of the constant gravity acceleration on the three axes. But tilt sensor does not allow the determination of the static yaw orientation [11].

However, the range of motion provides by the tilt sensor may not always be conducive to tilt control due to the ergonomic limitations of human wrists and the difficulty in obtaining visual feedback in “extreme” angles (e.g., when a device’s screen is turned leftward or rightward from a horizontal posture to vertical). But under the eyes-free condition, users can control the tilt by proprioception and memory without relying on visual feedback; arms can be used besides wrists to enlarge the tilt space. In other words, the visual and ergonomic limitations are alleviated under the eyes-free condition.

<figure><img src="image_3.png" /><figcaption>Fig. 1: The three axes of the device rotations and the three corresponding angles.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2: The tilt space is divided into 4, 6, 8, and 12 TOLs (from left to right) together with 2 TMLs.</figcaption></figure>

## 3.1 Apparatus

The experiment was conducted using a Huawei honor 5c smartphone with a built-in 3-axis accelerometer running Google's Android 6.0 operating system. The display resolution was 1920 × 1080 pixels. In addition, a Lenovo YT3-X50F 10.1-inch tablet with android 5.1.1 OS was used to inform the user a target under the no-visual-feedback condition.

The experimental software was developed in Java using the Android SDK. The accelerometer was used to detect tilt magnitude and tilt orientation of the smartphone, sampling at a rate of 50 samples per second.
## 3.2 Participants

Twelve volunteers, 10 males and 2 females, ranging in age from 22 to 32 years (Mean = 25.3, SD = 3.8), participated in the experiment. All the participants were recruited from the computer department in the local university, and they were all familiar with manipulation of smartphones and right-handed. All of them had normal or corrected to normal vision. Although none of the participants had ever used tilt to perform subtle pointing or selection tasks before, they all had prior experience with tilt-based interfaces such as gravity sensor games.

## 3.3 Task and Stimuli

The human control of tilt on handheld devices was decomposed into the control of tilt orientation and the control of tilt magnitude. The tilt orientation (0° to 360°) was linearly divided into four levels: 4, 6, 8, and 12 with angular intervals of 90°, 60°, 45°, and 30°, respectively. The motion space of tilt magnitude (0° to 90°) was linearly divided into three levels: 2, 3, and 4 with angular intervals of 45°, 30°, and 22.5°, respectively. A circle, standing for the tilt space, was divided into *m* (the number of tilt orientation levels, *TOLs*, tilt orientation hereafter referred to as *TO*) × *n* (the number of tilt magnitude levels, *TMLs*, tilt magnitude hereafter referred to as *TM*) districts (e.g., 4*TOLs* × 2*TMLs* = 8 districts, see Fig. 2 left). One of these districts was randomly (across trials) selected to be a target. There were three visual feedback levels (visual feedback and its level hereafter referred to as *VF* and *VFL*, respectively, see Fig. 3): fully visual (*FV*), partially visual (*PV*), and no visual (*NV*).

<figure><img src="image_5.png" /><figcaption>Fig. 3: Different visual feedback conditions. *FV*: a target district was filled in green and a district holding the pointer in gray, the blue pointer is always visible; *PV* (the dashed lines were not shown in the experiment): a target district was filled in green, the blue pointer disappears once tilting began; *NV*: districts were shown statically during a trial in the tablet PC.</figcaption></figure>
Under the *FV* and *PV* conditions, the cursor was displayed as a blue pointer, which was painted from the center of the device's display (i.e., the center of the circle, denoted as *C*). Tilt magnitude and tilt orientation were computed to the pointer's length and orientation, respectively. The tilt magnitude of device was linearly mapped to the pointer's length which varied between 0 and the maximum (the radius length of the largest circle) when a participant tilted the phone from horizontal to vertical. Under the *NV* condition, the pointer dimensions were calculated using the same method, but the pointer had no visual feedback (i.e., the pointer was hidden).

To confirm selection of a target, a participant manipulated tilt input to adjust the pointer's orientation and length to make it enter the target district, and kept the pointer in the district over the time threshold. The threshold was set to be one second vs. the largest selection delay, 500ms, in [34]: the dwell time was intentionally doubled to discriminate target selection from unintentional entry and disambiguate target selection especially for *NV* condition. Similarly, to minimize unintentional selection and reduce selection difficulty, minute tilt magnitude variation (within 5°) was also eliminated from target selection. A non-target district was determined whether selected using the same method.

Under *FV* condition (Fig. 3a), visual feedback was continuous during the whole process of a trial. All the districts were painted with gray-lined borders; a target district was filled in green, while the current district (holding the pointer) in gray. The pointer was painted in blue. When being selected, a target district was filled in red, while a non-target in yellow.

Under *PV* condition (Fig. 3b), only a target district was continuously displayed and the pointer was visible only at the beginning of a trial. Once tilting began, the pointer disappeared. A participant manipulated the hidden pointer to meet the target upon their proprioception and spatial perception and cognition. Once the pointer entered the target, the target color changed from green to gray. When the trial was performed correctly, the target was colored in red; otherwise, the selected non-target district became visible and was filled in yellow.

There was no visual feedback under *NV* condition (Fig. 3c), i.e., the participants performed trials using an eyes-free method. A target was displayed on the tablet screen using a *FV* method except the pointer, but the surface was static during a trial (Fig. 4b). The tablet was connected to the smartphone via Blue-tooth. The trial sequence in the tablet was synchronized with that in the mobile phone. The participants selected a target completely upon their proprioception, kinesthetic memory, and spatial perception and cognition. This scenario was similar to the expert mode of the Marking Menus [19].

Under any condition, a pleasure or alarm sound was played when a district was selected correctly or incorrectly, respectively.

## 3.4 Procedure and Design

The experiment used a within-subjects full factorial design. The experimental factors and their levels were:
<figure><img src="image_6.png" /><figcaption>Fig. 4: A participant was performing experimental trials.</figcaption></figure>

VF: FV, PV, NV.

TO: 4, 6, 8, 12.

TM: 2, 3, 4.

Before the experiment, each of the participants had been told they had the right to freely quit the experiment at any time. The participants were allowed to have a rest between any two experimental conditions, and they had been told they were not allowed to do the experiment when they felt tired. A longer rest between experimental blocks was mandatory.

After given a detailed description about the task, participants spent a few minutes in familiarizing themselves with tilt control and the three visual feedback conditions. The participants performed the experiment seated, and all of them held the device with their right hands (see Fig. 4).

In the formal experiment, the participants were required to complete three blocks of trials. A Latin square was used to counterbalance the order efforts of the three *VFLs*. For each feedback condition, the participants performed target selection beginning from *TOL4* and *TML2*. The rationale for the sequential design is that interaction performance does reduce significantly with the increase of levels of the two factors according to the previous work [26] and our observation. So the major objective of this study is not to determine whether there exists significantly different effects between these factor levels on interaction performance measures but to find out the upper bound of human ability to control tilt input according to the two experimental factors. The target appeared randomly among all the districts with 2 repetitions at the same location. The *TOLs* and the *TMLs* were always presented in ascending (smallest to largest) order for easy learning. When a wrong selection happened, a failed attempt was recorded. Not was the experiment proceeded to next trial until a trial was done correctly. Participants were allowed to try for no more than 5 times for each trial before success, otherwise, the trial was aborted. Totally, there were:

12 *participants* ×

3 *blocks* ×
$$ 
\begin{aligned}
& 3 \text{ VFLs} \times \\
& (4 + 6 + 8 + 12) \text{ TOLs} \times \\
& (2 + 3 + 4) \text{ TMLs} \times \\
& \text{2 repetitions} \\
& = 58320 \text{ target selection trials.}
\end{aligned}
$$

The participants were instructed to complete each trial “as quickly and accurately as possible”. After performing all experimental trials, the participants were investigated for their subjective comments on the three VFLs with a questionnaire where they rated the usability, ease of learning, hand and eye fatigue along a Likert scale (each aspect with seven rating levels, where 1 represented the worst and 7 the best). In total, the experiment lasted approximately 3 hours for each participant.

## 3.5 Performance Measures

The experimental measures were selection time (*ST*), error rate (*ER*, there was ONE error when participants failed at their first attempt), number of aborted trials (*AT*, i.e., number of trials that had failed for 5 times), and number of crossings (*NC*, the times of the cursor enters a target without selecting it, i.e., when dwell time in the target was less than the given time threshold). *ST* was defined as the time consumed from tilting the phone to selecting a target. *NC* was used to reveal the difficulty degrees to dwell the pointer in a target district, it was 1 in a perfect selection task. In the next section, these measures were calculated to averages per trial from the primitive experimental records.

<table><caption>Table 1: Statistical effects for performance measures.</caption><thead><tr><th colspan="2">Effect</th><th colspan="3">ST</th><th colspan="3">ER</th><th colspan="3">AT</th><th colspan="3">NC</th></tr><tr><th>Name</th><th>df</th><th>F</th><th>p</th><th>η<sub>p</sub><sup>2</sup></th><th>F</th><th>p</th><th>η<sub>p</sub><sup>2</sup></th><th>F</th><th>p</th><th>η<sub>p</sub><sup>2</sup></th><th>F</th><th>p</th><th>η<sub>p</sub><sup>2</sup></th></tr></thead><tbody><tr><td>block(b)</td><td>2, 22</td><td>33</td><td>.000</td><td>.750</td><td>24.3</td><td>.000</td><td>.689</td><td>0.9</td><td>.44</td><td>.072</td><td>16.8</td><td>.000</td><td>.604</td></tr><tr><td>VFL(v)</td><td>2,22</td><td>119</td><td>.000</td><td>.915</td><td>197.4</td><td>.000</td><td>.947</td><td>13</td><td>.000</td><td>.541</td><td>186.3</td><td>.000</td><td>.944</td></tr><tr><td>TOL(o)</td><td>3,33</td><td>95.6</td><td>.000</td><td>.897</td><td>45.3</td><td>.000</td><td>.804</td><td>12.7</td><td>.000</td><td>.535</td><td>7.7</td><td>.001</td><td>.411</td></tr><tr><td>TML(m)</td><td>2,22</td><td>291.7</td><td>.000</td><td>.964</td><td>62.6</td><td>.000</td><td>.851</td><td>10.7</td><td>.001</td><td>.494</td><td>150.9</td><td>.000</td><td>.932</td></tr><tr><td>b × v</td><td>4,44</td><td>2.3</td><td>.08</td><td>.170</td><td>6.9</td><td>.000</td><td>.385</td><td>0.7</td><td>.57</td><td>.064</td><td>7.8</td><td>.000</td><td>.416</td></tr><tr><td>v × o</td><td>6,66</td><td>8.2</td><td>.000</td><td>.428</td><td>24.4</td><td>.000</td><td>.689</td><td>12.6</td><td>.000</td><td>.533</td><td>2.7</td><td>.022</td><td>.195</td></tr><tr><td>v × m</td><td>4,44</td><td>26</td><td>.000</td><td>.702</td><td>38.9</td><td>.000</td><td>.780</td><td>107</td><td>.000</td><td>.492</td><td>78.7</td><td>.000</td><td>.877</td></tr><tr><td>o × m</td><td>6,66</td><td>1.6</td><td>.17</td><td>.126</td><td>2.5</td><td>.03</td><td>.183</td><td>1.6</td><td>.15</td><td>.130</td><td>0.5</td><td>.79</td><td>.046</td></tr><tr><td>v × o × m</td><td>12,132</td><td>1.1</td><td>.41</td><td>.087</td><td>0.5</td><td>.93</td><td>.042</td><td>1.7</td><td>.07</td><td>.137</td><td>0.9</td><td>.53</td><td>.077</td></tr></tbody></table>

“×” means interaction effects.
# 4 Results

A 3 *blocks* × 3 *VFLs* × 4 *TOLs* × 2 *TMLs* RM-ANOVA was conducted on mean data of *ST,ER,AT*, and *NC*. Statistical reports for the four performance measures are shown in Table 1.

## 4.1 Selection Time

Learning effects were observed from the significant effect of *block* on *ST*. There was no significant interaction effect between *feedback* and *block*, Fig. 5 shows the similar learning effects under different visual feedback conditions.

<figure><img src="image_7.png" /><figcaption>Fig. 5: Average *ST* by *blocks* and *VFLs*.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6: Average *ST* by *TOLs* and *VFLs*.</figcaption></figure>

There was a significant effect of *VF* (*F*(2, 22) = 119.04, *p* < 0.001, *η*<sub>p</sub><sup>2</sup> = 0.915) on *ST*. Pairwise comparisons showed significant differences between all visual feedback levels (at *p* < 0.001). The participants performed the fastest under the *NV* condition, but the most slowly under the *PV* (Fig. 5). Based on our observations, we speculate that this may because, under the *FV* and *PV* conditions, the participants tended to tilt the device from a small angle little by little to a large one and kept watching the visual feedback during a trial. But under the *NV* condition, the participants tended to tilt the device directly to the location they thought to be appropriate based on proprioception, kinesthetic memory, and spatial perception and cognition, since it was impossible to rely on any visual feedback.

Although the participants had more practice before the trials of the subsequent *TOLs* (Fig. 6) and *TMLs* (Fig. 7), tests of within-subjects contrasts revealed general *ST* increases with those of level numbers of the two factors. These trends indicated the significant increase of manipulation difficulty in tilt control when the number of divided districts increased, and the upper bounds of human ability to manipulate tilt input.
<figure><img src="image_9.png" /><figcaption>Fig. 7: Average *ST* by *TMLs* and *VFLs*.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 8: Average *ERs* by *blocks* and *VFLs*.</figcaption></figure>

## 4.2 Accuracy

We explore the effects of experimental factors on accuracy from three aspects: *ER*, *NC*, and *AT*. Statistical results of these metrics are shown in Table 1.

**Error Rate.** A significant effect of *block* on *ER* (*F*(2, 22) = 24.34, *p* < 0.001, *η*<sub>p</sub><sup>2</sup> = 0.689) revealed that the participants improved tilt input ability with limited exercise, as shown in Fig. 8. There was a significant effect of *feedback* on *ER*(*F*(2, 22) = 197.45, *p* < 0.001, *η*<sub>p</sub><sup>2</sup> = 0.947). Pairwise comparisons showed there were significant differences between *VFLs* (at *p* < 0.001). It was the most and least error-prone under the *NV* and the *FV* conditions, respectively. Figure 8 also reveals that there was a significant interaction effect between *block* and *VF*, and *ER* decreased the fastest under the *NV* condition. This indicated that although it was the most error-prone to perform tasks without any visual feedback, the participants improve their ability to control tilt input more quickly with an eyes-free method.

The similar variation trends and human ability upper bounds with *orientation resolution* (Fig. 9) and *magnitude* (Fig. 10) on *MT* were also observed on *ER*. We will further analyze these results in the subsequent subsections.

**Number of Crossings.** *NC* indicates how difficult to dwell the cursor (pointer) in a district. As shown in Table 1, the main and interaction effects (except that of *TO* × *TM* and that of *VF* × *TO* × *TM*) of all the experimental factors had significant impacts on *NC*. Figure 11 shows there was a significant learning effect across the three blocks (*F*(2, 22) = 16.75, *p* < 0.001, *η*<sub>p</sub><sup>2</sup> = 0.604) and a significant *VF* effect (*F*(2, 22) = 186.32, *p* < 0.001, *η*<sub>p</sub><sup>2</sup> = 0.944) on *NC*. A significant interaction effect between *block* and *feedback* is also observed in Fig. 11. *PV* possessed the largest *NC* among the three *VFLs* indicated that partially visual feedback imposed difficulty on keeping a mobile device still with a 3D angle.
<figure><img src="image_11.png" /><figcaption>Fig. 9: Average *ERs* by *TOLs* and *VFLs*.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 10: Average *ERs* by *TMLs* and *VFLs*.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 11: Average *NC* by *blocks* and *VFLs*.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 12: Average *AT* by *blocks* and *VFLs*.</figcaption></figure>

**Number of Aborted Trials.** *AT* indicates the difficult degree to complete the experimental tasks. As shown in Fig. 12, the differences of *AT* across the blocks were not significant (F(2, 22) = 0.85, NS). Figure 12 shows all *ATs* were approximately 0 under the *FV* and *PV* conditions. A minor decrease of *AT*, from 4.8% to 3.5%, under *NV* condition was observed (Fig. 12).

## 4.3 Orientation Effects under the NV Condition

Generally, experimental task performance metrics of speed (according to *ST*, see Fig. 13), accuracy (according to *ERs*, see Fig. 14), difficulty to dwell in the target (according to *NC*, see Fig. 15), and difficulty to complete a selection (according to *AT*, see Fig. 16) decreased with the increase of *TOLs* and *TMLs*. But we observed an exception (from *TOL6* to *TOL8*) for *TO* from the figures. We conducted another 3 *blocks* × 4 *TOLs* × 2 *TMLs* RM-ANOVA on data under *NV* condition to get further insights. Figure 13 shows *MT* decreased minutely from *TOL6* (1888ms) to *TOL8* (1876ms) (F(1, 11) = 0.514, *p* = 0.488, *η*<sub>p</sub><sup>2</sup> = 0.045). *ERs* also decreased slightly from *TOL6* (31.3%) to *TOL8* (27.3%) (F(1, 11) = 3.28, *p* = 0.097, *η*<sub>p</sub><sup>2</sup> = 0.23), see Fig. 14. Similarly, there was also a slight decrease of *NC* from *TOL6* (1.42) to *TOL8* (1.4) (F(1, 11) = 0.406, *p* = 0.537, *η*<sub>p</sub><sup>2</sup> =
<figure><img src="image_15.png" /><figcaption>Fig. 13: Average *ST* by *TOLs* and *TMLs* under the *NV* condition.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Fig. 14: Average *ERs* by *TOLs* and *TMLs* under the *NV* condition.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 15: Average *NC* by *TOLs* and *TMLs* under the *NV* condition.</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 16: Average *AT* by *TOLs* and *TMLs* under the *NV* condition.</figcaption></figure>

0.036), see Fig. 15. Figure 16 shows that *AT* also decreased minutely from *TOL6*(3.9%) to *TOL8*(2.8%) (*F*(1, 11) = 1.068, *p* = 0.324, η<sup>2</sup> = 0.088). Figure 17 illustrates the distribution of *ERs* at *TOL12* in detail: *ERs* in the device's *axial* directions were lower than in the other directions, similarly Guo and Paek [9] reported the similar results on smart watches.

## 4.4 Discernible Numbers of Orientation Resolution Levels and Magnitude Levels

The participants had to keep watching the screen during a trial under *FV* and *PV*, but *PV* was defeated by the other two *VFLs* on both speed and accuracy. This indicates that *PV* made no contribution for the experimental tasks. In the following of this subsection, we determine the participants’ performance based on the data gathered when they had limited practice with tilt input, so only the data of block 3 were analyzed. A 3 *VFLs* × 4 *TOLs* × 2 *TMLs* RM-ANOVA was conducted on mean data of *ER*, *AT*, and *NC* in block 3.

The largest *NC* in *block 3* was 1.4 and 1.6 under the *FV* (see Fig. 18) *NV* conditions (see Fig. 19), respectively. So *NC* imposed no limitations on
<figure><img src="image_19.png" /><figcaption>Fig. 17: Average *ERs* at *TOL12*.</figcaption></figure>

discernible levels of the two factors. Similarly, all the percentages of *ATs* in *block 3* under the *FV* condition were approximately 0% (see Fig. 20), and the largest percentage of *AT* under the *NV* condition was 11% (see Fig. 21). Thus *AT* had no influences on discernible levels of the two factors, either.

<figure><img src="image_20.png" /><figcaption>Fig. 18: Average *NC* by *TOLs* and *TMLs* in *block 3* under the *FV* condition.</figcaption></figure>

<caption>Fig. 19: Average *NC* by *TOLs* and *TMLs* in *block 3* under the *NV* condition.</caption>

Figure 22 shows the largest *ER* in *block 3* was 4.2% under the *FV* condition. So all the explored levels for both *TO* and *TM* are discernible, i.e., *TOLs* were 12 and *TMLs* were 4.

There was only one *ER* (8.3%) less than 10% under the *NV* condition, as shown in Fig. 23. In that case, *TOLs* and *TMLs* were 4 and 2, respectively. These two are discernible numbers for the two factors under *NV*. If we extend
<figure><img src="image_21.png" /><figcaption>Fig. 20: Average *AT* by *TOLs* and *TMLs* in *block 3* under the *FV* condition.</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>Fig. 22: Average *ERs* by *TOLs* and *TMLs* in *block 3* under the *FV* condition.</figcaption></figure>

<caption>Fig. 23: Average *ERs* by *TOLs* and *TMLs* in *block 3* under the *NV* condition.</caption>

*ER* tolerance to 15%, the discernible numbers of *TOLs* and *TMLs* were 8 and 2, respectively.

## 4.5 Subjective Feedback

The survey was to determine how the participants felt using tilt control for target acquisition and selection tasks under different *VFLs*. Participant's subjective ratings of the three levels are shown in Fig. 24. The ratings include usability, ease of learning, and hand and eye fatigue.

The participants rated *FV* the highest regarding measures of usability and ease of learning; while *NV* was rated the best for eye and hand fatigue. According to our observations, under the *FV* and the *PV* conditions, the participants tried hard to keep watching visual feedback while tilting the device, and to keep their wrists in a special posture to keep the screen in a visible range: this limited the movement of wrists and arms. While under the *NV* condition, the participants manipulated the device to perform trials with more comfortable gestures. Although the participants rated *NV* the lowest regarding usability, they believed that using a mobile with an eyes-free method is irreplaceable for
<figure><img src="image_23.png" /><figcaption>Fig. 24: Likert scale ratings by VFLs.</figcaption></figure>

some special application scenarios, e.g., while walking on the road. NV was rated the lowest for ease of learning, but it is conflicting with the quantitative statistical results. We speculate that, under the NV condition, the difficulty of tilt manipulation made the participants think it was also difficult to obtain the ability of tilt manipulation.

Other open comments include: “Tilt to the upper right corner is very uncomfortable.” Given that all the participants were right-handed, this is easy to understand the comment and consistent with the aforementioned *orientation effects*.

# 5 Concept Designs of Tilt-based Interaction Using an Eyes-Free Method

Building on the experimental results and our observations, we now explore the tilt-based design space using an eyes-free method.

We often receive a phone call when walking across a crossroad, sometimes holding a bag in one hand. At that case, it is difficult to answer the call. Furthermore, we might put us in danger if we watch the mobile phone on the road. According to our experimental results, we can easily deal with the phone call with tilt input using an eyes-free method and by one hand (see Fig. 25a). Typically, we have four choices to deal with a phone call. We may first get to know the calling-number by an audio prompt message, and then determine to accept or deny the call, or send an automatic message and deny the call. The division of the tilt space with four *TOLs* and one *TML* is suitable for this case (See Fig. 25a).

Figure 25b shows the concept design of a music player supporting eyes-free interaction. Since it is easier to be controlled for “X-Y axial” orientation districts, more commands are assigned in these directions (see Fig. 25b). There are two *TOLs* in these axial directions, but only one in the diagonal. In the design, opposite functions are mirrored in one line, most functions are located alone X-Y axes. By default, the surface of the player is hidden. Users tilt a mobile phone into a certain 3D tilt district and keep its tilting posture for one second, then a function is chosen, and then the player works according to the chosen command.
<figure><img src="image_24.png" /><figcaption>Fig. 25: Concept designs for tilt space using an eyes-free method. (a) Dealing with a phone call with tilt input. (b) Mapping between the tilt space districts and the music player commands. (c) Manipulating a toy car with tilt input. (d) A smart toy car that can be manipulated by a mobile APP through Blue-tooth.</figcaption></figure>
In the market, there are many smart toy cars that can be manipulated by a
mobile APP through Blue-tooth, see Fig. 25d. We typically manipulate a smart
toy car using the GUI of an APP. But that is not the best method, since we had
better keep visually tracking with the toy car when it is running on the road,
otherwise it may run into some obstacles or even be crashed. Based on our study
results, we build a pie menu that is suitable for eyes-free interaction, see Fig. 25c.
We employ tilt input to make a toy car run forward by tilting a mobile phone
outward (at approximately twelve o'clock) or make the toy speed up by tilting
the phone further in the same direction. When we want to stop the toy, we just
tilt the phone backward (at approximately six o'clock); and if we tilt back the
phone further, we make the toy run reversely. We also manipulate the toy turn
leftward or rightward by tilting the phone leftward or rightward, respectively.
During the whole process, we keep on watching the toy car and don't need to
glimpse the screen of the mobile phone.

# 6 Discussion

The results indicate that the participants could distinguish at least 12 *TOLs* and 4 *TMLs* under the *FV* condition. But there exist some spectra of *TM* in which the usability is not good enough, e.g., the spectrum where the *TM* is near 90°, it is difficult to keep a device in that posture and get visual feedback from the screen for our wrists and eyes, respectively. These *TM* spectra should not be used in applications or be used as mappings to some special least use commands, e.g., the command of APP exit.

For the *NV* condition, although the discernible numbers are small, *NV* had the highest speed and were the best in terms of hand and eye fatigue. *NV* is not bothered by “fat finger” [31] and hand occlusion [36], which generally exist for all direct-touch screens. And the eyes-free method can reduce some safety risks when we use a mobile phone on the road. Although *NV* has the potential to address the aforementioned issues of a mobile phone, it limits the available number of tilt input commands. *NV* is more suitable for some special interaction scenarios, where one’s vision is unavailable and/or one of her/his hands is occupied and fewer interfacial commands are necessary.

The previous research [26] had probed discretizing raw angular space using
linear, quadratic, and sigmoid functions to improve angular tilt control. We
adopted a uniform linear discrete function for better spatial sense and cognition,
especially for the *NV* condition where there is no other cues to identify a target
district in a 3D tilt space.

We have found out that there are orientation effects of tilt input, especially
under the *NV* condition. We speculate that these results may be caused by the
bisection method. Bisection is the simplest dividing method. We can utilize *n* (a
positive integer) times of bisection to divide something into 2ⁿ equal parts. The
numbers of 4 and 8 are both integer exponents of 2, under these dividing con-
ditions the participants could perform trials more easily, so better performance
was found at *TOL4* and *TOL8*. Especially, the impact of dividing method was
magnified under the *NV* condition, since the participants had no other cues be-
sides dividing the space upon their own sense and cognition. As for the condition
of *TOL12*, the *axial* directions also conformed to the least times of bisection. The
best performance in the *axial* directions may also be caused by our experience
in other disciplines, e.g., the knowledge about system of rectangular coordinates
from mathematics.

7 Conclusion and Future Work

In our study, a quantitative experiment was conducted to determine the effects
of *visual feedback* on tilt control of a mobile phone and the upper bounds of
human ability to control the tilt input under different *VFLs*. The experimental
results have some indications for tilt-based interaction design. First, partially
visual feedback (*PV*) performed not so well as fully visual feedback (*FV*) and
no visual feedback (*NV*) in terms of both speed and accuracy. This reveals that
*PV* has no help for tilt input. Second, the participants could distinguish at least
12 tilt orientation levels (*TOLs*) and 4 tilt magnitude levels (*TMLs*) under the
*FV* condition. Third, the participants could distinguish 4 *TOLs* and 2 *TMLs*
or 8 *TOLs* and 2 *TMLs* under the *NV* condition with an error rate tolerance
of 10% or 15%, respectively. Four, the ability of the participants on tilt control
was related with *orientation* direction.

There exist some limitations of our work. First, the experiment was conducted
when the participants were seated. But some external factors, e.g., motion sta-
tus of the subjects, could influence the results. We had considered some other
factors during the experiment design, but they had finally been excluded from
the experiment to prevent it from becoming too complicated. Some other factors
should be considered in our future work. Second, we only employed dwell time
(1s) to confirm a target selection, but different selection modes may also have
impacts on the results. In the future work, we will investigate impacts of different
selection modes. Third, tilt gestures have the potential to serve the blind, since
the techniques support eyes-free interaction. We will explore the use of tilt for
people with visual impairments. Fourth, we only utilized audio feedback under
none visual feedback condition. But it is not clear whether there are some other
kinds of feedback that are more suitable for eyes-free interaction. In our future
work, we will explore different feedback, e.g., tactile and auditory feedback, and
find out the most suitable feedback mechanism for the *NV* condition.

Acknowledgment

This work is supported by Science Foundation of Guangxi (AA17204096, AD16380076).
We thank all the anonymous reviewers for their generous comments and good
advice on the paper.
References

1. Baglioni, M., Lecolinet, E., Guiard, Y.: Jerktilts: using accelerometers for eight-choice selection on mobile devices. In: Proceedings of the 13th international conference on multimodal interfaces - ICMI '11. pp. 121–128. ACM Press, New York, NY, USA (2011). https://doi.org/10.1145/2070481.2070503

2. Bartlett, J.F.: Rock'n'scroll is here to stay. IEEE Computer Graphics and Applications 20(3), 40–45 (2000). https://doi.org/10.1109/38.844371

3. Browne, K., Anand, C.: An empirical evaluation of user interfaces for a mobile video game. Entertainment Computing 3(1), 1–10 (2012). https://doi.org/10.1016/j.entcom.2011.06.001

4. Cho, S.J., Murray-Smith, R., Kim, Y.B.: Multi-context photo browsing on mobile devices based on tilt dynamics. In: Proceedings of the 9th international conference on Human computer interaction with mobile devices and services - MobileHCI '07. pp. 190–197. ACM Press, New York, NY, USA (2007). https://doi.org/10.1145/1377999.1378006

5. Dunlop, M.D., Roper, M., Imperatore, G.: Text entry tap accuracy and exploration of tilt controlled layered interaction on smartwatches. In: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI '17. pp. 23:1–23:11. ACM Press, New York, NY, USA (2017). https://doi.org/10.1145/3098279.3098560

6. Eslambolchilar, P., Murray-Smith, R.: Tilt-based automatic zooming and scaling in mobile devices – a state-space implementation. pp. 120–131. Proceedings of Mobile Human-Computer Interaction - MobileHCI 2004, Springer Berlin Heidelberg (2004). https://doi.org/10.1007/978-3-540-28637-0

7. Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47, 381–391 (1954). https://doi.org/http://view.ncbi.nlm.nih.gov/pubmed/13174710

8. Geronimo, L.D., Canonica, A., Husmann, M., Norrie, M.C.: Continuous tilting interaction techniques on mobile devices for controlling public displays. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems - EICS '17. pp. 21–26. ACM Press, New York, NY, USA (2017). https://doi.org/10.1145/3102113.3102120

9. Guo, A., Paek, T.: Exploring tilt for no-touch, wrist-only interactions on smartwatches. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI '16. pp. 17–28. ACM Press, New York, NY, USA (2016). https://doi.org/10.1145/2935334.2935345

10. Harrison, B.L., Fishkin, K.P., Gujar, A., Mochon, C., Want, R.: Squeeze me, hold me, tilt me! an exploration of manipulative user interfaces. In: Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '98. pp. 17–24. ACM Press, New York, NY, USA (1998). https://doi.org/10.1145/274644.274647

11. Hinckley, K., Pierce, J., Sinclair, M., Horvitz, E.: Sensing techniques for mobile interaction. In: Proceedings of the 13th annual ACM symposium on User interface software and technology - UIST '00. pp. 91–100. ACM Press, New York, NY, USA (2000). https://doi.org/10.1145/354401.354417

12. Homaeian, L., Goyal, N., Wallace, J.R., Scott, S.D.: Group vs individual: Impact of TOUCH and TILT cross-device interactions on mixed-focus collaboration. In: Mandryk, R.L., Hancock, M., Perry, M., Cox, A.L. (eds.) Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. pp. 73–73. ACM Press, New York, NY, USA (2018). https://doi.org/10.1145/3173574.3173647
13. Hynninen, T.: First-person shooter controls on touchscreen devices: A heuristic evaluation of three games on the iPod touch. Thesis, University of Tampere (2012)

14. ISO: Iso 9241-9 ergonomic requirements for office work with visual display terminals (vdts) - part 9: Requirements for nonkeyboard input devices (2000)

15. Jones, E., Alexander, J., Andreou, A., Irani, P., Subramanian, S.: Ges-text: accelerometer-based gestural text-entry systems. In: Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. pp. 2173-2182. ACM Press, New York, NY, USA (2010). https://doi.org/10.1145/1753326.1753655

16. Karlson, A.K., Bederson, B.B., SanGiovanni, J.: AppLens and launchTile: two designs for one-handed thumb use on small devices. In: Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '05. pp. 201-210. ACM Press, New York, NY, USA (2005). https://doi.org/10.1145/1054972.1055001

17. Kratz, S., Brodien, I., Rohs, M.: Semi-automatic zooming for mobile map navigation. In: Proceedings of the 12th international conference on Human computer interaction with mobile devices and services - MobileHCI '10. pp. 63-72. ACM Press, New York, NY, USA (2010). https://doi.org/10.1145/1851600.1851615

18. Kurosawa, H., Sakamoto, D., Ono, T.: Myotilt: a target selection method for smart-watches using the tilting operation and electromyography. In: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI '18. pp. 43:1-43:11. ACM Press, New York, NY, USA (2018). https://doi.org/10.1145/3229434.3229457

19. Kurtenbach, G., Buxton, W.: The limits of expert performance using hierarchic marking menus. In: Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93. pp. 482-487. ACM Press, New York, NY, USA (1993). https://doi.org/10.1145/169059.169426

20. MacKenzie, I.S., Teather, R.J.: Fittstilt: the application of fitts' law to tilt-based interaction. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction Making Sense Through Design - NordiCHI '12. pp. 568-577. ACM Press, New York, NY, USA (2012). https://doi.org/10.1145/2399016.2399103

21. Medryk, S., MacKenzie, I.S.: A comparison of accelerometer and touch-based input for mobile gaming. Proc. MHCI pp. 117.1-117.8 (2013)

22. Oakley, I., O'Modhrain, S.: Tilt to scroll: Evaluating a motion based vibrotactile mobile interface. In: Proceedings of First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. pp. 40-49. IEEE (2005). https://doi.org/10.1109/whc.2005.138

23. Partridge, K., Chatterjee, S., Sazawal, V., Borriello, G., Want, R.: Tilttype: accelerometer-supported text entry for very small devices. In: Proceedings of the 15th annual ACM symposium on User interface software and technology - UIST '02. pp. 201-204. ACM Press, New York, NY, USA (2002). https://doi.org/10.1145/571985.572013

24. Pietroszek, K., Wallace, J.R., Lank, E.: Tiltcasting: 3d interaction on large displays using a mobile device. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST '15. pp. 57-62. ACM Press, New York, NY, USA (2015). https://doi.org/10.1145/2807442.2807471

25. Poupyrev, I., Maruyama, S., Rekimoto, J.: Ambient touch: designing tactile interfaces for handheld devices. In: Proceedings of the 15th annual ACM symposium on User interface software and technology - UIST '02. pp. 51-60. ACM Press, New York, NY, USA (2002). https://doi.org/10.1145/571985.571993
26. Rahman, M., Gustafson, S., Irani, P., Subramanian, S.: Tilt techniques: investigating the dexterity of wrist-based input. In: Proceedings of the 27th international conference on Human factors in computing systems - CHI 09. pp. 1943–1952. ACM Press, New York, NY, USA (2009). https://doi.org/10.1145/1518701.1518997

27. Rekimoto, J.: Tilting operations for small screen interfaces. In: Proceedings of the 9th annual ACM symposium on User interface software and technology - UIST '96. pp. 167–168. ACM Press, New York, NY, USA (1996). https://doi.org/10.1145/237091.237115

28. Sad, H.H., Poirier, F.: Evaluation and modeling of user performance for pointing and scrolling tasks on handheld devices using tilt sensor. In: Proceedings of Second International Conferences on Advances in Computer-Human Interactions. pp. 295–300. IEEE (feb 2009). https://doi.org/10.1109/achi.2009.15

29. Sazawal, V., Want, R., Borriello, G.: The unigesture approach one-handed text entry for small devices. In: Proceedings of International Conference on Mobile Human-Computer Interaction. pp. 256–270. Human Computer Interaction with Mobile Devices, Springer Berlin Heidelberg (2002). https://doi.org/10.1007/3-540-45756-9

30. Shima, K., Onishi, K., Takada, R., Adachi, T., Shizuki, B., Tanaka, J.: Investigating accuracy of tilting operation on wrist-worn devices with touchscreens. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '16. pp. 2705–2711. ACM Press, New York, NY, USA (2016). https://doi.org/10.1145/2851581.2892377

31. Siek, K.A., Rogers, Y., Connelly, K.H.: Fat finger worries: How older and younger users physically interact with pdas. In: Proceedings of Human-Computer Interaction - INTERACT 2005. pp. 267–280. Springer Berlin Heidelberg (2005). https://doi.org/10.1007/11555261

32. Small, D., Ishii, H.: Design of spatially aware graspable displays. In: Proceedings of CHI '97 Extended Abstracts on Human Factors in Computing Systems. pp. 367–368. ACM, New York, NY, USA (1997)

33. Sun, K., Wang, Y., Yu, C., Yan, Y., Wen, H., Shi, Y.: Float: One-handed and touch-free target selection on smartwatches. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. pp. 692–704. ACM Press, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3026027

34. Teather, R.J., MacKenzie, I.S.: Position vs. velocity control for tilt-based interaction. In: Proceedings of Graphics Interface Conference 2014. pp. 51–58. ACM Press, New York, NY, USA (2014)

35. van Tonder, B.P., Wesson, J.L.: Improving the controllability of tilt interaction for mobile map-based applications. International Journal of Human-Computer Studies 70(12), 920–935 (dec 2012). https://doi.org/10.1016/j.ijhcs.2012.08.001

36. Vogel, D., Casiez, G.: Hand occlusion on a multi-touch tabletop. In: Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems. pp. 2307–2316. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2208276.2208390

37. Wang, J., Zhai, S., Canny, J.: Camera phone based motion sensing: interaction techniques, applications and performance study. In: Proceedings of the 19th annual ACM symposium on User interface software and technology - UIST '06. pp. 101–110. ACM Press, New York, NY, USA (2006). https://doi.org/10.1145/1166253.1166270

38. Wigdor, D., Balakrishnan, R.: Tilttext: Using tilt for text input to mobile phones. In: Proceedings of UIST 2003. pp. 81–90 (2003). https://doi.org/10.1145/964696.964705
39. Yeo, H.S., Phang, X.S., Castellucci, S.J., Kristensson, P.O., Quigley, A.: Investigating tilt-based gesture keyboard entry for single-handed text entry on large devices. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. pp. 4194–4202. ACM Press, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025520

40. Zaman, L., MacKenzie, I.S.: Evaluation of nano-stick, foam buttons, and other input methods for gameplay on touchscreen phones. In: Proceedings of International Conference on Multimedia and Human-Computer Interaction-MHCI. vol. 1, pp. 1–8. Citeseer (2013)