Please tell us how you feel about using swipe gestures to reveal *swhidgets*.

I1 I enjoy manipulating *swhidgets*.

I2 When I interact with *swhidgets*, the interface responds to my inputs in a way that feels satisfactory.

I3 Swipe gestures are difficult to perform.

I4 The system often misinterprets my swipe gestures.

I5 The animation of *swhidgets* is sometime unpredictable.

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

<figure><img src="image_65.png" /></figure>

<figure><img src="image_66.png" /></figure>

<figure><img src="image_67.png" /></figure>

*Swhidgets* behave like “real-world” objects that you physically manipulate: they slide on flat surfaces, they are bound to other objects that gets dragged or pushed with them, and they can hide under other objects. Please tell us how you feel about the use of this metaphor in the design of *swhidgets*.

I6 I do not expect controls to hide under interface elements or bezels.

I7 I can tell from the interface which on-screen objects could be moved to reveal *swhidgets*.

I8 I can tell from the interface in which direction I should move objects to reveal *swhidgets*.

I9 This design is coherent with the design of other iOS interactions.

<figure><img src="image_68.png" /></figure>

<figure><img src="image_69.png" /></figure>

<figure><img src="image_70.png" /></figure>

<figure><img src="image_71.png" /></figure>

Now, we would like to know about the possibility of leveraging the knowledge of specific *swhidgets* to discover interaction possibilities in other contexts. Please complete the sentence below with the following statements and evaluate your agreement to them:

*When I know a specific swhidget in an application, I...*

I10 ...suspect other places with a similar screen layout to provide a similar *swhidget*.

I11 ...do not expect I can swipe it in another direction to reveal other functions.

I12 ...play with it to find all the possible ways to use it.

<figure><img src="image_72.png" /></figure>

<figure><img src="image_73.png" /></figure>

<figure><img src="image_74.png" /><figcaption>Table 8.3: Questions in the last part of the questionnaire, with histograms of responses from totally disagree (left) to totally agree (right) and box pots showing quartiles.</figcaption></figure>
We will now consider how the placement and type of *swhidgets* relate to the functions they provide: can you find this logic and make it yours?

<figure><img src="image_75.png" /></figure>

Can you use your knowledge of how to accomplish tasks with *swhidgets* to find other situations you could possibly use them?

Is your knowledge of how to accomplish tasks with *swhidgets* transferable to other situations?

Please evaluate your agreement with the following statements.

<figure><img src="image_76.png" /><figcaption>Table 8.4: (Continuation of Table 8.3) Questions in the last part of the questionnaire, with histograms of responses from totally disagree (left) to totally agree (right) and box pots showing quartiles.</figcaption></figure>
Does using *swhidgets* in your daily tasks make you feel in control of the situation? Please evaluate your agreement to the following statements.

<figure><img src="image_77.png" /></figure>

<figure><img src="image_78.png" /></figure>

<figure><img src="image_79.png" /></figure>

<figure><img src="image_80.png" /></figure>

<figure><img src="image_81.png" /></figure>

Does using *swhidgets* in your daily tasks make you feel like an advanced user? Please evaluate your agreement to the following statements.

<figure><img src="image_82.png" /></figure>

<figure><img src="image_83.png" /></figure>

<figure><img src="image_84.png" /></figure>

<figure><img src="image_85.png" /></figure>

<figure><img src="image_86.png" /></figure>

Finally, we would like to know if *swhidgets* have an impact on your ability to organize your life so as to be in harmony with your values. Please evaluate your agreement to the following statements.

<figure><img src="image_87.png" /></figure>

<figure><img src="image_88.png" /></figure>

<figure><img src="image_89.png" /></figure>

<figure><img src="image_90.png" /></figure>

<figure><img src="image_91.png" /><figcaption>Table 8.5: (Continuation of Table 8.4) Questions related to the Self-Determination Theory in the last part of the questionnaire, with histograms of responses from totally disagree (left) to totally agree (right) and box plots showing quartiles.</figcaption></figure>
"When I know a specific *swhidget* in an application, I suspect other places with a similar screen layout to provide a similar *swhidget*." (I10: Q1=4, median=6, Q3=7).

Most participants agreed that when they discover a *swhidget*, they “play with it to find all the possible ways to use it” (I12: Q1=4, median=5, Q3=7). Answers to this item were also well correlated to the answers to the previous item about other places with a similar *swhidget* ($\rho=.540, p<10^{-11}$) and negatively correlated with participants’ age ($\rho=-.231, p=.0053$). Self-reported knowledge of iOS interaction was slightly correlated to this item’s answers ($\rho=.283, p<.001$), which suggests that users who “play” with *swhidgets* to explore interaction possibilities have more chance to know about iOS interaction or, at least, to feel more confident about their knowledge.

Participants are divided on the idea that having discovered a *swhidget*, they should expect it can be swiped in another direction to reveal other functions (I11: 38.2% disagree, 31.9% disagree, median=4, Q1=2, Q3=5). This is not necessarily surprising considering that *swhidgets* are not systematically symmetric. Typically, lists in many applications have a delete *swhidget* on the right of the items but do not necessarily have *swhidgets* on the left.

### MENTAL MODELS AND THE LOGIC OF SWHIDGETS

We asked participants to evaluate their agreement with seven statements concerning "how the placement and type of *swhidgets* relate to the functions they provide [and if participants could] find this logic and make it [theirs]". There was a global agreement (Q1=4, median=5, Q3=6) for the following six statements: I13 participants understand the logic prevailing to the different types of *swhidgets* and their placement in the interface; I14 they can use this logic to guide their search for a *swhidget* with the function they need; I15 they sometime feel that different parts of an application failed to follow a common logic for their *swhidgets*; I16 they notice *swhidgets* that follow a similar logic in different applications or systems; I17 once they learned how to do something with a *swhidget*, they cannot think anymore about another way to do it; I18 they sometimes forget that there are *swhidgets* they can use to do a task. As one can expect, there is a strong correlation between items I13 and I14 ($\rho=.787, p<10^{-30}$), which were both correlated with participants’ self-estimated knowledge of iOS ($\rho=.279$ and $.217, p^*=.0043$ and $.047$ respectively) and interest for technology news ($\rho=.268$ and $.308, p^*=.0012$ and $.0002$ respectively). There was however no agreement on I19 "I sometimes need to be reminded where are the *swhidgets*".

### 8.6.5 General perception of *Swhidgets*

### INTERACTING WITH SWHIDGETS

Participants appreciate interacting with *swhidgets*, reporting that they enjoy manipulating them (I1:
Q1=5, median=6, Q3=7, 85.3% scored 5-7) and that the interface re-
sponds in a way that feels satisfactory (I2: Q1=5, median=6, Q3=7,
88.1% scored 5-7). Answers to these two items are well correlated
(Spearman's ρ=.75, p<10⁻²⁵), and also correlate with participants'
interest in technology news (ρ=.306, p<10⁻³, and ρ=.231, p=.0054
respectively).

Participants globally rejected the difficulty of manipulating *swhidgets*, at 82.5% for I3 "Swipe gestures are difficult to perform", 64.3% for I4 "the system often misinterprets my swipe gestures", and 62.2% for I5 "the animation of *swhidgets* is sometime unpredictable". There was nonetheless respectively 9.8%, 15.4% and 13.3% of participants who agreed with these statements. These three items can be regarded as measuring the perception of difficulties in the manipulation of *swhidgets* with good internal consistency (Cronbach's α=0.88).

REASONS NOT TO ADOPT SWHIDGETS The main reason reported for not adopting a *swhidget* was *habits* ("It is unadapted to my habits and/or preferences", 38.6% of all evaluations), followed by *context* ("It is unadapted to my contexts of use", 26.6%), *perform* ("It is difficult to perform", 14.5%), *worth* ("It is not worth changing my habits", 11.2%), and *learn* ("It is difficult to learn", 5.4%). Distinguishing the three types of *swhidgets* revealed a statistically significant difference for *context* (*p* = .043), with system *swhidgets* having respectively 1.37, and 1.38 times greater odds than view *swhidgets* (*p* = .044), and item *swhidgets* (*p* = .010). The few open comments regarding reasons not to adopt varied considerably from one respondent to another. As an example, one respondent reported that she had difficulties revealing the Notification Center, as her swipe was often not recognized by the system, which changes her preference of this *swhidget*, whereas as seen above, the statement "the system often misinterprets my swipe gestures" was in majority rejected by respondents. Interestingly, even though it was not asked in the survey, several participants insisted for some *swhidgets* that they did not see any reason not to adopt them (reported by 13 respondents for *flashlight* and 11 for locking orientation through the control panel).

8.6.6 *Satisfaction of basic needs*

Table 8.5 shows the results of the last questions in the final question-
naire of the study, which were inspired by the Self-Determination
Theory (See Section 7.4.2) and targeted participants' feelings of Com-
petence (through its two components control and mastery) and Au-
tonomy.

FEELING OF BEING IN CONTROL There was no clear consensus on
the two questions related to participants' confidence in using *swhidgets*
in a way that match their intents. For I22 "I sometimes worry about triggering *swhidgets* by mistake", 33.6% of participants disagreed and 43.4% agreed with the statement (Q1=3, median=4, Q3=6). For I24 "I swipe when I want to use *swhidgets*, but the commands it reveals are not always the ones I thought", 24.5% disagreed but 39.9% agreed (Q1=4, median=4, Q3=5).

There was however a relative consensus on the two other items in favor of a positive feeling of being in control when manipulating *swhidgets*: agreement with "I use *swhidgets* only when I am sure of what I am doing" (I23: Q1=4, median=5, Q3=6, 59.4% agreed, 13.3% disagreed), and disagreement with "Using *swhidgets* requires some attention and distracts me from my tasks" (I25: Q1=2, median=4, Q3=4, 46.2% disagreeing and 22.4% agreeing).

## FEELING OF MASTERY

There was a clear consensus among participants for a positive answer to the question "I consider *swhidgets* as advanced techniques" (I27: Q1=4, median=5, Q3=7, 11.2% disagree, 63.6% agree). There was also a clear consensus among participants that they express mastery of *swhidgets* and the interface in general, as manifested by their knowledge with "In general, I know when it is possible to use a *swhidget* to achieve a task I want to do" (I28: Q1=4, median=5, Q3=7, 4.9% disagree, 71.3% agree), and by their performance with "*Swhidgets* are the fastest way for me to do my tasks" (I29: Q1=4, median=6, Q3=7, 7.7% disagree, 66.4% agree). On the other side, there was no consensus on participants' ability to memorize all *swhidgets*: "I cannot remember all these *swhidgets*" (I26: Q1=3, median=4, Q3=5, 30.1% disagree, 39.9% agree).

## FEELING OF AUTONOMY

Participants expressed a clear consensus on the fact that *swhidgets* enable them to do some tasks with their devices generally or in particular situations: "There are tasks I could not or would not do without *swhidgets*" (I30: Q1=4, median=5, Q3=7, 14.7% disagree, 57.3% agree). "There are situations I could not or would not use my phone to do some task, without *swhidgets*" (I31: Q1=4, median=6, Q3=7, 11.2% disagree, 66.4% agree).

There is also a relative consensus on the fact that a good knowledge of *swhidgets* is a quality that is or should be socially recognized: "Knowing many *swhidgets* is a legitimate source of pride" (I33: Q1=4, median=4, Q3=7, 15.4% disagree, 49.7% agree).

On the other hand, participants expressed different opinions on their interest in learning *swhidgets per se*, as shown by the lack of consensus on "I do not want to remember so many *swhidgets*" (I32: Q1=3, median=4, Q3=5, 30.1% disagree, 33.6% agree).
## 8.7 DISCUSSION OF THE RESULTS OF THE TWO STUDIES

I reported on the results of the first two studies conducted on current usage of *swhidgets*. The first study conducted in a laboratory setting investigated which *swhidgets* were spontaneously used by participants when prompted to perform certain operations on an iOS device. The second study conducted via an online survey platform, investigated which *swhidgets* users reported to know and use. Combined, our studies provide the following main insights on awareness, usage and discovery of *swhidgets* by middle-aged and technology-friendly smartphone users.

### 8.7.1 Awareness of Swhidgets

Awareness of *swhidgets* was of 64.1% in the laboratory study and 80.2% in the online study. While unsurprisingly lower than awareness of navigation controls, awareness rate of our participants was relatively high, yet slightly different between both studies. One possible explanation of this difference may be the fact that it was *observed* in study 1 and *self-reported* in study 2. Regardless, these results suggest that the integration of *swhidgets* in touch-based operating systems, despite the lack of signifier to inform users of their availability, is well executed and that users are still able to discover their existence.

Interestingly though, awareness rate also happened to vary depending on the operation. Notably, 42.9% and 50% of participants in study 1 and 2, respectively, were not aware of the view *swhidget* for revealing the time at which a SMS was sent in Message. It is of particular interest since there is no real alternative to obtain this information, and that messaging was one of the main activity reported to be conducted by our participants with their smartphones. Unaware participants provided mixed feedbacks about this feature when this item *swhidget* was demonstrated to them, either commenting that they were simply not aware of it where others commented that they never need to know the exact time of a message. Awareness also depends on the type of *swhidgets*, with users significantly more aware of *system* than *item* and *view swhidgets*.

In the end, similarly to hotkeys that are more often learned for frequent commands [Lan+05; Mal+13], it is likely given their complete lack of signifier that users are more aware of *swhidgets* they feel the need to use, which probably motivates their discovery in the first place.

### 8.7.2 Usage of (And Reasons Not to Use) Swhidgets

Usage of *swhidgets* was on average of 56.0% and 66.0%, in study 1 and 2, respectively (once again, usage rate was lower for *swhidgets*
than for navigation controls, as one could expect). Overall usage
rates are relatively high, especially considering awareness rates as
they show that our participants tend to use a high proportion of the
swhidgets they are aware of (median usage rate of 83.1% in study 2,
up to 94.6%). Even more interestingly, we observed that swhidgets are
extremely often favored over alternative inputs. In study 1, 62.1% of
all completed trials were performed using first a swhidget, with 36%
only for navigation. In study 2, swhidgets were preferred on average
by 57.8% of participants who knew how to complete the operation,
against 39.2% only for navigation controls (swhidgets favored over
navigation for 5 operations). This result can once again be explained
by the fact that swhidgets may appear as a shortcut alternative to
navigation controls in many applications, and as such, are favored
over techniques that would take longer or would be more tedious to
use. However, proportion of primary or preferred usage once again
depends on the operation performed. Typically, in study 2, only 37.5%
of participants who completed the delete e-mail operation preferred
the swhidget over the navigation methods. Similarly, in the laboratory
study, only 36% preferred the swhidget for replying to an e-mail. The
main reasons reported were that these operations are often performed
right after reading the e-mail, where conventional navigation controls
are ready-to-hand, whereas using the swhidget would require to come
back to the e-mail list. Once again, these results suggests that usage is
also likely to be higher for swhidgets that appear as shortcut alternative
to conventional input methods.

In the laboratory study, 21.4% of participants did not consider that
swhidgets help them with their everyday tasks. We further investigated
the reasons behind this in the online study, where 38.6% of respon-
dents reported that the main reason not to adopt a swhidget would be
because it is unfit to their habits, and 26.6% because it is unsuitable in
their context of use, confirming the assumptions made above about
swhidgets considered mostly useful as shortcuts. However, it is known
that users have difficulties to estimate the actual costs and benefits of
switching to a theoretically more efficient alternative [Coc+14; Mal+13].
This was confirmed by our observations, where several users had a
wrong perception of the performance gain provided by swhidgets. For
instance, in the laboratory study, several participants who knew how
to delete emails with an item swhidgets still preferred to tap on the
visible button, because they believe it is faster to select all the emails
that they want to delete and then put them into the trashcan together.
Although this is only our perception that swhidgets would be faster in
this case, and that they are indeed designed to be faster, an implication
for design would be to increase the visibility of this fact so that the
speed difference is more accurately perceived by users.
8.7.3 Discovery of Swhidgets

Study 2 highlighted the role of self-exploration of the interface in the discovery of *swhidgets*, where we asked how users discovered each *swhidgets* they were aware of. 42.6% of the answers were that the *swhidget* was discovered by exploring the interface, and 15.9% that it was from accidental activation. These results are in line with the assumptions made by Schramm et al. in their work on performance of hidden toolbars, that users are becoming more familiar with touch-based devices and *swhidgets*, and tend to explore the interface to discover more of them [SGC16].

Schramm et al.'s assumption is also in line with the results collected in the online study regarding the understanding of the physical metaphor of *swhidgets*. While we found no consensus regarding a complete understanding of which objects could be moved to reveal *swhidgets*, respondents clearly reported that they suspect to find *swhidgets* they know at similar location in different applications, which probably encourages interface exploration, and therefore, the discovery of *swhidgets* that way. That being said, our results suggest that knowledge of *swhidgets* transfers only in limited amounts from one application to another, between *swhidgets* that rely on similar interface layouts. For instance, the similitude of participants' knowledge about deleting mails and SMS with *swhidgets* (in both studies), as well as the similitude of PRIMARY usage of *swhidgets* for searching mails and SMS in study 1 are compatible with such a transfer. However, the significant difference between PRIMARY usage of SMS.delete and mail.markAs - which use basically the same kind of *swhidgets* - suggests that the transfer of knowledge does not rely only on the type of *swhidgets* involved, but also on the nature of the task and direction of the swipe gestures (participants of study 2 differed in opinions about expecting different swipe directions).

Finally, the low numbers of *swhidgets* discovered following a demonstration of the input method (4.9%) or remembering seeing someone else using it (3.1%) was however surprising. Indeed, social interaction had been found to play a major role in disseminating useful system capabilities [Per+04]. Similarly to usage of *swhidgets*, it is possible that users have a wrong estimation of the performance benefits of a *swhidget* when witnessing someone using it. In addition, it is possible that our respondents discovered the *swhidgets* months or years before completing the survey, therefore it remains uncertain that participants remembered perfectly how they did discover them. Even though a "I do not remember" option was available, it is likely that users sometimes preferred to assume how they discovered them rather than answering they did not remember. Finally, it is also possible that participants who learned about a *swhidget* by watching someone do
it still considered that they discovered it "by exploring the interface" when they tried to reproduce what they saw on their own device.

8.7.4 *Future work*

While informative, our studies could not explore all smartphone ecosystems, as well as focus on specific populations. As such, it leaves interesting opportunities for future work.

First, we chose to focus our studies on the iOS ecosystem for several reasons: it is a coherent and consistent OS, whereas Android allows manufacturer to create software overlays that change the look and feel of the OS. Because of this, default applications, advanced input methods (including *swhidgets*), hardware inputs, etc. are likely to differ much more between Android devices than between iOS devices. That being said, Apple recent smartphones and tablets are not equipped with a hardware home button anymore. The home feature is now performed with an upward swipe from the bottom of the display, which interferes with the previously control panel *swhidget*. Consequently, on these devices, downward swipes from the top edge are now used both for revealing Control Panel and Notification Center, depending on which corner they are performed from. Our studies did not investigate these novel gesture mappings for this system *swhidget*, but remain valid for all other *swhidgets* tested. Future work should therefore focus on investigating *swhidgets* on other platforms, and possibly investigating the discovery and awareness of the localized *swhidgets* on recent iOS devices.

Second, even though we recruited participants of various ages (18 to 75 years old), both of our studies were conducted in majority with middle-aged adults familiarized with novel technologies. As such, it is likely that our participants, while representative of many smartphone users, correspond to a relatively high limit in term of knowledge of interactive capabilities of their device and interest in latest technologies. It is unlikely that this limited pool of participants would diminish in any way the validity of our results, since many smartphone users are likely to correspond to our stereotypical participant. Yet, future work should investigate different populations. While we informally tested the effect of age on awareness and usage of *swhidgets* but did not find any strong effect (possibly due to the low number of participants over 65), we still believe that future work should investigate usage of input features on smartphones with different age groups. Indeed, seniors and teenagers have significantly different background with technologies and it would be interesting to investigate how these difference in technology exposure may impact.
8.7.5 Dataset

Data collected for this work has been anonymized and is available
to the community through our institution address<fnref n="10" />. We collected
a significant amount of information in order to compare interaction
with *swhidgets* with navigation and voice control, but focused our
analysis on *swhidgets* which is the core of this thesis. That being said,
we remind that this data set contains information regarding many of
the points discussed (typically regarding discovery, reasons to adopt,
etc.) for navigation and voice controls as well.

<footnote n="10">10 http://ns.inria.fr/discovery/swhidgets/</footnote>
<figure><img src="image_92.png" /></figure>
Part V

PERSPECTIVE AND CONCLUSION
<figure><img src="image_93.png" /></figure>
<figure><img src="image_94.png" /></figure>
<figure><img src="image_95.png" /></figure>
In Section 9.1, I will discuss the perspectives for future works follow-
ing the studies I did in this thesis and the theoretical models that I
presented. In Section 9.2, I will shortly present an ongoing design
work to improve the discovery of *swhidgets* in order to show how the
theoretical models that I have introduced can point to unexplored
areas of the design space of *swhidgets*. Finally, I will conclude this
thesis in Section 9.3.

9.1 FROM RESULTS BACK TO MODELS

In this dissertation, for the clarity of the discourse and because it is a
first work on *swhidgets*, I chose to start with the definitions of affor-
dances, signifiers, and signifier-less designs, so that I could introduce a
model of discovery and proficiency improvement in Chapter 7, which
would bring useful concepts for the design of the studies presented
in Chapter 8 as well as for the discussion of their results. In other
words, I presented the concepts that allowed to define signifier-less
designs and to devise experiments beyond the simple measure of what
proportion of the population knew about them. In doing so, my goal
was to provide foundations not only for my own work but also for
future works on *swhidgets* and signifier-less designs. I will now discuss
how these concepts may be the building blocks of further studies.

Assuming they generalize to the whole population of iOS users, the results of the two experiments bring a few observations: users have a moderately high awareness of iOS *swhidgets*, use most of those they are aware of, and often prefer *swhidgets* over alternative input methods when their performance benefits are clear. While relying on a signifier-less design, *swhidgets* are relatively well discovered by users through exploration of the interface.

These results need to be confirmed by other experiments and ex-
tended to larger populations of touchscreen-based devices, but they
provide early observations that raise new questions. I will address
some of these questions in the next sections as perspective for future
research, discussing how the model I proposed in Chapter 7 as well
as the notions of affordances, signifiers, and semiotics can bring light
to the questions or suggest ways to approach them.
9.1.1 *Transfer of Knowledge*

*What role plays the transfer of knowledge between applications in the discovery of swhidgets in particular, and interaction techniques in general?*

This question was identified as one of the general research ques-
tions related to *swhidgets* under the concern of *consistency with other
interactions* (Section 2.6.1), and as one possible answer to a question
investigated in this thesis, *how can users know swhidgets despite their lack
of signifiers?* (Section 2.7.3). The two studies I conducted during this
thesis were thus designed to approach this question but did not focus
on it, and provided mixed results: On one hand, transfer of knowledge
seems to be a common way to discover *swhidgets*. On the other hand,
it does not seem to depend only on the design of the interface but also
on users' ability to relate the two applications – which was apparently
not the case for the conversation list in the Messages application and
the thread list in the Mail application, despite similar layouts.

These studies thus call for additional experiments addressing this
question with a narrower focus, to determine aspects of signifier-less
designs that affect user's ability to transfer their knowledge to other
contexts of use. However, experiments with a narrow focus need
precise hypotheses, which require detailed models and theories. I
will thus now review how the notions I discussed in this thesis can
suggest topics of studies investigating the phenomenon of discovering
signifier-less designs with a transfer of knowledge.

The analysis of signifiers that I have done in the framework of
Peirce's semiotics (see Chapter 5) suggests that transfer of knowledge
can sometime be modeled as recognizing a specific layout or other
features of the interface and treat them as iconic signifiers, which
objects are the affordances to be discovered – or, if not directly the
affordances, at least generic interface patterns that include them, such
as “a list view with *item swhidgets***” (See also Section 7.3.1).

In this model, users first need to understand that there is a rule
in the interface connecting the affordance to the layout or feature
that serves as a signifier. Such a rule would be knowledge at the *UI level* (Section 7.2.1). It can be taught explicitly by tutorials, or will
more likely be discovered by users through an inductive process –
essentially by noticing the pattern after having been exposed to some
number of its instances. Is there something in the pattern that allows
users to notice it more easily? Could the noticeability of the pattern be
improved by means of careful design? To notice the pattern, users need
to make generalizations at the *UI level* from pieces of knowledge at the
*command level* (the instances of the pattern) in a form of dependency
across levels (see Section 7.2.3), which may first need to acknowledge
the relevance of these pieces of knowledge (see Section 7.5.2).
Are design means such as interface layouts the main factor affecting the likeliness of users noticing the pattern, or do users actually notice patterns that encompass design means but also knowledge improvement aspects? For instance, users might recognize that a problem they have is similar to one they already had, and thus look for a similar solution. In that case, the similarity of design means could help recognize the similarity of problem contexts, but the design means may not be recognized as similar if the user was not facing a similar problem. Such a phenomenon could explain why there seems to be little transfer of knowledge between the Mail *item swhidgets* and the Message ones, since deleting a mail or thread might be a frequent operation while deleting a full conversation with a contact might only happen rarely. If a type of problem is frequent enough, users may form strategies to deal with it, and shift from what Rasmussen describes as knowledge-based behaviors treating design means as symbols (see Section 7.3.1) to rule-based behaviors treating design means as signs (with strategies such as "If I need to delete an item in a list, test if *item swhidgets* are available").

Beside the noticeabiltty of the pattern, effects of (a lack of) reliability on their adoption also need to be studied – i. e., will the recognized design mean of the pattern always be associated with the affordances it conveys? Indeed, comments from the participants of the two studies I did revealed that for some users, a lack of consistency in the use of *swhidgets* across applications was a reason not to adopt *swhidgets*. This behavior can be explained in the framework that I presented in Chapter 7, and especially in Section 7.5.1 about the interactions between knowledge and performance-oriented motivations. Indeed, when users believe a *swhidget*-hinting pattern lacks of consistency, they may conclude they will have to learn on a case by case basis which applications provide *swhidgets* and which ones do not. In other words, the cost of learning to use the interface in a way that relies on *swhidgets* may seem higher for users, which can lead them to discard *swhidgets* as globally demanding too much effort to learn. The results of the online study (See Section 8.6.4) suggest that difficulty of learning *swhidgets* can be a rare (5.4%) reason not to adopt them, despite overall results suggesting *swhidgets* are well understood by users and recognized across applications.

On the other hand, this negative effect of the lack of consistency on adoption of *swhidgets* can also not happen, being replaced with a positive effect on user experience when users decide to playfully explore the diverse applications to find which ones provide *swhidgets* (results of the online study suggest users tend to play with a *swhidget* after they have discovered it). In that case, the discovery of a *swhidget* can be considered as a random reward, and there is a great amount of literature in Psychology about the role of such rewards in conditioning human and animal behavior – in the field of video game design, it
is often referred to by designers and critics with expressions like "Skinner boxes" or "loot boxes" (e. g., [Sci11; Blo10; Ext12; LWW12]).

As a conclusion on the topic of transfer of knowledge, I would like to first recall how important the topic is for HCI in general, but even more for the study of signifier-less designs like *swhidgets*, since their discovery rely more heavily on external sources of knowledge (22.9% in the online survey, see Section 8.6.4) and transfer of knowledge from other areas of the interface (8.8%). However, as I have discussed in this section, the topic is complex and there are many aspects of the interaction and interface that affect transfers of knowledge, such as the recognize-ability and notice-ability of user interface patterns as signifiers, analogical reasoning in goal-oriented interaction problem resolution, user strategies, consistency of patterns across applications and views, users' ability to correctly evaluate this consistency, playful user behaviors and behavior conditioning. The model that I have presented in Chapter 7 and the concepts of affordances and signifiers are useful tools to discuss these questions theoretically and to identify precise hypotheses on the mechanisms of transfer of knowledge that can be tested experimentally. In addition, integrating the above list of concerns into more pragmatic research tools like questionnaires or usability evaluation methods is an area of research that, in my mind, deserves more attention. I created the questionnaire for the online study with these concerns in mind, knowing that, having not been formally validated before the study, the questionnaire could only suggest user behavior trends rather than provide conclusive observations – which was acceptable because it was used in a preliminary study that had no such high-level objectives and also intended to test the usefulness of such a questionnaire as a research tool. In my mind, this questionnaire did bring interesting results and should be refined to develop a well-validated research tool that could be instrumented in other studies.

### 9.1.2 Hedonic aspects of *Swhidgets*

#### Can pragmatic benefits justify to not use signifiers in a design?

The concept of transfer of knowledge – and more generally the idea of sources of knowledge that are outside the application concerned by a piece of knowledge – helps understanding how users can learn about *swhidgets* despite their lack of signifiers. It thus explains how this design can possibly work for users, providing a possible answer to my research question *how can users know swhidgets despite their lack of signifiers?* (Section 2.7.3), but it does not explain the benefits of not including signifiers in the design of *swhidgets*, which was the topic of my last my research question (Section 2.7.4).
I have discussed in this thesis a few (rare) plausible reasons for not providing signifiers that rely on pragmatical expected benefits for users, such as making interfaces easier to read and improving the consistency of swipe-based interaction techniques in the system. While such benefits cannot be dismissed, it can also be doubted that they are strong enough to justify not providing signifiers at all (rather than simply minimizing their visual salience). Indeed, it seems that these benefits would be overcome by the improved discoverability of useful *swhidgets* that could be obtained by adding signifiers – although validating this hypothesis would require a dedicated study, which would itself require means to quantify these benefits and measure them.

I have thus developed during this thesis the conviction that to justify the signifier-less design of *swhidgets*, which is questionable from the point of view of pragmatics, considerations for the hedonic aspects of interaction (see Section 7.4.3) were at least equally important than considerations for the pragmatic aspects. Hassenzahl comments that pragmatic and hedonic concerns can be in conflict: “Strikingly, taking the need for novelty and change into account might unavoidably imply a reduction of usability. Usability and joy of use might be partially incompatible, because the former requires consistency and simplicity, whereas the latter requires surprise and a certain amount of complexity” [HBB01]. Although research on UX is conscious of this possible trade-off between pragmatics and hedonic qualities of an interface or product, I believe *swhidgets* are a good case study to better understand the consequences of favoring hedonic qualities over pragmatic ones. In the remainder of this section, I discuss how this general research question on UX-oriented designs can be informed by the study of *swhidgets*.

*What types of benefits can we expect, based on the hedonic aspects of user experience?*

Discovery of advanced interaction techniques can create a strong stimulation, and as we have seen in the previous section, such a stimulation can even incite users to engage into playful searches for *swhidgets* – finding hidden things is after all the core task of many games and playful activities, from hide-and-seek to adventure video games. The stimulation created by a discovery can also reinforce the emotional attachment to a device or system: our relation to things become more personal when we have discovered one of their aspects that was hidden.

Conversely, as the online study showed for *swhidgets* (see Section 8.6.6), we easily get proud of our knowledge of “tricks” to interact with electronic devices or everyday objects. We also like to share it with our relatives. This knowledge is thus instrumented to satisfy our innate needs for both Competence and Relatedness (see Section 7.4.2 about the Self-Determination Theory).
The results of the online study (Section 8.6.6) also show that partici-
pants attributed to *swhidgets* their ability to do some tasks with their
device or to use the device in specific situations: the discovery of new
affordances can indeed bring new opportunities to change the way
we organize our lives so that it better matches with our values and
who we want to be, satisfying our needs for Autonomy – although
the relationship between theses higher goals and the affordance dis-
covered might be quite indirect. For instance, one could realize that
with Mail's *item swhidgets*, it becomes actually not too unpractical to
manage a great amount of mails on the small screen of the phone,
and could decide to do this activity during the commute to work in
subway instead of doing it in the office, saving time either to focus
on other work at the office, or to leave for work later and enjoy shar-
ing a breakfast with beloved ones – or simply to distract from the
unpleasant aspects of the commute.

*Does accounting for the hedonic aspects of user experience require a shift in how we understand the roles of signifiers and discovery of affordances?*

Historically, there has been a transition from computers as profes-
sional tools evaluated on their capacity to effectively increase workers'
productivity at reasonable costs, to the personal and always-available
smartphones that we have today and use not only for work but also
for many other aspects of our lives. This transition naturally causes
a shift in the considerations about what are the benefits of a device
or interaction technique, from pragmatic concerns to hedonic ones:
it is less about what I can do with the computer and more about
how it supports me in being who I want to be. In this context, the
focus of design may shift from designing products that communi-
cate clearly what they are for and how to use them (typically using
signifiers) to designing products that satisfy consumers' needs for
self-improvement.

Discovering new affordances and developing mastery have always
been a concern of user interface design, but it was mostly valued for its
consequences on user performance as extrinsic reward rather than for
the user experience of doing so, valued for itself as an intrinsic reward.
Providing users with opportunities to experience the discovery of new
affordances and of developing mastery might have become a new
design concern that could justify the signifier-less design of *swhidgets*.

Such an approach to design suggests new ways to formulate usabil-
ity concerns. For instance, a question like "is this affordance signified
clearly to the user?" might be replaced by the more general question
"does the design invite users to engage in an exploration that will lead
them to discover this affordance?" Some designers may already have
started more or less consciously to ask such questions, but the conse-
quences of such approaches on users and on the quality of designed
products are hard to foresee, and should thus be studied quickly.
There seems to be currently little evidence that user interfaces are designed with such a question in mind. It has however become frequent in the design of video games, as the player experience of discovering an unknown world and the way it works is a primary focus of game design and there are more opportunities to orient the player experience through the design of a world and the challenges it provides than for a user interface. As the studies of video game design tend to be more common in HCI, it is likely that HCI practitioners and researchers will get inspiration from video game design philosophies and focus more on this type of questions in the future.

*How can we study such approaches to design?*

Investigating such approaches to design requires new conceptual tools (both for design and HCI research) [HBB01]. Although there already exist conceptual models such as Hassenzahl's model of UX (see Section 7.4.3) and evaluation tools like the AttrakDiff questionnaire [HBK03; Lal+15], such models and tools are not always adapted to studies on user adoption of interaction techniques. For instance, the AttrakDiff questionnaire evaluates how users perceive a product or interface and is usually used to investigate users' perception of a product after their first contact with it. Although the questionnaire can be instrumented to investigate how this perception evolves in time with repeated measures [Kar+09], it is not made to investigate how a design contributes in the long term to how a product fits in its users' lives and satisfies their evolving needs. It can however bring important information, and I plan to use it in a study under preparation, in order to complement and confirm some aspects of the interaction with *swhidgets* that have been suggested by the online study, such as the fact that participants enjoyed interacting with *swhidgets*.

I have myself attempted to integrate questions inspired by the Self-Determination Theory at the end of the online study final questionnaire (see Table 8.5), although constraints on the length of the study did not allow me to go as deep as would have been necessary. I also faced the issue that existing SDT-based questionnaires – such as the *Intrinsic Motivation Inventory* (IMI) [RRP06] – tend to evaluate participants’ satisfaction of needs during an activity without targeting the aspects of the activity (or of the design of artifacts involved in the activity) that are believed to cause such satisfaction of needs. Although there exist such targeted questionnaires in specific areas of design – such as the *Player Experience of Need Satisfaction* (PENS) questionnaire for video games with specific questions on topics such as presence in the game world and intuitive controls [RRP06] – I am not aware of any questionnaire that would be suitable for the analysis of *swhidgets*. Since some of the questions I integrated in the questionnaire brought clear and interesting results despite their limitations, I firmly believe that the creation and validation of a SDT-based questionnaire to evaluate signifier-less designs is an important research perspective.
*What are the consequences of designing for the hedonic aspects of user experience?*

A perspective for future works around this approach of design is to study its implications from the point of view of ethics. When usability is decreased to create opportunities for stimulation and fulfill users' need for personal growth, don't we artificially instrument a product to satisfy a basic need instead of truly giving users a way to self-improve? Indeed, the increase of knowledge concerns the knowledge of the product itself instead of concerning knowledge that may be more important to the user. If the design focus is more concerned by "who I want to be" than by "what I can do with the product", maybe the answer to the question "who do I want to be?" should not be "an expert in this particular product". This ethical question is connected to Hassenzahl's distinction between "experiences mediated through a product versus experience of a product" [Has18].

## 9.2 IMPROVING SWHIDGETS

### 9.2.1 Revealing Swhidgets with Animated Transitions

From the analysis of *swhidgets* conducted in Chapter 2 and the results of the experiment presented in Chapter 8, I derive the need to improve on some aspects of the design of *swhidgets* with the following considerations:

1. Obviously, improved designs should foster the discoverability of *swhidgets*;

2. I do not want to change the affordances and interactive behaviors of *swhidgets* themselves, nor to create new interaction patterns for users. My goal is therefore not to create new interactions or to redesign interactions with *swhidgets*,<fnref n="1" /> but to build upon their design;

3. I want to preserve the benefits brought by the absence of signifiers, and will thus avoid the obvious solution of adding permanent signifiers. In other words, I stay in the paradigm of signifier-less designs defined in Chapter 6.

My approach is to explore the design space for sources of knowledge that have the smallest possible distance (as defined in Section 7.7.1)

<footnote n="1">¹ In a first attempt in the direction of the work on animation exposed later in this section, I thought about simulating gravity so that the *item swhidgets* in a list view like Mail's thread list would be revealed by tilting slightly the device (the items of the list sliding sideways a little under the effect of gravity). However, informal tests with users quickly revealed they thought it was a new way to interact with the *swhidgets* rather than a way to inform about their presence, and spent a considerable time trying to exploit this new affordance.</footnote>
without being direct signifiers (as defined in Section 6.1), and that foster discovery of *swhidgets* out of problem-solving contexts (as defined in Section 7.7.2).

To keep a small *distance* without using direct signifiers, designers may use temporary signifiers that would be displayed only when 1) the *swhidgets* concerned by these signifiers are available (or about to be) and potentially useful for users,<fnref n="2" /> but 2) the user is unlikely to interact with these *swhidgets* at that moment, and 3) the temporary signifiers would not distract users in their tasks. Points 1 and 2 precise the idea of a small distance while point 3 extends the concerns of an uncluttered interface to temporary signifiers.

I therefore propose to use *animated transitions* as a way to give *swhidgets*-unaware users a chance to discover the *swhidgets* as they interact with the device. The goal is to raise users’ awareness of interface features they do not know and stimulate their curiosity, in order to motivate them to perform the sliding gestures corresponding to these *swhidgets*. Animations are already used in many places in mobile touch-based systems and applications as a way to “increase understanding”.<fnref n="3" /> Their use is believed to facilitate the understanding of changes [CU93] and has been repeatedly demonstrated as helping to understand the spatial relationship between views (see [Che+10] for a brief review).

### 9.2.2 Considerations for the Design of Animated Transitions

**ELIGIBLE TRANSITIONS** Animated transitions are often used when a new view is *loaded* (a UI view replaces another one), or *updated* (an element in this view is added, removed, or otherwise changed at the user's or system's initiative). Since *swhidgets* are usually hidden outside of the view's boundary, I believe that user-triggered animated transitions between views could expose the space around the views and let users glimpse at the *swhidgets* lying there before they get hidden. I advocate to focus on animated transitions triggered by user events, as opposed to transitions triggered by external events that change the state of an element (e.g., when a message transitions from the *sent* state to the *read by recipient* state). Indeed, in addition to the

<footnote n="2">2 This point excludes the situation in which the user triggers a command provided by a *swhidget* through alternative methods of activation, which is a case similar to the hotkey hints provided in menus, as discussed in Section 6.2.3. There is however room for using animations in such situations too. For instance, deleting mails with the Edit button rather than *swhidgets* already uses an animation to remove the corresponding list items. This animation is also used when deleting emails with an *item swhidget*, although in that case the sws are still displayed in the item removed. The animation of item removal with the Edit button could be extended to show the *swhidget* as if they had been used, to expose these *swhidgets* and suggest the appropriate gesture to reveal them.</footnote>

<footnote n="3">3 https://developer.apple.com/design/human-interface-guidelines/ios/visual-design/animation/</footnote>
risk that users interact with the animated elements by mistake because they did not see them move or could not stop an engaged gesture in time, understanding such animations may also have a higher cognitive load because it requires users to understand why something changes in a first place. Moreover, such on-update animations will most often be redundant with other animated transitions between views.

### CONFLICTING PURPOSES
Animated transitions serve a primary purpose, which can broadly be described as “helping the user understanding what is happening”. In the case of animated view transitions, it is “helping the user understanding that the interface is switching to a new view”. When using animated transitions to expose *swhidgets* and give users a chance to discover them, we introduce a secondary purpose of the animation, and we should then make sure it does not take precedence over the primary one or impede it. A good rule of thumb seems therefore to minimize the amount of information provided to the user for this secondary purpose only, by both reducing the visual salience of the *swhidgets* during the animation, and by avoiding redundancy (multiple animations exposing the same widgets). Indeed, as we have seen in Section 7.3.1, users’ informational needs and abilities to process the information conveyed by the animation evolve as they gain more experience with the interface, and can shift from the primary purpose of the animation to the secondary one (although they can also learn to discard information that seem irrelevant for them).

### COMPLEXITY OF ANIMATIONS
When not used judiciously though, animations and motion effects can potentially be distractive or slowing down interaction [BWCo3], which is why we should constrain new designs to animations that are relatively short and do not involve complex movement patterns (elements moving in different directions simultaneously, non-linear trajectories, etc.)

### INTEGRATION IN USERS' WORKFLOWS
Attention should also be payed not to impact users' workflows. For instance, in some applications the most important content can be expected to be in certain areas of the view (e.g., newest mails are on top of the list in the Mail application), and I then consider animations where contents move less in these areas or stop moving earlier than in other areas, so that users can read them as soon as possible.

### 9.2.3 Proposed Animated Transitions

The interface of touch-based systems like iOS and Android already makes great use of animated transitions. To leverage the existing transitions in these systems to help users discovering *swhidgets*, we need to distinguish the different types of animations they use. For
<figure><img src="image_96.png" /><figcaption>Figure 9.1: Examples of animated transitions that can facilitate the discovery of *swmidgets*.</figcaption></figure>
instance, iOS uses animations based on three types of geometrical
transformations: Scale variations (when opening an application or
locking the screen), 2D translations (when navigating within an
application) and perspective rotations (when selecting an element
in a ** view, for instance to change the current tab in Safari). I
propose the following three types of slightly modified versions of these
transformations in order to increase the discoverability of *swhidgets*:

* *Scale-in* progressively scales the new view but differs from the
  iOS built-in transition in that it does not clip the view's content
during the transition, thus giving the opportunity to see during
the animation the *swhidgets* that will end hidden outside the
screen (Figure 9.1c).

* *Slide-in* applies additional 2D translations to the new view and *swhidgets* during the animated transition to let users see behind which bezel of the display the *swhidgets* will finally hide (Figures 9.1a and 9.1d).

* *Perspective* applies a perspective rotation but, similarly to scale-in, does not clip the view's content to reveal the *swhidgets* during the animation (Figure 9.1b).

*Scale-in, slide-in* and *perspective* could be instantiated for all the
tasks that were tested in the first study (listed in Table 8.1) whenever
possible (scale-in and perspective-based cannot be applied to the
System unlock scenario without significantly changing the metaphor
currently employed in iOS). I detail only a subset of these instances
below.

SYSTEM UNLOCK SLIDE-IN On unlock, iOS currently translates-out the lock-screen above the top edge of the display, which is extremely rapid (and hard to perceive). By simply sliding-out the Control and Information panels at the same time, but at a relatively lower speed, users could notice the panels progressively translating below the top and bottom phone bezels (Figure 9.1a). Using a simple easing-out timing function, the information and control panels quickly move in the vicinity of the edges of the display, thus not hiding the main view, but slowing down before getting completely out of view so that users can still perceive this motion.

MAIL APP PERSPECTIVE When an application is launched from
the home screen, iOS scales-in the application's view from its icon
to full-screen. By changing this animation for a perspective rotation
without clipping the view's content at the border of the email list
(Figure 9.1b), users have the opportunity to see the *item swhidgets*
located on both the left and right sides of each e-mail in the list. The
small perspective distortion at the top of the screen also allows them
to start reading the first (most recent) mail before the transition ends.
MESSAGE APP SCALE-IN AND SLIDE-IN iOS scales the view of an application when it is launched, but uses translations to animate view transitions inside a same application. I propose to apply scale-in to the discussion list when the Message application is launched, so that users can notice the associated *item swhidgets* (Figure 9.1c). I also propose to increase the amplitude of the existing slide-in transition when a discussion is opened, to create a kind of "overshoot" that temporarily reveals the message time *view swhidgets* (Figure 9.1d), before they slowly slide back past the right edge of the display – giving users an opportunity to discover these *view swhidgets* and suggesting the leftward sliding gesture that brings them back into the view.

### 9.2.4 Testing the animations

Mock-ups of the proposed animations have been realized in HTML5 and JavaScript, along with *swhidgets* that reproduce as precisely as possible the iOS lock screen interface and Mail and Message applications – although the data displayed are not real data and the commands triggered with the *swhidgets* have no actual effect.

While these mock-ups allow to test the animation and fine-tune their parameters, an intended use was also to actually compare the new animations with the current design of *swhidgets*, in controlled studies. These studies could unfortunately not be realized during my PhD due to a lack of time, but also due to methodology issues. I thus plan to experimentally validate the efficiency of these prototypes in fostering the discovery of *swhidgets* as future work.

Such a validation is a however difficult task, as no ready-to use validation procedure has been proposed in the literature to test the discovery of features from commercial systems. Indeed, the discovery of system features and input methods is something that should be assessed on relatively long term and can hardly be evaluated with a laboratory experiment. It requires fully functional applications similar to the ones they are compared to. Moreover, it would require to first measure the level of knowledge of participants regarding the tested input methods, while avoiding that participants understand what the experiment is about to prevent any demand characteristics issue [Orno9]. It might alternatively require to find participants that have currently no experience with touch-based devices, but such people are getting rare and might not be representative of other users.

## 9.3 CONCLUSION

During this thesis, I studied the question of why designers would create interfaces that do not clearly expose some of their interaction possibilities, taking the case of *swhidgets* as an example and focus of inquiry. My contributions to this research are:
* An analysis of the fundamental notions required to define precisely this research question, including affordances, signifiers and semiotics.

* A definition of *signifier-less designs* and the observation that this type of design is common in user interfaces and deserves more attention from HCI research.

* A model of user discovery and adoption of interaction techniques, relying on the categorizations of three dimensions and their relationships: user's current knowledge and skills, user's motivations, and the design means of informing users provided by the interface.

* The notions of Sources of Knowledge and Degrees of Knowledge derived from this model, that can be instrumented in experiments.

* Two studies on *swhidgets* that investigated how well users known them, how they discovered them, how they feel about them, and how they integrate them in the way they think about the interaction. These studies revealed that *swhidgets* were globally appreciated and relatively well known by users, although there is still room for improvement, notably for some specific *swhidgets*.

These contributions are however limited in that the object of this study – i.e., interfaces that do not clearly expose some of their interaction possibilities – has traditionally been seen in HCI as a failure of design that was acceptable only for interaction possibilities targeting expert users. I therefore faced a lack of conceptual tools to study such designs beyond denouncing their flaws. As such, an important aspect of the work done during my PhD has been to identify and unify the conceptual tools needed for the study of *swhidgets* and *signifier-less designs*. As a result from this focus on identifying conceptual tools and research questions, the studies that I have done only provide preliminary and incomplete answers to the main research question of this PhD and to the question of the suitability of the conceptual tools that I have identified for conducting such a research. Further research is required to confirm these preliminary results, to extend them to other categories of systems, users, and signifier-less designs, and to better understand the design space of *swhidgets*. The research presented in this PhD however opens a few new perspectives for original research, as I have discussed at the beginning of this chapter.
<figure><img src="image_97.png" /></figure>
<figure><img src="image_98.png" /></figure>
<figure><img src="image_99.png" /></figure>
<figure><img src="image_100.png" /></figure>
Part VI

APPENDIX
<figure><img src="image_101.png" /></figure>
<figure><img src="image_102.png" /></figure>
<figure><img src="image_103.png" /></figure>
J

FIRST QUESTIONNAIRE OF THE LABORATORY
STUDY

Participant ID

2018

Welcome and thank you for taking this test with us!

Please tell us about you:

- What is your age:

- What is your gender:

☐ man

☐ woman

☐ other/don't want to answer

- What is your education level?

- What is your professional domain and current activity?

- Have you worked in a different domain before? If so, in which domain(s)?

- What is your preferred mobile phone operating system?

☐ Android

☐ Windows

☐ iOS

☐ Others (please precise)

☐ No preference

- How familiar are you with the following mobile phone systems?

<table><thead><tr><th rowspan="2">System</th><th colspan="5">not familiar</th><th colspan="1">very familiar</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th></tr></thead><tbody><tr><td>☐ Android, __________</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>☐ iOS __________</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>☐ Windows __________</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>☐ Others (please precise) __________</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>☐ Others (please precise) __________</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr></tbody></table>
- Do you also own a tablet?

<figure><img src="image_104.png" /></figure>

Yes

<figure><img src="image_105.png" /></figure>

No

- If you do, what is the system of your tablet?

<figure><img src="image_106.png" /></figure>

Android

<figure><img src="image_107.png" /></figure>

Windows

<figure><img src="image_108.png" /></figure>

iOS

<figure><img src="image_109.png" /></figure>

Others (please precise)

- How familiar are you with the following systems on tablets?

<table><thead><tr><th rowspan="2">System</th><th colspan="5">not familiar</th><th colspan="5">very familiar</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th></tr></thead><tbody><tr><td>Android</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>iOS</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>Windows</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>Others (please precise)</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr></tbody></table>

- What activities do you perform on your touch-based devices?

<figure><img src="image_110.png" /></figure>

accepting phone calls

<figure><img src="image_111.png" /></figure>

drawing

<figure><img src="image_112.png" /></figure>

taking pictures/videos

<figure><img src="image_113.png" /></figure>

calculating

<figure><img src="image_114.png" /></figure>

writing/taking note

<figure><img src="image_115.png" /></figure>

programming

<figure><img src="image_116.png" /></figure>

recording voice/singing

<figure><img src="image_117.png" /></figure>

listening to music

<figure><img src="image_118.png" /></figure>

checking mails

<figure><img src="image_119.png" /></figure>

watching tv/video

<figure><img src="image_120.png" /></figure>

browsing the web

<figure><img src="image_121.png" /></figure>

text messaging

<figure><img src="image_122.png" /></figure>

chatting

<figure><img src="image_123.png" /></figure>

video conferencing

<figure><img src="image_124.png" /></figure>

playing games

<figure><img src="image_125.png" /></figure>

lifelogging

<figure><img src="image_126.png" /></figure>

other activities

- How much time do you spend daily on your touch-based devices?

<figure><img src="image_127.png" /></figure>

within 30 minutes

<figure><img src="image_128.png" /></figure>

between 3 to 5 hours

<figure><img src="image_129.png" /></figure>

between 30 mins to 1 hr

<figure><img src="image_130.png" /></figure>

more than 5 hours

<figure><img src="image_131.png" /></figure>

between 1 to 3 hours
## Just a few last questions!

We are almost there!

- How many smartphones did you own since 2008?

- Which platform and which version were they running on?

<figure><img src="image_132.png" /></figure>

- Do you consider yourself as a person who is highly interested in new technology ?

<table><thead><tr><th>No, I don't</th><th>I don't know</th><th>Yes, I do</th></tr></thead><tbody><tr><td>1</td><td>2</td><td>3</td></tr></tbody></table>

- Do these statements describe you?

<table><thead><tr><th>Description</th><th colspan="2">Totally disagree</th><th colspan="3">Totally agree</th></tr></thead><tbody><tr><td>• I feel that new technology improves my life</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I always want to know the latest news on technology</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I have close friends who keep me informed on new technology</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr></tbody></table>
<table><thead><tr><th>Participant ID</th><th colspan="5">2018</th></tr></thead><tbody><tr><td>• I am always the one who tells others about the latest technology news</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I frequently search for news on technology</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I follow technical web blogs</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I always want to have the latest version of devices</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I sometime support innovative projects</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• I am willing to try on all types of upcoming applications/devices</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• When I get a new device/feature, I try/check almost every new feature</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr><tr><td>• In discussions, talking about technology is the best way to get my attention</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr></tbody></table>
<figure><img src="image_133.png" /></figure>
K

SECOND QUESTIONNAIRE OF THE LABORATORY
STUDY

Participant ID

2018

This is the final section of the interview!

Congratulations for reaching so far :-)

Here are some questions that require some imagination from you:

- Do you think the existing hidden controls help you on your daily tasks?

☐ Yes

☐ No

- Where does your knowledge of these hidden features come from? (Check all items that apply)

• From the Internet

☐ From bloggers' reviews

☐ From video tutorials

☐ From a forum

☐ Other: _________________________

• From other people

☐ I asked for help from professionals

☐ I heard people talking about it

☐ Other: _________________________

• From friends

☐ I have knowledgeable friends who can teach me

☐ My friends were talking about it

☐ Other: _________________________

• From the device itself

☐ I discovered it myself by searching the feature in the interface

☐ I discovered it myself by randomly exploring the device

☐ I discovered it myself by accident
Other :

• From another system

I can guess it because I use another system

I can guess it because it's just opposite from Android

• From the application itself

I just discovered it on my own by exploring

I discovered it on my own by accident

I discovered it with help from an icon or other hint

Other :

• During this interview

By myself (I observed it on my own)

From the investigator (Nicole taught you)

Other :

- If you are a designer interested in adding this kind of hidden features to your design, please tell us:

• on what platform/device?

• with which kind of application/software?

• what kind of gesture is used to reveal the feature?

Thank you so much for your participation!
<figure><img src="image_134.png" /></figure>
Alzayat, Ayman, Mark Hancock, and Miguel A. Nacenta (2017). "Measuring Readiness-to-Hand through Differences in Attention to the Task vs. Attention to the Tool." In: *Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces*. ISS '17. Brighton, United Kingdom: Association for Computing Machinery, pp. 42–51. ISBN: 9781450346917. DOI: 10.1145/3132272.3134114.

Andersen, Peter Bøgh (1991). *A Theory of Computer Semiotics: Semiotic Approaches to Construction and Assessment of Computer Systems*. USA: Cambridge University Press. ISBN: 0521393361.

— (Dec. 2001). "What Semiotics Can and Cannot Do for HCI." In: *Know.-Based Syst*. 14.8, pp. 419–424. ISSN: 0950-7051. DOI: 10.1016/S0950-7051(01)00134-4.

Apple (2019). *Apple Human Interface Guidelines*. Retrieved July 8, 2019. URL: https://developer.apple.com/design/human-interface-guidelines/ios/overview/themes/.

Avery, Jeff and Edward Lank (2016). "Surveying Expert-Level Gesture Use and Adoption on Multi-Touch Tablets." In: *Proceedings of the 2016 ACM Conference on Designing Interactive Systems*. DIS '16. Brisbane, QLD, Australia: ACM, pp. 577–581. ISBN: 978-1-4503-4031-1. DOI: 10.1145/2901790.2901895.

Bærentsen, Klaus B. and Johan Trettvik (2002). "An Activity Theory Approach to Affordance." In: *Proceedings of the Second Nordic Conference on Human-Computer Interaction*. NordiCHI '02. Aarhus, Denmark: Association for Computing Machinery, pp. 51–60. ISBN: 1581136161. DOI: 10.1145/572020.572028.

Bailly, Gilles, Emmanouil Giannisakis, Marion Morel, and Catherine Achard (2018). "Characterize the Transition from Menus to Hotkeys." In: *Proceedings of the 30th Conference on l'Interaction Homme-Machine*. IHM '18. Brest, France: Association for Computing Machinery, pp. 30–41. ISBN: 9781450360784. DOI: 10.1145/3286689.3286699.

Barr, Pippin, Robert Biddle, and James Noble (2004). "A Semiotic Model of User-Interface Metaphor." In: *Virtual, Distributed and Flexible Organisations: Studies in Organisational Semiotics*. Ed. by Kecheng Liu. Dordrecht: Springer Netherlands, pp. 189–215. ISBN: 978-1-4020-2162-6. DOI: 10.1007/1-4020-2162-3_13.

Bartram, Lyn, Colin Ware, and Tom Calvert (2003). "Moticons: detection, distraction and task." In: *International Journal of Human-Computer Studies* 58.5, pp. 515–545.

Bau, Olivier and Wendy E. Mackay (2008). "OctoPocus: A Dynamic Guide for Learning Gesture-based Command Sets." In: *Proceedings*
of the 21st Annual ACM Symposium on User Interface Software and Technology. UIST '08. Monterey, CA, USA: ACM, pp. 37–46. ISBN: 978-1-59593-975-3. DOI: 10.1145/1449715.1449724.

Beaudouin-Lafon, Michel (2000). "Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '00. The Hague, The Netherlands: Association for Computing Machinery, pp. 446–453. ISBN: 1581132166. DOI: 10.1145/332040.332473.

— (2004). "Designing Interaction, Not Interfaces." In: *Proceedings of the Working Conference on Advanced Visual Interfaces*. AVI '04. Gallipoli, Italy: ACM, pp. 15–22. ISBN: 1-58113-867-9. DOI: 10.1145/989863.989865.

Benyon, D. (1994). "A Functional Model of Interacting Systems: A Semiotic Approach." In: CSCW and Artificial Intelligence. Ed. by John H. Connolly and Ernest A. Edmonds. London: Springer London, pp. 105–125. ISBN: 978-1-4471-2035-3. DOI: 10.1007/978-1-4471-2035-3_8.

Bérard, François and Amélie Rochet-Capellan (2015). "The Transfer of Learning as HCI Similarity: Towards an Objective Assessment of the Sensory-Motor Basis of Naturalness." In: *Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems*. CHI '15. Seoul, Republic of Korea: Association for Computing Machinery, pp. 1315–1324. ISBN: 9781450331456. DOI: 10.1145/2702123.2702359.

Bhavnani, Suresh K. and Bonnie E. John (2000). "The Strategic Use of Complex Computer Systems." In: *Human-Computer Interaction* 15.2–3, pp. 107–137. DOI: 10.1207/S15327051HCI1523\_3.

Blow, Jonathan (2010). *Video Games and the Human Condition*. blog post: http://the-witness.net/news/2010/10/a-new-lecture-about-game-design/, video of the lecture: https://youtu.be/SqFu50-oPmU. Retrieved July 25, 2020.

Bonderup Dohn, Nina (June 2009). "Affordances revisited: Articulating a Merleau-Pontian view." In: *International Journal of Computer-Supported Collaborative Learning* 4.2, pp. 151–170. DOI: 10.1007/s11412-009-9062-z.

Brühlmann, Florian, Beat Vollenwyder, Klaus Opwis, and Elisa D. Mekler (2018). "Measuring the "Why" of Interaction: Development and Validation of the User Motivation Inventory (UMI)." In: *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems*. CHI '18. Montreal QC, Canada: Association for Computing Machinery, pp. 1–13. ISBN: 9781450356206. DOI: 10.1145/3173574.3173680.

Brunner, Edgar and Madan L. Puri (Jan. 2001). "Nonparametric methods in factorial designs." In: *Statistical Papers* 42.1, pp. 1–52. ISSN: 1613-9798. DOI: 10.1007/s003620000039.
Card, Stuart K., Allen Newell, and Thomas P. Moran (1983). *The Psychology of Human-Computer Interaction*. USA: L. Erlbaum Associates Inc. ISBN: 0898592437.

Carroll, John M and Mary Beth Rosson (1987). *Paradox of the active user*. The MIT Press.

Chang, Bay-Wei and David Ungar (1993). "Animation: From Cartoons to the User Interface." In: *Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology*. UIST '93. Atlanta, Georgia, USA: ACM, pp. 45-55. ISBN: 0-89791-628-X. DOI: 10.1145/168642.168647.

Chevalier, Fanny, Pierre Dragicevic, Anastasia Bezerianos, and Jean-Daniel Fekete (2010). "Using Text Animated Transitions to Support Navigation in Document Histories." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '10. Atlanta, Georgia, USA: ACM, pp. 683-692. ISBN: 978-1-60558-929-9. DOI: 10.1145/1753326.1753427.

Cockburn, Andy, Carl Gutwin, Joey Scarr, and Sylvain Malacria (Nov. 2014). "Supporting Novice to Expert Transitions in User Interfaces." In: *ACM Computing Surveys* 47.2, 31:1-31:36. ISSN: 0360-0300. DOI: 10.1145/2659796.

Cooper, Alan, Robert Reimann, David Cronin, and Christopher Noesel (2014). *About Face: The Essentials of Interaction Design*. John Wiley & Sons.

Coppers, Sven, Kris Luyten, Davy Vanacken, David Navarre, Philippe Palanque, and Christine Gris (June 2019). "Fortunettes: Feedforward about the Future State of GUIWidgets." In: *Proc. ACM Hum.-Comput. Interact*. 3.EICS. DOI: 10.1145/3331162.

De Bruijn, Oscar and Robert Spence (May 2008). "A New Framework for Theory-Based Interaction Design Applied to Serendipitous Information Retrieval." In: *ACM Trans. Comput.-Hum. Interact*. 15.1. ISSN: 1073-0516. DOI: 10.1145/1352782.1352787.

deSouza, Clarisse Sieckenius (2005). *The Semiotic Engineering of Human-Computer Interaction (Acting with Technology)*. The MIT Press. ISBN: 0262042207.

Diefenbach, Sarah and Marc Hassenzahl (2008). "Give Me a Reason: Hedonic Product Choice and Justification." In: *CHI '08 Extended Abstracts on Human Factors in Computing Systems*. CHI EA '08. Florence, Italy: Association for Computing Machinery, pp. 3051-3056. ISBN: 9781605580128. DOI: 10.1145/1358628.1358806.

Djajadiningrat, Tom, Kees Overbeeke, and Stephan Wensveen (2002). "But How, Donald, Tell Us How? On the Creation of Meaning in Interaction Design through Feedforward and Inherent Feedback." In: *Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques*. DIS '02. London, England: Association for Computing Machinery, pp. 285-291. ISBN: 1581135157. DOI: 10.1145/778712.778752.
Djajadiningrat, Tom, Stephan Wensveen, Joep Frens, and Kees Overbeeke (Sept. 2004). "Tangible Products: Redressing the Balance between Appearance and Action." In: *Personal Ubiquitous Comput. 8.5*, pp. 294–309. ISSN: 1617-4909. DOI: 10.1007/s00779-004-0293-8.

ExtraCredits, ed. (2012). *The Skinner Box – How Games Condition People to Play More*. Youtube. Retrieved July 25, 2020. URL: https://youtu.be/tWtvrPTbQ_c.

Fitts, Paul M and Michael I Posner (1967). "Human Performance. Brooks." In: *Cole, Belmont, CA 5*, pp. 7–16.

Gaver, William W. (Mar. 1989). "The SonicFinder: An Interface That Uses Auditory Icons." In: *Hum.-Comput. Interact.* 4.1, pp. 67–94. ISSN: 0737-0024. DOI: 10.1207/s15327051hci0401_3.

— (1991). "Technology Affordances." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '91. New Orleans, Louisiana, USA: Association for Computing Machinery, pp. 79–84. ISBN: 0897913833. DOI: 10.1145/108844.108856.

Gentner, Don and Jakob Nielsen (Aug. 1996). "The Anti-Mac Interface." In: *Commun. ACM* 39.8, pp. 70–82. ISSN: 0001-0782. DOI: 10.1145/232014.232032.

Gibson, James J (1977). "The Theory of Affordances." In: *Perceiving, Acting, and Knowing: Toward an Ecological Psychology*. Hillsdale. Lawrence Erlbaum, pp. 67–82.

— (1979). *The ecological approach to visual perception*. Boston, MA, US. Houghton, Mifflin and Company.

Grossman, Tovi, Pierre Dragicevic, and Ravin Balakrishnan (2007). "Strategies for Accelerating On-Line Learning of Hotkeys." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '07. San Jose, California, USA: Association for Computing Machinery, pp. 1591–1600. ISBN: 9781595935939. DOI: 10.1145/1240624.1240865.

Grossman, Tovi and George Fitzmaurice (2015). "An Investigation of Metrics for the In Situ Detection of Software Expertise." In: *Human-Computer Interaction* 30.1, pp. 64–102. DOI: 10.1080/07370024.2014.881668.

Grossman, Tovi, George Fitzmaurice, and Ramtin Attar (2009). "A Survey of Software Learnability: Metrics, Methodologies and Guidelines." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '09. Boston, MA, USA: Association for Computing Machinery, pp. 649–658. ISBN: 9781605582467. DOI: 10.1145/1518701.1518803.

Grudin, Jonathan (Oct. 1989). "The Case against User Interface Consistency." In: *Commun. ACM* 32.10, pp. 1164–1173. ISSN: 0001-0782. DOI: 10.1145/67933.67934.
— (Mar. 2006). "The GUI Shock: Computer Graphics and Human-Computer Interaction." In: *Interactions* 13.2, pp. 45–47, 55. ISSN: 1072-5520. DOI: 10.1145/1116715.1116751.

Guillon, Maxime, François Leitner, and Laurence Nigay (2015). "Investigating Visual Feedforward for Target Expansion Techniques." In: *Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems*. CHI '15. Seoul, Republic of Korea: Association for Computing Machinery, pp. 2777–2786. ISBN: 9781450331456. DOI: 10.1145/2702123.2702375.

Halasz, Frank and Thomas P. Moran (1982). "Analogy Considered Harmful." In: *Proceedings of the 1982 Conference on Human Factors in Computing Systems*. CHI '82. Gaithersburg, Maryland, USA: Association for Computing Machinery, pp. 383–386. ISBN: 9781450373890. DOI: 10.1145/800049.801816.

Hartson, Rex (2003). "Cognitive, physical, sensory, and functional affordances in interaction design." In: *Behaviour & Information Technology* 22.5, pp. 315–338. DOI: 10.1080/01449290310001592587.

Hassenzahl, Marc (2003). "The Thing and I: Understanding the Relationship Between User and Product." In: *Funology: From Usability to Enjoyment*. Ed. by Mark A. Blythe, Kees Overbeeke, Andrew F. Monk, and Peter C. Wright. Dordrecht: Springer Netherlands, pp. 31–42. ISBN: 978-1-4020-2967-7. DOI: 10.1007/1-4020-2967-5_4.

— (2018). "The Thing and I: (Summer of '17 Remix)." In: *Funology 2: From Usability to Enjoyment*. Ed. by Mark Blythe and Andrew Monk. Cham: Springer International Publishing, pp. 17–31. ISBN: 978-3-319-68213-6. DOI: 10.1007/978-3-319-68213-6_19.

Hassenzahl, Marc, Andreas Beu, and Michael Burmester (Jan. 2001). "Engineering Joy." In: *IEEE Softw*. 18.1, pp. 70–76. ISSN: 0740-7459. DOI: 10.1109/52.903170.

Hassenzahl, Marc, Michael Burmester, and Franz Koller (2003). "At-trakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität." In: *Mensch & Computer 2003: Interaktion in Bewegung*. Ed. by Gerd Szwillus and Jürgen Ziegler. Wiesbaden: Vieweg+Teubner Verlag, pp. 187–196. ISBN: 978-3-322-80058-9. DOI: 10.1007/978-3-322-80058-9_19.

Hofmeester, Kay and Dennis Wixon (2010). "Using Metaphors to Create a Natural User Interface for Microsoft Surface." In: *CHI '10 Extended Abstracts on Human Factors in Computing Systems*. CHI EA '10. Atlanta, Georgia, USA: Association for Computing Machinery, pp. 4629–4644. ISBN: 9781605589305. DOI: 10.1145/1753846.1754204.

Hope, Computer (2019). *Definition of Charms*. Retrieved July 8, 2019. URL: https://www.computerhope.com/jargon/c/charms.htm.

Horn, Michael S. (2013). "The Role of Cultural Forms in Tangible Interaction Design." In: *Proceedings of the 7th International Conference*
*on Tangible, Embedded and Embodied Interaction*. TEI '13. Barcelona, Spain: Association for Computing Machinery, pp. 117–124. ISBN: 9781450318983. DOI: 10.1145/2460625.2460643.

Hornbæk, Kasper and Antti Oulasvirta (2017). "What Is Interaction?" In: *Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems*. CHI '17. Denver, Colorado, USA: Association for Computing Machinery, pp. 5040–5052. ISBN: 9781450346559. DOI: 10.1145/3025453.3025765.

Hutchins, Edwin L., James D. Hollan, and Donald A. Norman (1985). "Direct Manipulation Interfaces." In: *Human-Computer Interaction* 1.4, pp. 311–338. DOI: 10.1207/s15327051hci0104\_2.

Jacob, Robert J.K., Audrey Girouard, Leanne M. Hirshfield, Michael S. Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum (2008). "Reality-based Interaction: A Framework for post-WIMP Interfaces." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '08. Florence, Italy: ACM, pp. 201–210. ISBN: 978-1-60558-011-1. DOI: 10.1145/1357054.1357089.

Jain, Mohit and Ravin Balakrishnan (2012). "User Learning and Performance with Bezel Menus." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '12. Austin, Texas, USA: ACM, pp. 2221–2230. ISBN: 978-1-4503-1015-4. DOI: 10.1145/2207676.2208376.

Janlert, Lars-Erik (2014). "The ubiquitous button." In: *Interactions* 21.3, pp. 26–33. URL: https://interactions.acm.org/archive/view/ may-june-2014/the-ubiquitous-button.

Johnson, Jeff (2014). *Designing with the Mind in Mind, Second Edition: Simple Guide to Understanding User Interface Design Guidelines*. 2nd. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. ISBN: 0124079148.

Kaptein, Maurits Clemens, Clifford Nass, and Panos Markopoulos (2010). "Powerful and Consistent Analysis of Likert-type Ratingscales." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '10. Atlanta, Georgia, USA: ACM, pp. 2391–2394. ISBN: 978-1-60558-929-9. DOI: 10.1145/1753326.1753686.

Kaptelinin, Victor (2014). *Affordances and Design*. DNK: The Interaction Design Foundation. ISBN: 8792964125.

Kaptelinin, Victor and Bonnie Nardi (2012). "Affordances in HCI: Toward a Mediated Action Perspective." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '12. Austin, Texas, USA: Association for Computing Machinery, pp. 967–976. ISBN: 9781450310154. DOI: 10.1145/2207676.2208541.

Karapanos, Evangelos, John Zimmerman, Jodi Forlizzi, and Jean-Bernard Martens (2009). "User Experience over Time: An Initial Framework." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '09. Boston, MA, USA: Associa-
tion for Computing Machinery, pp. 729–738. ISBN: 9781605582467.
DOI: 10.1145/1518701.1518814.

Kirsh, David (1997). "Interactivity and multimedia interfaces." In: *Instructional Science* 25.2, pp. 79–96. ISSN: 00204277, 15731952. DOI: https://doi.org/10.1023/A:1002915430871.

Kurtenbach, G., T. P. Moran, and W. Buxton (n.d.). "Contextual Animation of Gestural Commands." In: *Computer Graphics Forum* 13.5 (), pp. 305–314. DOI: 10.1111/1467-8659.1350305.

Kurtenbach, Gordon (Oct. 1991). "Making Marks Self-revealing." In: *SIGCHI Bull*. 23.4, pp. 60–61. ISSN: 0736-6906. DOI: 10.1145/126729.1056048.

Kurtenbach, Gordon and William Buxton (1991). "Issues in Combining Marking and Direct Manipulation Techniques." In: *Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology*. UIST '91. Hilton Head, South Carolina, USA: ACM, pp. 137–144. ISBN: 0-89791-451-1. DOI: 10.1145/120782.120797.

Kurtenbach, Gordon Paul (1993). "The Design and Evaluation of Marking Menus." UMI Order No. GAXNN-82896. PhD thesis. Toronto, Ont., Canada.

Lallemand, C., V. Koenig, G. Gronier, and R. Martin (2015). "Création et validation d'une version française du questionnaire AttrakDiff pour l'évaluation de l'expérience utilisateur des systèmes interactifs." In: *European Review of Applied Psychology* 65.5, pp. 239–252. ISSN: 1162-9088. DOI: https://doi.org/10.1016/j.erap.2015.08.002.

Lane, David M, H Albert Napier, S Camille Peres, and Aniko Sandor (2005). "Hidden costs of graphical user interfaces: Failure to make the transition from menus and icon toolbars to keyboard shortcuts." In: *International Journal of Human-Computer Interaction* 18.2, pp. 133–144.

Lewis, Chris, Noah Wardrip-Fruin, and Jim Whitehead (2012). "Motivational Game Design Patterns of 'ville Games." In: *Proceedings of the International Conference on the Foundations of Digital Games*. FDG '12. Raleigh, North Carolina: Association for Computing Machinery, pp. 172–179. ISBN: 9781450313339. DOI: 10.1145/2282338.2282373.

Lidwell, William, Kritina Holden, and Jill Butler (2010). *Universal principles of design, revised and updated: 125 ways to enhance usability, influence perception, increase appeal, make better design decisions, and teach through design*. Rockport Pub.

*Lime Survey* (2019). Retrieved July 8, 2019. URL: https://www.limesurvey.org/.

Malacria, Sylvain, Joey Scarr, Andy Cockburn, Carl Gutwin, and Tovi Grossman (2013). "Skillometers: ReflectiveWidgets That Motivate and Help Users to Improve Performance." In: *Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology*.
UIST '13. St. Andrews, Scotland, United Kingdom: ACM, pp. 321-330. ISBN: 978-1-4503-2268-3. DOI: 10.1145/2501988.2501996.

Mayer, Sven, Lars Lischke, Adrian Lanksweirt, Huy Viet Le, and Niels Henze (2018). "How to Communicate New Input Techniques." In: *Proceedings of the 10th Nordic Conference on Human-Computer Interaction*. NordiCHI '18. New York, NY, USA: ACM. DOI: 10.1145/3240167.3240176.

McGrenere, Joanna and Wayne Ho (May 2000). "Affordances: Clarifying and Evolving a Concept." In: *Proceedings of Graphics Interface 2000*. GI '00. Montreal, Quebec, Canada, pp. 179-186.

Nielsen, Jakob (Apr. 1993). "Noncommand User Interfaces." In: *Commun. ACM* 36.4, pp. 83-99. ISSN: 0001-0782. DOI: 10.1145/255950.153582.

Noguchi, Kimihiro, Yulia Gel, Edgar Brunner, and Frank Konietschke (2012). "nparLD: An R Software Package for the Nonparametric Analysis of Longitudinal Data in Factorial Experiments." In: *Journal of Statistical Software, Articles* 50.12, pp. 1-23. ISSN: 1548-7660. DOI: 10.18637/jss.v050.i12.

Norman, Donald A. (1988). *The Psychology of Everyday Things*. Basic Books.

— (May 1999). "Affordance, Conventions, and Design." In: *Interactions* 6.3, pp. 38-43. ISSN: 1072-5520. DOI: 10.1145/301153.301168.

— (2004). *Emotional design : why we love (or hate) everyday things*. New York: Basic Books. ISBN: 0465051367.

— (Nov. 2006). "Logic versus Usage: The Case for Activity-Centered Design." In: *Interactions* 13.6, pp. 45-46. ISSN: 1072-5520. DOI: 10.1145/1167948.1167978.

— (2008). "The way I see IT: signifiers, not affordances." In: *Interactions* 15.6, pp. 18-19. URL: https://interactions.acm.org/archive/view/november-december-2008/the-way-i-see-itsignifiers-not-affordances1.

— (2009). *The design of future things*. Basic Books.

— (May 2010). "Natural User Interfaces Are Not Natural." In: *Interactions* 17.3, pp. 6-10. ISSN: 1072-5520. DOI: 10.1145/1744161.1744163.

— (2013). *The Design of Everyday Things*. Basic Books.

O'Neill, Shaleph (2008). *Interactive Media: The Semiotics of Embodied Interaction*. Springer Publishing Company, Incorporated. ISBN: 9781848000353.

O'Neill, Shaleph, David R Benyon, and SR Turner (2002). "Semiotics and interaction analysis." In: *Proceeding of the 11th European Conference on Cognitive Ergonomics*. Citeseer, pp. 44-50.

Orne, Martin T (2009). "Demand characteristics and the concept of quasi-controls." In: *Artifacts in Behavioral Research: Robert Rosenthal and Ralph L. Rosnow's Classic Books* 110, pp. 110-137.
Peres, S. Camille, Franklin P. Tamborello, Michael D. Fleetwood, Phillip Chung, and Danielle L. Paige-Smith (2004). "Keyboard shortcut usage: The roles of social factors and computer experience." In: *Proceedings of the Human Factors and Ergonomics Society Annual Meeting*. Vol. 48. 5. SAGE Publications Sage CA: Los Angeles, CA, pp. 803-807. DOI: 10.1177/154193120404800513.

Pong, Nicole Ke Chen and Sylvain Malacria (2019). "Awareness, Usage and Discovery of Swipe-Revealed Hidden Widgets in IOS." In: *Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces*. ISS '19. Daejeon, Republic of Korea: Association for Computing Machinery, pp. 193-204. ISBN: 9781450368919. DOI: 10.1145/3343055.3359713.

Ramakumar, Adhithya (2017). *An analysis of the interactions on iOS 11*. Retrieved September 1, 2019. URL: https://uxdesign.cc/analysis-of-the-interactions-on-ios-11-2abc072d42ed.

Raskin, Jef (2000). *The Humane Interface: New Directions for Designing Interactive Systems*. USA: ACM Press/Addison-Wesley Publishing Co. ISBN: 0201379376.

Rasmussen, J. (May 1983). "Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models." In: *IEEE Transactions on Systems, Man, and Cybernetics SMC-13.3*, pp. 257-266. ISSN: 0018-9472. DOI: 10.1109/TSMC.1983.6313160.

Roth, Volker and Thea Turner (2009). "Bezel Swipe: Conflict-free Scrolling and Multiple Selection on Mobile Touch Screen Devices." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '09. Boston, MA, USA: ACM, pp. 1523-1526. ISBN: 978-1-60558-246-7. DOI: 10.1145/1518701.1518933.

Ryan, Richard M and Edward L Deci (2000a). "Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being." In: *American psychologist* 55.1, p. 68.

Ryan, Richard M, C Scott Rigby, and Andrew Przybylski (2006). "The motivational pull of video games: A self-determination theory approach." In: *Motivation and emotion* 30.4, pp. 344-360.

Ryan, Richard M. and Edward L. Deci (2000b). "Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions." In: *Contemporary Educational Psychology* 25.1, pp. 54-67. ISSN: 0361-476X. DOI: https://doi.org/10.1006/ceps.1999.1020.

Schramm, Katherine, Carl Gutwin, and Andy Cockburn (2016). "Supporting Transitions to Expertise in Hidden Toolbars." In: *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*. CHI '16. San Jose, California, USA: ACM, pp. 4687-4698. ISBN: 978-1-4503-3362-7. DOI: 10.1145/2858036.2858412.

Scimeca, Dennis (2011). *Radoff: Games Are Not Skinner Boxes*. Ed. by Gamasutra. Retrieved July 25, 2020. URL: https://www.gamasutra.
com/view/news/126470/Radoff_Games_Are Not Skinner Boxes.
php.

Serrano, Marcos, Eric Lecolinet, and Yves Guiard (2013). "Bezel-Tap Gestures: Quick Activation of Commands from Sleep Mode on Tablets." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '13. Paris, France: ACM, pp. 3027-3036. ISBN: 978-1-4503-1899-0. DOI: 10.1145/2470654.2481421.

Shneiderman, Ben (Aug. 1983). "Direct Manipulation: A Step Beyond Programming Languages." In: *Computer* 16.8, pp. 57-69. ISSN: 0018-9162. DOI: 10.1109/MC.1983.1654471.

— (1997). *Designing the User Interface: Strategies for Effective Human-Computer Interaction*. 3rd. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc. ISBN: 0201694972.

Spolsky, Avram Joel (2008). *User Interface Design for Programmers*. Apress.

Svanæs, Dag (Apr. 2013). "Interaction Design for and with the Lived Body: Some Implications of Merleau-Ponty's Phenomenology." In: *ACM Trans. Comput.-Hum. Interact*. 20.1. ISSN: 1073-0516. DOI: 10.1145/2442106.2442114.

Tak, Susanne, Piet Westendorp, and Iris Van Rooij (2013). "Satisficing and the use of keyboard shortcuts: being good enough Is enough?" In: *Interacting with computers* 25.5, pp. 404-416.

Torenvliet, Gerard (July 2003). "We Can't Afford It! The Devaluation of a Usability Term." In: *Interactions* 10.4, pp. 12-17. ISSN: 1072-5520. DOI: 10.1145/838830.838857.

Tyack, April and Elisa D. Mekler (2020). "Self-Determination Theory in HCI Games Research: Current Uses and Open Questions." In: *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems*. CHI '20. Honolulu, HI, USA: Association for Computing Machinery, pp. 1-22. ISBN: 9781450367080. DOI: 10.1145/3313831.3376723.

Vermeulen, Jo, Kris Luyten, Elise van den Hoven, and Karin Coninx (2013). "Crossing the Bridge over Norman's Gulf of Execution: Revealing Feedforward's True Identity." In: *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*. CHI '13. Paris, France: ACM, pp. 1931-1940. ISBN: 978-1-4503-1899-0. DOI: 10.1145/2470654.2466255.

Vicente, Kim J. and Jens Rasmussen (1990). "The Ecology of Human-Machine Systems II: Mediating 'Direct Perception' in Complex Work Domains." In: *Ecological Psychology* 2.3, pp. 207-249. DOI: 10.1207/s15326969eco0203\_2.

Ware, Colin (2012). *Information Visualization: Perception for Design*. 3rd ed. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. ISBN: 9780123814647.

Watson, Diane, Mark Hancock, Regan L. Mandryk, and Max Birk (2013). "Deconstructing the Touch Experience." In: *Proceedings of*
the 2013 ACM International Conference on Interactive Tabletops and Surfaces. ITS '13. St. Andrews, Scotland, United Kingdom: Association for Computing Machinery, pp. 199–208. ISBN: 9781450322713. DOI: 10.1145/2512349.2512819.

Weinschenk, Susan (2011). 100 things every designer needs to know about people. Pearson Education.

Wensveen, S. A. G., J. P. Djajadiningrat, and C. J. Overbeeke (2004). "Interaction Frogger: A Design Framework to Couple Action and Function through Feedback and Feedforward." In: *Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques*. DIS '04. Cambridge, MA, USA: Association for Computing Machinery, pp. 177–184. ISBN: 1581137877. DOI: 10.1145/1013115.1013140.

Wigdor, Daniel and Dennis Wixon (2011). *Brave NUI world: designing natural user interfaces for touch and gesture*. Elsevier.
<figure><img src="image_135.png" /></figure>
COLOPHON

This document was typeset using the typographical look-and-feel
*classicthesis* developed by André Miede and Ivo Pletikosić. The
style was inspired by Robert Bringhurst's seminal book on typography
*"The Elements of Typographic Style"*. *classicthesis* is available for both
*LATEX* and *LyX*:

<https://bitbucket.org/amiede/classicthesis/>

Final Version as of January 18, 2021().