CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

Preemptive Action: Accelerating Human Reaction
using Electrical Muscle Stimulation Without
Compromising Agency
Shunichi Kasahara∗Sony CSLTokyo, Japankasahara@csl.sony.co.jp

Jun Nishida†
University of Chicago
Chicago, United Statesjunnishida@uchicago.edu

ABSTRACTWe enable preemptive force-feedback systems to speed up
human reaction time without fully compromising the user’s
sense of agency. Typically these interfaces actuate by meansof electrical muscle stimulation (EMS) or mechanical actuators; they preemptively move the user to perform a task,such as to improve movement performance (e.g., EMS-assisted
drumming). Unfortunately, when using preemptive forcefeedback users do not feel in control and loose their sense ofagency. We address this by actuating the user’s body, using
EMS, within a particular time window (160 ms after visual
stimulus), which we found to speed up reaction time by 80
ms in our first study. With this preemptive timing, when
the user and system move congruently, the user feels thatthey initiated the motion, yet their reaction time is fasterthan usual. As our second study demonstrated, this particular timing significantly increased agency when compared tothe current practice in EMS-based devices. We conclude byillustrating, using examples from the HCI literature, how toleverage our findings to provide more agency to automatedhaptic interfaces.
CCS CONCEPTS• Human-centered computing → Haptic devices; Gestural input.
∗ Author affiliated also with the University of Tokyo, Japan.

† Author affiliated also with Tsukuba University, Japan.

‡ Author affiliated also with Hasso Plattner Institute, Germany.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
CHI 2019, May 4–9, 2019, Glasgow, Scotland Uk© 2019 Copyright held by the owner/author(s). Publication rights licensed
to ACM.
ACM ISBN 978-1-4503-5970-2/19/05…$15.00
https://doi.org/10.1145/3290605.3300873

Paper 643

Pedro Lopes‡
University of Chicago
Chicago, United States
pedrolopes@cs.uchicago.edu

KEYWORDS
Author Keywords: agency; electrical muscle stimulation; reaction time; human augmentation
ACM Reference Format:
Shunichi Kasahara, Jun Nishida, and Pedro Lopes. 2019. Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency. In CHI Conferenceon Human Factors in Computing Systems Proceedings (CHI 2019),
May 4–9, 2019, Glasgow, Scotland Uk. ACM, New York, NY, USA,
15 pages. https://doi.org/10.1145/3290605.3300873

1 INTRODUCTION
While computer interfaces have excelled at supporting intellectual augmentation [16], only recently have these interfaces started to provide physical augmentations, such as
assisting motion (e.g., learning drumming [14] or learning
medical procedures [63]). This type of augmentation happens through the assistance of force feedback devices thatare able to actuate the user’s body, such as by means of electrical muscle stimulation (e.g., [14]) or mechanical actuators(e.g., [63]). These haptic actuated systems offer the potential
to speed up the user’s physical reaction time by means ofpreemptive actions—the system actuates the user faster thantheir normal speed to perform a task that the user also wants
to perform. For instance, these systems have been used topreemptively steer the user into safety (e.g., Pedestrian Navigation [50]) or to correct the user in timing-sensitive tasks(e.g., keeping a steady rhythm [14]).
However, an interactive system that acts automaticallyand preemptively (be it physical or cognitive) will reduce
the user’s sense of being in control [5, 42]—the user loses
their sense of “I did this”. Speed et al. argued that automatedinteractions with objects (e.g., as in Affordance++ [38] wherean object forces the user to act) cause a shift in agency, removing the user’s agency [54]In this paper, we investigate how to enable preemptive
haptic systems to actuate users in a way that accelerates
their reaction time, yet, also maximizes their sense of agency.

Page 1CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

2 OUR APPROACH: PREEMPTIVE ACTION
At first glance our proposal might seem paradoxical: if a system actuates so the user moves faster than they do normally,
how could the user still feel as if they initiated the movement?
We propose that the solution to this paradox lies in thefact that our conscious awareness of intention takes a moment to arise, around 200 ms [30, 31, 40, 53]. These findings
suggest a time window between a volitional action and oursense of agency for this action [40]. We exploit the fact that
this time window offers flexibility to attribute agency to apreemptive action (Figure 1), i.e., one that was externallygenerated by a haptic device (e.g., electrical muscle stimulation) and is aligned with the user’s intention. For this hypothesis to work, one needs find out exactly when a preemptive actuation should be delivered so that it maximizes theuser’s agency.

electrical muscle stimulation

sense ofagencypreemptive action

preemptive gain

Figure 1: We reveal the relationship between a user’s agency
and the preemptive gain enabled by automated haptic interfaces (here, a system that speeds up the user’s tapping using
electrical muscle stimulation). This allows researchers to design preemptive actuation systems that now maximize theuser’s agency.

As depicted in Figure 1, we found and validated the key
relationship between agency and the gain in preemption(i.e., how much a system speeds up a user). We did this bymeans of two user studies that we present in this paper. This
agency-preemption relationship (depicted by the model on
the right side of Figure 1) allows researchers to optimize
their haptic actuated systems accordingly to their needs, i.e.,
maximize either for agency, reaction time or even both.
3 RELATED WORK
Our approach builds on the areas of human augmentation,
contemporary theories of agency in neuroscience and cognitive psychology, and electrical muscle stimulation.
Sense of agency
The sense of agency, or sense of control, is a neural mechanism that drives one’s awareness of initiating, executing,

Paper 643

and controlling our own voluntary actions in our surroundings [24]. The sense of agency allows us to recognize ourselves as the agent of a particular behavior, enabling us tobuild a self that is independent from the external world [24];it is thus one of the most primal mechanisms.
Wegner proposed three principles that condition the senseof agency, which they demonstrated to affect agency in VR
experiences [59]. These principles are: “(1) priority: conscious
intention to perform an act must immediately precede theaction, which should precede the outcome; (2) consistency:“the sensory outcome must fit the predicted outcome”; and,(3) exclusivity: “one’s thoughts must be the only apparent
cause of the outcome”. Jeunet et al. showed that a manipulation of any of these three principles changes the perceived
causality of an event, which in turn affects agency [26].
The neural origin of the sense of agency
In the field of neuroscience there are different views thatsuggest a neural mechanism that handles our sense of agency.
One view, which is of relevance to our research, places the
motor system (i.e., the neural apparatus that controls voluntary movements) at the center of the question of agency [18].
In their experiments, Frith et al. showed that participants
with lesions on the motor cortex (but no lesions on the sensory organs or pathways that allow to feel touch, etc.) exhibited a decreased sense of agency [18]. This theory statesthat the agency is dependent on the comparison between
the prediction of a movement and its outcome.
While much is still not understood regarding the innerworkings of our sense of agency, we do know it is not a staticand immutable state as many experiments have shown, suchas: the rubber hand illusion [7, 32, 62], randomly generated
feedback [15, 26], inversion of user’s movements [26], distortion of the haptic feedback [60], deception [48], and delays or sped-ups in feedback [15, 26]. The latter is especially
important for our investigation of agency in preemptive interactive systems. We believe the time window in which thesystem provides its preemptive feedback offers the biggest
design opportunity.
Interaction between time and the sense of agency
A series of studies have shown that the perception of timeis what binds together the degree of association we creating between a voluntary action (our intention) and outcome(the result of our intention); this has been shown to drive thesense of agency by Libet et al [31] and Haggard et al. [21]—just to cite a few. These findings (e.g., Libet’s clock experiment) found the existence of a time delay between a volitional action and our sense of agency for this action [40].
One key conclusion of all these studies in neuroscience isthat “temporal sensorimotor discrepancies reduce the sense

Page 2

CHI 2019 Paperof agency” [12]. This has been shown countless times by offsetting the onset of the expected outcome in visual, tactile
or auditory modalities [15]. Thus, time is a key in preserving the sense of agency. We argue that this is the parameterthat preemptive interactive systems must tune in order toprovide more agency to their users.
Designing for the sense of agency (Agency in HCI)
While designing interactive systems, the sense of agency is
key to achieve a user experience that grants a sense of control to the user. Unfortunately, achieving this becomes morechallenging with assistive systems, i.e., those that automatically change the system to facilitate interactions [41, 42].
For instance, Coyle et al., found that, when a user controls amouse cursor, changing the amount of assistance the interactive system provides (e.g., predictive mouse acceleration)
has a significant impact on the user’s sense of agency [11].
In fact, understanding this relationship between time andagency is so crucial to designing interactive systems thatthe HCI community has deepened that understanding: McEneaney et al. found that machine-induced mouse clicks affect the user’s perceived agency [41]; Coyle et al. found thatthe mouse cursor speed manipulations can affect agency [11];and, Bergstrom-Lehtovirta et al. found that on-skin interaction improves the sense agency vs. a traditional buttonpress [6].
Besides investigations that probe how feedback timing affects the user’s sense of agency, it has also been shown thatthe feedback modality has an impact as well. For instance,
Martinez et al. found that the sense of agency varied during
a mid-air touching task, across different feedback modalities, such as visual, audio or vibration [9]. They found thatboth auditory and haptic feedback modalities were superior
to the visual-only baseline in terms of increased sense ofagency. Likewise, Limerick et al. showed that, in speechbased interfaces, the voice input modality reduces agency
when compared to keyboard input [33].
Unfortunately, while these projects have investigated therole of timing and feedback in HCI systems (e.g., mousebased or mid-air interactions), nothing is known about theeffect that timing has on the user’s sense of agency in haptic
systems capable of actuating the user.
Haptic actuation systems
The type of haptics we discuss in this paper are strong force
feedback systems that are capable of moving a user’s body,even against their own volitional force (this effect is also often referred to as force feedback). There are two kinds of actuators that provide enough output force to actuate humanlimbs: mechanical actuators (e.g., large robotic arms or exoskeletons) or electrical muscle stimulation. The traditional
type of mechanical force feedback devices consists of having

Paper 643

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

users hold on to a handle, which is then actuated by a roboticarm or by pulley system (e.g., SPIDAR [44]). An alternative
to stationary haptic devices are ungrounded haptic devices:
these are mechanically actuated devices that provide forcesby pulling the user’s limbs against the mounting point on
the user’s body. Exoskeletons are the canonical example ofwearable ungrounded haptic devices; they can actuate theuser’s arms [57] or fingers [19].
Electrical Muscle Stimulation
Electrical Muscle Stimulation (EMS) originated in rehabilitation medicine as a means to restore lost motor functions,e.g., after spinal cord injuries [55]. Only more recently haveresearchers in HCI started to explore EMS, e.g. Kruijff et al.’s
EMS desktop gaming [29] or the Possessed Hand [57].
When surveying interactive systems based on EMS, wefound a clustering around three research topics: (1) increasing realism in virtual experiences by creating involuntary
force feedback [17, 34, 36, 39]; (2) tutorial/training systems [14,38, 46, 50, 57] and (3) information access [37, 47].
From these EMS-based devices, the area of training systems is of particular relevance to our case of preemptive actuation since most of these scenarios involve actuating theuser quicker than their normal reaction time. For instance,in Stimulated Percussions, an EMS-based system actuates
to user’s hands, as they hold on drumsticks, to automatically make the user drum on the correct tempo [14]. Also,
the aforementioned Affordance++ [38] actuated users to automatically perform certain tasks, such as to shake a spraycan. Lastly, as an additional example, Wired Muscle [46] is asystem that enables speeding up a user’s reaction time with
EMS so that they can pass the pen-drop test, a common fitness test also known as the ruler drop test, in which one
must grab a falling ruler, reacting only to the visual stimuli
of another user dropping the ruler.
Loss of agency in actuated haptic systems
While researchers are excited about the prospects of EMS
as a way to miniaturize haptics [35], it is not known what
effects the electrically induced involuntary actions have on
the user’s agency. This shortcoming is where we draw inspiration for our research question. In fact, the studies conducted on these EMS-based interactive systems point to aloss in agency. For example, in Affordance++ participants attributed agency to the object they manipulated (e.g., a spraycan that shakes) rather than to themselves [38]. Likewise,in Proprioceptive Interaction, participants often voiced not
feeling in control, e.g., “it was so remarkable to see my handmoving without my intention” [37].

Page 3

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

4 USER STUDY 1: MEASURING AGENCY IN
PREEMPTIVE HAPTIC SYSTEMSThe objective of our first experiment was to understand how
timing affects perceived agency in a preemptive haptic system. In the particular case of this experiment, we created apreemptive system that actuates the user using EMS. We designed a tapping study based a canonical psychophysics task
to measure reaction time (e.g., [23, 33, 45]). We engineered
the system to deliver the stimuli preemptively, i.e., before
the user has time to react to it. Our first hypothesis (H1)is that any preemptive stimulation will result in a shorter
reaction time, but will also decrease agency. Our second hypothesis (H2) is that there is a timing sweet spot that still
offers preemption (i.e., faster than ordinary reaction time)
without drastically decreasing agency.
Apparatus
Our experimental setup is depicted in Figure 2. To assist
readers in replicating our experiment, we provide the necessary technical details and the complete source code1 .

fingerbase

EMS ground electrodevibration
motorsmuscle stimulator

EMS electrodes(under the strap)

Figure 2: Setup for our preemptive action experiment.

Stimulation apparatus: Participants were actuated using a bioSync EMS stimulator (same as in [47], but removedof wireless communication to reduce latency). The stimulator’s intensity was adjusted per participant to operate painfree, yet rapidly and robustly actuate their ring finger.
Electrode placement: Two pairs of electrodes are used:
One pair, which sits at the flexor digitorum profundus muscle, provides sufficient intensity for flexing the user’s ring
finger. We actuated the ring finger since we can found (viapilot testing) that we could robustly actuate it without any

parasitical motion of neighboring muscles. The second electrode pair, attached to the flexor pollicis longus muscle (justa centimeter away from the first electrode pair), is used in
the sham-condition, providing a typical tingling sensation
without causing any finger flexion. During pilot testing we
confirmed that this sensation mimics effectively the tingling
effect of the first electrode pair.
Calibrating the EMS stimulation: We iteratively adjust the stimulation parameters by adjusting the pulse width(from 30 – 500 µsec), the pulse frequency (from 10 – 100 Hz),
and, lastly, the amplitude (from 1 mA to 10 mA) for each participant. The stimulation was one EMS pulse only. Thus, thestimulation duration was same as the selected pulse-width,
i.e., 30 – 500 µsec: we chose this since prior research suggests that short stimulations are less noticeable [36]. Prior
to starting the trials, we guaranteed that participants’ finger
was actuated via the EMS to tap the surface and was always
recognized by the touchscreen.
Masking the “tingling”: To allow our study to conclude
precisely on what preemptive EMS actuation does to theuser’s sense of agency, we controlled for external confounding factors, such as the EMS’ tingling sensation on the participant’s skin [37, 57]. Using the approach described in [37],we masked the tingling sensation by attaching two large vibration motors (with off-axis weights) to the participant’sarm. We used the following procedure to confirm that thisvibration effectively masks the EMS tingling sensation: (1)We calibrate EMS to tingle the skin but not strongly enoughto actuate the arm, (2) We run the vibration motors which
vibrate the participant’s arm. (3) We ask the participant to
say when they feel skin tingling among the vibrations. (4)We turn on the EMS at a random point in time. (5) If participants still feel the skin tingling, we lower the EMS (never
below the point of tingling) and increase the intensity of thevibrations. (6) We ensured all participants’ responses to thiswere below or around random (<=50%).
Touchscreen: Below the participants’ hands we placeda Razer Blade’s touch screen (model from 2017, running at
120Hz).
Finger support: We created a 3D printed finger base that
supports participants’ hands as they hover the touchscreen;
the base leaves the ring finger free to flex and touch thescreen. This approach is commonly used in psychophysics
too, as means to ensure that only one particular muscle causes
the observable result [2]. Furthermore, this prevents participants’ muscles from getting tired quickly.
System’s latency: As with any computer-based system,
our implementation has an intrinsic latency of less than 40 ms
between the software generated stimulation command andthe muscle activation (this accounts for the complete loop:

1 https://github.com/shks/PreemptiveAction

Paper 643

Page 4

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

placement of the EMS did not result in a precise tapping motion. Participants received 17 USD as compensation for their
time.

3-5s
-200

0

Rate
1234567+400[ms]

1 = I did not do it

7 = I did it

Figure 3: Our tapping task.

USB, stimulator’s microcontroller and human muscle contraction time). Also, our system has a latency of 60 ms in
logging the touch events, which we will later subtract.
Task
The task was a simple tap test, depicted in Figure 3; our task
was modeled after the psychophysics task that measures reaction time [23, 45]. We asked participants to tap a target
that appears on screen as fast as possible.
The target location was stationary and the period between
trials was randomized (between 1 and 4 s) to keep participants attentive. After each tap on the screen, participants
were presented with a questionnaire regarding their perceived sense of agency for this screen tap. We follow the typical agency questionnaire, i.e., a Likert scale question with
1 = “I did not do it” and 7 = “I did it”.
The time window of preemptive stimulation ranged from
200 ms (i.e., prior to the target showing up) to 400 ms after
the target shows up. To sample this time interval as muchas possible, we randomized the preemptive stimulation forevery trial.
Procedure
Before engaging in the study task, participants performed a
training phase consisting of a maximum of 10 trials. After
this, participants were asked to perform 10 trials to record
their average reaction time. Then, we asked participants toperform the task, i.e., 50 taps on the target. Each trial was followed with the aforementioned agency questionnaire. Afterall trials were completed, we gave participants an opportunity to provide open-ended feedback.
Participants
We recruited 12 participants (1 female, avg = 26.5 years old;std.dev. = 5.4) from our local institution. With their priorwritten consent, we transcribed their comments. An additional 3 participants were recruited, but excluded since thePaper 643

Results
We collected 600 trials from all participants, with two data
points per trial: reaction time and assessment of agency. Also,
we found participants’ average reaction time at the start ofthe study to be 267.3 ms (std.dev. = 32 ms), which is alignedwith findings in psychophysics research that found a reaction time of 250 ms in response to visual stimuli [23]. Furthermore, we did not observed a decrease of EMS stimulation efficacy over time, since we found no correlation between the trial index and the time between EMS trigger andscreen tap (R 2 = 0.0042).
7
6
5
4
3
2

Perceived agency score

Touch

1
-200

-100

0

100

EMS offset time [ms]200
300

Figure 4: Perceived agency score vs. EMS preemptive time.
0 ms denotes the moment the visual stimulus appeared andthe vertical lines represent a particular participant’s typicalreaction time. Colored data points represent the different
participants.

H1: preemption decreases agency. As Figure 4 depicts,
we found that preemption impacts agency, i.e., as we spedup the reaction time by providing more preemption to thesystem (i.e., EMS acted earlier), participants perceived lessagency. To illustrate this we can pick two time windows in
Figure 4, such as [–10, 10] ms and [290, 310] ms. In the firstcase, between [–10, 10] ms (i.e., actuation in sync with visualtarget), the average perceived agency was barely none (avg
= 1.25; std.dev = 0.43). Here, the participants’ reaction time
was shortened under the typical human reaction time (avg
= 47.5 ms; std.dev. = 20.2 ms). On the second case, when the
EMS was offset to act between [290, 310] ms after the visual
stimulus (i.e., system stimulated close to the human reaction time), the perceived agency approached the maximum(avg= 6.81; std.dev. = 0.39). However, there was no preemptive gain (reaction time avg = 281.9 ms;std.dev. = 43.6 ms).
This validates H1, which is in conformity with previous research in neuroscience for visual tasks, but had not beenshown with haptic actuation.

Page 5

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

300
200
100

Reaction time

400 [ms]6543

Perceived agency score

7

2
1
-100

y = 0.99 x + 41.6
R2= 0.91
-100

0

100

200

300

[ms]

0.5

Figure 5: EMS offset time vs. reaction time. Due to the limitation of experimental system, measured reaction time of lessthan -60 ms (with the system latency already subtracted),then the data point was capped at -60 ms.
0Now, as we have established the positive effect of the EMS
actuation in shortening reaction time, let us observe how
the agency scores distribute over total reaction time; Figure 6 depicts this relationship. Again, we observed that alarge percentage of maximum agency ratings occurred earlier than the participants’ usual reaction time (i.e., high ratings before the vertical bars).

Paper 643

Reaction time [ms]
300
400

200

These findings confirm our H2; they suggest that there
is a point in which we can speed up reaction time withoutcompromising the sense of agency. However, we can also
observe that each participant’s data points seem to fall on aslightly offset curve, which we decided to investigate further
in order to establish more precisely how early is the ideal
preemption that does not compromise agency relative to aparticipant’s usual reaction time?
Investigating the precise curve per participant. We
modeled each participant’s perceived agency as a coloredcurve, depicted in Figure 7.

EMS offset time
-200

100

Figure 6: Perceived agency score vs. preempted reaction time(Tp ). The vertical lines indicates each participants’ usual reaction time (Tb ).

1

0

0

Perceived agency score (normalized)

H2: there is a sweet spot that preserves agency. Now,
what we are looking for is for a point in between “maximumagency but no speed up” and “no agency due to too muchspeed up”. First, let us examine Figure 4, this time from theperspective of the vertical lines, which denote a participant’s
typical reaction time. For every participant there are manyhigh scored points (e.g., maximum ratings of agency) before their usual reaction time; this implies there is a point inwhich EMS can speed their reaction time and provide moreagency—this is the focus of our next analysis.
Let us confirm that indeed this EMS actuation did resultin an overall speed up in reaction time. Figure 5 depicts therelationship between EMS offset and total reaction time. We
found this relationship to be clearly linear between -100 and
250 ms, i.e., the earlier the EMS acts, the faster the reaction
time is; this implies that EMS is solely responsible for the
speed up in reaction time. Later than 250 ms, as we approach
the participants’ usual reaction time, the data variance increases since it is not only the EMS that is responsible forthe reaction time at these speeds. Note that all our measuresof reaction time were obtained with the same experimental setup and thus contain approximately 60 ms of latency.
Therefore, from here onward, we subtracted the aforementioned touch screen latency (60 ms) from the reaction time.

P6

P3

400

300

200

100

0

Preemptive gain [ms]
-100
-200

Figure 7: Normalized perceived sense of agency [0-1] vs. resulting preemptive gain. The solid lines represent regression
curves derived from a logistic regression for each participant.

To achieve this, (1) we normalized the data by subtracting
each participant’s baseline reaction time to each reaction
time (with EMS actuation). This allows us to instead depict
the time gained by means of preemption, which we denoteas Pд . Then, (2) we normalized the agency axis from 0 to 1
and the horizontal axis to depict -400 to 400 ms. Lastly, (3)Page 6

CHI 2019 Paper

the logistic regression computes the relationship between
the perceived agency and the preemptive gain, per participant. Then, (4) from all the curve fits we computed a mean
fit of R 2 = 0.62 (with std=0.086, min=0.50, max=0.79) for
our logistic model. This demonstrates that the curves are in
most cases very similar and relatively consistent among allusers (as suggested by the low standard deviation), but withslight shifts in the preemptive gain (i.e., they start in a different timing).
As we can observe, these curves afford a similar drop-off
rate, which seems to cluster around a preemptive gain of
80 ms. This suggests that there is a dramatic drop in agency
if we set up the EMS stimulus in a way that the preemptive gain is larger than 80 ms. To demonstrate this, let us
illustrate the average agency in a particular window before
and after 80 ms: we found that the average agency found in
the window between 80–0 ms is 5.13 (std.dev. = 2.16) in [17] scale. On the other hand, the agency dramatically drops
in the window after 80 ms of preemptive gain, for instanceat 160 ms we recorded a perceived agency of 1.89 (std.dev. =1.51) in [1-7] scale. This result reinforces our H2, pointing to
the time window in which it is feasible for a preemptive system to offer faster-than-normal reaction time without dramatically compromising agency. Our model suggests thatsuch a system should actuate 160 ms after visual target, resulting in 80 ms speed up of the user’s reaction time.
Obtaining a model of agency under preemption. Our
simple model to assist in explicating how the sense agency
is affected by a preemptive stimulus is:
A = 1/(1 + e −k (Pд −to ) )The logistic regression allowed us to model the sense ofagency (A) normalized from 0–1 as a sigmoid function depending on the parameters Pд (the preemptive gain), t 0 (timeshift of the curve’s starting point) and k(models the flexibility of a participant’s sense of agency, i.e., whether is it
binary, such as agency vs. no-agency, or continuous).
We observed that the parameter k has an actual world
cause: it models how participants opted for different strategies to evaluate their sense of agency. To illustrate we pick
two extremes of the k parameter, depicted in Figure 7’s callouts: P6’scurve (k=0.033, t 0 =128 ms) suggests a clear distinction between having agency or not and affords a longer preemption gain; during the post-task interview, P6 stated to“have confidence with their agency judgment at all times”.
On the other hand, P3’s curve (k=0.012, t 0 =118 ms) depicts
a more ambiguous sense of agency; during the interview, P3
noted: “I was not always confident about my answers”.

Paper 643

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

Qualitative results
The most exciting of our findings is that many participants
felt in control even when the action was triggered by EMS asvoiced by the participants’ comments at the end of the task.
We summarize these: 8 participants voiced remarks at theend of the study such as “even I did not notice that I’ve got
EMS” or “I rated 6 or 5 when I felt some EMS sensation but
still I believe I control my finger”, “When I rated 7, I think
there was not EMS and I solely touched the screen by myown”. In particular, P2 noted that, with an early-stimulationof -200 ms, “it’s too fast, it moves before I do”. This further
emphasizes that, at this timing, participants are able to distinguish the order of these two events, i.e., they realized the
EMS moved them before they wanted to move—aligned with
Wegner’s agency principles. Conversely, when stimulated
in a timing around 160 ms after the target appeared, thesame P2 stated “I know I felt some tingling but I thought
it was some weak feedback, but, now [in hindsight] I actually believe I moved my finger too”; we observed analogous
comments from other participants—this depicted a timingalignment that enabled participant’s “thought” to first arise(preserved the priority principle) and to be consistent withthe outcome, which was the on-screen feedback after tapping the screen (consistency principle). P5 even stated “Iknow this [the EMS] moved my finger, at some point, but I
feel this [EMS] and my movement to get integrated, rather
than just moved by it”. Interestingly, two participants did
not realize that, during the trials, the stimulus timing was
being manipulated, as P6 puts it: “I did not notice that timing was controlled. I just thought that only difference is thestrength of this [EMS]”. P6 added “When I answered 7, I did
not notice that I was moved by the EMS [although it was the
EMS that responded faster than P6, in many of these trials].
When answered 6, I partially felt the sensation but I still believed that I was the one who moved [although in many ofthese trials, the EMS moved faster than the participant]”.
5

MODELLING PREEMPTIVE ACTION WITH
AGENCY
While in Study 1 we obtained one model per participant, thegoal here is to inform future interactive systems with a moregeneralizable model. Ultimately, this will allow future preemptive actuation systems to maximize for agency based on
how much they want to sacrifice in reaction time (or viceversa).
We start by decomposing a user’s reaction time it into its
constituents (based on Card’s Human Processor Model [8]),as depicted in Figure 8. In a voluntary action our reaction
time is comprised of perception, cognition, and motor action. However, if we are preempted to act (e.g., by means of
EMS) we can gain a speed up in the final reaction time of,Page 7

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

for instance, 80 ms (if we select our previously found preemptive gain).
The Human
Processor perception
Model

Tb (msec)baseline
reaction time

Touch

〜
〜

Voluntary

motor action

cognition

Pg (msec)
Preemptive gain

Tp (msec)

TEMS (msec)

preempted
reaction time

Time oftriggering EMS

〜
〜

EMS
Preemptive
Actionde
EMS actuation Delay

Visual Stimuli〜〜

EMS Trigger Touch
160

200

280

Figure 8: Decomposing the user’s reaction time into its constituents: perception + cognition + motor action. Our approach speeds up total reaction time by advancing the motor
reaction component ahead of time (i.e., by preempting it).

1

0.5

0

Perceived agency score (normalized)

From this timeline the key parameter that enables the preemption is Pд (preemptive gain). In an ideal setting, as we
did in Study 1, this parameter is dependent on the user’s
own reaction time. However, if we are looking for a moregeneralizable result to be applied to a broader set of applications, we must create one single model that generalizes all
the data. To do this, we analyze data from all participants
via the logistic regression to yield a single model function,
which uses the preemptive gain and the perceived sense ofagency as independent variables (x and y, respectively). Figure 9 shows the yielded sigmoid curve function that we willtake as our working model. Here, we found a model fit of R 2= 0.56.

y = 1 / {1+ e -k(x-t0) }k = 0.017, t 0 =77.6 ms

Pg* = optimal Preemptive gain

400

300

200

100

0

Preemptive gain [ms]
-200
-100
-300

Figure 9: Our model derived from Study 1’s data. We highlight a critical point where both agency and reaction time
are maximized, allowing for an 80 ms gain in the user’s reaction time.

and reaction time we would select 80 ms of preemptive gain
, which offers a 50-50 ratio between agency and speed gain
(marked by the label (Pд∗ ) in Figure 9). This preemptive gainof 80 ms means our actuation is 80 ms faster than the user’s
own reaction time; this is the value we will verify in Study 2
in order to understand whether it outperforms the existing
approaches and baselines.
Once we decided the preemptive gain Pд = 80, we candetermine the time of triggering EMS from the visual onset(TEM S ). For instance, assuming average of reaction time is
280 ms and the latency (d EM S ) from triggering EMS to touch
is 40 ms, TEM S is determined as:
Tb − Pд − d EM S = 160ms
6

USER STUDY 2: USING OUR MODEL TO
OPTIMIZE FOR BOTH AGENCY AND REACTION
TIME
While our first study focused solely on understanding how
the preemptive EMS timings affect agency, the objective of
our second experiment was threefold: (1) to provide a comparison between our newly found optimal timing (from Study
1) and the approach used in the EMS state-of-the-art; (2) toisolate and explain the effect that preemptive actuation has
on the user’s own agency, independent of the skin-sensations
caused by EMS; and, (3) to unveil how the user’s agency
is impacted by the usage of preemptive EMS stimuli, whencompared to their own volition.
For this, we setup a similar experiment as in Study 1 byasking participants to tap on a screen as fast as possible. Thistime, we utilized only two timings for delivering the haptic stimuli: our previously found timing sweet spot against
the traditional approach in preemptive interactive systems,
which is to actuate the user as early as the interface needs
it.
Hypotheses
In this experiment our main hypotheses were as follows: H1:
our approach, using optimal preemptive timing, speeds up
standard reaction time by actuating the user. H2: our approach provides more agency than the common practice in
EMS today, which is to actuate the user as early as the interface needs it. To gain a deeper understanding of the mechanisms that underlie the perception of agency under EMS,
we added: H3: our EMS-based actuation was responsible forspeeding up reaction time and not just merely cueing theuser when to move. H4: our approach is able to maintainagency because the user’s intention is also to move theirfinger.

This model is useful because it allows a designer to choose
how to optimize for agency and/or reaction time. For instance, if we wanted to equally optimize for both agency

Paper 643

Page 8

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK“Touch or Relax”
Rate
1234567

Preemptive
Action1 = I did not do it

160ms“Touch”

7 = I did it

EMS
160ms

EMS Only“Relax”

EMS
160ms

Sham
“Touch”

Sham EMS

Baseline“Touch”

EMS

Time

Figure 10: Summary of procedure for Study 2.

The four conditions were parameterized as follows:
A. Preemptive Action: Both the participant and the EMS
stimulation contract the user’s finger (System+User move).
The EMS is triggered at TEM S = 160 ms, which should yield
an improvement of 80 ms in reaction time, according to ourmodel.
B. EMS-Only: Only the system moved the finger and participants were asked to relax (System moves) using the same
preemptive timing.
C. Sham:The participant moves their finger voluntarily(User moves). They also felt an EMS impulse, triggered at thesame preemptive timing, but this impulse was calibrated toonly provide skin sensation (tingling) and was not enoughto actuate—we use this as a sham condition [22]. Note that
we utilized a secondary pair of electrodes (1cm apart fromthe actuating pair) since our hardware does not allow tostimulate the same pair of electrodes with two different intensities after the calibration phase. We verified during pilot
studies that the tingling sensations from sham and actuation
are perceived as equivalent.
D. Baseline: Both the participant and the EMS stimulation contract the user’s finger (System+User move). However, here, EMS timing is triggered exactly when the application needs it, i.e., when the visual target shows up. This
depicts the current approach in interactive systems basedon EMS.
It is important to emphasize that participants were notaware of four distinct interface conditions; we simply informed participants that the stimulation would assist themin different ways to tap the target.
Experimental design
This study was a within-subject design with randomized
condition order and 10 trials per condition.

Paper 643

Participants
We recruited 12 new participants (2 female, avg = 24.6 years
old, std.dev. = 4.85) from our local institution; none had partaken in Study 1. Participants received 17 USD as compensation for their time. Furthermore, we measured the participants nominal reaction time prior to the study by averaging 10 practice trials (M = 229 ms, std.dev. = 46.5 ms, max =350 ms, min = 180 ms); this value serves as a purely voluntary baseline.
Results
We analyzed the dependent variables reaction time and reported agency score in our 4 interface conditions. Regarding normality using the Shapiro-Wilk Test: all reaction time
data was found to be normal, with the exception of the Baseline condition; and, only the agency scores of Preemptive
Action were found to be normal. Therefore, we utilized anon-parametric test. A Friedman test revealed a significant
effect of all conditions on the reaction time (χ 2 (3)= 32.8; p<0.01) and on the reported agency score (χ 2 (3)= 32.3; p<0.01).
All pairwise comparisons were Bonferroni-adjusted for all
variables. As in the previous study, we found no correlation
between trial index and reaction time in all EMS-related conditions (with R 2 < 0.005 for baseline and Preemptive Action
and R 2 < 0.01 for EMS-only); this suggests that EMS efficacy was not decreased over the duration of the study. We
now investigate each of our hypotheses:

*

[ms]
350

*

300
250
200
150

n.s.

Reaction time

Interface conditions
Participants touched the screen in four distinct conditions
for the same simple button press task, which are depicted in
Figure 10.

100
50
0

Preemptive
Action

EMS
Only

Sham

Baseline

Figure 11: Results for the user’s reaction time (median) ineach condition for our tapping task.

H1: our approach improves human reaction time.
Figure 11 shows the average reaction time per condition. A
post-hoc test using Wilcoxon signed-rank showed the significant differences between all combination (p < 0.05), except between the reaction time of Preemptive Action and
EMS-Only (p > 0.05). As expected, the fastest condition was
Baseline (Md = 19.1 ms; avg= 17.4 ms; std.dev. = 11.92 ms)since the EMS was triggered 160 ms earlier than in all others
conditions. Also, we found that our Preemptive Action wasPage 9

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

significantly faster than participants’ own reaction time, i.e.,
it outperformed the Sham condition (p = 0.01); this validates
our H1.
H2: our approach improves agency. Figure 12 depicts
the average perceived agency per condition. A post-hoc test
using Wilcoxon signed-rank found statistically significant
differences between all combinations (p < 0.05), except between EMS-Only and our the Baseline (p > 0.05); this indicates that the current practice in HCI provides no moreagency than asking the user to stand still and let the EMS
act. As expected, our Preemptive Action scored significantly
higher on agency than Baseline (p = 0.01); this validates our
H2.

*
*

6
5
4
3
2

*

Perceived agency score

*
7

*

n.s.

1
Preemptive
Action

EMS
Only

Sham

Baseline

Figure 12: Results for the user’s perceived agency (median),for each condition.

Also as expected, we found Preemptive Action to provide
less agency than Sham (p < 0.05) because actions during
the Sham condition were voluntary action. Additionally, theperceived agency in the Sham condition (Md = 6.95; avg= 6.4;
std.dev. = 1.18) suggests that some users loss agency simply
due to feeling the tingling sensation.
While the two hypotheses above were our main premise(i.e., our approach does speed up the user beyond their usual
reaction time while providing more agency), we wanted toshed light into what drives the faster reaction (H3) and what
drives the sense of agency (H4).
H3: EMS-based actuation is responsible for speeding
up: When comparing the reaction time in the Preemptive Action with the Sham, we found that Preemptive Action is significantly faster (p = 0.01): this demonstrates that the EMS
tingling is not sufficient to cue a participant and speed up
their reaction time. This further suggests that the reaction
time is driven by the EMS actuation. In addition, it is notablethat there is no significant difference in reaction time between Preemptive Action and EMS-only (p>0.05), suggestingthat the user’s voluntary action did not affect the reaction

Paper 643

time, placing the EMS as the main driver to speeding up theaction; these results validate our H3.
H4: Requirement of participant’s congruent intention. We observed barely any perceived sense of agency in
the EMS-only condition (Md = 1.0; avg= 1.1; std.dev. = 0.23);
in fact, EMS-only was significantly lower than Preemptive
Action (p=0.01) and Sham (p = 0.01); this suggests the expected result: without the user’s intention to do an action,
there is no possibility for agency to arise.
7 LIMITATIONS OF OUR STUDY DESIGNSWhen researchers apply our findings to new domains andapplications, it is worth keeping in mind the inherent limitations of our study designs.
Agency measurement: In our study we measured theuser’s perceived agency via a questionnaire, i.e., by directly
inquiring the user regarding their experience. We opted fora direct measure instead of using other study paradigms
that indirectly measure agency, such as Intentional Binding (IB) [9, 10]. The IB paradigm uses a delay between theuser’s action and the feedback (usually a auditory feedback).
If this delay is perceived shorter by user than it actually is,
the user feels more agency regarding this pair of action andoutcome. However, our preemptive action includes a EMSinduced tap (involuntary) that might be followed by a later(voluntary) finger tap as users still press the screen after the
EMS has tapped it. Even if IB paradigm does find a shift
in the perceived delay between action and outcome, there
is an ambiguity in whether the binding was with the involuntary movement (EMS) or in the later voluntary movement (the extra screen push). This could present a severe
confound to the study as the side effects are unknown at
this point. Therefore we opted for a direct measure; however, we believe it might be interesting to explore whether
IB can still be used during EMS preemption in the future.
Lastly, it is worth noting that previous research has shownthat both indirect and direct measures are valid ways to assess agency [13, 43].
EMS-specific: Both our studies were focused on EMS as
force feedback generator. We chose EMS since: (1) it allows
us to actuate the user rapidly yet safely at speeds faster thanhuman reaction time, and (2) this is a emergent area in HCI,
which appears to most researchers as the most promising
approach in miniaturization of haptic actuation [34, 36, 38,
39].
No-choice experiment: Experiments that present subjects with a possible choice found that participants rationalize choices after they made their decision; this widely
studied phenomenon is called postdiction [52, 56, 59]. While
we believe haptic actuation can be used as means of actuating a forced-choice that participants will later justify using
postdiction, we designed our studies to not be confounded

Page 10

CHI 2019 Paper

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

by these effects. Thus, in our experiments, we purposely removed any possibility of choice.

Now, if we re-implement the Wired Muscle system we
could take our findings as a way to maximize agency without sacrificing functionality. Researchers found that 200 ms
8 DEMONSTRATING HOW TO APPLY OUR
reaction time is required to successfully catch the pen. Thus,
FINDINGS
computing backwards from the maximum time needed to
We now illustrate the applicability of our findings to researchers still grab the pen, we obtain through our model a new 50 ms
working on haptic interfaces at the example of improving an
preemptive gain (Pд ) that still allows the user to catch theexisting interactive system based on EMS to provide more
pen while providing a sense of agency of 0.65 (normalized
agency to its user. As an example, let us use the aforemenscale). The improved system actuates the user 150 ms aftertioned Wired Muscle [46], depicted in Figure 13 with conthe pen was released, instead of the old 10 ms. Thus, using
sent of the authors.
our model we tripled the perceived agency without compromising the Wired Muscle system’s functionality, as depicted in Figure 14.
maximum reaction time
for grabbing

grab
Human baseline

0

grab

0 10

60

190
Naive EMS Preemption

250
50

Figure 13: Wired Muscle, an EMS-based interface for speeding up reaction time. While in its original form it did notconsider the loss in sense of agency, using preemptive action, it can still retain functionality and boost the perceived
agency.

0.65

0.5

0.18

0

Optimal EMS Preemption
Reaction time0

Expected Agency Score (normalized)

1

400

While the original system achieves the goal of speeding
up reaction time so that an untrained user passes the pendrop test, it did not consider the loss in the user’s sense ofagency. The system uses an electromyography sensor (EMG)
attached to the extensor muscle of the user holding the pen.
The EMG records the extensor’s electrical activity and thus
measures the onset of movement that releases the pen. The
system uses a fixed threshold to detect this peak in the extensor’s EMG activity and immediately issues the command to
stimulate the flexor muscle of the user trying to grab the pen.
This results in an EMS preemptive actuation that is delayed
only by 60 ms from the moment of peak detection of the
EMG. This latency includes 10 ms of communication delay
and 50 ms until the EMS-induced grasp is completed (datafrom [46]).
Recalling our model, stimulating 60 ms after the stimulus onset, results in a preemptive gain of 190 ms (i.e., thatmuch faster than the standard reaction time of 250 ms). Unfortunately, as depicted in Figure 14, this is a point in the
curve with very low perceived agency of 0.18 (0–1 normalized scale). Thus, this system in its original form does notprovide their users with a sense of “I grabbed this pen”.

Paper 643

300

150

200

100

200 250[ms]

Preemptive Gain
0

[ms]

-100

Figure 14: A researcher can optimize their system’s agency
by choosing a new timing for their preemptive actuation;
here, we illustrate this at the example of improving theagency in the Wired Muscle system.

The same reasoning could be used to improve other systems from the prior art, such as Affordance++ [38] or Stimulated Percussions [14], just to mention a few. In the caseof Affordance++, the original system is based on trigger actions (e.g., “user grabs the spray can”), detected via opticaltracking, which in turn activate muscle stimulations (e.g.,using EMS “the spray can shakes itself”). Again, this system
immediately triggers the EMS when the user grabs the spraycan, even if the user actually wants “to shake it”. Authors
found in their studies that these automated actions shifted
agency from the user to the object (e.g., “it [the spray can]wants to shake” [38]. Now, if we re-implement such a system, we could tune its temporal behavior to trigger the muscle stimulation later in the preemption-agency curve. In fact,
in the case of Affordance++, we could select a timing aroundthe user’s average reaction time (e.g., 300 ms), which would
grant much more perceived agency for the case in which

Page 11

CHI 2019 Paper

both the user and the system decide to perform the sameaction.
9

GROUNDING OUR RESULTS IN THE
LITERATURE
We now discuss how our results can both find validity inexisting research and help shape current theories of agency.
How can someone feel in control with EMS
actuation?
The most intriguing and exciting of our findings is not that
our approach provides more agency than the current practice, but that so many participants felt in control even when
the action was triggered by EMS. These participants voiced
remarks at the end of the study such as “I did not notice that
I’ve got EMS”, “I felt some EMS sensation but still I believe
I control my finger”, and so forth. To this end, the cognitive
neuroscience literature provides a plausible explanation: as
demonstrated by Libet et al., the “desire to act” can arise, inconsciousness, after the motor commands have been issued,i.e., after starting to move [31]. Thus, by designing our preemptive timing to provide a stimulus 80 ms before the participants’ average reaction time, we have purposely inserted
the involuntary muscle motion in Libet’s range, i.e., around200-300 ms before the conscious act manifests itself. Thus,
the origin of the motion (EMS) gets obfuscated in this timeline.
The role of causality in our sense of agency
We now analyze our findings in light of Wegner’s three agency
principles [59]. Regarding the consistency principle, Wen etal. challenge it with their findings [61], they found that performance (i.e., quality of the outcome) was more importantto participants’ sense of agency than the action-feedback
association. With regards to the priority principle, Davidet al. found that, in an AR experiment where both the outcome (system’s auditory response to a movement) and theparticipant’s own movement were temporally manipulated,“participants were more sensitive to delays of outcome thanto delays of movement execution”. This speaks in favor of
our approach, which provides an active manipulation of the
movement execution (by preempting it), but attempts to preserve an optimal time in outcome presentation (by not preempting it too much). Also, it is worth noting that fact thatour preemptive action improved agency but never provided
a sense of complete agency for all participants in all trials;
this suggests that our results are in line with Haggard et al.’s
four stages of agency [21], which postulates a stage in which
our brain can veto a motor response [20].

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

Sense of agency vs. sense of ownership
Most contemporary theories argue that the sense of agency(“I did that”) influences and is influenced by the sense ofownership (“that is me”). However, research suggests thatthese are distinct neural mechanisms [27, 58, 64]. Much likeagency, the ownership can be modulated using virtual representations of one’s own body (e.g., avatars in VR [4] orproxy objects such as a rubber hand [7, 32, 62]). This manipulation of the sense of ownership has been shown in somecases to also affect the sense of agency [3, 4].
In our investigations, we explored how to actuate users
while maximizing their sense of agency. In fact, our choiceof using electrical muscle stimulation as a haptic actuator,
granted that users always felt (via their proprioception) that
it was their own body moving—we did not received commentary typical of participants experiencing a loss in senseof ownership. Thus, the choice of EMS as our haptic actuator puts aside, in broad terms, the concern of compromising
the sense of ownership. However, if researchers would apply our ideas to other haptic actuators that might be decoupled from the user’s own body (e.g., teleoperating a roboticarm [49]), the interaction effects between preemption andsense of ownership might be worth investigating.
10 OUTLOOK AND FURTHER IMPLICATIONS
We believe our findings will inspire areas in which interfaces have not benefited from any possibility of haptic automation. On the most immediate level, we strongly believe
our findings will provide more agency to the status-quo ofinteractive systems based on EMS; this includes possible improvements of commercially available force-feedback devices
based on EMS (e.g., UnlimitedHand 2 ).
Further along, we believe these findings will inspire research that will extend their impact to mechanically actuated haptics. Here, the potential is to enable exoskeletonlike interfaces to provide agency to the user. For instance,in the automated haptic juggling system by Ruffaldi [51],
our findings could be used to tune their control loop to provide more agency to the user as the robotic actuator throws
the balls for them. This opens a broader scope of haptic systems that might be subject to improvement through our findings, especially those that deal with haptic training, such as
sports (e.g., catch-ball [25], Ping-Pong [17, 28], boxing [36],juggling [51], biking [1] or music [14, 57].
11 CONCLUSIONS
We demonstrated that it is possible to tune preemptive forcefeedback systems to provide their users with more agency.
In our first study, we uncovered a particular timing that is
optimal to deliver the haptic actuation (by means of EMS)
2 Unlimited Hand, http://unlimitedhand.com/, last accessed in 17/09/2018

Paper 643

Page 12

CHI 2019 Paper

without drastically reducing the user’s sense of agency. With
this preemptive timing, when the user and system movecongruently, the user feels that they initiated the motion,
yet their reaction time is faster than usual. Then, in our second experiment, we found that our preemptive timing significantly increased agency when compared to the current
practice in EMS-based devices. We concluded by illustrating,using examples from the HCI literature, how to leverage ourfindings to provide more agency to automated haptic interfaces.
ACKNOWLEDGMENTS
This work was partially supported by Grant-in-Aid for JSPS
Research Fellow (JP16J03777).
We kindly thank our colleague Jas Brooks (University of
Chicago) for helping us proofread this work and Daisuke
Tajima for his advice with our experimental design.
REFERENCES[1] Josh Andres, Julian de Hoog, Jürg von Känel, Julian Berk, Bach Le,
Xizi Wang, Marcus Brazil, and Florian ’Floyd’ Mueller. 2016. Exploring
Human: EBike Interaction to Support Rider Autonomy. In Proceedingsof the 2016 Annual Symposium on Computer-Human Interaction in Play
Companion Extended Abstracts (CHI PLAY Companion ’16). ACM, New
York, NY, USA, 85–92. https://doi.org/10.1145/2968120.2987719[2] A. Angel. 1973. Input-output relations in simple reaction time experiments. The Quarterly Journal of Experimental Psychology 25, 2 (May1973), 193–200. https://doi.org/10.1080/14640747308400338[3] Tomohisa Asai. 2016. Agency elicits body-ownership: proprioceptive
drift toward a synchronously acting external proxy. Experimental
Brain Research 234, 5 (May 2016), 1163–1174. https://doi.org/10.1007/
s00221-015-4231-y[4] Domna Banakou and Mel Slater. 2014. Body ownership causes illusory
self-attribution of speaking and influences subsequent real speaking.
Proceedings of the National Academy of Sciences 111, 49 (Dec. 2014),
17678–17683. https://doi.org/10.1073/pnas.1414936111[5] Bruno Berberian, Jean-Christophe Sarrazin, Patrick Le Blaye, andPatrick Haggard. 2012. Automation Technology and Sense of Control: A Window on Human Agency. PLOS ONE 7, 3 (March 2012),
e34075. https://doi.org/10.1371/journal.pone.0034075[6] Joanna Bergstrom-Lehtovirta, David Coyle, Jarrod Knibbe, andKasper Hornb\a ek. 2018. I Really Did That: Sense of Agency with
Touchpad, Keyboard, and On-skin Interaction. In Proceedings of the2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, USA, 378:1–378:8. https://doi.org/10.1145/
3173574.3173952[7] M. Botvinick and J. Cohen. 1998. Rubber hands ’feel’ touch that eyessee. Nature 391, 6669 (Feb. 1998), 756. https://doi.org/10.1038/35784[8] Stuart K Card, Thomas P Moran, and Allen Newell. 1983. The psychology of human-computer interaction. L. Erlbaum Associates, Hillsdale,
N.J. OCLC: 9042220.
[9] Patricia Ivette Cornelio Martinez, Silvana De Pirro, Chi Thanh Vi,
and Sriram Subramanian. 2017. Agency in Mid-air Interfaces. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). ACM, New York, NY, USA, 2426–2439. https://doi.org/10.1145/3025453.3025457[10] Patricia I. Cornelio Martinez, Emanuela Maggioni, Kasper Hornb\a ek,
Marianna Obrist, and Sriram Subramanian. 2018. Beyond the Libet

Paper 643

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK

Clock: Modality Variants for Agency Measurements. In Proceedingsof the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). ACM, New York, NY, USA, 541:1–541:14. https://doi.org/
10.1145/3173574.3174115[11] David Coyle, James Moore, Per Ola Kristensson, Paul Fletcher, andAlan Blackwell. 2012. I Did That! Measuring Users’ Experience ofAgency in Their Own Actions. In Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems (CHI ’12). ACM, New York,NY, USA, 2025–2034. https://doi.org/10.1145/2207676.2208350[12] Nicole David, Stefan Skoruppa, Alessandro Gulberti, Johannes
Schultz, and Andreas K. Engel. 2016. The Sense of Agency Is MoreSensitive to Manipulations of Outcome than Movement-Related Feedback Irrespective of Sensory Modality. PLOS ONE 11, 8 (Aug. 2016),
e0161156. https://doi.org/10.1371/journal.pone.0161156[13] John A. Dewey and Günther Knoblich. 2014. Do Implicit and Explicit
Measures of the Sense of Agency Measure the Same Thing? PLOS
ONE 9, 10 (Oct. 2014), e110118. https://doi.org/10.1371/journal.pone.
0110118[14] Ayaka Ebisu, Satoshi Hashizume, Kenta Suzuki, Akira Ishii, Mose
Sakashita, and Yoichi Ochiai. 2016. Stimulated Percussions: Techniques for Controlling Human As Percussive Musical Instrument byUsing Electrical Muscle Stimulation. In SIGGRAPH ASIA 2016 Posters(SA ’16). ACM, New York, NY, USA, Article 37, 2 pages. https://doi.org/10.1145/3005274.3005324[15] Kai Engbert, Andreas Wohlschläger, and Patrick Haggard. 2008. Who
is causing what? The sense of agency is relational and efferenttriggered. Cognition 107, 2 (May 2008), 693–704. https://doi.org/10.
1016/j.cognition.2007.07.021[16] Douglas C. Engelbart and William K. English. 1968. A Research Center
for Augmenting Human Intellect. In Proceedings of the December 9-11,
1968, Fall Joint Computer Conference, Part I (AFIPS ’68 (Fall, part I)).
ACM, New York, NY, USA, 395–410. https://doi.org/10.1145/1476589.
1476645[17] Farzam Farbiz, Zhou Hao Yu, Corey Manders, and Waqas Ahmad.
2007. An Electrical Muscle Stimulation Haptic Feedback for Mixed
Reality Tennis Game. In ACM SIGGRAPH 2007 Posters (SIGGRAPH’07). ACM, New York, NY, USA, Article 140. https://doi.org/10.1145/
1280720.1280873[18] C. D. Frith, S. J. Blakemore, and D. M. Wolpert. 2000. Abnormalities
in the awareness and control of action. Philosophical Transactions of
the Royal Society of London. Series B, Biological Sciences 355, 1404 (Dec.
2000), 1771–1788. https://doi.org/10.1098/rstb.2000.0734[19] Xiaochi Gu, Yifei Zhang, Weize Sun, Yuanzhe Bian, Dao Zhou, and
Per Ola Kristensson. 2016. Dexmo: An Inexpensive and Lightweight
Mechanical Exoskeleton for Motion Capture and Force Feedback in
VR. In Proceedings of the 2016 CHI Conference on Human Factors in
Computing Systems (CHI ’16). ACM, New York, NY, USA, 1991–1995.
https://doi.org/10.1145/2858036.2858487[20] Adrian G. Guggisberg and Anaïs Mottaz. 2013. Timing and awarenessof movement decisions: does consciousness really come too late? Frontiers in Human Neuroscience 7 (July 2013). https://doi.org/10.3389/
fnhum.2013.00385[21] Patrick Haggard, Sam Clark, and Jeri Kalogeras. 2002. Voluntary action and conscious awareness. Nature Neuroscience 5, 4 (April 2002),
382–385. https://doi.org/10.1038/nn827 IB Origin.
[22] Uwe Herwig, Lizbeth Cardenas-Morales, Bernhard J. Connemann,
Thomas Kammer, and Carlos Schönfeldt-Lecuona. 2010. Sham orreal—Post hoc estimation of stimulation condition in a randomized
transcranial magnetic stimulation trial. Neuroscience Letters 471, 1(Feb. 2010), 30–33. https://doi.org/10.1016/j.neulet.2010.01.003

Page 13

CHI 2019 Paper[23] Ray Hyman. 1953. Stimulus information as a determinant of reaction
time. Journal of Experimental Psychology 45, 3 (1953), 188–196. https://doi.org/10.1037/h0056940[24] Marc Jeannerod. 2003. The mechanism of self-recognition in humans.
Behavioural Brain Research 142, 1-2 (June 2003), 1–15. https://doi.org/
10.1016/S0166-4328(02)00384-4[25] Seungzoo Jeong, Naoki Hashimoto, and Sato Makoto. 2004. A Novel
Interaction System with Force Feedback Between Real - and Virtual
Human: An Entertainment System: ”Virtual Catch Ball”. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in
Computer Entertainment Technology (ACE ’04). ACM, New York, NY,
USA, 61–66. https://doi.org/10.1145/1067343.1067350[26] C. Jeunet, L. Albert, F. Argelaguet, and A. Lécuyer. 2018. “Do You
Feel in Control?”: Towards Novel Approaches to Characterise, Manipulate and Measure the Sense of Agency in Virtual Environments.
IEEE Transactions on Visualization and Computer Graphics 24, 4 (April2018), 1486–1495. https://doi.org/10.1109/TVCG.2018.2794598[27] Andreas Kalckert and H. Henrik Ehrsson. 2012. Moving a Rubber
Hand that Feels Like Your Own: A Dissociation of Ownership andAgency. Frontiers in Human Neuroscience 6 (2012). https://doi.org/10.
3389/fnhum.2012.00040[28] Benjamin Knoerlein, Gábor Székely, and Matthias Harders. 2007.
Visuo-haptic Collaborative Augmented Reality Ping-pong. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (ACE ’07). ACM, New York, NY, USA, 91–94.
https://doi.org/10.1145/1255047.1255065[29] Ernst Kruijff, Dieter Schmalstieg, and Steffi Beckhaus. 2006. Using
Neuromuscular Electrical Stimulation for Pseudo-haptic Feedback. In
Proceedings of the ACM Symposium on Virtual Reality Software andTechnology (VRST ’06). ACM, New York, NY, USA, 316–319. https://doi.org/10.1145/1180495.1180558[30] Benjamin Libet. 1985. Unconscious cerebral initiative and the role ofconscious will in voluntary action. Behavioral and Brain Sciences 8, 4
(Dec. 1985), 529–539. https://doi.org/10.1017/S0140525X00044903[31] BENJAMIN LIBET, CURTIS A GLEASON, and ELWOOD W WRIGHT.
[n. d.]. TIME OF CONSCIOUS INTENTION TO ACT IN RELATIONTO ONSET OF CEREBRAL ACTIVITY (READINESS-POTENTIAL).
([n. d.]), 20. Libet Time.
[32] Jakub Limanowski and Felix Blankenburg. 2016. That’s not quite me:limb ownership encoding in the brain. Social Cognitive and Affective
Neuroscience 11, 7 (July 2016), 1130–1140. https://doi.org/10.1093/
scan/nsv079[33] Hannah Limerick, James W. Moore, and David Coyle. 2015. Empirical
Evidence for a Diminished Sense of Agency in Speech Interfaces. In
Proceedings of the 33rd Annual ACM Conference on Human Factors in
Computing Systems (CHI ’15). ACM, New York, NY, USA, 3967–3970.
https://doi.org/10.1145/2702123.2702379[34] Pedro Lopes and Patrick Baudisch. 2013. Muscle-propelled Force Feedback: Bringing Force Feedback to Mobile Devices. In Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, USA, 2577–2580. https://doi.org/10.1145/
2470654.2481355[35] P. Lopes and P. Baudisch. 2017. Interactive Systems Based on Electrical Muscle Stimulation. Computer 50, 10 (2017), 28–35. https://doi.org/10.1109/MC.2017.3641627[36] Pedro Lopes, Alexandra Ion, and Patrick Baudisch. 2015. Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. In Proceedings of the 28th Annual ACM Symposium on User Interface Software &#38; Technology (UIST ’15). ACM,
New York, NY, USA, 11–19. https://doi.org/10.1145/2807442.2807443

Paper 643

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK[37] Pedro Lopes, Alexandra Ion, Willi Mueller, Daniel Hoffmann, Patrik
Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in
Computing Systems (CHI ’15). ACM, New York, NY, USA, 939–948.
https://doi.org/10.1145/2702123.2702461[38] Pedro Lopes, Patrik Jonell, and Patrick Baudisch. 2015. Affordance++:
Allowing Objects to Communicate Dynamic Use. In Proceedings of the
33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). ACM, New York, NY, USA, 2515–2524. https://doi.org/10.
1145/2702123.2702128[39] Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, andPatrick Baudisch. 2017. Providing Haptics to Walls &#38; Heavy
Objects in Virtual Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems (CHI ’17). ACM, New York, NY, USA, 1471–1482.
https://doi.org/10.1145/3025453.3025600[40] Masao Matsuhashi and Mark Hallett. 2008. The timing of the conscious intention to move. The European journal of neuroscience 28,
11 (Dec. 2008), 2344–2351. https://doi.org/10.1111/j.1460-9568.2008.
06525.x[41] John McEneaney. 2013. Agency Effects in Human-Computer Interaction. International Journal of Human-Computer Interaction 29 (Sept.
2013), 798–813. https://doi.org/10.1080/10447318.2013.777826[42] Christopher A. Miller and Raja Parasuraman. 2007. Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control. Human Factors 49, 1 (Feb. 2007), 57–75.
https://doi.org/10.1518/001872007779598037[43] James W. Moore and Sukhvinder S. Obhi. 2012. Intentional binding
and the sense of agency: a review. Consciousness and Cognition 21, 1(March 2012), 546–561. https://doi.org/10.1016/j.concog.2011.12.002[44] Jun Murayama, Laroussi Bougrila, Yanlin Luo, Katsuhito Akahane,
Shoichi Hasegawa, Béat Hirsbrunner, and Makoto Sato. 2004. G : A
Two-Handed Haptic Interface for Bimanual VR Interaction.
[45] Pekka Niemi and Risto Näätänen. 1981. Foreperiod and simple reaction time. Psychological Bulletin 89, 1 (1981), 133–162. https://doi.org/10.1037/0033-2909.89.1.133[46] Jun Nishida, Shunichi Kasahara, and Kenji Suzuki. 2017. Wired Muscle: Generating Faster Kinesthetic Reaction by Inter-personally Connecting Muscles. In ACM SIGGRAPH 2017 Emerging Technologies (SIGGRAPH ’17). ACM, New York, NY, USA, Article 26, 2 pages. https://doi.org/10.1145/3084822.3084844[47] Jun Nishida and Kenji Suzuki. 2017. bioSync: A Paired Wearable
Device for Blending Kinesthetic Experience. In Proceedings of the2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, USA, 3316–3327. https://doi.org/10.1145/
3025453.3025829[48] Jay A. Olson, Mathieu Landry, Krystèle Appourchaux, and Amir Raz.
2016. Simulated thought insertion: Influencing the sense of agency using deception and magic. Consciousness and Cognition 43 (July 2016),
11–26. https://doi.org/10.1016/j.concog.2016.04.010[49] D. Pamungkas and K. Ward. 2015. Immersive teleoperation of a robot arm using electro-tactile feedback. In 2015 6th International Conference on Automation, Robotics and Applications (ICARA). 300–305.
https://doi.org/10.1109/ICARA.2015.7081164[50] Max Pfeiffer, Tim Dünte, Stefan Schneegass, Florian Alt, and Michael
Rohs. 2015. Cruise Control for Pedestrians: Controlling Walking Direction Using Electrical Muscle Stimulation. In Proceedings of the 33rd
Annual ACM Conference on Human Factors in Computing Systems (CHI
’15). ACM, New York, NY, USA, 2505–2514. https://doi.org/10.1145/
2702123.2702190

Page 14

CHI 2019 Paper[51] Emanuele Ruffaldi, Paolo Tripicchio, Carlo Alberto Avizzano, andMassimo Bergamasco. 2011. Haptic Rendering of Juggling with Encountered Type Interfaces. Presence: Teleoperators and Virtual Environments 20, 5 (2011), 480–501. https://doi.org/10.1162/PRES_a_00067
arXiv:https://doi.org/10.1162/PRESa 0 0067
[52] Shinsuke Shimojo. 2014. Postdiction: its implications on visual awareness, hindsight, and sense of agency. Frontiers in Psychology 5 (March2014). https://doi.org/10.3389/fpsyg.2014.00196[53] Chun Siong Soon, Marcel Brass, Hans-Jochen Heinze, and John-Dylan
Haynes. 2008. Unconscious determinants of free decisions in the human brain. Nature Neuroscience 11, 5 (May 2008), 543–545. https://doi.org/10.1038/nn.2112[54] Chris Speed and Duncan Shingleton. 2012. Take Me I’M Yours: Mimicking Object Agency. In Proceedings of the 2012 ACM Conference
on Ubiquitous Computing (UbiComp ’12). ACM, New York, NY, USA,1167–1170. https://doi.org/10.1145/2370216.2370465[55] P. Strojnik, A. Kralj, and I. Ursic. 1979. Programmed Six-Channel
Electrical Stimulator for Complex Stimulation of Leg Muscles During
Walking. IEEE Transactions on Biomedical Engineering BME-26, 2 (Feb
1979), 112–116. https://doi.org/10.1109/TBME.1979.326520[56] Matthis Synofzik, Gottfried Vosgerau, and Martin Voss. 2013. The experience of agency: an interplay between prediction and postdiction.
Frontiers in Psychology 4 (2013). https://doi.org/10.3389/fpsyg.2013.
00127[57] Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011. PossessedHand:
Techniques for Controlling Human Hands Using Electrical Muscles
Stimuli. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems (CHI ’11). ACM, New York, NY, USA, 543–552.
https://doi.org/10.1145/1978942.1979018

Paper 643

CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK[58] Manos Tsakiris, Gita Prabhu, and Patrick Haggard. 2006. Having
a body versus moving your body: How agency structures bodyownership. Consciousness and Cognition 15, 2 (June 2006), 423–432.
https://doi.org/10.1016/j.concog.2005.09.004 BO and SOA.
[59] Daniel M Wegner. 2003. The mind’s best trick: how we experience
conscious will. Trends in Cognitive Sciences 7, 2 (Feb. 2003), 65–69.
https://doi.org/10.1016/S1364-6613(03)00002-0[60] Sébastien Weibel, Patrick Eric Poncelet, Yvonne Delevoye-Turrell, Antonio Capobianco, André Dufour, Renaud Brochard, Laurent Ott, andAnne Giersch. 2015. Feeling of control of an action after supra andsubliminal haptic distortions. Consciousness and Cognition 35 (Sept.
2015), 16–29. https://doi.org/10.1016/j.concog.2015.04.011[61] Wen Wen, Atsushi Yamashita, and Hajime Asama. 2015. The Sense
of Agency during Continuous Action: Performance Is More Important than Action-Feedback Association. PLOS ONE 10, 4 (April 2015),
e0125226. https://doi.org/10.1371/journal.pone.0125226[62] Andrew Wold, Jakub Limanowski, Henrik Walter, and Felix Blankenburg. 2014. Proprioceptive drift in the rubber hand illusion is intensified following 1 Hz TMS of the left EBA. Frontiers in Human Neuroscience 8 (2014). https://doi.org/10.3389/fnhum.2014.00390[63] Vibol Yem, Hideaki Kuzuoka, Naomi Yamashita, Shoichi Ohta, and Yasuo Takeuchi. 2014. Hand-Skill Learning Using Outer-Covering Haptic Display. In Haptics: Neuroscience, Devices, Modeling, and Applications (Lecture Notes in Computer Science), Malika Auvray and Christian Duriez (Eds.). Springer Berlin Heidelberg, 201–207.
[64] Regine Zopf, Vince Polito, and James Moore. 2018. Revisiting the link
between body and agency: visual movement congruency enhances
intentional binding but is not body-specific. Scientific Reports 8, 1
(Jan. 2018), 196. https://doi.org/10.1038/s41598-017-18492-7

Page 15