DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation
by Combining it with Brakes
Romain Nith

Shan-Yuan Teng

Pengyu Li

University of Chicagornith@uchicago.edu

University of Chicago
tengshanyuan@uchicago.edu

University of Chicago
pengyulee@uchicago.edu

Yujie Tao

Pedro Lopes

University of Chicago
yujiet@uchicago.edu

University of Chicago
pedrolopes@uchicago.edu

Figure 1: DextrEMS is a haptic device designed to improve the dexterity of electrical muscle stimulation (EMS). It achieves thisby combining EMS with a mechanical brake on all finger joints. These breaks allow us to solve two fundamental problems
with current EMS devices: (1) lack of independent actuation (i.e., when a target finger is actuated via EMS, it also often causes
unwanted movements in other fingers); and (2) unwanted oscillations (i.e., EMS cannot stop a finger at a precise angle without
oscillations, which originate from the fact that to stop a finger, EMS needs to continuously contract the opposing muscle).
Using its brakes, dextrEMS achieves unprecedented dexterity, in both EMS finger flexion and extension, enabling applications
not possible with existing EMS-based interactive devices. For instance, (a) we demonstrate a haptic finger spelling application,that actuates the user’s fingers to pose the “K” sign (in American sign language, from which dextrEMS can only render a verysmall subset); (b) the quality of the same pose via EMS alone suffers from oscillations and unwanted movements; or, lastly, (c)a haptic guitar tutorial that actuates the user’s fingers to form an E-minor chord.

ABSTRACTElectrical muscle stimulation (EMS) is an emergent technique thatminiaturizes force feedback, especially popular for untethered haptic devices, such as mobile gaming, VR, or AR. However, the actuation displayed by interactive systems based on EMS is coarse andimprecise. EMS systems mostly focus on inducing movements inlarge muscle groups such as legs, arms, and wrists; whereas individual finger poses, which would be required, for example, to actuate
a user’s fingers to fingerspell even the simplest letters in sign language, are not possible. The lack of dexterity in EMS stems from two

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
UIST ’21, October 10–14, 2021, Virtual Event, USA© 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-8635-7/21/10. . . $15.00
https://doi.org/10.1145/3472749.3474759

fundamental limitations: (1) lack of independence: when a particular finger is actuated by EMS, the current runs through nearby
muscles, causing unwanted actuation of adjacent fingers; and, (2)unwanted oscillations: while it is relatively easy for EMS to startmoving a finger, it is very hard for EMS to stop and hold that finger
at a precise angle; because, to stop a finger, virtually all EMS systems
contract the opposing muscle, typically achieved via controllers(e.g., PID)—unfortunately, even with the best controller tuning, thisoften results in unwanted oscillations. To tackle these limitations,
we propose dextrEMS, an EMS-based haptic device featuring mechanical brakes attached to each finger joint. The key idea behind
dextrEMS is that while the EMS actuates the fingers, it is our mechanical brake that stops the finger in a precise position. Moreover,
it is also the brakes that allow dextrEMS to select which fingers are
moved by EMS, eliminating unwanted movements by preventing
adjacent fingers from moving. We implemented dextrEMS as anuntethered haptic device, weighing only 68g, that actuates eight
finger joints independently (metacarpophalangeal and proximal
interphalangeal joints for four fingers), which we demonstrate in awide range of haptic applications, such as assisted fingerspelling,UIST ’21, October 10–14, 2021, Virtual Event, USAa piano tutorial, guitar tutorial, and a VR game. Finally, in our
technical evaluation, we found that dextrEMS outperformed EMS
alone by doubling its independence and reducing unwanted
oscillations.

KEYWORDS
electrical muscle stimulation, exoskeleton, dexterity, force feedback,haptics
ACM Reference Format:
Romain Nith, Shan-Yuan Teng, Pengyu Li, Yujie Tao, and Pedro Lopes.
2021. DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by
Combining it with Brakes. In The 34th Annual ACM Symposium on User
Interface Software and Technology (UIST ’21), October 10–14, 2021, Virtual
Event, USA. ACM, New York, NY, USA, 17 pages. https://doi.org/10.1145/
3472749.3474759

1

INTRODUCTIONEMS is a popular technique for force-feedback and haptic-actuation
because of its small form factor when compared to mechanicalactuators (e.g., exoskeletons [75]). As such, non-implanted EMS
has been increasingly used to replace traditional mechanical forcefeedback devices and enable a wide range of untethered/mobile applications, such as: force-feedback in virtual [21, 52] or augmented
reality [53] (VR/AR), moving a user’s wrist to tap to a beat [18, 19],teaching users how to manipulate unfamiliar objects [50], or mobile
information access [49].
However, all these aforementioned non-implanted EMS systems,
and all others in the literature, are limited in that they either do
not actuate the user’s fingers (instead they actuate larger muscle
groups, e.g., arms [30, 52], wrists [47, 50, 52–54] or legs [66]) or
actuate the user’s fingers very coarsely and not independently ofeach other; in other words, current interactive systems based on
EMS display two key issues: (1) lack of independent finger actuation: when a particular target finger is actuated by EMS, theother fingers are actuated as well, causing a lack of dexterity—thisexplains why most authors only envision EMS to enable dexterous
applications, such as for playing guitar, but we have yet to see
any interactive EMS application realized with this level of finger
independence; and, (2) unwanted oscillations: while it is relatively easy for EMS to start actuating the user’s finger muscles, it
is very hard for EMS stop and hold one particular finger at a precise
angle/pose, because to prevent a finger from moving, virtually all
EMS systems continuously contract the opposing muscle, typicallyrelying on a PID controller that regulates this isometric muscle
contraction [37, 43, 49, 72, 74, 88]—unfortunately, even the best
tuning still results in unwanted oscillations as the controller stops
the finger at the target pose, these oscillations are detrimental to theuser experience as they send unnecessary proprioceptive signals tothe user.
This lack of dexterity in EMS-induced finger movements waswell documented in the PossessedHand [84], which pioneered theuse of EMS to engineer interactive devices. In this system, theauthors found that they could only control five out of 16 joints
independently of the other fingers; in other words, 11 out of 15
joints moved together (11 joints with unwanted movements). Although 10 years have passed since the PossessedHand, it still stands

Romain Nith et al.

as the most dexterous finger actuation in all interactive EMS devices. While optimizing electrode layouts can minimize some of theunwanted movements that limit EMS’ independence [82], this has
only been applied to full flexions around the metacarpophalangeal
joints (MCP), while proximal interphalangeal joints (PIP) and extensions of any fingers were not considered; as a result, it can only
flex the full finger but cannot hold it at any precise angle.
The root of non-implanted EMS’s lack of dexterity stems from:(1) forearm muscles that control fingers are densely packed, as they
all meet at the elbow with the humerus bone as a shared anchor;
therefore, stimulating one of these muscles by electrodes attachedto the skin causes currents to run also via adjacent muscles, causing
other muscles contract unwantedly; and, (2) forearm muscles that
control movement around the different joints of the same finger(e.g., MCP vs. PIP) are layered, with muscles that control MCP
movement at one depth (flexor digitorum profundus) and muscles
that control PIP at multiple depths (jointly digitorum superficialis
and digitorum profundus); as such, stimulating a finger around aparticular joint (e.g., MCP) by electrodes attached to the skin usuallycauses currents to run through other layers, resulting in unwanted
movements in other joints (e.g., PIP joint or even the wrist).
To illustrate the importance of this problem, we found that, in justthe last 17 years of EMS research in HCI/Haptics, 54 publications[1, 7, 11, 13–19, 21, 23, 24, 26, 28, 30, 33–41, 43, 45, 47, 50, 54, 55, 61–
65, 67–71, 77–79, 83–89] use EMS for finger movements but do notexhibit any more dexterity in the resulting finger movements than
the PossessedHand, neither they surpassed these two key limitations.
In this paper, we demonstrate a haptic device that increases
the dexterity of EMS-based finger actuation. Our approach, which
we call dextrEMS (a contraction of dexterous-EMS), is a haptic
device that actuates eight finger joints (MCP and PIP) providing an
unprecedented performance compared to existing non-implanted
EMS devices. It addresses the limitations of EMS by combining
it with a mechanical brake at every finger joint. This brake (1)holds the fingers when they reach a desired target angle/pose; and(2) prevents adjacent fingers from moving, by locking fingers not
involved in the target pose. Moreover, because our brake mechanism(a custom-made ratchet) is only used to halt the fingers and not
to actuate them, the resulting device is still lightweight (68g) for
mobile applications.

2

DEXTREMS’S KEY PRINCIPLE: EMS +BRAKE = PRECISE PER-FINGERACTUATION

To tackle the aforementioned limitations that prevent EMS fromdexterity, we propose combining each finger moved by EMS witha brake added at every joint, which is depicted in Figure 2 (b). We
demonstrate this principle at the example of tackling dexterous
EMS finger actuation, a long-sought challenge in EMS. Applying
our principle to the fingers results in a compact wearable device,
which is depicted in Figure 2 (a), that generates force feedback withmore dexterity (i.e., the quality of the resulting pose) than existing
EMS devices.
The key principle behind dextrEMS is depicted in Figure 3 at theexample of fingerspelling the letter “K” in American Sign Language(ASL); it is important to note that dextrEMS does not solve any

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

UIST ’21, October 10–14, 2021, Virtual Event, USA

Figure 2: (a) DextrEMS is stand-alone via a custom PCB that drives the brakes, battery, and wireless. EMS electrodes are placedon the dorsal side for finger extension, and on the palmar side for flexion. (b) Our brake mechanism. (c) We use one ratchet
per MCP and PIP joint on all four fingers, which allows us to stop the finger without the oscillations observed in typical EMS
control loops.
inherent challenge faced by the hard of hearing or deaf community,
not only because of its limitations (only renders a small subset offingerspelled letters, mostly simple poses that do not cross fingers
or use thumb grips) but, more importantly, because no technological solution should “just solve” the unique experience of these
individuals. Rather, we were inspired by the ASL alphabet and use
it to demonstrate the dexterity of dextrEMS when compared to only
using EMS.
Figure 3 decomposes dextrEMS’ actuation into three phases: (a)brakes lock joints that are not meant to move; (b) EMS actuates
the fingers, but its lack of independence is not experienced as ourbrakes prevent unwanted movements in non-target fingers; finally,(c) when each finger reaches the fingerspelling “K” pose (measured
by an external hand-tracking system) the mechanical brakes lockand halt the finger—this results in hand poses without the typicaloscillations seen as EMS-only systems attempt to halt a limb in a
precise position (e.g., [37, 49]).

Figure 3: Working principle behind our haptic device. In thisexample, the target pose is the letter “K” in American Sign
Language. (a) Braking mechanism locks finger joints thataren’t supposed to move. (b) EMS on the flexor actuates thefingers, bringing down free-moving finger joints. (c) Brakes
on the moving fingers lock when the target angle is achieved,
sensed with an external camera, stopping the finger in a
steady state. Our device allows posing without complex EMS
control loops and less EMS stimulation.
Moreover, because our brake mechanism is implemented usinga small ratchet and a matching pawl, it only requires a very small
motor to control the brake (0.13A at 4.2V per brake for only 12.5ms)

while only weighing 68g. This is a stark contrast with the typical size, weight, and power consumption of active exoskeletons,
which require powerful and heavy motors with sufficient force tophysically stop the finger, but, more importantly, it provides animprovement in dexterity when compared to the dexterity offeredby using EMS alone.
This unique combination of EMS and mechanical brakes is the
key to allowing dextrEMS to improve EMS’ dexterity. To better
illustrate this, in Figure 4 we depict the same pose from Figure 3
but using EMS alone. Here, to fingerspell the “K” pose using EMS
alone, it first stimulates the finger flexors that control the pinky,
ring, and middle finger. Unfortunately, these muscles are adjacent
to one another inside the forearm, and EMS currents run throughadjacent muscles causing unwanted movements, which is depicted
in Figure 4 (b). More dramatically, while it is relatively easy for EMS
to start moving a finger, it is very hard for EMS to stop and holdthat finger at a precise angle; because, to stop a finger, virtually all
EMS systems contract the opposing muscle, typically achieved via
controllers (e.g., here we use a dual-sided PID that stimulates both
flexors and extensors)—unfortunately, even with best controller
tuning, this still often results in unwanted oscillations, as depicted
in Figure 4 (c).

Figure 4: (a) Traditional EMS device actuates the fingers withgroups because of layered muscles under the skin, resulting
in moving unwanted finger joints. (b) EMS devices utilize actuation of the opposing muscle group to attempt at slowing
down/stopping the moving limb, this often results in (c) an
unstable pose that oscillates around the target.

UIST ’21, October 10–14, 2021, Virtual Event, USA

3

BENEFITS, CONTRIBUTIONS, AND
LIMITATIONSThe key contribution of our work is a new way to minimize twoof the key limitations of current interactive devices based on EMS;
we achieve this by contributing a new haptic device that combines
EMS with mechanical brakes.
The benefit is that this results in: (1) higher dexterity than current EMS-based haptic devices in both EMS finger flexions andextensions. Our device doubles the independence of EMS (i.e., it
can move a specific finger without unwanted movement of otherfingers) and reduces the amount of unwanted oscillations; (2) dextrEMS enables new interactive applications not possible before with
EMS, for instance, fingerspelling simple letters in ASL, or actuating
a user’s hand to form a chord on a guitar’s fretboard. (3) When
compared to fully actuated exoskeletons, which tend to have more
dexterity than EMS alone, our device is lighter because our brakes
are based on a ratchet, which locks mechanically and not by forcing
against the user.
Our haptic device is not without limitations: (1) like any otherdevice based on EMS it requires electrode placement and calibration. Moreover, like in other EMS devices with electrodes at theforearm, turning the wrist might also break the alignment between
the electrodes and the muscles, hindering the actuation; (2) as anyother passive exoskeleton, it covers part of the user’s hand; note
we intentionally designed our mechanics as such to minimize thisinterference; in fact, unlike most exoskeletons, we ensured that theuser’s fingerpads are not covered in any way to enable maximum
tactile feedback (e.g., one can comfortably type on a mobile phone
while wearing dextrEMS); (3) while it controls flexions and extension of all four fingers around both MCP and PIP joints (eight per
hand in total, two directions, thus 16DoF), it does not actuate thedistal interphalangeal joints nor the thumb; lastly, (4) while our device doubles the independence of finger actuation when comparedwith EMS alone, it is limited by the resolution of the ratchet that
implements the brake. Our laser-cut ratchet has 24 teeth, which
results in a brake position every 15°. One can simply double (or evenmore) the resolution of dextrEMS by CNC-machining the ratchetout of stronger materials (e.g., metal rather than acrylic).

4

RELATED WORKThe work presented in this paper builds primarily on the field ofwearable haptics, in particular exoskeletons and electrical muscle
stimulation.

4.1

Towards wearable force-feedback devices

Force-feedback devices apply force on the user’s body to create
sensations of weight or resistance or even actuate their limbs. Typically, force-feedback devices require substantial power and are thus
constructed from grounded motors, e.g., SpidarG&G [60], EXO-UL7[73], or SensAble PHANTOM Omni [59]. With much of today’s interactions taking place anywhere and anytime, i.e., mobile settings,
researchers have been exploring how to miniaturize force-feedback
devices. The most popular approach is to make devices wearable, i.e.,
attach an actuator to the user and push against the user’s body—thisis called an exoskeleton [3, 5, 6, 25, 29, 57, 73, 81].

Romain Nith et al.

4.2

Wearable force feedback using motors(active exoskeletons) vs. brakes (braking
exoskeletons)

Using glove-like exoskeletons worn on the user’s hand provides
force feedback to the fingers, which is a popular approach in rehabilitation, haptic guidance, and feedback for immersive experiences.
Exoskeleton gloves enjoyed this popularity for a number of decades,
for instance, the CyberGrasp [12], Haptic Telexistence [76], and RML
Glove [57] are canonical examples that used strong motors that canactuate the user’s limbs to provide force feedback. These exoskeletons create force feedback on the fingers by actively resisting their
movement using motors, which in order to provide sufficient force
to stop a finger, turn out to be large. As such, their final form factor
is prohibitively large, heavy, and often not standalone, e.g., power
supplies are external.
As an alternative to this, researchers turned to brake-based exoskeletons, i.e., these use very small motors coupled with passive
brakes, utilizing the fact that the resisting limb movements are realized passively through brake mechanism and not by motors pushingagainst the limb. Wolverine [10] and Grabity [9] are canonical examples that demonstrate how unidirectional brakes between threefingers and a thumb can provide force feedback in VR. Dexmo [25] isa lightweight exoskeleton using small servos combined with rachetsand linkages that can stop the finger movement interactively.
Other alternative mechanisms to resist finger movement also
exist, such as particle jamming using external vacuum pumps (e.g.,
in Jamming glove [92]), pistons [5] [20], layer jamming [8], electrostatic brakes [31], or even magnetorheological fluids [4]. All
these alternatives can generate high braking force with the helpof large equipment (high voltage supply or external pumps) orhave diminished performance when scaled down. These factors
make them unsuitable for portable or wearable applications. We areinspired by these previous works and designed a braking system
combined with EMS that does not require any heavy or powerhungry components—our brake mechanism uses only 0.13A for
12.5ms to halt a moving finger and is made from lightweight acrylic.

4.3

Electrical Muscle Stimulation

Electrical muscle stimulation (EMS) is a more recent alternative tothe long-standing challenge of actuating limbs. EMS involves solelyattaching electrodes to the user’s skin, atop a muscle. Then, passing
a small current through those electrodes causes the muscle fibers
to contract and, in turn, actuate the user’s muscle. As such, many
hail EMS as an increasingly promising technique to miniaturize
strong force feedback at a very small form factor [46].
EMS stems from a long history of developments in medical rehabilitation, back to the 1960s [80]. There, it has been one of the
primary techniques for restoring lost muscular functions (e.g., often as a result of spinal cord injury [80], stroke [32], or drop-footsyndrome [91]).
Only more recently, researchers explore the idea that EMS canreplace mechanical force feedback actuators [46]. Since then, EMS
has been permeating interactive devices, since not only does it
affords force feedback but it accomplishes this at an especiallylightweight form-factor [30, 47–50, 52–54, 66] much needed tobuild small/wearable haptic devices.

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

However, EMS suffers from limitations, one of the most striking
being: its lack of dexterity. This lack of dexterity is exacerbatedby the fact that to interactively stimulate the muscles of a user
with EMS, researchers typically put the electrodes on top of the skin(not implanted). As such, the electrodes cannot access a muscle
precisely without also letting currents pass through adjacent muscles
or even different depths. These limitations have been documented
in the early HCI works that pioneered the use of EMS in interactive
devices, for instance: “This contraction is hard to control” (fromKruijff et al. [42]) and, similarly, “We confirmed that PossessedHand
could control 5 independent and 11 linked joints [11 out of 16
joints have unwanted movements]” (from PossessedHand [84]).
Because of this limitation, it is difficult to individually control
each finger movement with dexterity. As such, most of the work
using EMS focuses on coarse body movement such as on the wrists[47, 49, 50, 52–54], arms [30, 52], and legs [66]. The applications of
EMS range from eyes-free interactions [49], mobile gaming [47],virtual [21, 52], or augmented reality [53], to augmenting object
affordance in daily life [50]. Our work aims to leverage the small
form factor of EMS and improve its dexterity in terms of finger
pose by using a lightweight exoskeleton.

4.4

Adding control loops to stabilize themovement induced by EMS-based actuators

To combat the difficulty of accurately controlling human limbs
through EMS, researchers have employed closed-loop controllers,
especially the proportional-integral-derivative controller (known
as PID), which regulates the EMS applied to opposing muscles so
as to stop the moving limb at a target angle [43, 49, 72, 74]. To
better illustrate how EMS-based interactive systems suffer fromoscillations typical of PID controllers, we look at two examples: (1)Kaul et al. [37] added a PID controller to their EMS system that actuated the user’s arm to point a target, but found overshooting often
affected the trajectories in their study; similarly, (2) Watanabe et al.
[88] added a PID control to their EMS system to actuate the MCP
and PIP joints of the user’s middle finger, but found oscillations
occurred when reaching the target joint angle. As we can observefrom prior work, even the best tuning by these expert researchers still
resulted in unwanted oscillations as the controller attempts to stop
the limb at the target pose. These oscillations are detrimental to theuser experience as they send unnecessary tactile and proprioceptive
signals to the user. Note that while PID tends to be the chosen controller by most researchers, likely due to its capabilities for handlingover/under-shooting, others have employed alternative methods
such as an extended Kalman filter used in Widjaja et al. [90].
Moreover, other advances in EMS are expected to improve thequality of EMS’ dexterity, such as optimizing the placement ofelectrodes that control finger muscles [1], automatic calibration ofhigh-density electrode arrays [39], or even, new electrode layoutsthat offer some minimization of the unwanted actuation [82]. However, the latter only allows to achieve a full flexion around the MCP(no PIP joint, nor extensions of any other fingers); in other words,
it can only flex the full finger at the MCP joint but not stop it at any
precise angle along the way. As such, the challenge of achieving
EMS dexterity without oscillations remains, which is the focus ofour device.

5

UIST ’21, October 10–14, 2021, Virtual Event, USA

IMPLEMENTATION

To help readers replicate our design, we now provide the necessary technical details. Furthermore, to accelerate replication, we
provide all the source code, firmware, 3D files, and schematics of
our implementation1.
DextrEMS is implemented as a self-contained haptic device (i.e., itworks on battery and is wireless) comprised of two key components:(1) a custom lightweight glove with eight brake-based joints and its
controller electronics of a total weight of 68g; and (2) any existing
EMS stimulator with at least four channels. Our haptic device issecured to the user’s hand with Velcro straps.

5.1

Mechanics: braking system

The majority of our brake mechanism was 3D printed (Form Labs
3) using clear resin, while the hinges at each joint were laser cut
out of 3mm clear acrylic. The see-through materials were used tominimize visual obstruction of the real world.
The mechanism of one of our brakes, depicted in detail in Figure 2, is a custom-made ratchet and pawl mechanism. This is astandard mechanism that allows for rotary motion in only one
direction because in the opposite direction the pawl jams against
the depression between the ratchet’s teeth. Unlike passive ratchetand pawl mechanisms (as found in many everyday tools, such as
wrenches) our pawl is controlled by a small DC motor (Vibration
Motor 11.6×4.6×4.8mm, Polulu). To activate the brake, the dextrEMS circuit drives the DC motor clockwise, which jams the pawl
into the ratchet, or counterclockwise to release it from the ratchet.
Like any device based on a ratchet mechanism, our brake is discrete
because the ratchet has a fixed number of teeth that dictate the final
resolution of the brake. Our laser cut ratchet has 24 teeth, which
results in a brake position every 15°; while this is the precision
limit of our current implementation, to surpass it, one only needs
to produce a new ratchet; for instance, a CNC machined ratchet
can feature the double (or more) teeth in the same form factor as
our laser cut one.
While our ratchet and pawl combination does not implementbidirectional mechanics, dextrEMS does achieve bidirectional braking by leveraging EMS and the biomechanics of the fingers to brakein both the flexion direction (the natural brake direction of ourratchet) as well as in the extension direction. To brake in the latterdirection, we preemptively actuate the pawl, earlier than in theflexion direction, i.e., we actuate the pawl one brake position earlier.
Because the finger is moving in the extensor direction, this does
not immediately cause the pawl to jam in the ratchet. Thus, we
subsequently, stop the actuation in the extensor direction, causing
the finger the recoil back to the resting pose, which anatomically is
below extension (neutral, towards flexor). This creates a momentary
flexion as the finger recoils back, which now actuates the pawl in
the correct direction to jam inside the ratchet—achieving braking
also in the extensor direction.
Our mechanics are not without their limitations. Any exoskeleton will exhibit some mechanical compliance (i.e., the so-called“slack”). To this end, we used acrylic, a stiff material, in our design
to mitigate some of this detrimental compliance. Furthermore, we
1 https://lab.plopes.org/#dextrEMS.

UIST ’21, October 10–14, 2021, Virtual Event, USA

Romain Nith et al.

Figure 5: (a) Final self-contained printed circuit board. (b) Electronics schematic of our haptic device.
added two Velcro straps along the metacarpal bone, as well as additional straps for the middle and proximal phalanx of each finger, tosecure the exoskeleton to the hand. These straps ensured an average 105mm2 of contact between the hook and loop, which reduced
mechanical slack. As such, after controlling for these variables, we
believe the main limitation of the mechanical resolution stems fromthe aforementioned ratchet teeth count. Again, fabricating a ratchetwith a higher resolution (e.g., from metal) will likely increase theaccuracy.
Finally, as with most exoskeletons, the fit is never universal.
However, dextrEMS accommodates various hand sizes by adjusting
the Velcro straps as well as our length-tunable hinges, which provide several pin-holes to adjust the length of each joint, as depicted
in Figure 2 (b, c).

5.2

Electronics: circuit design and printed
circuit board

The electronics of our device, depicted in Figure 5, are housedin our custom printed circuit board. Fitted on the back of thehand, its core is an nRF52811 microcontroller (Nordic Semiconductor) with on-chip Bluetooth Low Energy (BLE). Each of oureight 11.6×4.6×4.8mm DC motors (Polulu Vibration Motor, oneplaced at every PIP and MCP joints in all four fingers) is driven by
a DRV8837 H-bridge motor driver (Texas Instruments). Moreover,to better visualize the action of each brake joint, we added a redor green LED parallel to the positive and negative rail output of
the motor-driver, allowing us to visually see if a brake is locked or
unlocked (refer to our Video figure).
The electronics are all powered via a 500mAh 3.7V LiPo battery.
We measured a peak current draw of 1.02A when all eight motors
are stalling, which occurs when all pawls are being moved into
their respective ratchets. While this depicts the highest current
draw possible (all joints need to be locked at the same time) it isextremely brief (∼12.5ms) and quite low-powered since as soon as
the pawl is jammed into the ratchet, the motor drivers release theDC motor, relying on the mechanical force to keep the lock. Thissolution enables our device to be power efficient, especially whencompared to larger exoskeletons built using much larger DC or
servo motors that can consume much more than 1A per individual
motor [6, 22, 57].

5.3

Electrical muscle stimulation

The EMS stimulator we employ in dextrEMS is derived from previous EMS research, demonstrating its applicability to improve thedexterity of existing EMS systems. It implements a unidirectional
PID controller, tuned per joint (similar to [49]). The PID controller
takes as input the angles of each finger joint and outputs the pulsewidth of the EMS signal required to achieve this angle. Like any
EMS-based PID controller, its key contribution is to slow down thefinger’s inertia as it is approaching the target to avoid any brakeovershoot.
Using EMS, we address four muscle groups of the forearm, inparticular: (1) flexor digitorum superficialis at the location where
it flexes predominately the index finger (MCP and PIP); (2) flexor
digitorum superficialis at the location where it flexes predominately
the ring finger and middle finger (MCP and PIP); (3) flexor digitorum
superficialis at the location where it flexes pinky finger (MCP andPIP); and, (4) extensor digitorum, which extends all four fingers(MCP and PIP); all these placements are inspired to those used in
PossessedHand [84].
To actuate the fingers by means of EMS, we use a Rehastim3
medically-compliant muscle stimulator, this device is battery powered and communicates with dextrEMS using an additional hostcomputer with USB and BLE. Then, to enable mobile use, we modified the Rehastim3 library [51] to compile on ARM V7+ architecture (which we provide in1 for replication and the benefit otherresearchers), allowing us to control the EMS device and dextrEMS
from the 55.0mm x 35.0mm microprocessor from the PocketBeagle
[2].

5.4

Tracking and haptic communication
pipeline

Tracking. While our haptic device is self-contained, using dextrEMS requires finger tracking, which is typically supplied by theremainder interactive apparatus (e.g., VR or AR headsets, and soforth). In all our demos we leveraged existing tracking systems. In
particular, in our VR applications, we leveraged the built-in finger
tracking from the Oculus Quest, while in our mobile-phone applications, we used MediaPipe finger tracking [56]. Later, we will also
demonstrate an alternative tracking system that we integrated during the early stages of our device, which was based on flex sensors;

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

however, we found the latter to be less robust than the remainder
optical tracking systems.
Haptic communication pipeline. To illustrate our haptic
pipeline, we describe the steps involved in actuating a finger in our
VR piano tutorial: when the VR application intends to move theuser’s fingers to strike a note of the piano, it sends a “actuate” message to dextrEMS over BLE, which indicates that flexion is neededon the index finger. DextrEMS responds by locking all other fingers
and actuating the EMS channel corresponding to the index andmiddle finger. Then, the VR application tracks the finger’s movement, using its built-in Oculus Quest hand tracking and monitors
for any collision between the finger and any 3D object in the scene.
As the finger pushes the piano key to its final position, the VR application sends a “stop” message to dextrEMS, indicating to stop theindex finger. DextrEMS responds by locking the index finger anddisabling the EMS stimulation. Finally, our smartphone application(Android or iOS), such as the haptic fingerspelling application, use asimilar haptic pipeline but use the front-facing camera of the phone
and MediaPipe as their tracking system. To enable the detectionof simple sign language gestures, we pre-trained on a small set oftargeted (ASL) gestures. The tracking and gesture recognition runs
in real-time.

5.5

Alternative mechanical designs: (a)sideways PIP brake; (b) thumb-mechanism;(c) flex-sensors

While working on our implementation we also engineered threealternatives depicted in Figure 6 (a) a brake for the PIP joint that
did not use hinges that extrude vertically above the fingers; (b)a preliminary brake mechanism for the thumb; and, (d) a simple
integrated finger tracking system using flex sensors. We present
these mechanical designs as we believe they will assist future researchers in creating variations of dextrEMS for applications we
did not explore.

depicted in Figure 6 (a). This alternative mechanism results in aslimmer vertical form factor but wider horizontal form factor; assuch, it is harder to close the fingers (adduction), which is the reason why we opted for our vertical hinge design that affords much
more dexterity. Moreover, this mechanical design cannot be applied to the MCP joint, so it is only suited for applications thatmake use exclusively of PIP-actuation and do not require laterally
closing the fingers together. This alternative PIP brake follows asimilar mechanical principle as our main design showcased in the
Implementation section. However, this mechanical design cannot
be directly driven because the finger itself is in the way. As such,it uses a pulley-based redirection to brake: the pawl jams into theratchet as it is pulled by a wire attached to a small 1.8g linear servo
motor (SPMSA2005, Spektrum). To retract, we spring-loaded themechanism using a spring at the back of the pawl, which keeps it
disengaged when the linear servo motor loosens up the pulley.
Exploring a brake for the thumb. We also engineered a simple brake mechanism for the thumb based on our sideways PIP joint
and a simple pulley for the MCP joint. However, this pulley is not
as effective as the ratchet-pawl brake because it does not lock, i.e., it
relies on force rather than on a locking mechanism. This approachproved only sufficient for applications that do not require strong
forces on the thumb’s MCP. As such, we decided to not implement
this mechanism on our final device. Finally, it is worth noting thatthe thumb has an additional degree of freedom, achieved by anadditional joint: the carpometacarpal (CMC) joint. This joint was
not targeted by our brake system.
Integrating an on-board finger tracking using flex sensors.
While our main dextrEMS device relies on finger tracking suppliedby the interactive applications (e.g., VR applications on Oculus
Quest or phone applications that track via MediaPipe), we also
explored a simple integrated finger tracking system using flex sensors, which is depicted in Figure 6 (c). To implement this, we instrumented each MCP and PIP with two bend sensors, which tracked
the flexion of each joint. To compensate for the change in diameter
when the bend angle increases, the sensors are anchored on one
side of the exoskeleton and are freely sliding on the opposite side,
as depicted in Figure 6 (c).

6

Figure 6: The alternative mechanical designs we also explored while engineering our device: (a) a sideways brake,
mounted directly at the PIP joint, allowing for a smaller vertical form factor but restricting the lateral finger movement(finger adduction); (b) one preliminary design for the thumbjoint, which allows PIP locking but only weak MCP locking;and, (c) integrated sensing using simple flex sensors at every
joint.
Sideways PIP Brake. We implemented an alternative mechanical design for the PIP joint that did not use vertical hinges. Instead,
we implemented a ratchet and pawl configuration that was positioned laterally and, thus, directly at the PIP finger joints, as

UIST ’21, October 10–14, 2021, Virtual Event, USA

TECHNICAL EVALUATION

The goal of our technical evaluation was to compare the performance of dextrEMS to actuating the finger flexor/extensor muscles
using EMS alone. As such, we designed a test setup that allowed us
to measure the two key properties that we expected our device toimprove: (1) finger MCP and PIP independence, i.e., to which
degree can a haptic device actuate a finger around the MCP and PIP
joints independently of other fingers; and, (2) pose precision vs.
unwanted oscillations, i.e., to which degree of precision can ahaptic device pose a target finger to match a specific angle without
unwanted oscillations. Finally, to provide more insights into theapplicability of our system we also measured its: (3) end-to-end
latency, and (4) maximum braking force. To assist the reader
with replicating our technical evaluation, we provide all source
code2 (EMS/dextrEMS controllers, Optitrack script, analysis scripts,
and our custom-made image labeling software).
2 https://lab.plopes.org/#dextrEMSUIST ’21, October 10–14, 2021, Virtual Event, USA

Apparatus. We constructed a simple experimental apparatus,
comprised of a stand that secures the participant’s arm, allowing
us to stabilize the elbow and forearm. For our evaluations, we
recruited a participant per study, from our local institution, withno previous experience with our device. The participant’s reactions
to the interface were not the focus of our experiments nor do we
aim to generalize beyond it. Instead, this technical evaluation was
designed to measure how our braking mechanism improves thedexterity of EMS alone.

6.1

Technical evaluation#1: Finger
independence (i.e., moving each finger’s
MCP & PIP joint without other fingers)

Interface conditions. In this technical evaluation, we measured
finger independence for MCP and PIP flexion, comparing: (1) our
dextrEMS device, connected directly via USB; and, (2) the existing
EMS approach, inspired by the description in PossessedHand [84].
It is important to note that the dextrEMS condition uses precisely
the same EMS setup as the baseline (calibrated the same way), theonly difference being that the participant is additionally wearing
our braking-exoskeleton in combination with EMS. The goal was
to flex each finger (index, middle, ring, and pinky) at a target joint(MCP or PIP) and measure how much the other fingers moved
unwantedly.
Apparatus. To obtain the angles of all the MCP and PIP finger
joints, the participant’s hand was filmed from two angles by twocameras at 30fps. Camera images were corrected with checkerboardmethod for lens distortion. This allowed us to precisely measure
all the angles of all the MCP and PIP joints by manually labeling
each image using a custom script implemented using OpenCV. Notethat prior to deciding on a camera-based approach, we comparedthis and our Optitrack motion tracking system or, even, placing
MPU9250 9DOF inertial measurement units (IMUs) with Madgwick
filters [58] on all joints. We compared the angle estimation of eachtracking system against a ground truth protractor and found thatthe camera approach resulted consistently in the highest precision.
Calibration. We calibrated the EMS for each participant., perfinger joint. First, to determine the stimulation intensity (i.e., current, in mA) we started with an intensity of 0mA and a pulse-widthof 300 µs and slowly increased the intensity in 1 mA steps until the participant’s finger was fully flexed while minimizing any
additional movement on adjacent fingers; then, if necessary toachieve sub-mA adjustments, we fixed the intensity and subsequently adjusted the pulse-width. We repeated this for all jointsof the four fingers, essentially calibrating following a manual version of PossessedHand [84]: (1) attached electrodes to the target
muscles following anatomical guide; (2) per electrode, started withlow pulse-width and increased it step-by-step, confirming at eachstep with the participant that the stimulation was is pain-free; (3)repeated until no more flexion happens at the target finger or pain
has been reached (the latter never occurred). Finally, it is worth
noting that we have years of experience with this EMS calibrationthat follows PossessedHand’s [84].
Procedure. In this test, we recorded a total of 48 trials: four
fingers (index, middle, ring, and pinky) x two joints (MCP and
PIP) x three repetitions x two interface conditions. Per trial, theRomain Nith et al.

EMS actuated one target finger and the video cameras recorded
the movement. The order of the interface conditions and the orderof the finger actuation was randomized. Once all fingers had been
actuated, we moved onto the second interface and repeated for all
four fingers in a newly randomized order. As a result, each trial
depicts the final pose from the resulting actuation.
Accuracy metric: independence index. Our study uses the independence index (also known as I.I. in hand kinematics literature),a standard metric used to measure the amount of independent movement of a finger. This index denotes the ratio of movement between
a finger’s joint (measured in the angle of the joint’s movement) to
how much the other joints moved [44]. Thus, when consideringboth the MCP and PIP joints, the independent index of a target
finger i (IIi ) can be calculated as follows:1Í
k ∈Γi |Ak |IIi = 1 − 7,
where, Ak = ∫ θ k dt,|Ai |T
Here, i denotes a stimulated target finger and Γi is a set of nottarget fingers. An independence index of 0 or lower indicates that
actuating the target finger around this joint caused movement in
the other joints (including even joints in the same finger). Conversely, an index closer to 1 indicates more independence, witha perfect “1” depicting a complete independent movement of thetarget finger around this joint (with no unwanted movements fromother joints). However, it is critical to note that, anatomically, thehuman hand does not exhibit fully independent fingers. As experienced in daily life, many fingers bend when other fingers move.
Lang et al. measured an independence index of 0.84 for passive
finger flexions around the MCP joint when a participant’s finger
was bent passively by a motor to extract its natural limits [44]. For
the sake of visual clarity, we depicted these values of 0.84 (MCP)
as a dashed line in our charts (annotated as “maximum voluntary
independence”). Unlike MCP, finger movements around the PIP are
more independent from other joints [27], so we abstain from the
maximum voluntary independence for PIP to our plots, since theseare highly independent of MCP.

6.2

Result#1: dextrEMS doubled finger
independence of EMS

Figure 7 (a) depicts our results for all four target fingers, as they wereactuated around their MCP and PIP joints Overall, we found thatdextrEMS doubled the finger independence achieved with
EMS alone. We found an average independence of 0.25 (SD =0.31) when actuating MCP and PIP joints using EMS alone, and an
average independence of 0.60 (SD=0.25) when actuating MCP and
PIP joints using dextrEMS. This is our main result for this technicalevaluation.
Now, we analyze the overall independence for all four fingers
around each joint. For the MCP joint of all fingers, we found an
average independence of 0.18 (SD=0.17) when actuating MCP joints
using EMS alone, and an average independence of 0.56 (SD=0.16)
when actuating MCP joints using dextrEMS—as such, on average,
for all MCP joints, dextrEMS almost tripled the finger independence
of EMS. Secondly, for the PIP joint of all fingers, we found an average
independence of 0.31 (SD=0.40) when actuating PIP joints using
EMS alone, and an average independence of 0.65 (SD=0.31) when

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

UIST ’21, October 10–14, 2021, Virtual Event, USA

Figure 7: (a) Overall independence for all fingers in both conditions. (b) and (c) Result breakdown per joint (MCP vs. PIP) andfinger.
actuating MCP joints using dextrEMS—as such, on average, for all
PIP joints, dextrEMS doubled the independence of EMS.
For the MCP joint, we observed improvements in the independence of all fingers, except the index. Then, for the PIP joints, weobserved improvements in the independence of all fingers. Remarkably, even the ring finger, known to be “robust to actuate withoutparasitical motions of adjacent fingers” [35], was improved.
Finally, to give the reader a visual sense of the difference inquality between a pose rendered by dextrEMS when compared to
EMS-only, we provide exemplary situations taken directly from thecamera data. Figure 8 depicts an exemplary improvement over an
independent flexion of the ring finger at the PIP joint; this joint
is notoriously hard to address using EMS-only. Indeed, we foundthat using EMS-only, this finger (even with the best calibration andplacement) tends to unwantedly flex its own MCP and the MCP
and PIP of the adjacent ring finger.

Figure 8: An example of how (a) dextrEMS improved the dexterity of a PIP flexion of the ring finger when compared to(b) EMS-only.

Figure 9: An example of how (a) dextrEMS improved the dexterity of an MCP flexion of the index finger when comparedto (b) EMS-only.
different angles. We chose to actuate the ring finger around the MCP
joint, since it is easy to actuate (i.e., it is easy to move using EMS, as
used in [35]). This finger is also extremely sensitive to EMS currents
and hard to stop at a precise angle without oscillations; in otherwords, this finger is an ideal case to see if dextrEMS can improve
its performance at the MCP joint. Moreover, as our evaluation
involved 120 trials for a single finger, evaluating the performanceof all remaining four fingers would likely extend the study duration
dramatically and induce fatigue in participants.
Interface conditions. As such, we compared: (1) dextrEMS
device and, (2) the existing EMS approach, which is based on a
PID controller that regulates the contraction of opposing muscles
to halt the finger at a precise angle.; this approach is replicatedfrom [49] and used widely in interactive devices based of EMS[37, 43, 49, 72, 74, 88].
Apparatus. We use the same apparatus from our previous technical evaluation. Additionally, because the haptic devices aimed
to stop the finger at a precise angle, they required a closed-loop

Figure 9 depicts an exemplary improvement over an independent
flexion of the index finger at the MCP joint; this is an easier finger
to address using EMS-only but not independently of unwanted
PIP movements from other fingers. Indeed, we found that using
dextrEMS was able to reduce unwanted movements.

6.3

Technical evaluation#2: Pose precision (i.e.,
stopping the finger at precise angles
without oscillations)

In this test, we focus on measuring dextrEMS’ ability to stop thefinger at a target angle without oscillations. In this technicalevaluation, we measured the precision and unwanted oscillations
caused by dextrEMS and EMS-alone in stopping a finger at 12

Figure 10: (a) Amplitude of oscillations and (b) error to target
in stopping a ring finger with dextrEMS or EMS, for all 12angles.

UIST ’21, October 10–14, 2021, Virtual Event, USA

Romain Nith et al.

Figure 11: Accuracy (error to target) in stopping a flexing ring finger with dextrEMS or EMS, for six angles. Each plot is annotated with the frequency of any observed large oscillations (typically 2-2.5Hz with EMS-only condition), and a sidebar chartwith the amplitude of oscillations between 0.5-1s.
(between the current finger angle vs. target angle to reach). As
such, we supply both EMS and dextrEMS with the finger pose (angle of MCP) in real-time. To realize this, we used the Optitrack
motion capture system to retrieve the angle with two rigid bodies,
attached to the proximal phalanx of the target finger and a rigidbody baseline attached at the back of the hand. Note that, prior tothis, we compared between this Optitrack motion tracking systemand placing two of the aforementioned 9DOF IMUs combined with
Madgwick filters [58] and found that the Optitrack yielded a higher
accuracy and no drift over time.
Calibration procedure. For each angle measured, we calibrated
the EMS placement following our previous test. Then, we iteratively
calibrated the PID controllers for each condition. We varied thestimulation pulse-width from 100 to 450 µs, while using the same
current intensity (7 mA for flexor and 8 mA for extensor) in bothconditions. For the existing EMS condition (baseline), its dual-sided
PID (i.e., extensor and flexor) was tuned for maximum precision
with minimum oscillations, however, these are not always possible
to dissipate entirely. As for dextrEMS condition, a single-sided PID
was tuned, in the same manner, to reach the target with maximum
precision; moreover, we also calibrated the brake’s mechanicallatency to compensate for its triggering time. While both conditions
were calibrated, to the best of the expert experimenter’s knowledge,
with the same goal to minimize error, as with most studies involving

replication of a baseline by an experimenter, there is a small chanceof unconscious bias. Yet, we have ample experience in tuning thesetypes of PID controllers for EMS.
Procedure. We recorded a total of 120 trials: 6 angles x 2 starting positions (rested and open hand) x 5 repetitions x 2 interface
conditions. In each trial, the EMS actuated the ring finger to one of
the target angles for flexion (8°, 17°, 23°, 33°, 45°, 60°) and extension(0°, 8°, 17°, 23°, 33°, 45°). The angles were defined by the resolutionof dextrEMS’ ratchet and the anatomy of the participant since thefinger joint’s articulation does not translate linearly to the ratchet’s
rotation. The order of the interface conditions and, within it, theorder of angles was randomized. As a result, each trial depicts theerror between the desired angle and the final angle. Note that ourdevice and the baseline exhibit different stopping behaviors: (1) dextrEMS uses the brake to stop the finger at the target angle, when
this is locked, the angle thus remains final (with some variancedue to mechanical slack at the Velcro connections); and, (2) the
canonical EMS approach to stop a moving finger (as well as otherlimbs) relies on controllers (typically a PID, e.g., [49]) that regulate a dual-sided muscle stimulation, i.e., the controller regulates
the intensity of EMS contractions of the flexor and extensor, so
that, in theory, these cancel out and stop the finger from moving.
In practice, even with our best efforts and years of experience incalibrating PID for EMS, these tunings often still result in some

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

UIST ’21, October 10–14, 2021, Virtual Event, USA

Figure 12: Accuracy (error to target) in stopping an extending ring finger with dextrEMS or EMS, for six angles. Each plot
is annotated with the frequency of any observed large oscillations (typically 2-3Hz with EMS-only condition), and a sidebar
chart with the amplitude of oscillations between 0.5-1s.
oscillation, and not in an entirely “stopped” finger. To capture thisdynamic behavior and reveal more about the two haptic devices,
we recorded the angle data for 1.2 seconds.

6.4

Results#2: dextrEMS holds fingers without
oscillations experienced with EMS only

We found that for all 12 angles, dextrEMS minimized the oscillations
typical of PID controllers used in EMS; this is the main finding of
this experiment, which is depicted in Figure 10. Oscillations were
measured by analyzing the standard deviation of each trial andaveraging these per condition for the time frame between 0.5s-1.0s.
We found an average oscillation amplitude of 1.62° (SD=0.81°) when
actuated by dextrEMS, and an average oscillation amplitude of 8.37°(SD=7.39°) when actuated by EMS-only.
Then, also in Figure 10, we depict the average error to the target
for each condition, all angles. We found an average error of 9.72°(SD=10.37°) when actuated by dextrEMS, and an average error of
11.55° (SD=6.23°) when actuated by EMS-only. As such, we foundthese to be comparable. What is beneficial to our approach is thatwhile EMS-only oscillates around the target with an average errorof 11.55°, dextrEMS brakes around the target with an average errorof 9.72°; as such, the improvement is dextrEMS does so without
unwanted oscillations.

Moreover, as discussed before, dextrEMS average error to target
is derived from its ratchet’s resolution, which is 24 teeth in oursimple laser cut ratchet. One can easily double (or more) the resolution of dextrEMS by CNC-machining the ratchet out of stronger
materials (e.g., metal rather than simple acrylic).
In Figure 11, we depict the trajectories of the ring finger, as it
flexes around the MCP joint, for all six angles. Again, we found thatdextrEMS minimizes unwanted oscillations caused by the EMS’ PID
controllers. Moreover, it also depicts where dextrEMS excels andwhere it can still be improved in terms of halting finger flexions at
precise angles: dextrEMS was braking in a stable manner for threeof six angles (i.e., for 17°, 23°, and 33°) while in the at 45° and 60 °we observed the dextrEMS brake undershooting on occasion.
Finally, in Figure 12, we depict the trajectories of the ring finger,
as it extends around the MCP joint, for all six angles. We foundthat dextrEMS minimized unwanted oscillations from EMS’ PID
controllers. Moreover, we depicted where dextrEMS excels in termsof halting finger extensions at precise angles: dextrEMS provided
superior accuracy for 17° and 45° targets, which in case the EMS
suffered from oscillations. In the remainder angles, we observed
the dextrEMS and EMS to have similar performance for extensions,especially at 0° and 8° where both dextrEMS and EMS displayed asimilar undershooting profile.

UIST ’21, October 10–14, 2021, Virtual Event, USA

6.5

Technical evaluation#3: Latency

To measure the end-to-end latency of our device, we used a highspeed camera at 240 fps to film a particular joint of our device aswell as the screen of the VR application, which was set to flash
once a message to our device was sent; this message informed one
joint to lock using the brake. We obtained an end-to-end latency of
229.2ms, which we are able to break down into: 176.7ms of WiFi
communication latency (from VR to a PC via a WiFi router), 40ms
of BLE communication latency (from a PC to our microcontroller
receiving this message and blinking a built-in LED in response) and12.5ms of mechanical latency (from our motor driver LED turningon to the brake being fully locked with the ratchet). While 229.2ms
is not appropriate for fast haptic feedback, note that the key latency
here is only 52.5ms (from receiving a BLE message to locking a
joint), which is suitable for most haptic applications. The remaining
176.7ms of latency from the WiFi communication was not optimized
in our applications.

6.6

Technical evaluation#4: Maximum braking
force

Finally, we measured the maximum braking force of our mechanical brakes. To do so, we used a load cell rated for 5kg with an
ADC amplifier (HX711), which was sampled at 100Hz via a 16-bit
microcontroller. The base of the load cell was clamped onto a rigid
support structure, while the other end was floating with the exoskeleton secured on it. We tested both joints separately to identify
the failure point of the brake with their linkages set to the longest
setting. In both tests, we locked the joint’s brake and applied anincreasing force. We sampled the force sensor until the brake mechanism was broken. We found a maximum braking force of 1377.4g(13.5N) for the MCP joint and 1348.1g (13.2N) for the PIP with our
3D printed resin braking mechanism. Naturally, we expect thatprinting our design using robust materials such as carbon fiber oraluminum would yield a considerably larger braking force. On theother hand, our 3D printed mechanism already absorbs 13N, whichis reasonable for the proposed applications.
Additionally, to assist in understanding the braking force that ourmechanism provides, we also measured how much force a finger
that is stimulated by EMS exhibited. We found an average force of
11.9N (SD=0.25N) across all four fingers, using three repetitions

Romain Nith et al.

at an EMS intensity 1mA higher than the maximum used in ourevaluations. Furthermore, these measurements were done withboth PIP and DIP joints combined, which stands in contrast to ourbraking force evaluation, which tested for one joint at a time. To
sum up, the average forced-induced by EMS on the finger (11.9N)
was below our braking force (13.5N), which explained why no
dextrEMS broke in our evaluations.

7

APPLICATIONS

We created five applications that showcase our haptic device: (1)haptic fingerspelling of a small subset of ASL letters; haptic guidance for guitar chords; (3) VR piano; (4) whack-a-mole game; and,(5) VR bouldering simulator.

7.1

Application #1: haptic fingerspelling for asmall subset of American Sign Language(ASL)

Figure 13 depicts a mobile application that enables a user, without
prior knowledge of fingerspelling the manual ASL alphabet, to use
dextrEMS to communicate simple letters to a hard-of-hearing individual. As aforementioned, dextrEMS does not solve the inherent
challenges faced by the hard of hearing individual or deaf community, not only because of its limited abilities but, also, because
no technological solution should “solve” the unique experience of
these individuals. Instead, we were inspired by the ASL alphabet
and use it to demonstrate dextrEMS.
Figure 13 (a) shows the non-ASL cognizant user pointing theirsmartphone’s camera to their friend, who is speaking using lettersof a fingerspelling alphabet. The smartphone is running our custom
application to illustrate the portability of dextrEMS. Our application
leverages MediaPipe hand tracking [56] to recognize a fixed set ofpre-trained and simple ASL fingerspelled signs. Once a letter sign isrecognized, it is shown on the user’s smartphone as text, depictingthat their friend said, “what are you studying?” in ASL. Figure
13 (b), shows how the user, who unfortunately does not know
how to fingerspell, types “HCI” as a response using the phone’s
keyboard. Our application then translates the text, according to
ASL fingerspelling, into hardware instructions for dextrEMS. The
instruction is then sent to dextrEMS via Bluetooth. In response,
dextrEMS renders each letter’s corresponding pose by actuating the

Figure 13: Our custom smartphone application detects fingerspelling by using MediaPipe and translates it into text. (b) The
user types its response “HCI” onto the smartphone, which (b, c, and d) dextrEMS outputs by actuating the fingers to form afingerspelled “HCI”.

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

UIST ’21, October 10–14, 2021, Virtual Event, USA

Figure 14: In this simple application, dextrEMS assists a user in performing the E-minor chord, which we chose due to its pose
simplicity.
user’s muscles with sufficient dexterity to convey these, relatively
simple, letters. Figure 13 (b) shows dextrEMS outputting the “H” of“HCI”. After rendering the “H” letter for three seconds, dextrEMS
moves to the next letter (“C” in “HCI”), which is depicted in Figure
13 (c). To return the user’s hand to its resting state it releases all thelocks are released and dextrEMS makes use of all the extensors to“reset” the hand to its resting pose. Finally, as shown in Figure 13 (d),dextrEMS repeats the same process to fingerspell the “I” in “HCI”.
Again, we highlight that both the complete set of ASL fingerspelled
letters is of an immense complexity that neither dextrEMS nor any
device can achieve, but also sign language entails a vast richness
that simple devices like ours will never have the capacity to assist
with (including hand gestures, body language, facial expressions
and more).

7.2

Application #2: guitar chords using haptic
guidance

Our second application explores a different haptic tutorial: guitar
chords. Guitar requires great dexterity, especially when novice
players explore where to place their fingers on the fretboard to form
chords. In this application, depicted in Figure 14, we demonstrate
how dextrEMS assists a player in forming a chord by actuating
their fingers to the correct position on the fretboard to form an E
minor chord. Note that while this chord is relatively simple (e.g.,compared to a more complex chord, such as an E minor flat sixth),
it would not be easily achievable with EMS alone.
To position the fingers at the chord, dextrEMS first extends all
fingers (brakes open) so that none are touching the fretboard. Then,it locks the fingers that will not be used in the E-minor chord. Before
actuating the MCPs, it adjusts the fingers’ PIP joints’ angle using
EMS actuation and braking when the PIP joints are in place. Oncethe chord finger has been formed in mid-air, dextrEMS unlocks the
MCPs and pushes all fingers with EMS against the fretboard.

7.3

Application #3: a force feedback piano
chord tutor in VR

In Figure 15, we demonstrate how dextrEMS enables our VR piano
application to independently actuate each finger to achieve simple melodies or even multiple notes. First, dextrEMS actuates theuser’s fingers by locking those that should not move and actuating
the remainder via EMS to form the target notes; here, it actuates
the user’s hand to play a minor third interval over D, which is
depicted in Figure 15 (b). When the user’s fingers reach the bottomof the piano keys (i.e., the virtual keys’ mechanism is bottomedout and cannot move more), dextrEMS locks all fingers, providing the haptic feedback of the final position, which is depicted in
Figure 15 (c).

Figure 15: In this simple application, dextrEMS assists a user in performing a minor third interval of D.

UIST ’21, October 10–14, 2021, Virtual Event, USA

Romain Nith et al.

Figure 16: (a) In our whack-a-mole game, (b) players can feel the shape of the mole; (c) here, the mole fights back by extending
its arm upwards, which dextrEMS renders by actuating the index and locking it at the correct pose.

Figure 17: (a) This user swipes their hand sideways across the bouldering wall, trying to feel the next hold. (b) as their handreaches over the hold, it starts to be actuated by dextrEMS to pose the hand to (c) emulate the different hold shapes.

7.4

Application #4: feeling the changing shapeof the moles in a VR whack-a-mole gameWe provide an example of a more traditional use of EMS as forcefeedback in VR. However, instead of using EMS for large and coarse
movements (as typical of EMS in VR [48, 52]), we use dextrEMS for
rendering a changing “shape”, which requires more dexterous finger
actuation. To demonstrate this, we implemented a simple Whack-AMole game in VR, consisting of pushing the moles back to their holes
by whacking them on their heads. While in the typical renditionof this game moles disappear once hit, our moles fight back bypushing one of their arms up, pushing the user’s index finger up.
We make use of dextrEMS to render this individual motion of thefinger (dextrEMS locks all other joints and actuates the extensors,causing only the index finger to move up). As the mole fully extends
their arm, dextrEMS now locks the index as well, resulting in a
final pose that matches the expected shape of the mole. The userholds the mole for a few seconds, and it recoils to the hole. Now,
dextrEMS releases all brakes, and the user’s finger falls back down.
Finally, the user repeats this for the remaining moles, to win the
game.

7.5

Application #5: VR bouldering simulator

In this last VR experience, we demonstrate how dextrEMS allows
this user to feel the shapes of bouldering holds (the protuberancesthat stand out from a bouldering wall). Figure 17 (a) shows the user
moving their hand across the wall, trying to find the next hold tograb onto. As their fingers first hit the side of the hold, dextrEMS
actuates their index finger upwards and locks it into the heightof the shape, which is depicted in Figure 17 (b). Then, dextrEMS
continues to actuate and lock remainder fingers, until they are in

the correct geometry of the bouldering hold. Figure 17 (c) shows
the user’s fingers taking the shape of the hold. Conversely, as theuser continues moving to the other side, the process reverses anddextrEMS actuates the fingers down to the wall.

8

CONCLUSION

In this paper, we increased the dexterity of electrical muscle stimulation, by proposing and engineering an EMS-based haptic device
combined with a lightweight mechanical brake attached to each finger joint, which we called dextrEMS. The key idea behind dextrEMS
is that while the EMS actuates the fingers, it is our mechanical brakethat stops the finger in the correct pose. Moreover, dextrEMS alsouses its brakes to select which fingers are “free” to be moved bymeans of EMS, eliminating unwanted movements by preventing
adjacent fingers from moving.
We implemented dextrEMS as an untethered haptic device that
actuates eight finger joints independently (metacarpophalangeal
and proximal interphalangeal joints for four fingers), which wedemonstrate in a wide range of haptic applications, such as assisted
fingerspelling, a piano tutorial, guitar tutorial, and a VR game.
Finally, in our technical evaluation, we found that dextrEMS outperformed EMS alone by doubling its independence and reducing
unwanted oscillations.
As for future work, we expect researchers following up on dextrEMS to create variations attached to other body joints; in otherwords, dextrEMS is ripe for applications beyond just finger actuation, for instance, one can attach a brake to the elbow joint andprecisely brake the biceps without the need for EMS-PID controllers
for the triceps (as in [49, 52]). Moreover, there are several ways
researchers can expand on our mechanical design. For instance,DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes

researchers might explore replacing our ratchet-pawl mechanismwith a truly bidirectional ratchet, which involves more logic andmechanics (thus, potentially larger), but will enable even quicker
stopping in the extensor direction. Similarly, we expect other researchers to fabricate dextrEMS using even stronger material (e.g.,
CNC machined ratchets made from metal), which easily leads todoubling its resolution, which is primarily only limited by the resolution of the ratchet teeth. Finally, other researchers might explore
adding our mechanism to the DIP joint, to enable new applications
that use even finer motor skills of all three finger joints.

ACKNOWLEDGMENTS
This project initially started at McGill University under Prof. Cooperstock’s supervision in collaboration with Romain Nith, Clement
Fournier and Augustin Legrand. We would like to thank Prof. Cooperstock for his valuable feedback, and the rest of the team for theircontribution and support.

REFERENCES[1] Bao, X., Zhou, Y., Wang, Y., Zhang, J., Lü, X. and Wang, Z. 2018. Electrodeplacement on the forearm for selective stimulation of finger extension/flexion.
PLOS ONE. 13, 1 (Jan. 2018), e0190936. DOI:https://doi.org/10.1371/journal.pone.
0190936.
[2] BeagleBoard.org - pocket: https://beagleboard.org/pocket. Accessed: 2021-04-07.
[3] Bergamasco, M., Allotta, B., Bosio, L., Ferretti, L., Parrini, G., Prisco, G.M., Salsedo,
F. and Sartini, G. 1994. An arm exoskeleton system for teleoperation and virtualenvironments applications. Proceedings of the 1994 IEEE International Conferenceon Robotics and Automation (May 1994), 1449–1454 vol.2.
[4] Blake, J. and Gurocak, H.B. 2009. Haptic Glove With MR Brakes for Virtual
Reality. IEEE/ASME Transactions on Mechatronics. 14, 5 (Oct. 2009), 606–615.
DOI:https://doi.org/10.1109/TMECH.2008.2010934.
[5] Bouzit, M., Burdea, G., Popescu, G. and Boian, R. 2002. The Rutgers Master II-new
design force-feedback glove. IEEE/ASME Transactions on Mechatronics. 7, 2 (Jun.
2002), 256–263. DOI:https://doi.org/10.1109/TMECH.2002.1011262.
[6] Carignan, C., Tang, J. and Roderick, S. 2009. Development of an exoskeleton
haptic interface for virtual task training. 2009 IEEE/RSJ International Conferenceon Intelligent Robots and Systems (Oct. 2009), 3697–3702.
[7] Chen, K., Zhang, B. and Zhang, D. 2014. Master-Slave Gesture Learning System
Based on Functional Electrical Stimulation. Intelligent Robotics and Applications.
X. Zhang, H. Liu, Z. Chen, and N. Wang, eds. Springer International Publishing.
214–223.
[8] Choi, I., Corson, N., Peiros, L., Hawkes, E.W., Keller, S. and Follmer, S. 2018. A
Soft, Controllable, High Force Density Linear Brake Utilizing Layer Jamming.
IEEE Robotics and Automation Letters. 3, 1 (Jan. 2018), 450–457. DOI:https://doi.
org/10.1109/LRA.2017.2761938.
[9] Choi, I., Culbertson, H., Miller, M.R., Olwal, A. and Follmer, S. 2017. Grabity: A
Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality.
Proceedings of the 30th Annual ACM Symposium on User Interface Software andTechnology (New York, NY, USA, 2017), 119–130.
[10] Choi, I. and Follmer, S. 2016. Wolverine: A Wearable Haptic Interface for Graspingin VR. Proceedings of the 29th Annual Symposium on User Interface Software andTechnology (New York, NY, USA, 2016), 117–119.
[11] Colley, A., Leinonen, A., Forsman, M.-T. and Häkkilä, J. 2018. EMS Painter: Cocreating Visual Art using Electrical Muscle Stimulation. Proceedings of the Twelfth
International Conference on Tangible, Embedded, and Embodied Interaction (NewYork, NY, USA, Mar. 2018), 266–270.
[12] CyberGrasp: http://www.cyberglovesystems.com/cybergrasp. Accessed: 2019-0902.
[13] Dingler, T., Goto, T., Tag, B. and Kunze, K. 2017. EMS icons: conveying information
by analogy to enhance communication through electrical muscle stimulation.
Proceedings of the 2017 ACM International Joint Conference on Pervasive andUbiquitous Computing and Proceedings of the 2017 ACM International Symposium
on Wearable Computers (New York, NY, USA, Sep. 2017), 732–739.
[14] Duente, T., Pfeiffer, M. and Rohs, M. 2016. On-skin technologies for muscle
sensing and actuation. Proceedings of the 2016 ACM International Joint Conference
on Pervasive and Ubiquitous Computing: Adjunct (New York, NY, USA, Sep. 2016),
933–936.
[15] Duente, T., Pfeiffer, M. and Rohs, M. 2017. Zap++: a 20-channel electrical muscle
stimulation system for fine-grained wearable force feedback. Proceedings of the
19th International Conference on Human-Computer Interaction with Mobile Devices
and Services (New York, NY, USA, Sep. 2017), 1–13.

UIST ’21, October 10–14, 2021, Virtual Event, USA[16] Duente, T., Schneegass, S. and Pfeiffer, M. 2017. EMS in HCI: challenges andopportunities in actuating human bodies. Proceedings of the 19th International
Conference on Human-Computer Interaction with Mobile Devices and Services (NewYork, NY, USA, Sep. 2017), 1–4.
[17] Duente, T., Schulte, J., Pfeiffer, M. and Rohs, M. 2018. MuscleIO: Muscle-Based
Input and Output for Casual Notifications. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2, 2 (Jul. 2018), 64:1-64:21.
DOI:https://doi.org/10.1145/3214267.
[18] Ebisu, A., Hashizume, S. and Ochiai, Y. 2018. Building a feedback loop between
electrical stimulation and percussion learning. ACM SIGGRAPH 2018 Studio (Vancouver, British Columbia, Canada, Aug. 2018), 1–2.
[19] Ebisu, A., Hashizume, S., Suzuki, K., Ishii, A., Sakashita, M. and Ochiai, Y. 2017.
Stimulated percussions: method to control human for learning music by using
electrical muscle stimulation. Proceedings of the 8th Augmented Human International Conference on - AH ’17 (Silicon Valley, California, 2017), 1–5.
[20] ExoHand | Festo Corporate: https://www.festo.com/group/en/cms/10233.htm.
Accessed: 2021-04-06.
[21] Farbiz, F., Yu, Z.H., Manders, C. and Ahmad, W. 2007. An Electrical Muscle
Stimulation Haptic Feedback for Mixed Reality Tennis Game. ACM SIGGRAPH
2007 Posters (New York, NY, USA, 2007).
[22] Fontana, M., Ruffaldi, E., Salasedo, F. and Bergamasco, M. On the Integration of
Tactile and Force Feedback. Haptics Rendering and Applications.
[23] Fortin, P.E., Blum, J.R. and Cooperstock, J.R. 2017. Raising the Heat: Electrical
Muscle Stimulation for Simulated Heat Withdrawal Response. Adjunct Publicationof the 30th Annual ACM Symposium on User Interface Software and Technology(New York, NY, USA, Oct. 2017), 137–139.
[24] Grönvall, E., Fritsch, J. and Vallgårda, A. 2016. FeltRadio: Sensing and Making
Sense of Wireless Traffic. Proceedings of the 2016 ACM Conference on Designing
Interactive Systems - DIS ’16 (Brisbane, QLD, Australia, 2016), 829–840.
[25] Gu, X., Zhang, Y., Sun, W., Bian, Y., Zhou, D. and Kristensson, P.O. 2016. Dexmo:
An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture
and Force Feedback in VR. Proceedings of the 2016 CHI Conference on Human
Factors in Computing Systems (New York, NY, USA, 2016), 1991–1995.
[26] Gui, K. and Zhang, D. 2014. Influence of volitional contraction on muscle response to functional electrical stimulation. 2014 IEEE 19th International Functional
Electrical Stimulation Society Annual Conference (IFESS) (Sep. 2014), 1–4.
[27] Häger-Ross, C. and Schieber, M.H. 2000. Quantifying the Independence of Human
Finger Movements: Comparisons of Digits, Hands, and Movement Frequencies.
Journal of Neuroscience. 20, 22 (Nov. 2000), 8542–8550. DOI:https://doi.org/10.
1523/JNEUROSCI.20-22-08542.2000.
[28] Hanagata, S. and Kakehi, Y. 2018. Paralogue: A Remote Conversation System
Using a Hand Avatar which Postures are Controlled with Electrical Muscle
Stimulation. Proceedings of the 9th Augmented Human International Conference(New York, NY, USA, Feb. 2018), 1–3.
[29] HaptX | Haptic gloves for VR training, simulation, and design: https://haptx.com/.
Accessed: 2020-05-04.
[30] Hassib, M., Pfeiffer, M., Schneegass, S., Rohs, M. and Alt, F. 2017. Emotion Actuator:
Embodied Emotional Feedback Through Electroencephalography and Electrical
Muscle Stimulation. Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems (New York, NY, USA, 2017), 6133–6146.
[31] Hinchet, R., Vechev, V., Shea, H. and Hilliges, O. 2018. DextrES: Wearable Haptic
Feedback for Grasping in VR via a Thin Form-Factor Electrostatic Brake. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology(New York, NY, USA, 2018), 901–912.
[32] Hummelsheim, H., Maier-Loth, M.L. and Eickhof, C. 1997. The functional value ofelectrical muscle stimulation for the rehabilitation of the hand in stroke patients.
Scandinavian Journal of Rehabilitation Medicine. 29, 1 (Mar. 1997), 3–10.
[33] Ishimaru, T. and Saga, S. 2020. Virtual bumps display based on electrical muscle
stimulation. 2020 IEEE Haptics Symposium (HAPTICS) (Mar. 2020), 96–101.
[34] Jain, S., Sharma, S. and Babbar, D. 2017. Star-Force: A Playful Implementationof the Jedi-force. Proceedings of the Tenth International Conference on Tangible,
Embedded, and Embodied Interaction - TEI ’17 (Yokohama, Japan, 2017), 761–766.
[35] Kasahara, S., Nishida, J. and Lopes, P. 2019. Preemptive Action: Accelerating
Human Reaction Using Electrical Muscle Stimulation Without Compromising
Agency. Proceedings of the 2019 CHI Conference on Human Factors in Computing
Systems (New York, NY, USA, 2019), 643:1-643:15.
[36] Katoh, M., Nishimura, N., Yokoyama, M., Hachisu, T., Sato, M., Fukushima, S. andKajimoto, H. 2013. Optimal selection of electrodes for muscle electrical stimulation using twitching motion measurement. Proceedings of the 4th Augmented
Human International Conference on - AH ’13 (Stuttgart, Germany, 2013), 237–238.
[37] Kaul, O.B., Pfeiffer, M. and Rohs, M. 2016. Follow the Force: Steering the Index
Finger towards Targets using EMS. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (New York, NY, USA,May 2016), 2526–2532.
[38] Kitamura, T., Mizoguchi, H., Mizukami, N., Sakaino, S. and Tsuji, T. 2017. Chattering reduction of functional electrical stimulation with the smith compensator.
IECON 2017 - 43rd Annual Conference of the IEEE Industrial Electronics Society(Oct. 2017), 7577–7582.

UIST ’21, October 10–14, 2021, Virtual Event, USA[39] Knibbe, J., Strohmeier, P., Boring, S. and Hornbæk, K. 2017. Automatic Calibrationof High Density Electric Muscle Stimulation. Proceedings of the ACM on Interactive,
Mobile, Wearable and Ubiquitous Technologies. 1, 3 (Sep. 2017), 1–17. DOI:https://doi.org/10.1145/3130933.
[40] Kono, M., Ishiguro, Y., Miyaki, T. and Rekimoto, J. 2018. Design and Study of aMulti-Channel Electrical Muscle Stimulation Toolkit for Human Augmentation.
Proceedings of the 9th Augmented Human International Conference (New York,NY, USA, Feb. 2018), 1–8.
[41] Kono, M., Takahashi, T., Nakamura, H., Miyaki, T. and Rekimoto, J. 2018. Design
Guideline for Developing Safe Systems that Apply Electricity to the Human Body.
ACM Transactions on Computer-Human Interaction. 25, 3 (Jun. 2018), 19:1-19:36.
DOI:https://doi.org/10.1145/3184743.
[42] Kruijff, E., Schmalstieg, D. and Beckhaus, S. 2006. Using neuromuscular electricalstimulation for pseudo-haptic feedback. Proceedings of the ACM symposium on
Virtual reality software and technology (New York, NY, USA, Nov. 2006), 316–319.
[43] Kurita, Y., Ishikawa, T. and Tsuji, T. 2016. Stiffness Display by Muscle Contraction
Via Electric Muscle Stimulation. IEEE Robotics and Automation Letters. 1, 2 (Jul.
2016), 1014–1019. DOI:https://doi.org/10.1109/LRA.2016.2529689.
[44] Lang, C.E. and Schieber, M.H. 2004. Human Finger Independence: Limitations due
to Passive Mechanical Coupling Versus Active Neuromuscular Control. Journalof Neurophysiology. 92, 5 (Nov. 2004), 2802–2810. DOI:https://doi.org/10.1152/jn.
00480.2004.
[45] Limanowski, J., Lopes, P., Keck, J., Baudisch, P., Friston, K. and Blankenburg, F.
2020. Action-Dependent Processing of Touch in the Human Parietal Operculum
and Posterior Insula. Cerebral Cortex. 30, 2 (Mar. 2020), 607–617. DOI:https://doi.
org/10.1093/cercor/bhz111.
[46] Lopes, P. and Baudisch, P. 2017. Immense Power in a Tiny Package: Wearables
Based on Electrical Muscle Stimulation. IEEE Pervasive Computing. 16, 3 (2017),
12–16. DOI:https://doi.org/10.1109/MPRV.2017.2940953.
[47] Lopes, P. and Baudisch, P. 2013. Muscle-propelled Force Feedback: Bringing ForceFeedback to Mobile Devices. Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems (New York, NY, USA, 2013), 2577–2580.
[48] Lopes, P., Ion, A. and Baudisch, P. 2015. Impacto: Simulating Physical Impact by
Combining Tactile Stimulation with Electrical Muscle Stimulation. Proceedings ofthe 28th Annual ACM Symposium on User Interface Software & Technology (NewYork, NY, USA, 2015), 11–19.
[49] Lopes, P., Ion, A., Mueller, W., Hoffmann, D., Jonell, P. and Baudisch, P. 2015.
Proprioceptive Interaction. Proceedings of the 33rd Annual ACM Conference on
Human Factors in Computing Systems (New York, NY, USA, 2015), 939–948.
[50] Lopes, P., Jonell, P. and Baudisch, P. 2015. Affordance++: Allowing Objects to
Communicate Dynamic Use. Proceedings of the 33rd Annual ACM Conference on
Human Factors in Computing Systems (New York, NY, USA, 2015), 2515–2524.
[51] Lopes, P. and Tang, A. 2019. Rehamove Integration Lib.
[52] Lopes, P., You, S., Cheng, L.-P., Marwecki, S. and Baudisch, P. 2017. Providing
Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical
Muscle Stimulation. Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems (New York, NY, USA, 2017), 1471–1482.
[53] Lopes, P., You, S., Ion, A. and Baudisch, P. 2018. Adding Force Feedback to Mixed
Reality Experiences and Games Using Electrical Muscle Stimulation. Proceedingsof the 2018 CHI Conference on Human Factors in Computing Systems (New York,NY, USA, 2018), 446:1-446:13.
[54] Lopes, P., Yüksel, D., Guimbretière, F. and Baudisch, P. 2016. Muscle-plotter: An
Interactive System based on Electrical Muscle Stimulation that Produces Spatial
Output. (2016), 207–217.
[55] Lou, Z., Yao, P. and Zhang, D. 2012. Wireless Master-Slave FES Rehabilitation System Using sEMG Control. Intelligent Robotics and Applications (Berlin, Heidelberg,2012), 1–10.
[56] Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang,F., Chang, C.-L., Yong, M.G., Lee, J., Chang, W.-T., Hua, W., Georg, M. and Grundmann, M. 2019. MediaPipe: A Framework for Building Perception Pipelines.
arXiv:1906.08172 [cs]. (Jun. 2019).
[57] MA, Z. and Ben-Tzvi, P. 2015. RML Glove—An Exoskeleton Glove Mechanism
With Haptics Feedback. IEEE/ASME Transactions on Mechatronics. 20, 2 (Apr.
2015), 641–652. DOI:https://doi.org/10.1109/TMECH.2014.2305842.
[58] Madgwick, S.O.H. An efficient orientation filter for inertial and inertial/magnetic
sensor arrays. 32.
[59] Massie, T.H. and Salisbury, J.K. The PHANToM Haptic Interface: A Device for
Probing Virtual Objects. 7.
[60] Murayama, J., Bougrila, L., Luo, Y., Akahane, K., Hasegawa, S., Hirsbrunner, B.
and Sato, M. SPIDAR G&G: A Two-Handed Haptic Interface for Bimanual VR
Interaction. 9.
[61] Nagashima, Y. 2003. Bio-sensing systems and bio-feedback systems for interactive media arts. Proceedings of the 2003 conference on New interfaces for musicalexpression (SGP, May 2003), 48–53.
[62] Nishida, J., Kasahara, S. and Suzuki, K. 2017. Wired muscle: generating fasterkinesthetic reaction by inter-personally connecting muscles. ACM SIGGRAPH
2017 Emerging Technologies (New York, NY, USA, Jul. 2017), 1–2.

Romain Nith et al.

[63] Nishida, J. and Suzuki, K. 2017. bioSync: A Paired Wearable Device for Blending
Kinesthetic Experience. Proceedings of the 2017 CHI Conference on Human Factors
in Computing Systems (New York, NY, USA, May 2017), 3316–3327.
[64] Nishida, J., Takahashi, K. and Suzuki, K. 2015. A wearable stimulation device forsharing and augmenting kinesthetic feedback. Proceedings of the 6th Augmented
Human International Conference (New York, NY, USA, Mar. 2015), 211–212.
[65] Pfeiffer, M., Duente, T. and Rohs, M. 2016. A Wearable Force Feedback Toolkit
with Electrical Muscle Stimulation. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16 (San Jose,
California, USA, 2016), 3758–3761.
[66] Pfeiffer, M., Dünte, T., Schneegass, S., Alt, F. and Rohs, M. 2015. Cruise Control for
Pedestrians: Controlling Walking Direction Using Electrical Muscle Stimulation.
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing
Systems (New York, NY, USA, 2015), 2505–2514.
[67] Pfeiffer, M., Schneegaß, S. and Alt, F. 2013. Supporting interaction in public spacewith electrical muscle stimulation. Proceedings of the 2013 ACM conference on
Pervasive and ubiquitous computing adjunct publication (New York, NY, USA, Sep.
2013), 5–8.
[68] Pfeiffer, M., Schneegass, S., Alt, F. and Rohs, M. 2014. Let me grab this: a comparison of EMS and vibration for haptic feedback in free-hand interaction. Proceedingsof the 5th Augmented Human International Conference (New York, NY, USA, Mar.
2014), 1–8.
[69] Pfeiffer, M. and Stuerzlinger, W. 2015. 3D Virtual Hand Selection with EMS andVibration Feedback. Proceedings of the 33rd Annual ACM Conference Extended
Abstracts on Human Factors in Computing Systems (New York, NY, USA, Apr.
2015), 1361–1366.
[70] Pohl, H., Hornbæk, K. and Knibbe, J. 2018. Wanding Through Space: Interactive
Calibration for Electric Muscle Stimulation. Proceedings of the 9th Augmented
Human International Conference on - AH ’18 (Seoul, Republic of Korea, 2018), 1–5.
[71] Popović-Bijelić, A., Bijelić, G., Jorgovanović, N., Bojanić, D., Popović, M.B. andPopović, D.B. 2005. Multi-field surface electrode for selective electrical stimulation. Artificial Organs. 29, 6 (Jun. 2005), 448–452. DOI:https://doi.org/10.1111/j.
1525-1594.2005.29075.x.
[72] Qiu, S., He, F., Tang, J., Xu, J., Zhang, L., Zhao, X., Qi, H., Zhou, P., Cheng, X.,
Wan, B. and Ming, D. 2014. Intelligent algorithm tuning PID method of function
electrical stimulation using knee joint angle. 2014 36th Annual International
Conference of the IEEE Engineering in Medicine and Biology Society (Aug. 2014),
2561–2564.
[73] Rosen, J., Milutinović, D., Miller, L.M., Simkins, M., Kim, H. and Li, Z. 2014.
Unilateral and Bilateral Rehabilitation of the Upper Limb Following Stroke via an
Exoskeleton. Neuro-Robotics. P. Artemiadis, ed. Springer Netherlands. 405–446.
[74] Rouhani, H., Same, M., Masani, K., Li, Y.Q. and Popovic, M.R. 2017. PID Controller
Design for FES Applied to Ankle Muscles in Neuroprosthesis for Standing Balance.
Frontiers in Neuroscience. 11, (2017). DOI:https://doi.org/10.3389/fnins.2017.00347.
[75] Sandoval-Gonzalez, O., Jacinto-Villegas, J., Herrera-Aguilar, I., Portillo-Rodiguez,
O., Tripicchio, P., Hernandez-Ramos, M., Flores-Cuautle, A. and Avizzano, C.
2016. Design and Development of a Hand Exoskeleton Robot for Active andPassive Rehabilitation. International Journal of Advanced Robotic Systems. 13, 2(Mar. 2016), 66. DOI:https://doi.org/10.5772/62404.
[76] Sato, K., Minamizawa, K., Kawakami, N. and Tachi, S. 2007. Haptic Telexistence.
ACM SIGGRAPH 2007 Emerging Technologies (New York, NY, USA, 2007).
[77] Schneegass, S. and Rzayev, R. 2016. Embodied notifications: implicit notifications
through electrical muscle stimulation. Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct(New York, NY, USA, Sep. 2016), 954–959.
[78] Schneegass, S., Schmidt, A. and Pfeiffer, M. 2016. Creating user interfaces withelectrical muscle stimulation. Interactions. 24, 1 (Dec. 2016), 74–77. DOI:https://doi.org/10.1145/3019606.
[79] Shao, T., Li, X., Yokoi, H. and Zhang, D. 2016. FESleeve: A Functional Electrical Stimulation System with Multi-electrode Array for Finger Motion Control.
Intelligent Robotics and Applications (Cham, 2016), 191–199.
[80] Strojnik, P., Kralj, A. and Ursic, I. 1979. Programmed Six-Channel Electrical
Stimulator for Complex Stimulation of Leg Muscles During Walking. IEEE Transactions on Biomedical Engineering. BME-26, 2 (Feb. 1979), 112–116. DOI:https://doi.org/10.1109/TBME.1979.326520.
[81] Surakijboworn, M. and Wannasuphoprasit, W. 2015. Design of a novel finger
exoskeleton with a sliding six-bar joint mechanism. Proceedings of the 6th Augmented Human International Conference on - AH ’15 (Singapore, Singapore, 2015),
77–80.
[82] Takahashi, A., Brooks, J., Kajimoto, H. and Lopes, P. Increasing Electrical Muscle
Stimulation’s Dexterity by means of Back of the Hand Actuation. Proceedings ofthe 2021 CHI Conference on Human Factors in Computing Systems.
[83] Tamaki, E., Chan, T. and Iwasaki, K. 2016. UnlimitedHand: Input and Output Hand
Gestures with Less Calibration Time. Proceedings of the 29th Annual Symposium on
User Interface Software and Technology (New York, NY, USA, Oct. 2016), 163–165.
[84] Tamaki, E., Miyaki, T. and Rekimoto, J. 2011. PossessedHand: techniques forcontrolling human hands using electrical muscles stimuli. Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems (2011), 543–552.

DextrEMS: Increasing Dexterity in Electrical Muscle Stimulation by Combining it with Brakes[85] Tatsuno, S., Hayakawa, T. and Ishikawa, M. 2016. Comparison of reaction times
in response to electrical and Visual Stimulation using a high-speed camera. 2016
IEEE International Conference on Systems, Man, and Cybernetics (SMC) (Oct. 2016),
001251–001256.
[86] Tatsuno, S., Hayakawa, T. and Ishikawa, M. 2017. Supportive training system for
sports skill acquisition based on electrical stimulation. 2017 IEEE World Haptics
Conference (WHC) (Jun. 2017), 466–471.
[87] Tatsuno, S., Hayakawa, T. and Ishikawa, M. 2017. Trajectory adjustment system
for learning based on electrical stimulation. Proceedings of the 8th Augmented
Human International Conference (New York, NY, USA, Mar. 2017), 1–4.
[88] Watanabe, K., Oka, M. and Mori, H. 2019. Feedback Control to Target Joints
Angle in Middle Finger PIP and MP Joint Using Functional Electrical Stimulation.
Human Interface and the Management of Information. Information in Intelligent
Systems (Cham, 2019), 440–454.
[89] Watanabe, T., Iibuchi, K., Kurosawa, K. and Hoshimiya, N. 2003. A method ofmultichannel PID control of two-degree-of-freedom wrist joint movements by

UIST ’21, October 10–14, 2021, Virtual Event, USAfunctional electrical stimulation. Systems and Computers in Japan. 34, 5 (2003),
25–36. DOI:https://doi.org/10.1002/scj.10298.
[90] Widjaja, F., Shee, C.Y., Au, W.L., Poignet, P. and Ang, W.T. 2011. Using electromechanical delay for real-time anti-phase tremor attenuation system using
Functional Electrical Stimulation. 2011 IEEE International Conference on Robotics
and Automation (May 2011), 3694–3699.
[91] Yu-Luen Chen, Weoi-Luen Chen, Chin-Chih Hsiao, Te-Son Kuo, and Jin-Shin
Lai 2005. Development of the FES system with neural network + PID controller
for the stroke. 2005 IEEE International Symposium on Circuits and Systems (May2005), 5119-5121 Vol. 5.
[92] Zubrycki, I. and Granosik, G. 2017. Novel Haptic Device Using Jamming Principle
for Providing Kinaesthetic Feedback in Glove-Based Control Interface. Journal ofIntelligent & Robotic Systems. 85, 3 (Mar. 2017), 413–429. DOI:https://doi.org/10.
1007/s10846-016-0392-6.