FingerFlex: Shape Memory Alloy-based Actuation on Fingers forKinesthetic Haptic Feedback
Takuro Nakao

Kai Kunze

Megumi Isogai

Keio University
Kanagawa, Japana8111140@keio.jp

Keio University
Kanagawa, Japan
kai.kunze@gmail.com

NTT Media Intelligence Laboratory
Tokyo, Japan
megumi.isogai.ks@hco.ntt.co.jp

Shinya Shimizu

Yun Suen Pai

NTT Media Intelligence Laboratory
Tokyo, Japanshinya.shimizu.te@hco.ntt.co.jp

University of Auckland
Auckland, New Zealandyspai1412@gmail.com

Figure 1: FingerFlex. A: Top view in operation. B:Side view showing the finger before and after actuation. C: User typing a key
on a number pad (one of the application scenarios

ABSTRACTCCS CONCEPTS

The tactile and kinesthetic sensation of pushing a button is usually
lost when interacting with modern devices like touchscreens and/or
virtual reality platforms. We present FingerFlex, a standalone glove
wearable actuating the metacarpophalangeal joint (MCP) of eachfinger via shape memory alloy (SMA). SMA actuation is subtle,
silent, and light, making it ideal for actuation of the fingers which
we use to simulate the sensation of pressing a button.
For our first study, we evaluated the engineering performanceof FingerFlex by altering the current and triggering different levelsof stimuli to the user’s fingers. We show that users can perceive atleast 3 levels of actuation with an accuracy of 73%.
For our second study, we found FingerFlex to perform significantly better in terms of input error on a virtual numblock of a
keyboard with no significant change in perceived workload.

• Hardware → Emerging interfaces; • Computer systems organization → Embedded hardware.

KEYWORDS
Subtle haptics; shape memory alloy; kinesthetic feedback; finger
actuation
ACM Reference Format:
Takuro Nakao, Kai Kunze, Megumi Isogai, Shinya Shimizu, and Yun Suen
Pai. 2020. FingerFlex: Shape Memory Alloy-based Actuation on Fingers forKinesthetic Haptic Feedback. In 19th International Conference on Mobile andUbiquitous Multimedia (MUM 2020), November 22–25, 2020, Essen, Germany.
ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3428361.3428404

1

INTRODUCTION

Haptic devices are gaining popularity as an additional feedback
mechanism for augmented (AR), mixed (MR), and virtual reality(VR) applications. Haptics can be divided into tactile and kinesthetic
feedback [15, 18]. Tactile represents the pressure, vibration, andthermal feedback that we feel with our touch sensory receptors.
Kinesthetic feedback, on the other hand, refers to muscle and joint
actuation for our perception of weight, linked also to our proprioception [20] and cutaneous feedback [11]. In AR/MR/VR, bothtactile [8, 21] and kinesthetic feedback [9] have been explored assome form of hand-worn device or finger-mounted device. Furthermore, some research combines both tactile and kinesthetic feedback

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
MUM 2020, November 22–25, 2020, Essen, Germany© 2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-8870-2/20/11. . . $15.00
https://doi.org/10.1145/3428361.3428404

240

MUM 2020, November 22–25, 2020, Essen, Germany

Takuro Nakao, Kai Kunze, Megumi Isogai, Shinya Shimizu, and Yun Suen Pai

through haptic surrogates [1, 2]. These solutions are specially tailormade for virtual environment interactions. In this work, we present
FingerFlex, a wearable device that actuates each finger kinesthetically via shape memory alloy (SMA). SMA is a material that can provide tactile sensation via shrinking [3]. For our device, we instead
use it as a form of kinesthetic feedback for joint actuation. SMA
actuation provides subtle pulls to the joint to imitate the feedbackof pressing a button. We use the pulling force from the shrinkingof the SMA springs mounted on custom 3D-printed channels toactuate the finger about the metacarpophalangeal (MCP) joint. For
virtual applications, we also match the kinesthetic feedback withthe visuals of the button actuation to provide a more realistic experience. One issue with SMA is the slow actuation speed (especiallyafter continuous activation), we reduce this problem by using anovel SMA spring and applying active cooling. Still the system isonly useful for sporadic activation (every 1-3 seconds) not for tasks
like touch typing.
This paper’s contributions are the following: (1) We present FingerFlex a wearable haptic device that can actuate finger MCP joints
using SMA springs. It is subtle, silent, and light and can simulate
the kinesthetics of a single button press. The system schematics
and design will be open-sourced 1 . (2) We performed a user study
to evaluate user’s perception of the actuation. We found that users
can distinguish 3 levels of actuation with 73% accuracy. (3) We
evaluate the performance of FingerFlex in a VR environment. Users
make significantly fewer errors during input on a number pad with
FingerFlex than without it.

2

RELATED WORK

Haptic wearable devices have been explored using many different
kinds of approaches, either as a wearable [8], or attached to aphysical device for additional tactile feedback [14]. In the following,
we focus more on wearable approaches, as well as an in-depth lookat works using SMA specifically.
Most proposed haptic devices are quite large/heavy. Son et al.
[17] developed a haptic glove providing tactile and kinesthetic
feedback to the palm and fingers using a servo motor for each finger.
The sheer size and weight of it make the device unsuitable for use
cases outside of a controlled environment. The closest related work
is by Lopes et al. [10] using EMS for haptic sensation for walls
and heavy objects. EMS allows joint actuation of the arm, which is
why the author focuses on larger-scale feedback as opposed to ourimplementation that is more fine-tuned for finger haptics.
To our knowledge, there is very little related work regarding
the use of SMA for button press haptics at this point of writing.
However, SMA is a common material used for soft robotics [16]because of its non-linear actuation that is closer to actual organic
movement as opposed to rigid motors and mechanical joints.
Haptic-based research that looks at the usage of SMA can beseen by Suhonen et al. [19], Gupta et al. [5] and Chernyshov et al.
[3] who mainly explored its use for squeezing feedback and tactile
sensation on the wrist and finger. A more recent related work,
Springlets [6], uses SMA springs as an on-skin tactile interface tobe used in various scenarios because it is thin, flexible and silent.
PhantomTouch [12] and Touch Me Gently [13] used SMA to instead

simulate the sensation of touch via shearing forces. These works
focus on tactile sensation as opposed to kinesthetic feedback.

3

FINGERFLEX PROTOTYPE

FingerFlex uses 3D printed parts as well as Velcro for ease of attachment to the hand. The housing for the prototype is 3D printed.
The finger joints are fixed to the FingerFlex prototype using also
small Velcro rings connected to the SMA springs over strings. The
velcro and the length of the strings leading to the SMA spring canbe easily adjusted for each person (accommodating fro anthropometric differences in user hands see 1). The SMA actuation provides
a resistive kinesthetic actuation, with a gradual increase in pulling
force as the finger approaches the contact point for a haptic sensation. We implemented a single button press for our evaluation, yetdifferent and combined actuations are also possible. Instead of an
SMA wire, our initial testing shows that a spring-shaped SMA isable to generate a larger, faster pull with a higher recovery force.
Four SMA springs2 (excluding the thumb) are attached over strings
to each finger to pull them backwards. We use tensile springs with amean diameter of 5mm, a wire diameter of 0.8mm, and a maximum
compression force of up to 3N. The typical force is a maximum of30(N). The SMA springs work from an approximate temperature of
45degree. We need to limit activation time to maximum 7 seconds(crucial to not increase the temperature over 70degree). The springchanges its contraction speed depending on the temperature, it
exhibits maximum contraction within approximately 1.5 sec. For
a button-press illusion the feeling is pretty instantaneous (under
300 ms). Yet, it takes about 500 ms to cool the spring down again
for another activation. This time gets longer the more often thesystem is used and is a limitation of FingerFlex. The prototype will
not work for touch typing or other applications that require fast
feedback.

Figure 2: The compressed SMA spring when heated (on theleft), the SMA spring at room temperature
The outer frame is designed according to the shape of the handand shields the hand from the thermal properties of the SMA springs(heating up to 40-50degree for short periods of time (max 3 sec.).
Longer tests didn’t show an increase in temperature on the bottomof the 3D printed frame. The SMA springs are placed in a frame
designed at 6.5cm so that it operates within the range of 1 to 6cm
when it is contracted. We designed a custom printed circuit board(PCB). The prototype is only for the right hand, with the Velcro ringconnections extending from each of the SMA springs. Software 2 https://www.saesgetters.com/products-functions/products/shape-memory-alloys-

1 url blinded for review

nitinol241

FingerFlex: Shape Memory Alloy-based Actuation on Fingers for Kinesthetic Haptic Feedback

4

Figure 3: Glove Module with mounted springs and coolingfans(left)PCB board(Right)
The software uses the Unity Engine for the VR visualization and
PCB serial connection. The system is connected to a desktop PC
and controls the SMA springs via Unity and Arduino. The tensionof the SMA spring depends on the time and amount of the current. For FingerFlex, the tactile sensation is activated in the rangeof 0.8-1.6N. The sense of force is presented only max up to 7 seconds, as prolonged activation may break the SMA springs due tooverheating.

MUM 2020, November 22–25, 2020, Essen, Germany

STUDY 1: FINGERFLEX PERFORMANCEThis study is to investigate the user’s ability to differentiate andrecognize between several levels of stimuli from the SMA. Before
initiating the main study, we first conducted a pilot study to determine the minimum threshold where a user can feel the actuationfrom the SMA for each finger. We recruited 5 participants (3 male)between the age of 20 to 30 (mean: 24.3, std = 0). We then gradually
increase the amount of current that flows into the device. We foundthat users start to perceive the actuation at (2.2V,2.43A), with anactuation force of 0.8N. Therefore for our user study, we begin
actuation with 0.8N. We recruited 16 participants (12 males and4 Females) between the ages of 23 to 28 (mean: 24.8). They werealso asked to sign a consent form stating that at anytime, they may
withdraw from the experiment if they wish to do so. We performed
the experiment using a single glove that actuates the 4 fingers. A
desktop computer was used to run the Unity which instead wasused to control the prototype.
Experimental Design and Procedure- The level of each stimuli was designed to be maintained within the range of comfort andnot too exerting, which should range between the result of thepilot study of 0.3N to 4.8N [7]. For 3 granularity levels, the applied
stimulus are 0.8N, 1.2N and 1.6N. We apply the same stimuli for all
the fingers at the same time. Each participant was first informed
about the goal and nature of the study (also stated in the consent
form). They were then outfitted with the glove. The procedure isdivided based on the previously assigned granularity level, with a
training and testing phase. The participant experiences the 3 levelsof stimuli in an increasing order. Then, they enter the testing phase,where a random stimuli from the corresponding granularity level
was activated. The participant needs to select which of the stimuli
level they think they experienced. This is repeated 5 times, where
the stimulus was randomly activated (for three levels of granularity, a total of 5 x 3 = 15 trials in random order was tested on theparticipant).

Figure 4: Rotation Force Moment Diagram
Kinesthetic Joint Actuation- The SMA springs are mounted
at the back of the gloves to pull each finger individually via a stringthat is attached to the proximal at theta = 45 degree. This causes therotational point to be at about the MCP joint, with the generated
torque being equivalent to Fcos theta*d. The distance d refers to thelength of the proximal bone which is different for each finger aswell as between gender, ethnicity and various factors [4]. Our initial
measurements range between 0.045m to 0.056m. With the pulling
force F between 0.8N and 1.6N, the generated torque is equivalent
to a range between 0.025Nm and 0.063Nm. We illustrate this in the
moment diagram shown in Figure 4. A possible drawback regarding
SMA-based actuation is that the response can potentially be slower
after several actuations due to the high residual temperature. To
combat this, we introduce cooling fans for our prototype.
The airflow from the fans are piped through 3D printed lanes
which also houses the SMA springs. We use the BFB03505HHA-A
cooling fans manufactured by Delta Electronics3 with a maximum
speed of 8500 RPM and a power rating of 750mW. Future prototypes
can be improved using water/liquid cooling.

Figure 5: Study 1 Result, Confusion Matrix
Results and Discussion- Figure 5 shows the overall accuracy
for the perception of force feedback by our device. Participants
were able to discriminate between 3 different strength levels withan average 73 % accuracy. As seen from the confusion matrix, errors
happen just between adjacent levels. The applied force of 1.2N
(Level 2) seem to elicit the lowest accuracy when compared to 0.8N
(Level 1) and 1.6N (Level 3). It can also be observed that for Level 1,
21.3% perceived it to be level 2, whereas for Level 3, 20% perceived
it to be level 2. This is understandable because the lowest induced

3 https://www.mouser.com/manufacturer/delta-electronics/

242

MUM 2020, November 22–25, 2020, Essen, Germany

Takuro Nakao, Kai Kunze, Megumi Isogai, Shinya Shimizu, and Yun Suen Pai

force of 0.8N is the minimum perceivable force during our pilot
study and is easily distinguishable from higher forces, whereas
the highest induced force of 1.6N actuates the finger the furthest,
allowing one to easily perceive it. For Level 2, more participants
perceived it to be Level 1 (22.5%) compared to Level 3 (17.5%), though
not by a large margin. We believe this is attributed to Level 3
being more perceivable than Level 1, which can also be seen on its
achieved accuracy (80% versus 78.8%). As expected, there were no
participants who perceived Level 1 as Level 3, and vice versa.

5

STUDY 2: BUTTON PRESS FEEDBACK

For our second study, we evaluated the virtual button press performance as well as perceived workload when using FingerFlex. We
chose keypad input because, it is one of the most common interface that requires buttons and exists as a physical device, on touch
screen devices, and even AR/MR/VR environments. We recruited

Figure 7: Study Result: The Score of Input Error

Figure 8: Nasa TLX plot

Figure 6: The study setup : Evaluating SMA springs with Input test In Virtual Reality Environment
the same 16 participants from the previous study to participate inthis study. Information regarding the second study was alreadyincluded in the consent form that they previously signed.A desktop
computer was used to run the Unity game engine which instead
was used to control the prototype. For finger tracking, we use aLeap Motion sensor.
This experiment is also a within-subject study to obtain feedback
regarding input performance and perceived workload when using
FingerFlex. We chose to use a VR environment for the input task
as we can easily manipulate the complete visual experience of theparticipants (see Figure 8). We used a simplified virtual number
pad to allow single handed use. The first scenario is simply keypad
input on the virtual number pad with 2 patterns. The key havebeen activated 2 or 4 seconds for each scenario. And then totally
1 minute (2 seconds*30 times) and 2 minutes (4 seconds*30 times)
using our predefined number as the baseline. The seconds scenario
then requires the user to wear our prototype and repeat the sametask using a progressively increasing haptic sensation the more
the finger presses into the button. For all scenarios, each key lights
up during activation and deepening of the key during actuation isreflected on the virtual number pad. The order of the scenario wascounterbalanced to negate ordering effects.
At the end of the study, each user verbally reports their experience openly, as well as answering the provided immersion
questionnaire.

243

Results and Discussion. As shown in Figure 7, the mean Input
Error for the 4 seconds no-stimulation condition is 30.95 (SD:6.923)
without haptic feedback and 26.06 (SD: 6.628) with haptic feedback; For the two second condition 25.63(SD:8.180) without and22.63(SD:7.641)with the haptic feedback. Applying a t-test shows
significant differences within the activation time of 4 seconds.(t =3.917, d f = 15, p < .05). No significant difference was confirmedof the activation 2 seconds (t = 1.695, d f = 15, p > .05). Many
users reported confusion in the use case without haptic feedback,as they were not sure if the button was pressed or not (leading tomissed presses or multiple presses). Tactile stimulus feedback isprovided to the user at a fixed value of 1.2N. The Nasa TLX does
not show a significant difference in perceived workload index. The
users perform better in the 4 sec. activation interval, yet don’t feel
as if they do.

6

CONCLUSION AND FUTURE WORK

We present FingerFlex, a haptic wearable glove that can actuate
the MCP joint and provide kinesthetic feedback. We evaluated thesystem in a sensitivity study and showed that it can be used for
haptic feedback in a number input task. Users with FingerFlex makesignificant less errors during input.

FingerFlex: Shape Memory Alloy-based Actuation on Fingers for Kinesthetic Haptic Feedback

REFERENCES[1] Lung-Pan Cheng, Patrick Lühne, Pedro Lopes, Christoph Sterz, and Patrick
Baudisch. 2014. Haptic Turk: A Motion Platform Based on People. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(Toronto, Ontario, Canada) (CHI ’14). ACM, New York, NY, USA, 3463–3472.
https://doi.org/10.1145/2556288.2557101[2] Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Mutual
Human Actuation. In Proceedings of the 30th Annual ACM Symposium on User
Interface Software and Technology (Qu&#233;bec City, QC, Canada) (UIST ’17).
ACM, New York, NY, USA, 797–805. https://doi.org/10.1145/3126594.3126667[3] George Chernyshov, Benjamin Tag, Cedric Caremel, Feier Cao, Gemma Liu, andKai Kunze. 2018. Shape Memory Alloy Wire Actuators for Soft, Wearable Haptic
Devices. In Proceedings of the 2018 ACM International Symposium on Wearable
Computers (Singapore, Singapore) (ISWC ’18). ACM, New York, NY, USA, 112–119.
https://doi.org/10.1145/3267242.3267257[4] BT Davies, A Abada, K Benson, A Courtney, and I Minto. 1980. A comparison ofhand anthropometry of females in three ethnic groups. Ergonomics 23, 2 (1980),
179–182.
[5] Aakar Gupta, Antony Albert Raj Irudayaraj, and Ravin Balakrishnan. 2017. HapticClench: Investigating Squeeze Sensations Using Memory Alloys. In Proceedingsof the 30th Annual ACM Symposium on User Interface Software and Technology(Qu&#233;bec City, QC, Canada) (UIST ’17). ACM, New York, NY, USA, 109–117.
https://doi.org/10.1145/3126594.3126598[6] Nur Al-huda Hamdan, Adrian Wagner, Simon Voelker, Jürgen Steimle, and Jan
Borchers. 2019. Springlets: Expressive, Flexible and Silent On-Skin Tactile Interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing
Systems (Glasgow, Scotland Uk) (CHI ’19). ACM, New York, NY, USA, Article 488,
14 pages. https://doi.org/10.1145/3290605.3300718[7] Teng Han, Fraser Anderson, Pourang Irani, and Tovi Grossman. 2018. HydroRing: Supporting Mixed Reality Haptics Using Liquid Flow. In Proceedingsof the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). ACM, New York, NY, USA, 913–925. https://doi.org/10.1145/3242587.3242667[8] Ronan Hinchet, Velko Vechev, Herbert Shea, and Otmar Hilliges. 2018. DextrES:
Wearable Haptic Feedback for Grasping in VR via a Thin Form-Factor Electrostatic
Brake. In Proceedings of the 31st Annual ACM Symposium on User Interface Software
and Technology (Berlin, Germany) (UIST ’18). ACM, New York, NY, USA, 901–912.
https://doi.org/10.1145/3242587.3242657[9] Pedro Lopes, Alexandra Ion, and Patrick Baudisch. 2015. Impacto: Simulating
Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. In Proceedings of the 28th Annual ACM Symposium on User Interface
Software &#38; Technology (Charlotte, NC, USA) (UIST ’15). ACM, New York, NY,
USA, 11–19. https://doi.org/10.1145/2807442.2807443[10] Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick
Baudisch. 2017. Providing Haptics to Walls &#38; Heavy Objects in Virtual244

MUM 2020, November 22–25, 2020, Essen, Germany

Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI
Conference on Human Factors in Computing Systems (Denver, Colorado, USA)(CHI ’17). ACM, New York, NY, USA, 1471–1482. https://doi.org/10.1145/3025453.
3025600[11] Christine L MacKenzie and Thea Iberall. 1994. The grasping hand. Vol. 104.
Elsevier.
[12] Sachith Muthukumarana, Don Samitha Elvitigala, Juan Pablo Forero Cortes,
Denys JC Matthies, and Suranga Nanayakkara. 2019. PhantomTouch: Creating
an Extended Reality by the Illusion of Touch using a Shape-Memory Alloy Matrix.
In SIGGRAPH Asia 2019 XR. 29–30.
[13] Sachith Muthukumarana, Don Samitha Elvitigala, Juan Pablo Forero Cortes,
Denys JC Matthies, and Suranga Nanayakkara. 2020. Touch me Gently: Recreating
the Perception of Touch using a Shape-Memory Alloy Matrix. In Proceedings ofthe 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[14] Masa Ogata. 2018. Magneto-Haptics: Embedding Magnetic Force Feedback forPhysical Interactions. In Proceedings of the 31st Annual ACM Symposium on User
Interface Software and Technology (Berlin, Germany) (UIST ’18). ACM, New York,NY, USA, 737–743. https://doi.org/10.1145/3242587.3242615[15] Claudio Pacchierotti, Stephen Sinclair, Massimiliano Solazzi, Antonio Frisoli,
Vincent Hayward, and Domenico Prattichizzo. 2017. Wearable haptic systems forthe fingertip and the hand: taxonomy, review, and perspectives. IEEE transactions
on haptics 10, 4 (2017), 580–600.
[16] Daniela Rus and Michael T Tolley. 2015. Design, fabrication and control of soft
robots. Nature 521, 7553 (2015), 467.
[17] Bukun Son and Jaeyoung Park. 2018. Haptic Feedback to the Palm and Fingers for
Improved Tactile Perception of Large Objects. In Proceedings of the 31st Annual
ACM Symposium on User Interface Software and Technology (Berlin, Germany)(UIST ’18). ACM, New York, NY, USA, 757–763. https://doi.org/10.1145/3242587.
3242656[18] Lazar Stankov, Tatjana Seizova-Cajić, and Richard D Roberts. 2001. Tactile
and kinesthetic perceptual processes within the taxonomy of human cognitive
abilities. Intelligence 29, 1 (2001), 1–29.
[19] Katja Suhonen, Kaisa Väänänen-Vainio-Mattila, and Kalle Mäkelä. 2012. User
Experiences and Expectations of Vibrotactile, Thermal and Squeeze Feedbackin Interpersonal Communication. In Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers (Birmingham, United
Kingdom) (BCS-HCI ’12). British Computer Society, Swinton, UK, UK, 205–214.
http://dl.acm.org/citation.cfm?id=2377916.2377939[20] G Wyszecki, KR Boff, L Kaufman, and JR Thomas. 1986. Handbook of perception
and human performance. (1986).
[21] Jackie (Junrui) Yang, Hiroshi Horii, Alexander Thayer, and Rafael Ballagas. 2018.
VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools. In
Proceedings of the 31st Annual ACM Symposium on User Interface Software andTechnology (Berlin, Germany) (UIST ’18). ACM, New York, NY, USA, 889–899.
https://doi.org/10.1145/3242587.3242643