SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 10(3); December (2023), pp. 247–252
ISSN (Online): 2352-6386; ISSN (Print): 2405-9021
https://alife-robotics.org/jrnal.html

Research Article

Myoelectric Prosthetic Hand with Sensory Feedback that
Generates Discomfort
Taro Shibanoki1, Shunji Hashiguchi1, Kosuke Jin2

Graduate School of Natural Science and Technology, Okayama University, 3-1-1 Tsushima-naka, Kita-ku, Okayama, Okayama 700-8530, Japan
Graduate School of Science and Engineering, Ibaraki University, 1-4-1 Nakanarusawa-cho, Hitachi, Ibaraki 316-8527, Japan

1
2

ARTICLE INFO
Article History
Received 25 November 2021
Accepted 20 May 2024
Keywords
Electromyogram
Prosthetic hand
Pain sensation
Discomfort feedback

ABSTRACTThe aim of this study is to develop a new tactile feedback method for myoelectric prosthetic hands.
The sense of touch can be reproduced such as by vibrating actuators based on information from asensor attached to the fingertip of the myoelectric prosthetic hand. This paper attempted to use
delayed vibration feedback to create discomfort as a corresponding sensation to pain since a priorstudy has found that a time lag between sensory input and stimulation can cause discomfort. In
the experiment carried out, we first examine the relationship between the delay time D andsensation in vibration stimulation. The experimental results indicated that discomfort occurs
between D = 0.25 seconds and 0.65 seconds. Furthermore, to verify whether the proposed
feedback method can be used to discriminate between normal objects and dangerous objects suchas knives, we applied it to the haptic feedback of a prosthetic hand controlled by myoelectric
signals. The results showed that the participant could recognize the difference in feedback anddistinguish between the two types of objects with an accuracy of 95.0 ± 0.1%.
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/).

complex motion discrimination and continuous motion
recognition, and in terms of intuitive control using EMG
signals that can be measured from the human as a control
input, it can be said that a prosthetic hand that closely
resembles an actual human hand is being realized.
Although there are various studies on structures andcontrol methods of myoelectric prosthetic hands, tocreate a truly human prosthetic hand, it is necessary not
only to realize control that mimics human hand
movement, but also to equip the prosthetic hand with ameans of obtaining sensory information such as thetactile sensation that is inherent in the human arm. Oneof the reasons why there are so few myoelectric
prosthetic hand users, despite the development of various
advanced prosthetic hands, is that there is little sensory
feedback to the user.
To solve this problem, research has been conducted tofeed back the information obtained when the prosthetic

1. Introduction
Myoelectric prosthetic hands are capable of simulating
human
hand
movementbydiscriminating
electromyogram (EMG) signals, such as intended
movements and force information, therefore, they cancompensate for the lost functions of upper limb amputees.
Yang and Liu proposed deep learning based multi-DOF
wrist movement discrimination and achieved accurate
and complex control of various robotic platforms [1]. In
order to create a myoelectric prosthetic hand that
resembles a human arm, it is necessary to consider theconstruction of a multi-degree-of-freedom robot handand its control method, and several studies on patternclassification to estimate motions from myoelectric
signals. Various methods have been proposed to realize
movements that more closely resemble human hand
movements [1], [2], [3], [4], [5], [6]. These include

Corresponding author E-mail: shibanoki@okayama-u.ac.jp, hashiguchi_shunji@s.okayama-u.ac.jp, 19nm718s@vc.ibaraki.ac.jp
URL: https://www.okayama-u.ac.jp/user/mechatro/247

hand comes in contact with an object to the operator. For
sensory feedback, it is important to develop sensors andcontrol methods to imitate the human sense of touch [7],[8], [9], [10], [11], [12], and to reproduce the sense of
touch using various actuators [13], [14], [15], [16], [17].
For example, BioTac tactile sensor [7] can detect contact
force, micro-vibration and thermal fluxes. In this paper,
how to reproduce the sensation to a human is mainly
discussed.
Feedback for myoelectric prosthetic hands can bedivided into invasive and non-invasive methods, among
which it is important to know what kind of stimuli to use
and how to use them. Various types of stimuli such as
vibration [13], [14], temperature, pressure [15], [16] andelectrical stimulation [17] have been studied as possible
feedback. On the other hand, there is research that uses
electrical stimulation to present the pain of a localized
strong stimulus, and research that combines image
recognition to present the texture of an object, andphantom limb stimulation [18]. Pain, for example, may
be useful for presenting contact with objects that should
not be touched with a prosthetic hand, such as hot or
sharp objects, but it is usually difficult to provide direct
feedback of pain to the user. To present pain sensation,
we have investigated the relationship between the timingof vibration stimuli and human sensation based on theeffects of stimulus latency on human sensation [19]. [20].
The previous study revealed that the time-delay of
approximately 0.4 s from the timing of contact with anobject could cause discomfort to the user [21]. However,
we have not introduced the previously proposed feedback
method to the myoelectric prosthetic hand.
This paper proposes a myoelectric prosthetic hand withdiscomfort feedback. The proposed prosthetic hand aims
to create a sensation of discomfort that corresponds topain, and simulates such sensation through delayed
vibration stimulation when touching an object with theprosthetic hand.
The rest of the paper is organized as follows: Section 2
describes the concept of the proposed prosthetic handwith a sensory feedback system, and Sections 3 and 4
show experiments using the proposed system. Section 5
finally concludes the paper.

EMG sensor

P rosthetic handVibration motor
Microcontroller
and control circuits
Force sensor

Fig. 1. Overview of the myoelectric prosthetic hand
feedback system. The proposed prosthetic hand canbe controlled by myoelectric signals, and tactile
information can be presented by controlling avibration motor using information obtained from asensor attached to the fingertip.
corresponding to pain. The proposed prosthetic handconsists of a sensor for measuring myoelectric signals, a
microcontroller for estimating the user's intention fromthe measured signals and controlling each motor of the
myoelectric prosthetic hand, and a force sensor. The
detailed information about the proposed prosthetic handis described in the following subsections.
2.1. Structure
The structure of the proposed prosthetic hand is basedon OpenHand (see Fig. 1). A geared DC motor with an
encoder is used to control each finger, and a servo motor
is also used to control the base joint of the thumb. The
prosthetic hand control system consists of a Raspberry Pi
Zero WH (Raspberry Pi) with a high precision ad/daboard (onboard ADS1256 and DAC8552) and DRV8835
motor drivers. A membrane force sensor (diameter: 10.2mm, ALPHA) is also attached to the fingertip of theprosthetic hand, and the voltage value corresponding to
the magnitude of the force is measured by the
microcontroller via the AD board. Additionally, it isequipped with a small eccentric motor, and tactile
feedback is possible by controlling the magnitude andtiming of vibrations via the motor driver.
Thus, the proposed prosthetic hand proposed hand canacquire contact information from the external
environment, and can be controlled like the hand itself bycontrolling the motors of each finger. Next, the control
method of the proposed prosthetic hand is explained.

2. EMG-based prosthetic hand with discomfort
feedback
Fig. 1 shows the proposed myoelectric prosthetic handwith a sensory feedback system that simulates sensations248

2.2. EMG-based control

feedback vibration is changed depending on whether or
not the 𝐶𝐶(𝑡𝑡) exceeds a preset threshold.
The time taken for the sensory stimuli to be recognized
as contacting an object is approximately 50 to 300 [ms].
In a music game, the timing of the button operation andits feedback was varied, and the results were evaluatedby questionnaire, and the larger the deviation from theoperator's expected timing, the greater the discomfort.
Therefore, when the force sensor value exceeds theobject contact judgment threshold 𝐶𝐶th and the thresholdvalue 𝐶𝐶th during the contact judgment time S, thevibration stimulus is given after the delay time 𝐷𝐷 [s] fromthe end of the contact judgment time 𝑆𝑆 in this paper.

Although various sensors are available for acquiring
EMG signals, a Myo Gesture Arm Band (Thalmic Labs.)
is used in this paper. Eight pairs of stainless electrodes (L= 8) are attached to the forearm, and measured signals areimported to the microcontroller via Bluetooth LE
communication (sampling frequency: 𝑓𝑓𝑠𝑠 =200 Hz). The
signals are then full-wave rectified, smoothed using asecond-order Butterworth low pass filter (𝑓𝑓𝑐𝑐 =1 Hz), andnormalized as the maximum value of each channel is 1
using the maximum value for each channel and theaverage value for each channel at rest obtained inadvance. After that, the normalized value with the sum of
all channels as 1 is defined as a feature vector 𝒙𝒙(𝑡𝑡) ∈ ℜ𝐿𝐿
for classification, and the average value of all channels isdefined as force information 𝐹𝐹(𝑡𝑡) for determining
motion occurrence.
A multi-layered perceptron with the terminal learning isused to discriminate the user’s intended motions. Feature(𝑚𝑚)
vectors ⬚𝒙𝒙(𝑡𝑡)(𝑛𝑛) (𝑚𝑚 = 1, … , 𝑀𝑀; 𝑛𝑛 = 1, … 𝑁𝑁𝑚𝑚 ; M is thenumber of motions and 𝑁𝑁𝑚𝑚 is the number of samples formotion m) are used to train the network, and when newfeature vector 𝒙𝒙(𝑡𝑡) is input to the trained network, thenetwork outputs posterior probabilities of each motion.
The motion with the highest probability value is judged
as the motion performed by the user if the force
information 𝐹𝐹(𝑡𝑡) is greater than the pre-set value
𝐹𝐹th (𝐹𝐹(𝑡𝑡) ≥ 𝐹𝐹th ). Otherwise, motion discrimination is not
performed (𝐹𝐹(𝑡𝑡) < 𝐹𝐹th ).
According to the motion discrimination result, themotor of each finger is selected and controlled via PWM
modulation. The maximum equilibrium angle Θ𝑖𝑖 (𝑖𝑖 =1, … 𝐼𝐼; I is the number of joints for control) is determinedin advance, and the target angle at each time is calculated
by solving the equation of motion according to the force
information. Then, by determining the amount of traction
on the wire for each finger and adjusting the rotation
speed of the motor, each finger can be controlled
voluntarily using EMG signals.

3. Verification of the effect of vibration feedback
delay time on tactile sensation
3.1. Method

In order to verify the optimal delay time 𝐷𝐷 for
expressing pain as discomfort using the proposed method,an experiment was conducted on a healthy university
student whether the operator feels discomfort against theunexpected stimulus. In the experiment, the participant
initiated feedback at his timing, and after a random delayof 𝐷𝐷 = [0.1, 0.8] s (16 total in 0.05 s increments), a 2.0 s
vibration stimulus was generated. In this experiment, theparticipant was asked to evaluate how much discomfort
he felt by a questionnaire with a 10-point scale. The
experiment consisted of 80 trials (each delay time waspresented randomly five times), with one trial consistingof the process of the participant pressing a button to
generate a delay time feedback and answering aquestionnaire.
3.2. Results and discussion
Fig. 2 shows the results of the experiment. The verticalaxis is the evaluation value of the questionnaire, and thehorizontal axis is the delay time 𝐷𝐷 . From the graph,
although there are variations from each trial, it can beseen that the results change in a quadratic manner. Whenthe delay time is small, it is felt as normal vibration
feedback, and when the delay time is large, it is felt as avibration stimulus with a different timing than thestrength of the discomfort. However, when the delay time
is around 0.25-0.65 s, the vibration stimulation starts witha slight delay from the timing of contact, so the

2.3. Sensory feedback
Here, when performing a grip or pinch motion andcoming into contact with an object, the magnitude offorce applied to a fingertip can be obtained as a voltage
value 𝐶𝐶(𝑡𝑡) via a force sensor. In this paper, we assume
that the pain when coming into contact with a sharp
object is fed back as a feeling of discomfort, and the249Trial1

Trial2

Trial3

Trial4

Trial5

10

AnswerP articipant

EMG sensor

PC

Vibration motor

00Time delay [s]0.8Force sensor
P rosthetic handFig. 2. Relationships between delay time of vibrationand tactile sensation.

Fig. 3. Experimental scene. The fingertips of a
myoelectric prosthetic hand can be opened and closedby discriminating EMG signals measured from theparticipant’s skin surface. The system detects contact
between the prosthetic fingertip and an object andiddib i f db k
200

participant feels a sense of lag, similar to the time lag
between video and audio in a video conference.
From these results, the optimal delay time forexpressing discomfort by vibrotactile feedback can be
estimated at around 0.4 s.
4. Object discrimination experiment
EMG signals

4.1. Method

Feed- Force Discrim- Forceback sensor ination inforstate value results mation[V]

In order to verify the effectiveness of the proposed
tactile feedback method, object discrimination
experiments were performed using the developed EMGbased prosthetic hand with tactile feedback.
The number of as intended timing during the experimen
discrimination motions was set as M = 4 (opening,grasping, wrist flexion, and extension) and EMG signals
were measured from L = 8 pairs of electrodes. 𝐹𝐹th is set
as 𝐹𝐹th = 0.2 and the participant was asked to perform
opening/grasping motion t.
Fig. 3 shows the experimental scene. The participant
performs a grasping motion using the prosthetic hand
while the prosthetic hand is not visible. In the experiment,
contact with the object was detected by a force sensor
attached to the fingertip (𝐶𝐶th = 2.0), feedback with orwithout a delay of 0.4 s (determined from the previous
experiment) was randomly presented 20 times. The
participant was asked to answer whether the object was
intended to be a normal object (non-delayed feedback) ora sharp object such as a knife (discomfort presentation
with delayed feedback).

1.5
Fth

0.0
M4
M3
M2
M1
M0

2.5
2.0
1.5
1
0
0.0

Time [s]10.0

Fig. 4. An example of experimental results (trial 5).
After contact with an object is detected by a sensor
attached to the fingertip of the prosthetic hand,delayed vibration feedback is provided.

250

4.2. Results and discussion

such as the texture of an object, and participants were
only able to recognize the gap in vibration onset time.
While the authors do not consider that it is necessary to
directly feedback sensations corresponding to pain, it isnecessary to consider how information including the
texture of objects can be given to users to make theprosthetic hand more similar to a human hand.
In future research, we would like to increase the numberof subjects and conduct experiments, as well as propose
a new feedback method for myoelectric prosthetic handsthat utilizes illusions.

Fig. 4 shows an example of experimental results,
representing EMG signals, force information calculatedusing EMG signals, discrimination results, force sensor
value, and vibrotactile feedback information, from thetop. From the result, the participant performs an opening
motion to open the prosthetic hand, and then performs agrasping motion to try to grasp an object. In this trial, byusing delayed feedback assuming a sharp object, notethat there is a delay in the start of vibration after contact
with the object due to discomfort feedback. In this way,
the prosthetic hand can be controlled by EMG signals,
and information can be fed back when it touches anobject. The object discrimination rate was 95.0 ± 0.1 %.
The participant was able to recognize the differences
between objects through vibration feedback, and the
proposed prosthetic hand was able to accurately obtain
this information in situations where the prosthetic hand
was supposed to be in danger, such as touching a sharp
object.
5.

Acknowledgments
This work was supported by JSPS KAKENHI Grant
Numbers: 17K12723 and 20K20212.
References
1.

2.

Conclusion

In this paper, we proposed a myoelectric prosthetic handwith a sensory feedback system that can provide
discomfort to the user based on the time delay of tactile
stimulation. The proposed prosthetic hand measures
EMG signals from electrodes attached to the forearm andenables voluntary control by using the differences insignal characteristics for each motion to discriminate
them. Additionally, the proposed system can detect
contact with an object by a tactile sensor and then apply
a vibration stimulus at a timing according to themagnitude of the contact. In the experiments performed,
we first evaluated the time delay between vibration
feedback after contact determination and feedback. The
start time of the feedback by vibration stimulation wasvaried in 16 steps in 0.05-second increments between 0.1
and 0.8 seconds and presented randomly. The time delaythat caused the most discomfort to the participants was
0.4 seconds. Furthermore, we also conducted prosthetic
hand control and object discrimination experiments using
the developed EMG-based prosthetic hand withdiscomfort feedback function. Experimental results
showed that vibrotactile feedback with and without timedelay was presented randomly and the participant wasable to discriminate between two types of objects with anaccuracy of 95.0 ± 0.1%. However, although this paper
could evaluate the effect on sensation due to the temporalgap between contact and vibrotactile stimulation, thesystem was not able to accurately provide the sensory
information to the user possessed by the human fingertips,

3.
4.

5.
6.

7.

8.

9.

251

D. Yang and H. Liu. An EMG-Based Deep Learning
Approach for Multi-DOF Wrist Movement Decoding,IEEE Transactions on Industrial Electronics, vol. 69, no.
7, 7099-7108, 2022.
P. Sri-Iesaranusorn, A. Chaiyaroj, C. Buekban, S.
Dumnin, R. Pongthornseri, C. Thanawattano, and D.
Surangsrirat. Classification of 41 hand and wrist
movements via surface electromyogram using deepneural network. Frontiers in Bioengineering andBiotechnology, 9, 548357, 2021.
C.P. Swami, N. Lenhard, and J. Kang. A novel frameworkfor designing a multi-DoF prosthetic wrist control using
machine learning, Sci Rep 11, 15050, 2021.
Q. Zhang, T. Pi, R. Liu, and C. Xiong. Simultaneous andProportional Estimation of Multijoint Kinematics FromEMG Signals for Myocontrol of Robotic Hands,
IEEE/ASME Transactions on Mechatronics, vol. 25, no.
4, 1953-1960, 2020.
C. Dai, and X. Hu. Finger joint angle estimation based on
motoneuron discharge activities. IEEE J. Biomed. Health
Inform, 24, 760–767, 2019.
M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A-GM.
Hager, S. Elsig, G. Giatsidis, F. Bassetto, and H. Müller
Electromyography data for non-invasive naturallycontrolled robotic hand prostheses. Sci Data 1, 140053
2014.
Z. Su, JA. Fishel, T. Yamamoto, and GE. Loeb. Use oftactile feedback to control exploratory movements tocharacterize object compliance. Front Neurorobot. 6:7,
2012.
SY. Jung, HJ. Yoo, SG. Kim, SH. Park, JK. Park, JH. Kim,and H. Shin. Development of multi-degree-of-freedom
hand prosthesis cover with sensory recognition, Journalof Biomechanical Science and Engineering, vol. 16, 2,
2021.
K. Bae, J. Jeong, J. Choi, S. Pyo, and J. Kim, Large-Area,
Crosstalk-Free, Flexible Tactile Sensor Matrix Pixelated

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

by Mesh Layers, ACS Applied Materials & Interfaces, 13(10), 12259-12267, 2021.
Y. Qiu, , S. Sun, X. Wang, K. Shi, Z. Wang, X. Ma, W.
Zhang, G. Bao, Y. Tian, Z. Zhang, H. Ding, H. Chai, A.
Liu, and H. Wu. Nondestructive identification of softness
via bioinspired multisensory electronic skins integrated
on a robotic hand. NPJ Flex Electronics, 6, 45, 2022.
R. A. Romeo, C. Lauretti, C. Gentile, E. Guglielmelli and
L. Zollo, "Method for Automatic Slippage Detection
With Tactile Sensors Embedded in Prosthetic Hands," inIEEE Transactions on Medical Robotics and Bionics, vol.
3, no. 2, 485-497, 2021
H. Xu, Y. Rong, J. Ren, N. Zhang, Y. Zhao, X. Yang, Z.
Zhu and G. Gu. A Learning-Based Sensor Array for
Untethered Soft Prosthetic Hand Aiming at Restoring
Tactile Sensation, Advanced Intelligent Systems.
2300221, 2024.
T. Rosenbaum-Chou, W. Daly, R. Austin, P. Chaubey,
and D. A. Boone. Development and real world use of avibratory haptic feedback system for upper-limb
prosthetic users, Journal of Prosthetics and Orthotics, vol.
28, no. 4, 136-144, 2016.
F. Clemente, M. D ’Alonzo, M. Controzzi, B. B. Edin,and C. Cipriani. Non-invasive temporally discrete
feedback of object contact and release improves grasp
control of closed-loop myoelectric transradial prostheses,
IEEE Transactions on Neural Systems and Rehabilitation
Engineering, vol. 24, no. 12, 1314-1322, 2016.
M. Aziziaghdam and E. Samur. Providing contact
sensory feedback for upper limb robotic prosthesis,
Proceedings of IEEE Haptics Symposium (HAPTICS),
575-579, 2014.
T. Morita, T. Kikuchi, and C. Ishii. Development ofsensory feedback device for myoelectric prosthetic handto provide hardness of objects to users, Journal of
Robotics and Mechatronics, vol. 28, no. 3, 361-370, 2016.
M. A. Schweisfurth, M. Markovic, S. Dosen, F. Teich, B.
Graimann, and D. Farina. Electrotactile EMG feedback
improves the control of prosthesis grasping force, Journalof Neural Engineering, vol. 13, pp. 056010, 2016.
L. E. Osborn, A. Dragomir, J. L. Betthauser, C. L. Hunt,
H. H. Nguyen, R. R. Kaliki, N. V. Thakor. Prosthesis withneuromorphic multilayered e-dermis perceives touch andpain, Science robotics vol. 3, 19, 2018.
R. Eg, K. Raaen, and M. Claypool. Playing with Delay:
With Poor Timing Comes Poor Performance, andExperience Follows Suit, 2018 Tenth International
Conference on Quality of Multimedia Experience(QoMEX), Cagliari, Italy, 2018,
D. Halbhuber, A. Köhler, M. Schmidbauer, J. Wiese, and
N. Henze. The Effects of Auditory Latency on
Experienced First-Person Shooter Players. ACM, In
Mensch und Computer 2022 (MuC '22), Darmstadt,
Germany, 2022.
T. Shibanoki, and K. Jin. Relationship Between Delay
Time and Sensation in Tactile Feedback for Myoelectric
Prosthesis, Proceedings of the International Conferenceon Artificial Life and Robotics (ICAROB2022), Oita,
Japan, 275-277, 2022.

Authors Introduction
Dr. Taro Shibanoki

He received the B.E. degree from the
University of Tokushima in 2008, andthe M.E. and D.Eng. degrees from
Hiroshima University in 2010 and2012, respectively. He was a Research
Fellow of the Japan Society for the
Promotion of Science since 2011, an
Assistant
Professor(Special
Appointment) in Hiroshima University
since 2013 and an Assistant Professor, Lecturer, and an
Associate Professor in Ibaraki University since 2014,
respectively. He is currently an Associate Professor in
Graduate School of Natural Science and Technology,
Okayama University. His current research interests focus
on medical robotics, biological signal processing, andhuman-machine interface.
Mr. Shunji Hashiguchi
He received the B.E. and M.E. degrees
from Okayama University in 2022 and2024, respectively. His researchinterests focus on myoelectric
prosthetic hand.

Mr. Kosuke Jin

252

He received the B.E. and M.E. degrees
from Ibaraki University in 2019 and
2021, respectively. His researchinterests focus on myoelectric
prosthetic hand.