Naviarm: Augmenting the Learning of Motor Skills using a
Backpack-type Robotic Arm System
Azumi Maekawa∗

Shota Takahashi∗

MHD Yamen Saraiji

The University of Tokyo
Meguro-ku, Tokyo, Japanazumi@star.rcast.u-tokyo.ac.jp

Waseda University
Shinjuku, Tokyo, Japans.tbb1994-estl@asagi.waseda.jp

Keio University Graduate School ofMedia Design
Kohoku-ku, Kanagawa, Japanyamen@kmd.keio.ac.jp

Sohei Wakisaka

Hiroyasu Iwata

Masahiko Inami

The University of Tokyo
Meguro-ku, Tokyo, Japan
wakisaka@star.rcast.u-tokyo.ac.jp

Waseda University
Shinjuku, Tokyo, Japanjubi@waseda.jp

The University of Tokyo
Meguro-ku, Tokyo, Japaninami@star.rcast.u-tokyo.ac.jp

ABSTRACT1

We present a wearable haptic assistance robotic system for augmented motor learning called Naviarm. This system comprises two
robotic arms that are mounted on a user’s body and are used to transfer one person’s motion to another offline. Naviarm pre-records
the arm motion trajectories of an expert via the mounted roboticarms and then plays back these recorded trajectories to share theexpert’s body motion with a beginner. The Naviarm system is an
ungrounded system and provides mobility for the user to conduct
a variety of motions. In this paper, we focus on the temporal aspectof motor skill and use a mime performance as a case study learning
task. We verified the system effectiveness for motor learning using
the conducted experiments. The results suggest that the proposedsystem has benefits for learning sequential skills.

Our bodies are separate entities, and we rely on our sensory feedback to observe and learn from other people or instructors. In motor
learning, beginners rely on indirect methods to learn the body motions or performances of another person and offline learning usuallyrelies on prerecorded video and audio files. In other words, it isnecessary to rely on information obtained via observations of amodel or verbal instructions if the coaching cannot be obtained
thoroughly. Beginners need to understand the obtained information
and remember sequential motions, and then they need to remap
the motion to their own bodies. This process is often non-intuitiveand can yield incorrect motion trajectories for learners due to thelack of direct feedback on their body.
As a way to solve this problem, haptic feedback has been used tosupport other modalities. Multiple studies have aimed at assisting
and accelerating the motor learning process by enhancing information feedback via haptic interfaces [23, 28]. In these approaches,
tabletop mechanical devices often act as haptic displays. However,the use of such grounded approaches results in a motion range thatis limited to the degree of freedom (DoFs) of the end effector of thedevice and by its fixed point in the environment.
In this paper, we propose Naviarm, a haptic assistance system
for the augmented learning of motor skills that allows the user
to record and play over a relatively wide motion range with themobility. Naviarm is a backpack-type robotic arm device with seven
DoFs per arm (Figure 1). In this system, the motion trajectories of anexpert’s arms relative to the expert’s torso are recorded. The target
user of this system is a beginner who aims to learn motor skills
from the expert. The user can play back the prerecorded motion
trajectories via the robotic arms as a learning support.
In this paper, we test this system using a case study for the
motor learning of a mime performance. Mime artists require a highdegree of muscle control to convey a convincing performance. The
conducted user study verifies the degree of the learned motor skills
for beginner users.
Our contributions are summarized as follows.

CCS CONCEPTS• Human-centered computing → Haptic devices; Human computer interaction (HCI); • Computer systems organization →Robotics;

KEYWORDS
Augmented learning; Wearable Device; Motor Learning; Haptics;
Robotics;
ACM Reference Format:
Azumi Maekawa, Shota Takahashi, MHD Yamen Saraiji, Sohei Wakisaka, Hiroyasu Iwata, and Masahiko Inami. 2019. Naviarm: Augmenting the Learningof Motor Skills using a Backpack-type Robotic Arm System. In Augmented
Human International Conference 2019 (AH2019), March 11–12, 2019, Reims,
France. ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3311823.
3311849∗ Both authors contributed equally to this research.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
AH2019, March 11–12, 2019, Reims, France© 2019 Association for Computing Machinery.
ACM ISBN 978-1-4503-6547-5/19/03. . . $15.00
https://doi.org/10.1145/3311823.3311849

INTRODUCTION• We propose a wearable backpack-type haptic device withdual robotic arms.
1

AH2019, March 11–12, 2019, Reims, France

Maekawa and Takahashi et al.
approaches can provide the feeling of touching an object in virtual space, they are not suitable for conveying continuous motion
trajectories.
In an approach similar to our system, there have been studies
conducted on robotic exoskeletons. However, many of these studies
focus on power assistance and rehabilitation [3]. The work closest
to that of Naviarm was proposed by Saraiji et al. [26] for remote
guidance and collaboration by enforcing the posture of a surrogate
via a remote user. However, there was no evaluation of their systemwith respect to the transfer and learning of motor skills.
Our goal is to use the developed backpack-type haptic system
to assist in learning motor skills by actively simulating the user’s
body. The system evaluation presented in this paper verifies theeffect of the proposed approach on learning motor skills.

2.2

Previous studies have not completely proved the advantages of a
haptic guidance approach for learning motor skills. Several studies
have reported that haptic guidance can improve motor skills [2, 4];however, there are also results showing that haptic guidance does
not have a significant effect on learning motor skills [8, 30, 32].
Further, Lee et al. demonstrated that haptic guidance was actually
detrimental to learning motor skills and that haptic disturbanceswere more effective [17]. One likely reason for this is that the effectof haptic guidance depends on the type of learning task.
Conversely, in the case of the temporal aspects of a motor skill,e.g., timing, patterns, or sequences, several studies have shown thathaptic guidance does provide benefits. Morris et al. reported thathaptic guidance was effective for learning a sequence of forces [20].
Feygin et al. conducted an experiment to learn 3D hand trajectories.
Their experiment showed that the temporal aspects of a task wereeffectively learned with haptic guidance, while the spatial aspects
were not improved [6]. When playing musical instruments, timing
and rhythm are both important and haptic benefits have been reported [7, 10]. In our proposed system, we conduct an experimentto verify the effectiveness of Naviarm on the temporal aspects oflearning motor skills.
In addition, many approaches including those described above
have concentrated on the range of motion in a hand or on a desktop
range. Few verification studies have been done for systems that canhandle relatively wide ranges of motion and motion accompanying
whole body movement. We aim to construct a system that combines
a wide range of upper arm workspace and mobility.

Figure 1: (a) A system overview of the wearable haptic guidance system called Naviarm. (b) The hardware design of the
backpack-type robotic arms. (c) The wristband to provide
haptic feedback from the robot to the wearer.

• We develop a haptic assistance system for augmented learning of motor skills including mobility and a wide range of
motion.
• A case study verifies the benefits of the system for motor
skill training using a mime performance.

2

RELATED WORK

Naviarm is derived from wearable haptic devices, haptic guidance
for motor skill training, and systems for body performance training.

2.1

Haptic Guidance for Motor Skill Training

Wearable Haptic Devices

2.3

Many studies use grounded apparatuses for haptic guidance, such ascommercially available haptic devices or manipulators. Therefore,
the workspace available to a user is limited. One way to transfer
haptic information to a user preserving spatial mobility is to use
a wearable device. HapticGear and SPIDAR-W are backpack-typemechanical haptic feedback devices [14, 21]. Rotor thrust [12] andthe gyro effect [33] have been utilized for hand-held-type haptic
display devices. HapticSerpent [1] proposed a waist-worn robot toprovide direct feedback on the user’s body via a touch and pinch
system.
Lopes et al. proposed a wearable approach for generating haptic
feedback with electrical muscle stimulation (EMS) [19]. While theseTraining Assist System for Body
Performances

Many approaches for providing external self-images have been
proposed as methods to support the training of motor skills for
activities such as dancing [31], jogging [29], and playing sports[13]. These approaches encourage users to improve their motor
skills by allowing them to observe their own behavior. Lieberman
et al. proposed an approach for the motor skill transfer of upper
limb motions using a vibrotactile device [18]. They showed thatthe performance was improved via tactile feedback. Landin andHebert developed the method for improvement of the skill of tennis
players by enhancing mental ability[16]. Hiyama et al. proposed2Naviarm

AH2019, March 11–12, 2019, Reims, France

Figure 2: An overview of a common domain calculation for
the range of motion of a human arm and the robot trajectory domain. The gray, yellow, and red regions indicate theranges of motion in the human arm, Naviarm, and the common domain, respectively.

Figure 3: The position accuracy of proposed record-andreplay system without wearing.

an expert-skill transferal system based on first-person audiovisual
information and tactile feedback [15]. We aim to provide significant
information for skills that are difficult to transmit using conventional approaches.

3

3.3

SYSTEM OVERVIEW

In this section, we present a wearable haptic guidance system forlearning arm motor skills. Figure 1 shows an overview of the proposed device and system. The Naviarm system provides haptic
feedback for the wrist position trajectory from experts to beginners
using the medium of a wearable robotic arm. Our system for skill
learning is composed of backpack robotic arms, haptic-feedback
wristbands, and a record-and-replay system.

3.1

Backpack-type Robotic Arms

The hardware design of the backpack dual-arm robot is shown in
Figure 1(b). The robot can be carried on the back and has a total
weight of 5.75 kg. Each arm has six DoFs and a length of 787 mm.
Each joint of the robotic arms is driven by servo-motors (Kondo
B3M-SC-1070-A and B3M-SC-1170-A). The maximum torque of theshoulder joint is 15.2 N m, and the resolution is 0.088◦ . As shown in
Figure 2, the proposed robotic arm sufficiently covers the range of
motion of a human arm [22] and is longer than the average humanarm [24]. Therefore, it does not limit the movement of the wearer’s
arm motion and maintains the DoFs necessary to learn a motor
skill. For details on the design of the robotic arm, please refer to
[25].

3.2

Record-and-Replay System

The Naviarm system comprises a recording phase for the expert
motion trajectory and a replay phase. First, in the recording phase,an expert carries the robot arms and a haptic-feedback wristband is
attached to each of the expert’s wrists. When the expert performs
an arm motor skill movement, the joint angle data of each robot arm
are sequentially recorded as the 3D trajectory of the wrist positions
with respect to the wearer’s torso. These angle data are translated
into wrist position trajectories expressing the sequential motor skillof the expert. The sampling frequency of the system is set to 100 Hz.
During the recording phase, the motors of the robotic arms are set
to free rotation so that the system does not add extra force to thewearer and the expert can perform the motor skill performance inthe same way as when the robotic arm is not worn.
In the replay phase, the robotic arm is set to active control, andtherefore, all motors are activated. The arms move the beginner’s
wrists according to the sequential angle data of the recorded expert
motor skill movement. In this phase, the recorded data are also
queried at 100 Hz; therefore, the temporal resolution of the motion
is 0.01 sec. At that time, a beginner can learn the sequential motion
trajectory of the expert while a directed force is applied to the wrist
via the wristband.
To measure the accuracy of the record-and-reply system, we used
a HTC VIVE Tracker to make position measurements of the end
effector. The position accuracy of the Naviarm record-and-replay
system is shown in Figure 3. The maximum errors in the position
trajectory between record and replay were 47.8 mm (x-axis) and
42.4 mm (y-axis). We assume that the main source of this error isdue to the backlash in the gearing system of the motor. However,
the replayed trajectories were similar to the recorded paths without
sequential changes. Therefore, even though the proposed Naviarm
system can be effective for teaching sequential motor skills, it is
not suitable for motor skills that require high positional accuracy(such as painting).

Haptic-Feedback Wristband

We designed a custom wristband that connects the end effector of
the robotic arm to the wearer’s wrist (Figure 1(c)). This wristband
consists of 3D printed parts, bearings, and a cloth band. Its total
weight is 91 g, and the cloth band can be applied regardless ofbody differences. The bearing gives a one-DoF passive joint to thewristband. The proposed number of DoFs was chosen because our
feasibility study indicated that a seven-DoF robotic arm including
an end effector is suitable for skill extraction from an expert andskill transmission to a beginner.
3

AH2019, March 11–12, 2019, Reims, France

Maekawa and Takahashi et al.

Figure 4: Experimental flow diagram. The difference between the two groups is the method used in the training phase.

3.4

Naviarm System

for an evaluation because a mime moves relatively slowly. We compared learning using Naviarm to an existing motor skill learning
method for learning a mime performance.

Here, we present the main characteristics of the developed backpacktype robotic arm system for learning motor skills. First, the proposed Naviarm system can directly teach the temporal aspects of a
motor skill, e.g., timing, pattern, or sequence, via the dual-robotic
arms. During the acquisition of a motor skill, it is challenging tolearn sequential motor skills as a structure that smoothly connects
individual motor skills [27]. However, the Naviarm record-andreplay system first assists in conveying an expert’s motor skills
without sequential changes. Therefore, the wearer is able to learna sequential motor skill with motions involving both entire upper
limbs via direct haptic feedback. Second, the backpack-type roboticarms are applicable to a wide range of upper limb motor skills
while allowing for freedom of movement in the legs. Third, the proposed backpack-type robotic arms have higher mobility comparedto grounded apparatuses. Mobility is essential for training a wide
range of upper limb motor skills while involving the movement ofa user’s legs, for example, for activities such as dancing and sports.
Therefore, the mobility of Naviarm is an important characteristic
in terms of the variety of applicable motor skills. Next, Naviarm
provides moderate agency of the upper limbs to the user. Because
haptic feedback is only applied to the user’s wrists, the user’s otherjoints can move freely. Therefore, we assume that beginners canmaintain a certain level of active learning. Finally, Naviarm canbe used for learning motor skills both online and offline. As an
example of online use, using Naviarm, one teacher would be able
to teach sequential dance movements to numerous students in real
time.

4

4.1

Setup and Protocol

We conducted a case study to compare the effect of the Naviarm
system on learning a skill to a method based only on visual information. The experiment participants comprised 12 males and 1 female(ages: 22.9 ± 0.99, heights: 174 ± 4.66 cm), who had no previous
experience in miming. We recruited a 23-year-old man who had 10
years of mime and dance experience to record a mime performance
expressing an invisible wall as the expert motion trajectory. The
recorded performance involves whole body movements including
leg movements to use the mobility of Naviarm, and the recorded
time was approximately 20 s. In addition, we recorded a video ofthe expert motion and used it as a reference during the training
sessions. Then, the experiment participants were divided into twogroups: group V (6 people) and group V + H (7 people). In groupV, the participants were trained in the mime performance based
only on the reference video. Meanwhile in group V + H, the participants were trained using the robot guidance and the reference
video (visual and haptic cross modality).
The experimental procedure is described as follows (Figure 4).
(1) Each participant watched the reference video. Next, the participant performed the same operation with the HTC VIVE
Tracker attached to his/her hands for motion measurement
and evaluation (Figure 5(d)).
(2) Participants were instructed to practice the mime performance of the expert in the reference video for approximately
3 min. The video was projected onto a wall using a projector
and looped for the duration of the training time. During
training, both groups were instructed to focus on aligning
the palm sides of both of their hands. Participants in the V+ H group were trained using the Naviarm, worn as shownin Figure 5(a), while the V group were trained without the
robot, as shown in Figure 5(b). The robotic arm motion was

EXPERIMENTS

We verified the effectiveness of the Naviarm system for learning
motor skills. In this study, we set the motion of a mime, in particular
the motion used to express an invisible wall (Figure 5(c)), as thetarget learning task. This motor skill was selected for the following
reasons. 1) Sequential motor skills for the arms are important formime performances. 2) A mime requires the action to significantly
move both their arms and trunk. 3) Such a movement is suitable4

Naviarm

AH2019, March 11–12, 2019, Reims, France

4.2

Evaluation Criteria

We defined several evaluation criteria to evaluate the learning of the
motor skills involved in the mime performance. Because miming is
an artistic medium, a qualitative evaluation is important to indicate
whether other people feel that an invisible wall exists. At the sametime, from our interview of the mime expert, we found that one
criterion to judge the miming skill level for an invisible wall is thealignment of the palm sides of both hands. In addition, a subjective
evaluation focusing on not only the short-term skills being verified
but also on long-term practice is a crucial index. Accordingly, thethree evaluation criteria for the learning of the motor skill are
shown below.
a) Experts qualitative evaluation score
The visual impression of the mime motion was qualitatively
evaluated. Based on the interview with the expert and thegrading criteria of rhythmic gymnastics [11], we set threescoring criteria comprising technique, composition, and expression points. Technique points evaluate the beauty of themime’s invisible wall. Composition points are evaluated basedon the smoothness of the sequential connection of each motion. Finally, composition points involve the coincidence levelof the expert’s performance flow. Therefore, the composition
points also refer to the level of the mime’s sequential motor
skills. The expression points evaluate the dynamics of the
movement. In other words, we evaluated how much the wall
could be expressed in relation to the spatial viewpoint. If
participants express mobility in the reference video, highexpression points are acquired.
Concerning the above three points, three experts with 10
years of mime or dance experience judged the participant’s
mime performances on a scale of 1–10. To eliminate any
grading bias, we introduced a Z-score, which standardized
the scores of each judge. If a raw score by a judge is x, thenthe Z-score is calculated as follows:n
1 Õ xk − µk
,
(1)
Z=
n
σk

Figure 5: Overview of the experimental environment. (a) Participants in the V + H group practiced with Naviarm, which
simultaneously replays the performance of the experts on
the front video. (b) Participants in the V group practiced
without Naviarm. (c) The overview of the mime’s invisible
wall (d) Measuring the palms of the hands for the mime motion using the HTC VIVE tracker fixed with rubber bands.

k =1

where µ is the mean of the judges’ scores, σ is the standard
deviation, and n is the number of judges.
b) Quantitative evaluation based on the hand posture
To express an invisible wall in miming, it is important toalign the palm sides of both hands as if the performer weretouching the invisible wall. In other words, an evaluationis required for the level of coincidence between the pitch
angle and the yaw angle for the palm sides of both hands.
Therefore, we set the following evaluation index G based on
the inclination of the hands in the pitch and yaw directions:replayed in sync with the expert motion in the reference
video.
(3) Similar to step 1, the participants’ mime performance wasrecorded for evaluation using video playback and the tracker.
In this step, group V + H participants performed withoutwearing the robot to evaluate the spatial memorization of
the motion.
(4) Steps 2 and 3 were repeated three times (in three sets) forall participants to measure their temporal memorization.
(5) Finally, the participants filled in a questionnaire to provide
a subjective qualitative evaluation of their experience.

G=

T q
1Õ
(pi − p0 )2 + (yi − y0 )2 ,
T i=1

(2)

where p is the pitch angle, y is the yaw angle, and p0 and y0
are the initial values for calibration. G is a scalar value. A
smaller G value represents a smaller error from the coronalplane of the hand posture in the series of motions. The pose
was measured by the tracker (Figure 5(d)).
5AH2019, March 11–12, 2019, Reims, France

Maekawa and Takahashi et al.

Note that the Naviarm system is not able to assist in aligning the palm surface because wearers can move their wrist
joints freely. As described in experimental procedure (2),both groups were verbally instructed to focus on aligning thepalm sides of both of their hands. Therefore, by comparing
this evaluation index, the influence of the verbal information
provided for learning the skill can be examined for the twogroups.
c) Subjective evaluation for the motor skill training
From the viewpoint of the long-term use of a training system,
we conducted subjective questionnaires using the following
eight statements at the end of the experiment. For each question, we asked participants to respond using a seven-point
Likert scale from 1: “I do not think so at all” to 7: “I think so”.
(1) I knew how to improve the mime skill while practicing.
(2) My performance improved with practice.
(3) I was able to concentrate on learning the mime skill.
Figure 6: Results of the Z-score increase rate from before toafter practice for each of the three evaluation scores. The
Z-score was calculated using Eq. (1). The *p values were derived using the Bonferroni method (*p < 0.05).

(4) My motivation to learn the mime skill was high during
practice.
(5) I wanted to use the device when practicing alone.
(6) I did not feel uncomfortable during practice.
(7) My level of fatigue was low during practice.

expert’s performance in this study. In other words, Naviarm madeparticipants naturally learn the same sequence as that demonstrated
by the expert. Therefore, it is possible that learning motor skills
with Naviarm will be effective for learning sequential skills for activities such as dance choreography and assembly work proceduresin factories.

(8) Practicing the mime skill was enjoyable.

5

RESULTS AND DISCUSSIONS

The results of each evaluation criteria for each group (V and V +H) for the mime motor skill training are shown below.

5.1

5.2

Criteria A) Experts qualitative evaluation
score

Criteria B) Quantitative evaluation basedon the hand posture

Figure 7 shows the result of the hand pose error (evaluation indexG) measured by the tracker during the mime performances. In the V+ H group, there was no significant difference between the practice
repetitions, including the first and last sets. Conversely, in the Vgroup, a significant difference was observed between the first andlast sets (p < 0.05).
This result suggests that the V group efficiently acquired theskill of aligning their hands, which is a non-sequential fine motor
skill. The robotic arm guidance appears to have made participants
conscious of their arm trajectories via the haptic assistance; therefore, they could not concentrate on the skill of aligning the palmsof their hands. That is, if haptic guidance is used, the assisted skill
will be significantly recognized. Therefore, Naviarm is likely notsuitable for non-sequential fine motor skill acquisition, which is
not haptic guidance-based.

Figure 6 shows the results of the Z-score increase rate comparing
before training and after training for each evaluation score. To
exclude the bias with respect to the participants’ fundamental skill
levels, the increase rates of the scores are shown. Of the threescoring criteria, there was a significant difference (p < 0.05) in theincrease rate of the composition points when comparing the Vgroup and the V + H group. However, there was no significant
difference in the technical and expression points between the Vgroup and the V + H group.
The result for the composition points suggests that Naviarm
may be useful for acquiring sequential motor skills. As describedin Section 4.2(a), the composition points indicate the level of thesequential motor skills for an action, in this case, miming an invisible wall. Typically, it is difficult to communicate sequential motor
skills verbally and a beginner needs to remember the sequence ofthe movements to reproduce the motor skill. However, a Naviarm
wearer can directly access the sequential movements of an expert
via the robotic arms’ sequential trajectory information. Therefore,a novice does not need to remember the movement sequence. In
this way, Naviarm appears to efficiently transfer sequential motor
skills even for short-term training. Surprisingly, this occurred eventhough we did not tell participants to imitate the sequence of the

5.3

Criteria C) Subjective evaluation for the
motor skill training

The result of the subjective questionnaire after the motor skill training sessions is shown in Figure 8. Overall, the standard deviation
was large due to the influence of individual differences; however, asignificant difference was found between the two groups regarding
6

Naviarm

AH2019, March 11–12, 2019, Reims, France
due to wearing the proposed backpack-type wearable robotic arms,
the participants were motivated and enjoyed the mime training. We
assume that this is because the participants were able to maintainan active learning state due to the mobility of Naviarm and thehaptic feedback to their wrists. In addition to the novelty of therobot arm directing their bodies, motivation and enjoyment seems
to be induced by the sense that an expert resides in the robot andteaches them “hand in hand”. In other words, we suggest that skill
learning assistance using Naviarm in long-term training may beeffective to prevent from becoming tired of training and to maintainmotivation for beginners.

5.4

Limitations and Future WorkThe results obtained in this study only focused on skill acquisition
for the motion used to mime an invisible wall and short-term training. Additional verification will be necessary to examine whether
it is useful for learning other motor skills. In addition, Yang etal. [32] indicated that the effect of the feedback changes in longterm training. Therefore, it will be necessary to verify the effectsof Naviarm for longer-term training sessions. In addition, further
studies of control methods and systems are required to evaluate theeffectiveness of this method for learning motor skills. The haptic
feedback of the proposed system was continuous and constant;however, in a previous study, it was suggested that complicatedhaptic feedback including intermittent feedback [5] and impedancecontrol [9] are effective. A promising avenue for future research isa record-and-replay system based on an absolute spatial coordinate
system. In this study, the proposed system records the trajectoryof the position of the arms relative to the torso. By introducing
an absolute spatial position trajectory system, we will be able toprovide new interactions according to the environment. Finally, thecurrent system is not suitable for reproducing very quick physicalactions that require instantaneous power. To respond to high-speed
operations, it is necessary to review the hardware design and thecontrol method.

Figure 7: Results of the palm side poses of both hands in the
training sets. The G-score was calculated using Eq. (2). The*p values were derived using the Bonferroni method (*p <0.05).

6

CONCLUSIONSIn this paper, we proposed Naviarm, a novel haptic feedback system for augmenting the learning of motor skills. The system uses
backpack-type robotic arms to transmit haptic information andcan handle a larger range of motion compared to previous haptic
guidance systems. In the developed system, the arm trajectories ofa skilled expert are prerecorded with the robotic arms and then therecorded trajectories are used to allow beginners to learn the skill
with haptic assistance. In this paper, we used a mime performanceas the target motor skill to evaluate the proposed system. We conducted a user study to verify the benefits of the proposed system
for learning motor skills, and the results suggest that Naviarm has
benefits for learning sequential skills and helped increase users’
motivation. In the current system, the haptic assistance is generated
via the simple positional control of the servo-motors. In the future,
we would like to change the control method and verify its effecton users. In addition, we will explore the possibility of applications
such as remote work support or real-time hand-in-hand teaching.

Figure 8: Subjective questionnaire results regarding the motor skill training sessions. The *p values were derived using
the Bonferroni method (*p < 0.05).

motivation and enjoyment (p < 0.05). In particular, motivation received a high result of 6.00 ± 0.76 in the V + H group. Conversely,
the level of fatigue was not significantly different between the twogroups; however, the V + H group had a critically small result (2.71± 1.91).
The results suggest that Naviarm is effective for improving theparticipants’ level of motivation. Despite being physically fatigued
7

AH2019, March 11–12, 2019, Reims, France

Maekawa and Takahashi et al.

ACKNOWLEDGMENTS[22] National Institution of Technology and Evaluation. 2013. Human Characteristics
Database. (2013). http://www.tech.nite.go.jp/human/[23] Jim B Plunkett. 2010. Robotic golf swing trainer. (Oct. 5 2010). US Patent
7,806,780.
[24] Research Institute of Human Engineering for Quality Life. 2008. Japanese Anthropometric Database 2004-2006. (2008). https://www.hql.jp/database/cat/size/
size2004[25] MHD Yamen Saraiji, Tomoya Sasaki, Kai Kunze, Kouta Minamizawa, andMasahiko Inami. 2018. MetaArms: Body Remapping Using Feet-Controlled
Artificial Arms. In The 31st Annual ACM Symposium on User Interface Software
and Technology. ACM, 65–74.
[26] MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa, andMasahiko Inami. 2018. Fusion: full body surrogacy for collaborative communication. In ACM SIGGRAPH 2018 Emerging Technologies. ACM, 7.
[27] Misako Sawada, Shiro Mori, and Motonobu Ishii. 2002. Effect of metaphoricalverbal instruction on modeling of sequential dance skills by young children.
Perceptual and motor skills 95, 3_suppl (2002), 1097–1105.
[28] Roland Sigrist, Georg Rauter, Robert Riener, and Peter Wolf. 2013. Augmented
visual, auditory, haptic, and multimodal feedback in motor learning: a review.
Psychonomic bulletin & review 20, 1 (2013), 21–53.
[29] Junya Tominaga, Kensaku Kawauchi, and Jun Rekimoto. 2014. Around me: asystem with an escort robot providing a sports player’s self-images. In Proceedingsof the 5th Augmented Human International Conference. ACM, 43.
[30] Seijiro Tsutsui and Kuniyasu Imanaka. 2003. Effect of manual guidance on
acquiring a new bimanual coordination pattern. Research quarterly for exercise
and sport 74, 1 (2003), 104–109.
[31] Shuo Yan, Gangyi Ding, Zheng Guan, Ningxiao Sun, Hongsong Li, and Longfei
Zhang. 2015. OutsideMe: Augmenting Dancer’s External Self-Image by Using A
Mixed Reality System. In Proceedings of the 33rd Annual ACM Conference Extended
Abstracts on Human Factors in Computing Systems. ACM, 965–970.
[32] Xing-Dong Yang, Walter F Bischof, and Pierre Boulanger. 2008. Validating theperformance of haptic motor skill training. (2008).
[33] Hiroaki Yano, Masayuki Yoshie, and Hiroo Iwata. 2003. Development of a nongrounded haptic interface using the gyro effect. In null. IEEE, 32.

We would like to thank G. Ganesh, M. Kitazaki, M. Sugimoto, and
Y. Miyawaki for useful advice. This project was supported by JST
ERATO Grant Number JPM-JER1701, Japan.

REFERENCES[1] Mohammed Al-Sada, Keren Jiang, Shubhankar Ranade, Xinlei Piao, Thomas
Höglund, and Tatsuo Nakajima. 2018. HapticSerpent: A Wearable Haptic Feedback Robot for VR. In Extended Abstracts of the 2018 CHI Conference on Human
Factors in Computing Systems. ACM, LBW624.
[2] CA Avizzano, Jorge Solis, A Frisoli, and M Bergamasco. 2002. Motor learning skill
experiments using haptic interface capabilities. In Proc. of 11th IEEE International
Workshop on Robot and Human Interactive Communication. 198–203.
[3] Robert Bogue. 2015. Robotic exoskeletons: a review of recent progress. Industrial
Robot: An International Journal 42, 1 (2015), 5–10.
[4] Laura Marchal Crespo and David J Reinkensmeyer. 2008. Haptic guidance canenhance motor learning of a steering task. Journal of motor behavior 40, 6 (2008),
545–557.
[5] Takahiro Endo and Haruhisa Kawasaki. 2015. A fine motor skill training systemusing multi-fingered haptic interface robot. International Journal of HumanComputer Studies 84 (2015), 41–50.
[6] David Feygin, Madeleine Keehner, and R Tendick. 2002. Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill. In
Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002. HAPTICS
2002. Proceedings. 10th Symposium on. IEEE, 40–47.
[7] Katsuya Fujii, Sophia S Russo, Pattie Maes, and Jun Rekimoto. 2015. MoveMe: 3D
haptic support for a musical instrument. In Proceedings of the 12th International
Conference on Advances in Computer Entertainment Technology. ACM, 9.
[8] R Brent Gillespie, M OâĂŹModhrain, Philip Tang, David Zaretzky, and Cuong
Pham. 1998. The virtual teacher. In Proceedings of the ASME Dynamic Systems andControl Division, Vol. 64. American Society of Mechanical Engineers, 171–178.
[9] Diego Felipe Paez Granados, Breno A Yamamoto, Hiroko Kamide, Jun Kinugawa,
and Kazuhiro Kosuge. 2017. Dance Teaching by a Robot: Combining Cognitive
and Physical Human–Robot Interaction for Supporting the Skill Learning Process.
IEEE Robotics and Automation Letters 2, 3 (2017), 1452–1459.
[10] Graham Grindlay. 2008. Haptic guidance benefits musical motor learning. In
Haptic interfaces for virtual environment and teleoperator systems, 2008. haptics
2008. symposium on. IEEE, 397–404.
[11] Ration Internationale D E Gymnastique. 2017. 2017 - 2020 CODE OF POINTS
Rhythmic Gymnastics. January (2017).
[12] Seongkook Heo, Christina Chung, Geehyuk Lee, and Daniel Wigdor. 2018. Thor’s
Hammer: An Ungrounded Force Feedback Device Utilizing Propeller-Induced
Propulsive Force. In Proceedings of the 2018 CHI Conference on Human Factors in
Computing Systems. ACM, 525.
[13] Keita Higuchi, Tetsuro Shimada, and Jun Rekimoto. 2011. Flying sports assistant:external visual imagery representation for sports training. In Proceedings of the
2nd Augmented Human International Conference. ACM, 7.
[14] Michitaka Hirose, Koichi Hirota, Tetsuro Ogi, Hiroaki Yano, Naoyuki Kakehi,
Makoto Saito, and Mutsuhiro Nakashige. 2001. HapticGEAR: the developmentof a wearable force display system for immersive projection displays. In Virtual
Reality, 2001. Proceedings. IEEE. IEEE, 123–129.
[15] Atsushi Hiyama, Yusuke Doyama, Mariko Miyashita, Eikan Ebuchi, Masazumi
Seki, and Michitaka Hirose. 2011. Wearable display system for handing down
intangible cultural heritage. In International Conference on Virtual and Mixed
Reality. Springer, 158–166.
[16] Dennis Landin and Edward P Hebert. 1999. The influence of self-talk on theperformance of skilled female tennis players. Journal of applied sport psychology
11, 2 (1999), 263–282.
[17] Jaebong Lee and Seungmoon Choi. 2010. Effects of haptic guidance and disturbance on motor learning: Potential advantage of haptic disturbance. In Haptics
Symposium, 2010 IEEE. IEEE, 335–342.
[18] Jeff Lieberman and Cynthia Breazeal. 2007. TIKL: Development of a wearable
vibrotactile feedback suit for improved human motor learning. IEEE Transactions
on Robotics 23, 5 (2007), 919–926.
[19] Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick
Baudisch. 2017. Providing Haptics to Walls & Heavy Objects in Virtual Reality by
Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conferenceon Human Factors in Computing Systems. ACM, 1471–1482.
[20] Dan Morris, Hong Tan, Federico Barbagli, Timothy Chang, and Kenneth Salisbury.
2007. Haptic feedback enhances force skill learning. In EuroHaptics Conference,
2007 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator
Systems. World Haptics 2007. Second Joint. IEEE, 21–26.
[21] Kazuki Nagai, Soma Tanoue, Katsuhito Akahane, and Makoto Sato. 2015. Wearable 6-DoF wrist haptic device SPIDAR-W. In SIGGRAPH Asia 2015 Haptic MediaAnd Contents Design. ACM, 19.
8