Building a Feedback Loopbetween Electrical Stimulation and Percussion Learning
Ayaka Ebisu

Satoshi Hashizume

Yoichi Ochiai

Digital Nature Group,
University of Tsukuba
ayaka.ebisu@digitalnature.slis.
tsukuba.ac.jp

Digital Nature Group,
University of Tsukuba
hashizume@digitalnature.slis.
tsukuba.ac.jp

Digital Nature Group,
University of Tsukuba
Pixie Dust Technologies, Inc.
wizard@slis.tsukuba.ac.jp

(a)

(b)

1

5 reproduce
the movement
System ON4(c)2

3 remember

muscle moves

the movement

System OFF

Figure 1: (a) Process of Rhythm Learning, (b) The user plays the castanets with his right hand stimulated by the Stimulated
Percussions, (c) The user tries to remember the movement of her limbs and a drum rhythm pattern

ABSTRACT

1 INTRODUCTION

A sense of rhythm is essential for playing instruments. However,many beginners learning how to play musical instruments havedifficulty with rhythm. We have proposed "Stimulated Percussions," which is a musical instrument performance system using
electrical muscle stimulation (EMS) in the past. In this study, weapply it to the learning of rhythm. By the movement of muscles
stimulated using EMS, users are able to acquire what kind of arms
and legs to move at what timing. In addition to small percussion instruments such as castanets, users can play the rhythm patterns ofdrums that the require the simultaneous movement of their limbs.

A sense of rhythm is indispensable for playing instruments. However, many beginners learning how to play musical instruments
have difficulty with rhythm. In some cases, some people hesitate
to start playing instruments, because it is difficult to read a musicalscore and play the correct rhythm. If a player does not understandthe rhythm, the player will not be able to performance with confidence, the volume will be reduced, and the tones will change.

CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI);KEYWORDS
electrical muscle stimulation (EMS), rhythm, drums
ACM Reference Format:
Ayaka Ebisu, Satoshi Hashizume, and Yoichi Ochiai. 2018. Building a Feedback Loop between Electrical Stimulation and Percussion Learning. In Proceedings of SIGGRAPH ’18 Studio. ACM, New York, NY, USA, 2 pages.
https://doi.org/10.1145/3214822.3214824SIGGRAPH ’18 Studio, August 12-16, 2018, Vancouver, BC, Canada© 2018 Copyright held by the owner/author(s).
This is the author’s version of the work. It is posted here for your personal use. Notfor redistribution. The definitive Version of Record was published in Proceedings ofSIGGRAPH ’18 Studio, https://doi.org/10.1145/3214822.3214824.

With electrical muscle stimulation (EMS), the muscles contract involuntarily and the arms move. By stimulating the muscles according to the rhythm users want to play, EMS can act as ateacher and teach them the rhythm through their own body. There
are several studies that have applied EMS to music performance[Tamaki et al. 2011]. We focus on the learning of rhythm anddemonstrate rhythm learning as an application of "Stimulated Percussions," which was proposed in a previous study [Ebisu et al.
2017].

2 RHYTHM LEARNING ASSISTANCE BY
ELECTRICAL MUSCLE STIMULATION
2.1 Implementation
The basic hardware implementation is the same as "StimulatedPercussions" [Ebisu et al. 2017]. Arduino with a USB-MIDI device1
was used to control the electrical stimulation generation circuitand generate pulses. The system configuration is as shown in Figure 2.
We implemented a step sequencer user interface that allows users
1 dualMocoLUFA

Project Copyright (C) 2013,2014,2015 byhttps://github.com/kuwatay/mocolufa (last accessed May 16th, 2018)

morecat_lab

SIGGRAPH ’18 Studio, August 12-16, 2018, Vancouver, BC, Canada

Ayaka Ebisu, Satoshi Hashizume, and Yoichi Ochiai

Set the devices in the outputs

PCArduino(MIDI device)
Electrodes

power supply

Orange indicates step
A button isToggle ofa sixteenth-noteeach outputʼs on/oﬀcircuit

Figure 2: Configuration of the Stimulated Percussions
Figure 3: Screenshot of the sequencer user interface

to enter their own rhythm patterns using the WEB MIDI API (Figure 3). With this UI, it is possible to control multiple Arduinos. Inaddition, the rhythm input and the ON / OFF of the electric stimulation can be switched to each Arduino. By sending an electricalstimulus before the step is visually indicated, the hitting tone willsound at the timing of the step. Two examples of drum rhythm
patterns sent as input to the UI are shown in Figure 4 with theassociated score.

2.2

Rhythm Learning Process

Using this system, the muscles move according to the rhythm stimulated by the EMS. As a result, users will remember the timing ofthe movement. After users are able to remember the movement,
the movement can be reproduce without the system. Users repeat
this process to acquire rhythm as shown in Figure 1 (a).

(a)

Right hand
Left hand

Right footRight hand
Left hand
Right foot(b)

Right hand
Left hand

Right footRight hand
Left hand

By adding EMS while listening to a sound or reading musical notes
as a visual aid, the timing of the movement of each limb becomeseasy to understand. Once the timing of the movements is learned,
it will be easier for users to practice without the aid of this system.
Users who practiced drum performance using this system provided
the following feedback. "By using electrical stimulation, I was able
to feel the beat in my body and it is easy to understand when eachlimb should be moved. When I play the beat, my stroke matches thestimulus if I hit on the timing to hit, so I know that I struck at thecorrect timing. It felt more fun and easier to hit than when watchinga movie of lesson on how to play drums." "By using electrical stimulation, it was good that I was able to think separately from the rhythm
pattern of the foot and hand."3

APPLICATION

Users can not only learn the rhythm with one hand (Figure 1 (b))
but also the rhythm that uses both arms simultaneously. For example, polyrhythm (2:3, 3:4 etc.) where the right hand and theleft hand simultaneously beat different beat are generally difficultrhythms.
In addition, it is possible to learn playing drum rhythms that require a player to not only use the arms but also the feet at thesame time. As the order and timing of movement of limbs can be

Right footFigure 4: Drums rhythm patterns: (a) Represents eighth notebeats, (b) shows shuffle beatsacquired, this system is considered to be effective for a rhythm pattern in which multiple limbs, such as polyrhythm and drums, areseparately moved at the same time. Our study shows that this system even work for a user who is a non-drummer and has very littleexperience with playing musical instruments. By practicing withdozens of minutes of EMS, it became possible to hit eight beats.
The user shown in Figure 1 (c) has a little drum playing experience. She was not able to beat patterns such as a changing bass
drum pattern. By practicing with Stimulated Percussions, she wasable to learn the rhythm.

REFERENCES
Ayaka Ebisu, Satoshi Hashizume, Kenta Suzuki, Akira Ishii, Mose Sakashita, andYoichi Ochiai. 2017. Stimulated Percussions: Method to Control Human for Learning Music by Using Electrical Muscle Stimulation. In Proceedings of the 8th Augmented Human International Conference (AH ’17). ACM, New York, NY, USA, Article 33, 5 pages. https://doi.org/10.1145/3041164.3041202
Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011. PossessedHand: Techniquesfor Controlling Human Hands Using Electrical Muscles Stimuli. In Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). ACM,
New York, NY, USA, 543–552. https://doi.org/10.1145/1978942.1979018