SMASH: Synchronization Media of Athletes and Spectator through Haptic
Marie-Stephanie Iekura∗, Hirohiko Hayakawa, Keisuke Onoda, Yoichi Kamiyama, Kouta Minamizawa, Masahiko Inami
Graduate School of Media Design, Keio University

Figure 1: (a)Heartbeat Microphone, (b)Sport tool microphone, (c)Spectator Haptic Device

1Overview

What could we do if we were able to feel others experience in realtime. SMASH is a system that provides sports experience of a person from a remote area to spectators at the stadium and television
audience in real time. For example, in sports watching, the spectator, by holding the actuator build-in device in the hand, is able
to feel the heartbeat of the athlete and the tactile sensation that theplayer has during a game like shooting a ball, steps, or smashinga shuttle. Sports watching experience using player view point with
HMD is said to be the most close to the player, but it shuts theuser from other spectators. By feeling the players sensation in thepalm, the spectator is able to feel the player closer while enjoying
sports watching with others and share their emotions. This systemused with television broadcast and at stadium should bring different
stage of synchronization with the athlete depending on the situation. In 2020, with the Tokyo Olympic and Paralympic games, newsystem for sports watching are expected to emerge, using not only
tactile information, but information system using extension of anyof the human senses are expected to come out.

2sation based on the Coupled Body Motion”[Mizushina et al. 2014]
permits to relive the experience of the athlete and share it. Because
cables are needed for the visual and tactile feedback, it can be used
only in limited places and not in large stadium.

3 System Design
We are using here “TECHTILE toolkit” technique of treating haptic
information as audio signals. The system structure is, a microphonesystem with a microphone unit that is attached to the player’s chestwith an elastic band, getting heartbeat sound signal during the play
and to any other tool used during sport to get efficient tactile sensation. These two signal are sent to a diversity channel receiver thatis connected to microphone amplifier and audio mixer. The signal
is amplified and sent to the spectator device through a bluetoothtransmitter. The following conditions are thought to be importantwhen we consider sports watching at the stadium: two hands device, synchronization with the athlete experience and real time experience(Fig.2).

Previous work

In “heartbeat picnic: Workshop for Touching Heartbeats” [Watanabe et al. 2011]the users is able to feel his or the people aroundheartbeat. Its aim is to see the importance of life in a new light ina one-to-one format workshop. This system didn’t aim to share
the heartbeat sound with several people. With sports watchingin stadium, a lot of system to enliven sports are existing. “FreeViewpoint Video Synthesis and Presentation from Multiple Sporting Videos”[Inamoto and Saito 2005] is realizing football gamewatching with a Head Mounted Display(HMD). In this case theuser has the feeling to be in the game only while wearing the HMD
so he doesn’t need to be present in the stadium. “TECHTILE toolkit
A prototyping tool for design and education of haptic media”[Minamizawa et al. 2012] is a device that reproduce and extend thetactile sensation that was recorded from any object or body throughother object or body. This technology is used in ”Interactive Instant
Replay: Sharing Sports Experience using Visual and Haptic Sen∗ e-mail: iekura.marie@kmd.keio.ac.jp

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor commercial advantage and that copies bear this notice and the full citation on thefirst page. Copyrights for third-party components of this work must be honored. For allother uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
SA’15 Symposium on MGIA, November 02 – 06, 2015, Kobe, Japan.
ACM 978-1-4503-3928-5/15/11.
http://dx.doi.org/10.1145/2818427.2818439

Figure 2: System Overview

The two hands device design is needed to let the spectator concentrate on the game watching. Moreover, penalty kick in football, free
throw in basket ball are one of the most tense but also best scene insports watching. During theses tense scenes, not only the spectatorin the stadium but people watching the game on tv screen, are doing the gesture of praying with the two hands joined. From the facts
above, the device should fit when hands are joined, so that it can be
hold during the game without disturbing it but also the heartbeat canbe felt in real time to experience athletes feeling. Next point is play-ers experience sharing. With first person view as you are immersedin someone’s feeling you are less able to have your own experienceand feeling. For sports watching, by feeling the heartbeat and thesport tactile sensation, the spectator is close to the player while having his own feelings. The spectators in the stadium and the athlete
but also the people watching on tv, sharing the same time and place
can synchronize. To let the spectator live the athlete experience themost close to reality, it should be transmitted in real time as a sports
match are all unique. By doing so the spectators in the stadium
and the athlete but also the people watching on tv, sharing the sametime and place can synchronize. Synchronize means here to get thesame experience as the athlete you are watching so that you are able
to know about the player with information that you can t usually
have with other media such as television or newspaper. Sports experience with tactile and heartbeat provide you to know about athlete
body information. Letting you know about their feelings and theirplay style more deeply(Fig.3).

the inside part printed in white and outside part printed in transparent(60%) and white(40%) mixed to create a milky transparent casethat allows LED to shine through.

5 User Experience
We did a user test with a futsal player, wearing the heartbeat mic
and one in the foot to record heartbeat and shoot tactile feedback.
We could observe during this test that the player heartbeat changes
while running or making a goal. To prevent the risks of disturbing
the play with the mic wire we put the sound transmitter in the player
pocket and hide the wire in the player’s socks. Yet we observe thatthe mic disconnected from the jack during strenuous movement.
During another user test during a futsal game, we focused on the
spectator experience, by watching a game with the tactile feedback
spectator device in the hands. As we put the mic in one foot andnot inside the ball, the shoot tactile sensation didn’t come each time
the player kicks the ball, but, we observe that we could feel thepresence of the player, which is about 5 meters far, in the palm. We
had a comment that by feeling this presence in the palm, we could
get the feeling of cheering one player individually.

6 Conclusion

Figure 3: SMASH Concept Illustration

4Implementation

The speaker system used by the spectator is 91*109*47(mm),shaped like a handle. The bigger part is fitting in the palm along
to the fingers to give the best haptic experience with the hand.
The thinner part that goes above the back of the hand is design
to preserve the device from accidentally being thrown while enjoying sports watching(Fig.4). Moreover, since the amplitude of

By sharing the athlete experience with the audience, a sports watching where the spectators synchronize with the athlete and his experience and live it as its own is realized. In this research, we mainly
focused on the audience speaker system device and how to share accurate player experience. In future work we would also implement
athlete’s side microphone system. The one we are using now canbecome the cause of interference in the game because of the cables.
We will also measure the level of synchronization between athlete
and spectator depending on the situation to determine in which situation this system fits the most. For example a badminton matchwith 2 athletes and 5 spectators watching near the field, a family orgroup of friend watching a soccer game in front of the TV at home,or a 10 person group public viewing.

Acknowledgements
This work is supported by JST-ACCEL Embodied Media Project
under the collaboration of the Superhuman Sports Society.

References
I NAMOTO , N., AND S AITO , H. 2005. Free viewpoint video synthesis and presentation from multiple sporting videos.
M INAMIZAWA , K., K AKEHI , Y., NAKATANI , M., M IHARA , S.,
AND TACHI , S. 2012. Techtile toolkit: a prototyping tool for
design and education of haptic media.

Figure 4: Spectator Haptic Device
the players heartbeat sound signal is very small, as well as the athletes tactile sensation can be very subtle, and must be accurate tohave a proper experience sharing, a sound amplifier augmenting
the sound received by the speaker system is needed. Therefore asound amplifier has been specially designed for the device, permitting to amplify the tactile sensation. As this amplifier module size
is 30*20(mm) we designed a device that could fit the hand and yet
could contain all this module but also bluetooth receiver module,oscillator and full color LED switch system. For the design weused a 3D printed that allows us to make a double layer object withM IZUSHINA , Y., F ERNANDO , C. L., M INAMIZAWA , K., AND
TACHI , S. 2014. Interactive instant replay: Sharing sports experience using visual and haptic sensation based on the coupledbody motion.
N ISHIMURA , N., I SHI , A., S ATO , M., F UKUSHIMA , S., AND K A JIMOTO , H. 2012. Facilitation of affection by tactile feedbackof false heartbeat. In CHI EA ’12 CHI ’12 Extended Abstracts
on Human Factors in Computing Systems, ACM New York, NY,USA 2012, 2321–2326.
WATANABE , J., K AWAGUCHI , Y., S AKAKURA , K., AND A NDO ,H. 2011. “heartbeat picnic”: Workshop for touching heartbeats.