PickHits: Hitting Experience Generation with Throwing Motion
via a Handheld Mechanical Device
Azumi Maekawa

Research Center for Advanced Science and Technology
The University of Tokyoazumi@star.rcast.u-tokyo.ac.jp

Atsushi Hiyama

Research Center for Advanced Science and Technology
The University of Tokyohiyama@star.rcast.u-tokyo.ac.jp

Seito Matsubara

Graduate School of Information Science and Technology
The University of Tokyo
matsubara@star.rcast.u-tokyo.ac.jp

Masahiko Inami

Research Center for Advanced Science and Technology
The University of Tokyo
inami@star.rcast.u-tokyo.ac.jp

Figure 1: (a) PickHits system provides hitting a target experience computationally. (b) Handheld-type mechanical device torelease an object towards a target. (c) The developed system tracks and predicts the throwing motion in real time.

ABSTRACTKEYWORDS

Experiences of hitting targets cause a great feeling. We propose asystem for generating this experience computationally. This system consists of external tracking cameras and a handheld device
for holding and releasing a thrown object. As a proof-of-concept
system, we developed the system based on two key elements: lowlatency release device and constant model-based prediction. During
the user’s throwing motion, the ballistic trajectory of the thrownobject is predicted in real time, and when the trajectory coincides
with the desired one, the object is released. We found that we cangenerate a computational hitting experience within a limited range
space.

hitting a target, throwing, motion prediction, human-in-the-loopsystem

CCS CONCEPTS• Human-centered computing → Interactive systems and tools;• Computer systems organization → Sensors and actuators; Realtime systems.

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
SIGGRAPH ’19 Emerging Technologies, July 28 - August 01, 2019, Los Angeles, CA, USA© 2019 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-6308-2/19/07. . . $15.00
https://doi.org/10.1145/3305367.3327996

ACM Reference Format:
Azumi Maekawa, Seito Matsubara, Atsushi Hiyama, and Masahiko Inami.
2019. PickHits: Hitting Experience Generation with Throwing Motion via
a Handheld Mechanical Device. In Proceedings of SIGGRAPH ’19 Emerging
Technologies, July 28 – August 01, 2019, Los Angeles, CA, USA. ACM, New
York, NY, USA, 2 pages. https://doi.org/10.1145/3305367.3327996

1

INTRODUCTION

Experiences of hitting a target or making a goal cause a great feeling,for example, basketball shot, shooting games, and even throwing
garbage into the trash can. On the other hand, in order to hit thetarget, a certain degree of skill is required. If that difficulty level
goes up, it will fail out of the goal and make us annoyed in manycases.
In this paper, we propose a system for generating experiences ofhitting a target computationally. The system consists of externalmotion tracking cameras and a handheld mechanical device to holdand release a bullet: a thrown object. While the user is performing
throwing motion, this system is running the calculation of theballistic trajectory of a thrown object in real time. At the timing
when the trajectory of the object matches the desired one, themechanical hand is opened with the electric trigger signal, andthe object is thrown into the air. In other words, the object is not

SIGGRAPH ’19 Emerging Technologies, July 28 - August 01, 2019, Los Angeles, CA, USA

2.2

A. Maekawa et al.

Constant model-based prediction

We measured the time from getting the position of the tracked
object to the release of the object held in the device hand as thedelay of the whole system, and we found that this delay is about
50 ms. Since human throwing motion is highly dynamic movement, the position and velocity of the device can change by anamount sufficient for the ballistic trajectory of the thrown object
to change significantly within 50 ms. In this paper, we predict theuser throwing motion on the assumption that the arm movement
can be regarded as constant acceleration motion in a short time.
The system calculates the ballistic trajectory with the predicted 50
ms ahead motion.
Figure 2: (a) A proof-of-concept system overview. While
tracking the object position, the system is calculating theballistic trajectory of a thrown object in real time. Modelbased motion prediction is implemented to suppress error
due to the system delay.

released while doing a motion that misses the target. In this way,
the system allows the user to throw an object at the desired target.
Itoh et al. proposed the system visualizing the ballistic trajectoryfor supporting human performance [Itoh et al. 2016]. Our system
focuses on enhancing the dynamic ability such as throwing motion.
Our goal is to construct a human-in-the-loop system where computational fast-loop and user’s slow-loop are integrated such as The
SmartTool [Nojima et al. 2002].

2SYSTEM

Figure 2 shows the architecture of the developed proof-of-concept
system. The position of the handheld device is tracked at 300 Hz
by externally placed motion tracking cameras, and the velocityof the object held in the device is estimated based on this information. This system has two key elements: 1) low-latency releasehand implemented as the physical handheld device, and 2) constantmodel-based prediction to suppress the error due to the systemdelay.

We developed the system based on the above two elements andcarrying out a feasibility study. We found that we can generate acomputational hitting experience withing a limited range space.

3USER EXPERIENCEAt SIGGRAPH 2019 Emerging Technology exhibition we will demonstrate PickHits system with the proposed external tracking systemand the handheld device. Attendees can select the targets and thethrown objects, and experience a pleasant feeling with hitting thetarget. This system can also allow users to access to remote places orhigh positions. By employing a handheld device, we found that thissystem could give the users a sense that the generated hitting experience was caused by themselves to a certain extent, even though
the release was triggered computationally. Therefore, this system
will give attendees the experience as if the user cooperate withthe computational system. Attendees can perform difficult tasks
without the system such as no-look shot and hitting the moving
target.

4LIMITATION AND FUTURE WORKIn the current system, the predicted ballistic trajectory and theactual trajectory do not match in the order of a millimeter. It is considered that this is caused by various factors such as measurement
error, user motion difference, release noise and so on. In order toincrease the system accuracy, it will be necessary to implementstabilization like gimbal mechanism and control of active releases
support.

ACKNOWLEDGMENTS
2.1

Low-latency release hand

We designed a simple hardware device with one degree of freedom
as a proof-of-concept model. This device employs a holding handclosed using a ratchet mechanism, and when the pawl, which prevents movement of the ratchet, is unlocked by a solenoid actuator,
the hand is opened by the elastic energy stored in the torsion spring.
This mechanism is designed with the aim of minimizing the delay
time from when the trigger signal is received until the hand opens
by eliminating intermediate elements such as gears in the actuator and applying sudden force in a moment with storing energy.
Generally, if the time horizon of motion prediction increases, theprediction accuracy gets worse. Therefore, by minimizing the delay,
the accuracy of the motion prediction described next section canbe improved.

This project was supported by JST ERATO Grant Number JPMJER1701, Japan.

REFERENCES
Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016. Laplacianvision: Augmenting motion prediction via optical see-through head-mounted displays. In Proceedings of the 7th Augmented Human International Conference 2016.
ACM, 16.
Takuya Nojima, Dairoku Sekiguchi, Masahiko Inami, and Susumu Tachi. 2002. The
SmartTool: A system for augmented reality of haptics. In Virtual Reality, 2002.
Proceedings. IEEE. IEEE, 67–72.