International Conference on Artificial Reality and TelexistenceEurographics Symposium on Virtual Environments (2017)
T. Huang (Editor)

Poster

3D Ground Reaction Force Visualization onto Training Video forSprint Training Support System
Takafumi Taketomi1 , Yasuhide Yoshitake2 , Goshiro Yamamoto3 , Cristian Sandor1 , and Hirokazu Kato1
1 Nara Institute of Science and Technology, Japan
2 National Institute of Fitness and Sports in KANOYA, Japan
3 Kyoto University Hospital, Japan

Abstract
We propose a method for visualizing 3D ground reaction forces for sprint training. Currently, sprinters can check their 3Dground force data using a 2D graph representation. In order to check the relationship between 3D ground force and their sprint
form, they must check the 2D graph and a training video repeatedly. To allow simultaneous observation of the 2D graph andthe training video, we use a mixed reality technology to overlay 3D ground reaction force onto the training video. In this study,
we focus on 2D-3D registration between the image sequence and 3D ground reaction data. We achieved 2D-3D registration byusing a constrained bundle adjustment approach. In the experiment, we apply our method to the training videos. The results
confirm that our method can correctly overly 3D ground reaction force onto the videos.
CCS Concepts•Computing methodologies → Mixed / augmented reality; •Human-centered computing → Visualization;

1. Introduction
In sports training, information technologies such as motion capturesystems, force sensors, and heatbeat sensors are gradually being
adopted to improve performance by analyzing the athlete’s form.
Currently, several types of training data can be acquired by sensing
devices. Athletes can analyze results that are visualized as either a
2D graph that depends on a time sequence or 3D computer graphics
of joint skeleton motion [BHCK13, ITM∗ 13]. In order to analyze
the relationship between the obtained data and their form, athletesmust check the training video independently. In many cases, theobtained data is displayed in parallel with the training video. Athletes and their coaches must separately check different information.
We believe this type of visual feedback method is not an optimal
strategy for sports training activities.
To improve the effectiveness of sports training activities, we usea mixed reality technology that can display the training data andsensor data simultaneously. Mixed reality is a technique that canmerge real world and virtual world information. For example, we
can overlay the obtained sensor data onto the video sequence. Inorder to overlay the obtained sensor data, all data should be represented in the same coordinate system.
In this research, we focus on the sprint training activities of trackand field sports. The use of force plates, which measure the 3D
ground reaction force of a sprinter’s foot, has increased in recentyears. In the literature [RDS∗ 15], they reported a sprint speed isrelated to ground reaction forces. Currently, after running the trackc 2017 The Author(s)Eurographics Proceedings c 2017 The Eurographics Association.

with embedded force plates, the sprinter can check sequential 3D
ground reaction force data displayed as a 2D graph.
In order to confirm the feasibility of visualizing 3D ground reaction force data using mixed reality (MR) technology, we interviewed sprinters at the National Institute of Fitness and Sports inKANOYA. They concluded that MR-based visualization would be
useful for their training. In addition, we interviewed them about
their actual training activity. They use a video camera to check theirform. Usually, the camera is mounted on a tripod, and they do not
use the camera’s zoom feature during video capture. Primarily, they
use a panning camera movement to capture the sprinter. Based on
the results of the interview, we note the following assumptions forregistering the 3D ground reaction force data onto the video sequence.
• The camera is mounted on a tripod.
• The camera primarily rotates in one direction.
2. 3D Ground Reaction Force Visualization Framework
In order to overlay geometrically correct 3D ground reaction force
onto the video sequence, geometric relationship between the force
plate coordinate system and the camera coordinate system is necessary. In other words, camera motion must be estimated in advance.
In our framework, camera motion is estimated in two stages. In theinitial estimation process, we assume pure rotational motion as acamera motion model. In the parameter refinement process, we assume rotational motion and small translation. Using these two pro-T. Taketomi et al. / 3D Ground Reaction Force Visualization

Camera coordinate
system C

MFC

Force plate coordinate
system F

MTC

Tripod coordinate
system TMFT

Figure 1: Relationship between the force plate coordinate system,
camera coordinate system, and tripod coordinate system.

Figure 2: An example of visualization for the sprinter.

cesses, we can obtain accurate camera motion for overlaying 3Dground reaction force. The details of the camera model and the costfunction for the refinement process are described in the following
sections.

image. dist () represents the function for calculating the distance
between reprojected track line S and extracted track line s. L j represents the number of control points in the projected line in camera
frame j. The non-linear minimization technique is applied to costfunction Ebundle to refine the parameters.

2.1. Camera Model

3. Experiment

We consider the three coordinate systems shown in Fig. 1. In order to visualize the 3D ground reaction force in the camera frame,
the transformation matrix M FC , which is conveyed from the force
plate coordinate system to the camera coordinate system, is needed.
According to Fig. 1, M FC can be represented as follows.

To demonstrate the effectiveness of the proposed method, we confirmed the results of visualization of 3D ground reaction forces.
Fig. 2 shows the results of the visualization of 3D ground reaction
forces. The results confirm that the proposed method can correctly
overlay the 3D ground reaction forces onto the foot positions. Inaddition, a brake force can be confirmed in the bottom left image.
By using our visualization framework, this type of inappropriate
3D ground reaction force can be checked using the sprint form. We
believe this intuitive visualization technique can help increase theefficiency of sprint training.

M FC = M TC M FT

(1)

where M FT represents the transformation matrix from the force
plate coordinate system to the tripod coordinate system, and M TC
represents the transformation matrix from the tripod coordinate
system to the camera coordinate system. Eventually, to estimate thetransformation matrix, we must estimate the rotational componentof the geometric relationship between the force plate coordinate
system and the tripod coordinate system. In this case, the degreesof freedom of the matrix M FC is three.
2.2. Energy Function for Refinement
In this process, translation matrices M FTj for each frame and 3D
positions of feature points P i are updated using the bundle adjustment approach. We use the following cost function Ebundle for refining the parameters.
1∑ (E p j + El j )N | j∈N|NN(2)

2
1∑ wi x ji − proj M FC j , X i
P j i∈PP j(3)21∑ dist s jk , S
L j k∈LL j(4)Ebundle =Ep j =El j =

where N represents the set of camera frames. wi represents a weight
for the feature point. At the first frame, several feature points with
3D positions are detected. These 3D positions are measured using a total station. The weights for these points have large values.
On the other hand, we set the weights for other feature points at
1.0. proj () represents the function for projecting point X i onto the4. Conclusion
We proposed a 3D ground reaction force visualization framework
for a sprint training support system. By using the proposed method,the 3D ground reaction forces were overlaid onto the video sequence with reasonably high accuracy. In the future, we are planning to confirm the effectiveness of this visualization framework
for improving the performance of sprinters.
Acknowledgements
This work was supported by the MIC/SCOPE #162107006.
References[BHCK13] B INI R., H UME P., C ROFT J., K ILDING A.: Pedal force
effectiveness in cycling: a review of constraints and training effects. J.
of Science and Cycling 2, 1 (2013), 11–24. 1[ITM∗ 13] I NABA H., TAKI T., M IYAZAKI S., H ASEGAWA J., KOEDA
M., YAMAMOTO H., K ITAGAWA K.: Visualization of human body sensing for supporting sports motion analysis. J. of the Society for Art andScience 2, 3 (2013), 94–100. 1[RDS∗ 15] R ABITA G., D OREL S., S LAWINSKI J., DE V ILLARREAL
E. S., C OUTURIER A., S AMOZINO P., M ORIN J.: Sprint mechanics
in world-class athletes: a new insight into the limits of human locomotion. Scandinavian J. of Medicine and Science in Sports 25, 5 (2015),583–594. 1c 2017 The Author(s)Eurographics Proceedings c 2017 The Eurographics Association.