A New Approach for Regulating Person’s Athletic Abilityby the Pulse Rate for Realistic Sports Casting
Haruo Noma
Tsutomu Miyasato
ATR Media Integration & Communications Research Labs.
{noma,miyasato}@mic.atr.co.jp
Abstract
Sports casting on TV has become the most popular program. Generally, a TV audience wholly
watches a sports program from a spectator’s point of view. Therefore it is difficult for them todirectly sense the excitement or enjoyment that the athlete they are watching is feeling. Broadcasting
what the athlete is sensing, thinking or feeling while s/he is in the heat of competition to theaudience directly, can allow the audience to experience the athlete vicariously with the use of a
VR system. We call such a new broadcasting system a realistic sports cast (RSC). We have been
focusing on marathons for our pilot RSC system and, in this paper, we introduce a method ofbroadcasting information on how a professional athlete is feeling tired while running. By using
our athletic ability regulating method by the pulse rate, a system can adjust the running speed of
the viewer, and consequently, allow the viewer to share a feeling of fatigue with the athlete.
Keywords: Sports casting, Locomotion Interface, VR, Multimedia display

1. Introduction
When we engage in a sports activity, we can choose either “to participate in the sports” byourselves or “ to watch the sports casting” of professional athletes. In the latter case, since the TV
audience wholly watches the sports program from a spectator’s point of view, it is difficult forthem to sense the excitement or enjoyment that the athlete they are watching is feeling. Considering this, we are studying a new method for the spectator sports market, called “Realistic Sports
Casting: (RSC)”, which allows TV viewers to directly feel what their favorite sports stars are
feeling as if they are playing the sports as the stars. As the final goal of RSC, we want the system
to collect what an athlete is watching, hearing, feeling and sensing using body-worn sensors, andto have this information be broadcasted live to viewing homes. Then in a home, a viewer canexperience for himself/herself what an athlete is going through in the field through several VR
devices. Consequently we want the viewer to experience the athlete’s feeling, mindset, pleasure,or pain vicariously without a noisy sports commentator.
To achieve RSC for various sporting events, the system needs to be capable of handling various
type of intersensory information concerning an athlete. Furthermore, it needs many smart andsmall sensors and displays.
Therefore, as a pilot RSC system, we aimed at marathons, because the motions of marathon
runners remain relatively stable and the race process of marathons does not change so drasticallyrace to race[1,2]. Figure 1 shows the concept of our RSC system. Each athlete in a race wears asmall CCD camera, a set of microphones, and some motion and biometric sensors. Ideally, theobtained results should be integrated and broadcasted to all viewers in real time. In the pilot
system, we record the results into a VCR and PC for experimental and technical reasons. In thisprocess, the pilot system adds such information as how, where and what kind road each runner isrunning on manually. In each viewing home, viewer prepare a VR deck , that is composed of aProceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

Without running speed modification
20km/h

20km/hA spectator

An athlete

Do

Directly
display running
speed

wn

A spectator lag an athlete because of athletic ability gap.

With runnig speed modification

Relay
stationBroadcast

20km/hAn athlete

Athletes whowear sensor

Athletic field

Athletic abilityof athlete andspectator

10km/hA spectator

Running speed
controller

Virtual room at home or gym

Modifying running speed to share a feeling of fatigue

Fig. 1 Concept of a realistic sports
casting system

Fig. 2 Necessity of running speed
modification in RSC

large projection screen, a computer controlled treadmill, and a motion platform at the pilot system. The athletes view is displayed on the screen and the viewer runs on the treadmill, which isdriven at the athlete’s running speed. Moreover, when the athlete is running one slope or goingaround a curve, the motion platform recreate the same road condition.
In a recent study, we confirmed that the pilot system can give subjects more positive subjective
impressions than the traditional marathon sport casing way approach[2]. In the experiment, we
simply drove the treadmill belt at the same speed as the demo athlete in the model race. However,if the athlete were an Olympic champ, it would be impossible for us to run with the athlete evenfor a moment as shown in Figure 2. For this reason, we were concerned with how to share the painor the feeling of exhaustion between the athlete and the viewer. After careful thought, we havedeveloped a new method that can regulate the athlete’s and viewer’s fatigue by controlling therunning speed. Using this method, the viewer can enjoy an Olympic champ in his/her own athletic
ability. Here, we introduce the concept of “workload”, which can obliquely represent how therunner feels tired.
In the following section, an athlete who runs in the field is called a “player” and a viewer who
vicariously experience the player is called a “replayer.” First, we discuss how to estimate the rateof a runner’s fatigue in section 2. Then, we introduce the automatic regulation method for RSC in
section 3. In section 4, we show experimental results in two conditions, i.e. controlled and field
conditions.

2. Estimation of a Runner’s Fatigue
The purpose of the paper is allowing to allow a replayer to share a feeling of fatigue a player in
a RSC system. First of all, we have to consider what is fatigue and how to measure it. The problems are that the simple word “fatigue” includes many definitions and also that it is not easing
measured and controlled directly. In our proposal, we introduce a method of obliquely measuring
and regulating the rate of fatigue.
2.1 Measuring Method of a Runner’s Fatigue
The best method to share a feeling of fatigue between a player and a replayer in the RSC system
is to control the running speed according to subjective answers given by both the player and theProceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

replayer. Borg proposed the “Rating of Perceived Exertion (RPE)” to evaluate the subjective rateof fatigue while doing exercise[3]. Table 1 shows an RPE table with Japanese expressions made by
Onodera[4]. Since the subject has to answer RPE orally while running, the RPE method does not fit
our purpose. However, RPE shows a high correlation with other physiological measuring methods[4,5], and so we use the method for verification purposes in a later experiments.
As automatic and direct measurement methods, physiological measuring methods such as oxygen uptake methods, blood levels of lactic acid methods, and heart rate methods, have been studied widely. The amount of oxygen uptake shows the most incisive response to the loading state
while one is exercising. Therefore, such methods have been widely used in recent training program. These methods, however, put a huge strain on the subject and require complex equipments.
Accordingly, they are unfit for RSC. Blood levels of lactic acid methods also directly show responses to loading state, but the subject must undergo blood sampling several times[6]. Heart rate
methods show the relationship to the oxygen uptake level, and the heart rate can be measured
simply[7]. These methods, however, are greatly affected by some mental factors such as strain orcompletely unexpected events.
2.2 Proposed Method
Considering these conventional methods for measuring a runner’s fatigue, we employ a pulse
rate method to estimate a runner’s fatigue obliquely. The pulse rate is only slightly affected byvessel diameter and time lag like the heart beat. However, it shows a good mutual relationshipwith the heart rate, and can be measured more conveniently without stress. It should be noted thatthe pulse rate method is also affected by some mental factors like the above heart rate methods. To
solve this disadvantage, we introduce an indirect factor called the “workload,” which indicatesthe personalized runner’s loading state while s/he is running.
Here, we introduce two assumptions, i.e. a runner who is running at a higher workload feels
more fatigue subjectively, and when two subjects are not running at the same running speed but at
the same workload, they get the same feeling of exhaustion. On these assumptions, the proposed
method measures the workload with a characteristic function that is measured beforehand in eachrunner, and it can control the workload so that both the player and replayer can share the same
feeling of exhaustion without bad effects from other factors.

Table 1 Rating of Perceived Exertion (RPE)
Borg's English scale

20
19
18
17
16
15
14
13
12
11
10
9
8
7
6

Onodera's Japanese scale

Very very hard
Very hard
Hard
Somewhat hard
Fairly light
Very light
Very very light

Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

2.3 Definition of Workload
As mentioned above, the workload indicates a personalized loading state as ones maximum
allowable limit of exercise. We defined workload by the following enumeration function of thepulse rate referencing a function of the heart rate[5].

WL =PR − PRrest
× 100 [%]
PR max − PRrest(1)

Here, PR denote the pulse rate while running, PRrest is the pulse rate at rest, and PRmax is the
maximum pulse rate while doing exercise. To measuring PRmax exactly, an exercise tolerance
test is required. In this test, the subject must run 1500 m at full stride, and the maximum pulse ratein the process is taken as PRmax. However, since the test is done under dangerous conditions, weemploy the following widely used simplification technique using the subject’s age[5].

PRmax = 220 - (Subject's Age)

(2)

After obtaining the workloads from both the player and replayer, the system can adjust theregulated rate of the exercise load of the replayer while referencing the player’s load.
2.4 Relation between Workload and RPE
We tested the first assumption that “a runner who is running at a higher workload feels more
fatigue subjectively” on a treadmill. Five subjects were asked to run on a treadmill for six minutes
at constant speeds (three speeds from 9 km/h to 15 km/h per subject). The subjects were 22-35
year old males. While they were running, we measured their pulse rates in the last one minute andasked them to judge their rates of fatigue subjectively according to the RPE table.
Figure 3 shows the results. The abscissa means the workload and the longitudinal axis means
RPE. The workload was derived from the average pulse rate with PRmax and PRrest for eachsubject. Each marker in the graph indicates a subject. The coefficient of correlation between theworkload and RPE was 0.869. The results tell us that the workload is in proportion to RPE, andalso that it is possible to estimate one’s subjective fatigue using the workload.
20
Subject M
Subject A
Subject H
Subject D
Subject U

18

RPE

16
14
12
10
8
6
0

20

40
60
80
Workload (WL) [%]100

Fig. 3 Relationship between workload by pulse rate and RPE
while running on a treadmill

Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

3. Control Method of Sharing a Feeling of Fatigue
3.1 Characteristic Function of Running Speed versus Workload
Even when two people run at same speed, they have different pulse rates, in other words, they
have different athletic abilities. Moreover, even if they were to have the same pulse rate at thesame running speed, their workloads would not always match because the workload depends on
one’s personal PRrest and PRmax. Therefore, we need to confirm the relationship between therunning speed and workload to prove the second assumption.
Yamaji reported that the heart rate increases with the running speed assuming an ordinary
running speed[8,9]. As mentioned in section 2.2, the pulse rate shows a good mutual relationshipwith the heart rate. Therefore, we expect the workload driven from the pulse rate to show same
proportionality relationship with the running speed. In an experiment, we asked four males subjects 24-34 year old to run on a treadmill for six minutes at three constant paces. We did the sametest three times on different days to evaluate the variations of the individuals.
Figure 4 indicates results between the running speed and the workload for subject A. The
workload was derived from the average pulse rate in taking the last one minute of each trial. The
graph indicates that there was a proportionality relationship between the running speed and workload. Results from the other subjects showed the same tendency, but they were omitted due tospace limitations. We draw the running speed - workload linear regression line in the graph. The
result on 14 Feb. slightly deflected from the line. We considered that the workload was affectedby the subject’s physical condition because he mentioned that he felt below par during at that day.
Furthermore, some results of the workload during high speed running got up to 100%. Since thesubject jogged regularly, we considered that his true PRmax might have actually larger than PRmax
estimated from his age.
We confirmed that the workload derived from the pulse rate correlated with the subjective
feeling of fatigue in section 2.4. Therefore, if we can adjust the pulse rate by controlling therunning speed, it is likely that we can make a player and a replayer of the RSC system share a
feeling of fatigue. The running speed - workload line can be defined as

WL = a RS + b(3)Here, RS is the running speed and WL is the workload. The regression coefficients a and b can
be regarded as personalized parameters of one’s athletic ability. Generally, the personalized running speed - workload line of an excellent runner is mapped at a lower position in the graph. By
obtaining these parameters before RSC use, we can expect a player’s feeling of fatigue to beWorkload (WL) [%]

120
100
80
60
14 Feb.

40

18 Feb.

20
0

22 Feb.

4

8

12

16

Running speed (RS) [km/h]

Fig. 4 Relationship between workload and running speed.

Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

Workload [WL]

Replayer's line
100
Player's line
WL R
,WLP

0 0

RS R RSP
Running speed

Fig. 5 Method of workload control
estimated and a replayer’s feeling of fatigue to be controlled with the running speed.
3.2 Method for Sharing a Feeling of Fatigue
Now, we propose our method for sharing a feeling of fatigue between a player and a replayer.
In the following discussion, we call the running speed of the player as the “virtual speed” and themodified running speed of the replayer as the “real speed.”
Two modification methods can be proposed. One is to control the real speed using a feedback
loop from the workload. In this case, the reference is the player’s workload that is measured witha pulse rate sensor in real time. The pulse rate of the replayer is also measured in real time, and thefeedback controller adjusts the real speed of the replayer to reduce errors in the workload of both.
The second to modify real speed of the replayer using previously measured running speed - workload lines of the player and replayer. In this case, a speed controller at the replayer’s side justrefers to the running speed of the player in real time.
Using the former feedback method, the feedback controller can exactly match the workload of
the replayer. However, it is feared that the feedback loop can be greatly affected by some mental
factor such as strain or completely unexpected events. On the other hand, the latter method does
not use the pulse rate while running, so it leaves such errors. However, the player and replayer
have to have initiation phase to get the speed-workload line before using RSC. Since we consider
the stability to be important in RSC, we employ the latter method.
We shall discuss the details of the method. Equation 4 denote the personalized speed-workloadline of the player and replayer as:

WLP = aP RSP + bP
WLR = aR RSR + bR

(4)Here, the lower right subscripts “P” and ‘R” indicate the data names of the player and replayer.
Figure 5 helps to understand the method. We intend to match workloads, so

WLP = WLR
Consequently, substituting equation 4 for equation 5, we get

Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

(5)

RSR =aP RSP + bP − bRaR

(6)

By obtaining the parameters beforehand, the real speed of the replayer (RSR) can be calculated
from the virtual speed of the player (RSP) by equation 6.

4 Validity Evaluation
We tested the proposed method with a trial RSC. In the first experiment, we checked the method
using controlled conditions in a lab, and then we applied the method for a model road race.
4.1 Controlled Conditions
To evaluate the effectiveness of running speed modification for sharing a feeling of fatigue
purely, we did not give the subject any visual or auditory information in the experiment. We set
one subject as the player and the other subjects as replayers on the assumption that the player
would runs at a constant virtual speed.
In advance, we measured the athletic abilities of all subject with the method mentioned in
section 3.1. We had five male subjects 24-35 year old. They were divided into two groups, i.e.
higher ability group (Subjects D, U, H, called group 1) and lower ability group (Subjects M andA, called group 2) according to their athletic abilities.
At first, the subjects of group 1 were asked to try the system as replayers while regarding
subject A as player. Subject A showed the worst athletic ability, so the subjects in group 1 run atincreased speeds to share the same feeling of fatigue. Correspondingly, the subjects in group 2
were asked to act as replayers while regarding subject D as the player. Subject D has the best
athletic ability, so the subjects of group 2 had to run at reduced speeds. The virtual speed of theplayer was set as 8, 9, 10 km/h for group 1 and as 8,10, 12 km/h for group 2. The modified real
speed of each subject is listed in Table 2. The subjects ran at these speeds for six minutes on atreadmill. We did not give any visual or sound information enabling the subject to presume theplayer’s status. After the running, we measured the pulse rate in the last one minute and calculated
the workload from the average pulse rate by using equation 1.
The results are shown in Figure 6. The markers indicate the workloads of the individual subjects. The abscissa denotes the virtual running speed and the longitudinal axis means the workload. The lines indicate the regression lines of each subject. The gray solid line indicates theproperty of the player, the black solid lines indicate results of the subjects while they ran at modiTable 2 Modified running speeds under controlled conditions(a) Real speed of group 1
Virtual speed[km/h]

Subject 8.0
D 10.2
U
9.1
H
8.9

9.0
12.2
10.7
10.4

10.0
14.1
12.3
11.8

(b)Real speed of group 2
Virtual speed[km/h]

Subject
MA

8.0 10.0
7.5
8.5
6.8 7.8

12.0
9.5
8.9

Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

120

120

Subject-D(Virtual speed)

Subject-A

Subject-H(Virtual speed)

Subject-H
Subject-U

80
60

Subject-D

40

Subject-A(Virtual speed)

100PlayerWorkload (%)

Workload (%)

100

Subject-M

80
60

Subject-M(Virtual speed)

40

Subject-U(Virtual speed)

Player

20

20

0

0
4

8
Running speed

12

16

4

Fig. 6-a Workload versus runningspeed in group 1

8Running speed[km/h]

12

16[km/h]

Fig. 6-b Workload versus runningspeed in group 2

fied speeds, and the gray dotted lines indicate the original properties of the subjects.
The graphs tell us that the lines of subjects A, M, D ,and H are matched with the line of thetarget player except for subject U. As mentioned in section 2.3, the workload well correlates withsubjective feeling of fatigue, so we consider that a subject can share a feeling with a player by the
proposed method under the controlled condition in a lab.
In addition, the result of subject U means that he actually ran on easier terms than the player.
According to oral statements made by the subject, his physical condition improved a great deal
more in this test than when we measured his property. Therefore we examined whether the result
was affected by fluctuations in the physical conditions. To minimize the effects, we need to develop a new method to measure the personal property of one’s athletic ability instantly. By using
such a method just before a race, both a player and a replayer can share their physical status moreexactly.
4.2 Road Conditions
We also evaluated the effectiveness of the proposed method under road conditions. We recruited long-distance athletes university students and set them in a model marathon race. The race
course was designed on a road around our lab, one lap was 1.4 km, and we asked the runners to
run two laps. They were also asked to run as if running in a real marathon race without any whatif scenarios. One of them wore the sensor unit. The sensor recorded his pace, pulse rate, and what
he was watching and hearing while running. Moreover, we asked him to rate the RPE per section
after the race. His time was 566 seconds, so the average speed is 18.1 km/h. His running speed ateach point was estimated by a post process[2]. Moreover, the property of his athletic ability wasmeasured in the same way as mentioned above.
Using the data, we then asked six subjects, four males and two females from 18 to 24 years old,to try the race as replayer regarding the runner who wore the sensors as the player. Their properties of athletic abilities were measured beforehand, and preset into the system. In the experiment,
the visual image and sounds recorded by the sensors wore by the runner were displayed on the 75
inch projection monitor attached just in front of the treadmill. All of the subjects were not trained,
so their abilities were lower than the athlete’s. Therefore, they ran at slower speed than the actual
running speed of the athlete.
Four of the subjects could run the whole distance, but two of them (subject B and D) abandoned
race by themselves due to cramps in the legs. We considered that the reasons for these failures toProceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

120

Workload [%]

100

80

60

RunnerSubject-A
Subject-B
Subject-C
Subject-D
Subject-E
Subject-F

40

20

0

100

200

300

400

500

600

700

Time (sec)Fig. 7 Time series of workload under model race
20
RunnerRPE (%)

18

Average
RPE fromfinishers

16
14
12
Course map

10Start/Goal

(A)

(F)

(E)

(B)

8

1.4km/lap

(C)

(D)

6
-AStart

-B-

-C-

-D-

-E-

-F-

-A2- -B2- -C2- -D2- -E2-

-F2GoalFig. 8 RPE of a player and replayers under model race
be insufficient warming-up exercises or excessive estimation of their properties. Figure 7 shows
the time series of the workloads of the player and subjects. Although there is slight discordancearound the first rising phase and just before the goal, the workloads of all of the subject are about
equal to the player’s level. One finding of the experiment was that the subjects could share thesame workload level as the athlete by the proposed method.
By checking the agreement between the workload and subjective feeling of fatigue in the model
race, we also asked the subjects about their subjective rates of fatigue by RPE. Because it was
impossible to ask them for RPE while they were running, they answered about RPE while relying
on memory after they finished the race. According, the results were not exact results. We supposea margin of error of 2 RPE rating scale as acceptable: If the player gives a rating of “hard,” theacceptable range is from “very hard” to “somewhat hard.” Figure 8 shows RPE by athlete andaverage RPE of the subject who could run the whole distance. The abscissa denotes the section
where they ran on the course, the error bar means standard variation and the gray hatching area
indicates the error margin. The graph shows that subjective rates at seven sections were included
within the margin, that is to say, the subjects could share a feeling of fatigue with the athlete inthese sections using our method. Moreover, we considered that the reason for the wide discrepancy shown in start phase, was because the subjects had not yet gained familiarity with the treadmill in this phase, and the same tendency has been shown in workload in Figure 7, also. As for theProceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE

discrepancies in section F and section C in the second lap, we are unable to offer explanations at
this time. Comparing the previous results under the controlled conditions, we considered that
other factors such as visual or auditory information, race process, and level of familiarity might
affect the subject also.

5. Conclusion and Future WorkWe are concerned with how to share a feeling of pain or exhaustion between an athlete and aviewer in RSC. At first, we introduced the concept of workload as a personalized loading state ofone’s maximum allowable limit of exercise, and defined workload as a function of the pulse rate.
Then, we confirmed that the workload was in proportion to RPE, which indicates the rate of therunner’s fatigue, and also that it can be adjusted by controlling the running speed from experimental results. Using these results, we developed a new method that is able to regulate the workload of
an athlete and a viewer by controlling the running speed.
We conducted an experiment to the evaluate effectiveness of the method on a pilot system. We
could confirm that subjects who are not trained runner can share the workload with the longdistance athlete.
In the experiment, one subject showed a wide discrepancy in workload matching test. We
considered that this was because fluctuations in his physical condition. To minimize the effects,
we need to develop a method to measure the personal property of one’s athletic ability instantly inthe future. In addition, we employed a widely used simplification technique using the subject’s
age and a magic number. The technique, however, caused slight misses in estimations because
the technique is developed aimed at ordinary people, not at trained athletes. So that, it is also
necessary for the RSC to more exactly and instantly get the maximum pulse rate for the RSC user.

Reference[1] Kosuge, Noma, Miyasato, “New Method for Realistic Sports Casting based on Player’s View”,
Correspondences oh Human Interface, Vol. 1 No. 2, pp. 29-34, 1999, (in Japanese).
[2] Sugihara, Noma, Miyasato, Kawai, “Experimental Evaluation for Virtual Marathon System”,
Proc. of Human Interface Symposium 2000, pp. 415-418, 2000, (in Japanese).
[3] Borg, “Psychophysical bases of perceived exertion.”, Med. Sci. Sports Exerc. 14: pp. 377381, 1982.
[4] Onodera, Miyashita, “A Study on Japanese Scale for Rating of Perceived Exertion in Endurance
Exercise”, Jap. J. Phys. Educ., Vol. 21, No. 4, pp. 191-203, 1976, (in Japanese).
[5] Yamaji, Yokota, Onishi, Yamamoto, Yokoyama, “A Comparison of the Perceived and the ECF
Heart Rate in the Two or Three Different Modes before and after Perceived Heart Rate Training”,
The Bulletin of the Department of Education of Toyama University - B, No. 40, pp. 49-61, 1991,(in Japanese).
[6] Schurch “Diagnosis of sports ability, theory & fact (Japanese edition)”, Ohmsha, Ltd, pp. 532, 1991.
[7] Shaver, “Essentials of Exercise Physiology: Section Three, The Heart and Exercise”, Burgess
Publishing Company, pp. 74-93, 1981.
[8] Yamaji, Tanaka, Kitagawa, “Significance of The Heart Rare Threshold, Deflection Velocity,
HRmax and Running Time to exhaustion in Conconi’s Test”, JJBSE, Vol. 1, No. 3, pp. 228236,1997, (in Japanese).
[9] Conconi, Ferrari, Ziglio, Droghetti, Codeca, “Determination of the Anaerobic Threshold at a
Noninvasive Field Test in Runners”, J. App. Physiol., Respirat Environ Exercise Physiol 52, pp.
869-873, 1982.

Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01)
0-7695-1402-2/01 $17.00 © 2001 IEEE