Effect of calf muscle fatigue on person
identification using video sequences of body sway
Masaya Kojima∗ , Masashi Nishiyama∗ , Yoshio Iwai∗∗ Graduate School of Sustainability Science, Tottori University, Japan

Abstract—Body sway, which is a slight movement that naturally occurs when a person stands, has attracted attention as
an informative cue with which to identify people. In the field of
kinematics, analytical studies have revealed that the movement ofbody sway varies when calf muscles are fatigued. However, thestudies have not revealed whether the accuracy of identifying
people is reduced by calf muscle fatigue. We investigate theidentification accuracy for body sway signals taken from video
sequences when fatiguing the target’s calves. We find that thepower spectral density of each frequency band of body sway
is increased by fatigue and that the identification accuracy issignificantly decreased. We also find that the magnitude of thevelocity of body sway is greater in post-fatigue movement thanin pre-fatigue movement.
Index Terms—Body sway, Muscle fatigue, Calf, Identification,
Video sequences

Identification
CameraIdentification
CameraCameraCameraBody sway

Body sway

Body sway

Body sway

Pre-fatigue
QueryPre-fatigue
TargetPost-fatigue
QueryPre-fatigue
TargetI. I NTRODUCTION
There is a demand for person identification technology
using surveillance cameras with the aim to achieve a safe
and secure society [1]–[3]. The accurate identification ofpeople requires informative characteristics representing individuals to be obtained from video sequences acquired withsurveillance cameras. As one such characteristic, the bodysway of a standing person has recently attracted interest fromresearchers. Body sway is a slight movement that occurs
naturally even when the human body is stationary, as describedin [4]. Kamitani et al. [5] demonstrated that the time-series
signals of body sway are a characteristic that can be usedeffectively for person identification. However, they evaluatedthe identification accuracy under the condition that there wasno muscle fatigue.
The human body enters a state of muscular fatigue afterstrenuous exercise. This muscle fatigue causes changes in bodysway, as described in analytical studies of body kinematics [6],[7]. Thus, muscle fatigue might reduce the accuracy of person
identification using body sway signals, as shown in Fig. 1.
As an example, we might consider the case that a person
is incorrectly identified as a different person after engaging
in strenuous exercise at a fitness club or gym. However, therelationship between the accuracy of person identification andthe muscle fatigue of body sway has not been investigated.
In this paper, we evaluate the effect of muscle fatigue on theaccuracy of person identification using time-series signals ofbody sway. In terms of muscle fatigue, we consider the casethat a person’s calves are strongly loaded. In the following,
we refer to the state before calf loading as the pre-fatigue
state and the state after calf loading as the post-fatigue state.

Fig. 1. Effect of calf muscle fatigue on the accuracy of person identification
using body movements. The identification accuracy changes significantly
depending on the presence or absence of muscle fatigue.

Our evaluation results reveal that the identification accuracy issignificantly worse for post-fatigue query samples than for prefatigue query samples. The power spectral densities computedfrom the time-series signals of body sway show significant
differences among the components in various frequency bands.
We also find that the magnitude of the velocity of movement
for body sway in various directions is greater post-fatigue than
pre-fatigue.
II. R ELATED WORKIn the field of body kinematics [6]–[9], the relationshipbetween muscle fatigue and body sway has been analyzed
when a specific body part is loaded. When a load is applied to
the neck [9], there is no significant difference in the location of
the center of pressure (CoP) of body sway. Additionally, there
is no significant difference in the velocity of the CoP appears
when a load is applied to the upper arm [6]. In contrast, thevelocity of the CoP increases significantly when a load isapplied to the groin [8] or the calf [6], [7]. Body sway thus
varies depending on which body part is fatigued. In particular,
the slight movement of body sway is greatly affected by a load
applied to the lower part of a person’s body. In this paper, weinvestigate the effect of the fatigue of a body part of the lower
body on the accuracy of identification using body sway. We
used an overhead camera to observe body sway signals though
the analytical studies [6]–[9] used a footplate pressure sensor.

Time

・・・

S1. Acquire a color video
sequence
Color video sequenceTime
Fig. 2. Heel lift exercise in our experimentsTime
S2. Detect a silhouette
video sequence of thehead region

III. E VALUATION METHOD

Silhouette video sequence

A. Fatiguing a body partWe focus on the calf as the body part that is fatigued. The
gastrocnemius and soleus muscles are located under the skin of
the calf. As described in [10], the gastrocnemius and soleus
muscles are known as the triceps surae and are responsible
for flexing the ankle joint and maintaining a person’s standing
posture.
We evaluate the following hypothesis, assuming that the prefatigue features are stored as target samples.
• When post-fatigue features are input as query samples,the identification accuracy is lower than that in the casethat pre-fatigue features are input as query samples.

S3. Determine
the silhouette imageat the reference time
Silhouette imageat the reference timeTime
S4. Generate a
difference-value
video sequence
Difference-value
video sequence

B. Procedure of fatiguing the calves

Time

Front
S5. Divide the video
sequence
into local blocks

Left

Right
Back

S6. Calculate
the amount of movement
for each local block

Amount ofmovement

We describe the procedure of fatiguing the calves. Initially,
we had the participants stand on a floor without shoes. As
shown in Fig. 2, we had the participants perform a heel lift
exercise in which they alternately raised and lowered theirheels with a constant rhythm. The participants repeated thisexercise until they reached the post-fatigue state. The exercise
was terminated if a participant failed to match the constantrhythm in three attempts or voluntarily called a halt to theexercise. We instructed the participants to not bend their knees
as much as possible and raise their heels as high as possible
so as not to apply a large load to body parts other than thecalves.

Left Back Right Front

Time

Person identification is a technique for determining whether
a person in a query video sequence is contained in thetarget dictionary. To obtain high identification accuracy, it isimportant to extract features from the video sequences of bodysway. In this paper, we extract spatio-temporal features by
calculating the amount of movement of body sway at eachtime from the video sequence using an existing method [5].
Figure 3 shows the procedure of feature extraction, the detailsof which are described below.
In S1, we obtain a color video sequence of body sway using
a commercially available camera. To observe the head region,
which is the body part that moves the most in body sway,
we place a camera above the head. We assume that a person
stands in an upright posture under the overhead camera. In
S2, we detect a silhouette video sequence of the head region

S7. Extract a feature

Log power

C. Feature extraction for person identification
Left Back Right Front

Frequency
Fig. 3. Overview of feature extraction for person identification using thevideo sequences of body swayfrom the color video sequence. A pixel value is set at 1 for thehead region and 0 otherwise. In S3, the reference time of eachsilhouette video sequence is determined. In the silhouette video
sequence, we find a representative silhouette image similar to
the remaining silhouette images at various times. The time ofthis representative silhouette image is used as the referenceCameraParticipant

1.5 m

2.3 m

Landmark

2.7 m(a) Upright posture(b) Environment

Fig. 5. Part of a color video sequence acquired with the overhead camera
Fig. 4. Experimental setting

time. In S4, we generate a difference value image to extract
the movement of the head region at each time. We computethe difference in pixel values between the silhouette imageat each time and the representative silhouette image at thereference time. In S5, we divide the difference value image
into local blocks to observe the spatial difference in head
shape. Specifically, we divide the head region into I blocks
radially from the center of the head region. In S5 of Fig. 3,
the number of local blocks I = 4 is used as an example.
In S6, we calculate the amount of movement at each time to
extract the temporal change in the head region. In each local
block, the sum of the difference values is obtained and used
to represent the amount of movement at each time. In S7,
we extract features for person identification. In each block,the power spectral density (PSD) is calculated from the timeseries signal of the amount of movement. The PSDs of all
blocks are concatenated to determine the feature vector for
identification.
IV. E XPERIMENTS
A. Dataset
We acquired pre-fatigue and post-fatigue video sequences toevaluate the effect of calf fatigue on identification accuracy.
Twenty participants (age: 22.6 ± 1.5 years, height: 168.3 ±5.6 cm, weight: 60.4 ± 10.9 kg) participated in the experiment.
The participants wore the same work clothes and did not wear
shoes. We instructed the participants to maintain the upright
posture shown in Fig. 4(a), stand under the camera, and gazeat a landmark. The alignment of our experimental setting isshown in Fig. 4(b). We show part of a color video sequencein Fig. 5 and part of a silhouette video sequence in Fig. 6.
In the following procedure described in Fig. 7, we acquired
two pre-fatigue video sequences and one post-fatigue video
sequence. We refer to the first pre-fatigue video sequence as
VB1 , the second pre-fatigue video sequence as VB2 , and thepost-fatigue video sequence as VA . The time length of each
video sequence was set at 120 s. Each participant took a 60second break between VB1 and VB2 and another 60-second

Fig. 6. Part of a silhouette video sequence of the head region

break between VB2 and the heel lift exercise. We started theacquisition of VA within 15 seconds of the end of the exercise.
We used a camera having a resolution of 1920 × 1080 pixels
and a frame rate of 30 fps. The video sequence was cropped
to 300 × 300 pixels, centering on the head region.
We here give the statistics of calf fatigue for the heel lift
exercise described in Section III-B. The constant rhythm of theheel lift was set at 100 times per minute. Different participants
were able to raise their heels a different number of times, as
shown in Fig 8. More than half of the participants completed
the heel lift exercise fewer than 400 times. In contrast, five
participants completed the heel lift exercise more than 800
times.
B. Accuracy of person identification
We first describe the setup of our experiments for theidentification method described in Section III-C. The featuresof VB1 were used as the pre-fatigue target samples, the featuresof VB2 as the pre-fatigue query samples, and the features
of VA as the post-fatigue query samples. On the basis of
the hypothesis described in Section III-A, we calculated theidentification accuracy when the pre-fatigue VB2 and postfatigue VA were input. The identification accuracy depends

120 s
120 s
PrePre60 s
60 s
fatiguefatigue
𝑉𝐵1
𝑉𝐵2
Target60 - 1200 s
Heel lift
exercise

120 sWithin Post15 s
fatigue
𝑉𝐴

Query

TABLE II DENTIFICATION ACCURACIES FOR PRE - FATIGUE AND POST- FATIGUEQUERY SAMPLES . S IGNIFICANCE AT THE LEVEL OF p < .01 IS DENOTEDBY **.
Pre-fatigue query
92.2 ± 2.9 %Query

Log power

864(800 - 1000]

Fig. 8. Histogram of the number of heel lifts performed by each participant
in reaching calf muscle fatigue

on the individuals included in the target samples, and we thus
generated subsets of 18 individuals randomly selected fromthe 20 participants. We generated 10 subsets and calculated themean and standard deviation of the identification accuracy forthe subsets. The accuracy metric was the rank-1 correct-match
rate. We used the nearest neighbor based on the Euclidean
distance for identification.
Table I gives the accuracy when using pre-fatigue query
samples and post-fatigue query samples. We found that theaccuracy was about 59 points lower for the post-fatigue querythan for the pre-fatigue query. A statistical test was performed
to determine if the decrease in the identification accuracy wassignificant. Specifically, we used the Mann–Whitney U test,with a significance level of p < .01. As a result, we confirmedthat there was a significant difference in the accuracy between
pre-fatigue and post-fatigue states.
C. Visualization of changes in features
To investigate the reason for the decrease in identification
accuracy, we compared the pre-fatigue feature VB2 and thepost-fatigue feature VA . We used the feature extraction method
described in Section IV-B. The difference between the postfatigue feature and pre-fatigue feature was calculated for eachparticipant at each frequency. The number of blocks was set tofour, as shown in S5 of Fig. 3. We computed the absolute value
differences between the features for each of the four blocks.
Figure 9(a) shows the average of the pre-fatigue feature, (b)shows the average of the post-fatigue feature, and (c) shows
the absolute value difference between the pre-fatigue and postfatigue features. In the front block, we find that there werelarge differences in the components of various frequencies.

Log power

Number of times thateach participant was able to raise his or her heel

Statistical test
∗∗

14
Left
Back
Right
Front
12
10
8
6
4
2
0
-2 DC
15DC
15DC
15 DC
15
Frequency [Hz](a) Pre-fatigue feature

14
Left
Back
Right
Front
12
10
8
6
4
2
0
-2 DC
15DC
15DC
15 DC
15
Frequency [Hz](b) Post-fatigue feature

Difference in log power(600 - 800]

(400 - 600]

0(200 - 400]

2

(0 - 200]

Number of participants

Fig. 7. Procedure of video sequence acquisition

Post-fatigue query33.3 ± 3.7 %

1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0

Left

DC

Back

15DC

Right

15DC
15 DC
Frequency [Hz]

Front

15

(c) Difference between pre-fatigue and post-fatigue

Fig. 9. Comparison of average pre-fatigue and post-fatigue features

The same tendency is observed in the left, right, and backblocks. We thus believe that calf fatigue had an effect on bodysway and accounts for the decrease in identification accuracy.
V. E FFECT OF CALF FATIGUE ON BODY SWAY
A. Purpose
We further investigated what changes in body sway arecaused by calf fatigue. A previous study [6] analyzed theeffect of calf fatigue on body sway in mediolateral (ML)and anterior–posterior (AP) directions. Observations in the
ML direction correspond to viewing the participant from theRight(a) ML (left, right)

Back(b) AP (front, back)

Velocity magnitude[number of pixel / s]

Left

**

500
400
300
200
100
0

Velocity magnitude[number of pixel / s]

Front

500
400
300
200
100
0

Fig. 10. Local blocks for MP and AP directions

side whereas observations in the AP direction correspond toviewing the participant from the front. The analytical study [6]found that calf fatigue significantly increases the magnitudeof the velocity of movement in the ML and AP directions.
However, the study used a foot pressure sensor to observe themovement of body sway. In contrast, we used a camera toobserve the body sway.
B. Calculation of the magnitude of the velocity of movement
We describe a method for extracting the magnitude ofvelocity of movement in body sway from imagery taken witha camera. Feature extraction is first performed following S1 to
S6 in Fig. 3, and the amount of movement is then extracted.
We compute the magnitude of the velocity of movement by
taking the absolute value after differentiating the amount of
movement at each time. We analyze the magnitude of thevelocity of movement in ML and AP directions following theanalytical study [6]. In the ML direction, the magnitude of thevelocity of movement is determined by adding the left-block
magnitude and the right-block magnitude, as illustrated in
Fig 10(a). In the AP direction, the magnitude of the velocity of
movement is determined by adding the front-block magnitude
and the back-block magnitude, as illustrated in Fig 10(b).
C. Experimental results
We compared the magnitude of the velocity of movement
between pre-fatigue VB2 and post-fatigue VA , targeting theparticipants used as a query for the identification in Section IV-B. Figure 11 compares the magnitude of the velocity
of movement between pre-fatigue and post-fatigue states in
ML and AP directions. We performed the Mann–Whitney U
test with a significance level of p < .01. We see a significant
difference in the magnitude of the velocity of movement
between pre-fatigue and post-fatigue states for both ML and
AP directions. In agreement with the result of the analytical
study [6], the magnitude of the velocity of movement was
higher in the post-fatigue state than in the pre-fatigue state.
We consider that this difference increased the PSD in eachfrequency band, as described in Section IV-C.
VI. C ONCLUSIONS
We evaluated the effect of muscle fatigue on the accuracyof person identification using video sequences of body sway.
The evaluation results revealed that the identification accuracy
was significantly worse for the post-fatigue state than forthe pre-fatigue state. We confirmed that the power spectral
densities computed from the time-series signals of body sway

Pre-fatigue
Post-fatigue(a) ML**

Pre-fatigue

Post-fatigue(b) AP
Fig. 11. Comparison of the average velocity magnitude between pre-fatigue
and post-fatigue states. Significance at the level of p < .01 is denoted **.

show significant differences among the components in various
frequency bands. We also confirmed that the magnitude of thevelocity of movement in body sway was greater post-fatigue
than pre-fatigue when observed in AP and ML directions.
As future work, we intend to develop a robust method
against calf fatigue that will increase identification accuracy
and extend the evaluation of the effects of muscle fatigue todifferent body parts.
R EFERENCES[1] A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometricrecognition,” IEEE Transactions on circuits and systems for video
technology, vol. 14, no. 1, pp. 4–20, 2004.
[2] X. Wang, “Intelligent multi-camera video surveillance: A review,”
Pattern Recognition Letters, vol. 34, no. 1, pp. 3–19, 2013.
[3] R. Min, J. Choi, G. Medioni, and J. L. Dugelay, “Real-time 3dface identification from a depth camera,” Proceedings of the 21stInternational Conference on Pattern Recognition, pp. 1739–1742, 2012.
[4] A. Bottaro, M. Casadio, P. G. Morasso, and V. Sanguineti, “Body sway
during quiet standing: Is it the residual chattering of an intermittent
stabilization process?,” Human Movement Science, vol. 24, no. 4, pp.
588–615, 2005.
[5] T. Kamitani, H. Yoshimura, M. Nishiyama, and Y. Iwai, “Temporal andspatial analysis of local body sway movements for the identification ofpeople,” IEICE Transactions on Information and Systems, vol. 102, no.
1, pp. 165–174, 2019.
[6] Y. Gimmon, R. Riemer, L. Oddsson, and I. Melzer, “The effect ofplantar flexor muscle fatigue on postural control,” Electromyography
and Kinesiology, vol. 21, no. 6, pp. 922–928, 2011.
[7] N. Vuillerme, N. Forestier, and V. Nougier, “Attentional demands andpostural sway: the effect of the calf muscles fatigue,” Medicine andScience in Sports and Exercise, vol. 34, no. 12, pp. 1907–1912, 2002.
[8] E.J. Bisson, D. McEwen, Y. Lajoie, and M. Bilodeau, “Effects of ankle
and hip muscle fatigue on postural sway and attentional demands during
unipedal stance,” GAIT POSTURE, vol. 33, no. 1, pp. 83–87, 2011.
[9] M. Schieppati, A. Nardone, and M. Schmid, “Neck muscle fatigue
affects postural control in man,” Neuroscience, vol. 121, no. 2, pp.
277–285, 2003.
[10] J. Piscopo and J. A. Baley, Kinesiology The Science of Movement, John
Wiley & Sons Inc, 1981.