Calf Fatigue Recognition in Heel-lift Exercise
Using Video Sequences of Body Sway
Takuya Kamitani∗ , Masaya Kojima∗ and Masashi Nishiyama∗∗ Graduate School of Engineering, Tottori University, Tottori, Japan, {takuya.kamitani,nishiyama}@tottori-u.ac.jp

Calf fatigue
level is 75%

Heel-lift exercise
OverheadcameraFeedback
Calf fatigue recognition
Left-right

Front-back

Abstract—Previous analytical studies have investigated therelationship between calf fatigue and body sway measured using
a force plate. However, they did not consider multiple levels ofcalf fatigue. Here, we propose a method for recognizing multiple
levels of calf fatigue based on video sequences of body sway
acquired using an overhead camera after the heel-lift exercise.
For calf fatigue recognition, we extract a feature of body sway bygenerating a time-series signal of the head center position in theleft-right and front-back directions based on medical knowledge.
To evaluate the accuracy of our method, we created a datasetof 100 video sequences (20 participants × 5 calf-fatigue levels).
The results show that our method can correctly recognize thecalf-fatigue level with an accuracy of 40.0±1.8%. Furthermore,
we demonstrated that for calf fatigue recognition, the accuracyof our method is superior to those of existing methods designed
for human action recognition.

Observation

Body sway
measurement

Classification

I. I NTRODUCTION
The calf, which plays a signiﬁcant role in pumping blood tothe heart [1], can sometimes cause poor blood circulation [2].
The heel-lift exercise can be performed to improve poor blood
circulation in the calf. Here, we develop a method that gives
feedback on the fatigue level of calf muscles after the heel-lift
exercise based on video sequences acquired using an overheadcamera, as shown in Fig. 1.
To realize such feedback, a method for accurately recognizing the calf-fatigue level after the heel-lift exercise is required.
In this paper, body sway measured from video sequences isused for calf fatigue recognition. Previous analytical studies [3], [4] have investigated the relationship between calf
fatigue and body sway measured using a force plate. They
found that several factors of body sway show different traits
depending on the presence or absence of calf fatigue. However,these studies did not consider multiple levels of calf fatigue
on body sway.
In this paper, we propose a method for recognizing the calffatigue level based on body sway measured from video sequences acquired after the heel-lift exercise. Five levels of calf
fatigue are quantitatively represented via linear approximation
based on the maximum number of heel lifts that an individual
can perform. We design a method that integrates medical
knowledge [3], [4] to extract features from the video sequencesof body sway. Speciﬁcally, we extract a feature of body sway
for calf fatigue recognition by generating a time-series signalof the head center position in the left-right and front-backdirections. To evaluate the proposed calf fatigue recognition
method, we created a dataset of video sequences of body swayfor various calf-fatigue levels for 20 participants (18 males

Fig. 1. Example of providing feedback on fatigue level of calf muscles using
calf fatigue recognition.

and 2 females). This paper validated calf fatigue recognition
based on video sequences of body sway acquired after the
heel-lift exercise. The experimental results conﬁrmed that our
method improves the accuracy of calf fatigue recognition by
integrating medical knowledge into feature extraction.
II. P ROPOSED M ETHOD
A. Calf-Fatigue Level Representation
For calf fatigue recognition, it is necessary to quantitatively
represent the calf-fatigue level. Previous analytical studies [3],[4] represented calf fatigue in a binary manner, namely thepresence or absence of calf fatigue. In this paper, we represent
calf fatigue using multiple levels by assuming that the calffatigue level is directly proportional to the maximum numberof heel lifts performed by an individual. The calf-fatigue level
ranges from L0 to L100 . A level of L0 indicates that there
is no fatigue in the calves (i.e., the person has refrained
from vigorous exercise for a certain period). A level of L100
indicates that a person has just performed their maximum
number of heel lifts. A level of L25 to L75 indicates that aperson has just performed the heel-lift exercise up to 25% to
75% of their maximum number of heel lifts, respectively.
B. Validation Conditions of Heel-lift Exercise
We controlled the inﬂuence of the calf-fatigue level on bodysway by setting the conditions of the heel-lift exercise as

C. Procedure for Calf Fatigue Recognition
There are no existing methods for calf fatigue recognition using video sequences of body sway (see Section I).
A method [6] that uses video sequences of body sway torecognize the weight of baggage held by a person has beenreported. However, we do not expect this method to have highaccuracy in calf fatigue recognition because it was designed to
distinguish baggage weight based on body sway (i.e., it lacks
information on calf fatigue).

Video sequences Head regions

Left-rightCalculation of center
position from head region

Generation of time-series
signals from center
positions in 𝑥 and 𝑦 axes

Generation of PSD foreach time-series signal
Classification of calf
fatigue level based on
PSD feature

Front-back

Estimation of head regionin video sequences

Center position distribution
Left-right

Front-back

Time

Time

Leftright

FrontbackDistance

Initialization of calf fatigue level
The calves of the participant have no fatigue. A previous
analytical study [5] reported that muscle loading causes
muscle pain within 48 hours and that this muscle pain
continues for 5 to 7 days. Here, we assume that muscle
fatigue disappears when the associated muscle pain disappears. Thus, we set the rest period for the calves to ten days
to allow as much muscle pain as possible to disappear.
Speed of heel lift
We set the heel lift speed to 100 beats per minute. We
placed a device that emits a sound at a constant beat, similar to a metronome, near the participant. The participants
were asked to raise and lower their heels in sync with thissound.
Height of heel lift
We set the heel lift height to 30% of each participant’s
foot length. We instructed the participants to follow thefollowing three guidelines during the heel-lift exercise. (i)The head should touch a board placed at a height of the
body height plus 30% of the foot length when the heels
are lifted to prevent heel lift height variation. (ii) The eyes
should be kept on the designated marker during the heellift exercise to prevent height variation due to changes inhead orientation. (iii) Bending of the knees while lifting
the heels should be avoided to prevent height variation.
Stop condition for heel-lift exercise
We set two stop conditions for the heel-lift exercise.
The participants were asked to stop the heel-lift exercise
when at least one of the conditions was met. The ﬁrst
condition was that the participant could not reach the headboard with their head three consecutive times or that theparticipant voluntarily decided to stop the exercise. The
second condition was that the participant performed apredetermined number of heel lifts (associated with L25 ,L50 , or L75 ).
Order for observing calf-fatigue levels
We determined the following order for observing calffatigue levels. First, we observed L0 . Then, we observed
L100 and counted the maximum number of heel lifts
required to reach this level. Then, we observed L25 , L50 ,and L75 . To prevent the inﬂuence of previous calf-fatigue
levels on the observation of the fatigue level of interest,
we set the rest period between observations to 10 days.

Center position

Acquisition of video
sequences of body sway
using overhead camera

PSD

follows.

Frequency

・𝐿0 ・𝐿25 ・𝐿50
・𝐿75 ・𝐿100
Classifier Calf fatigue level

Fig. 2. Procedure of proposed method for calf fatigue recognition withmedical knowledge integrated into feature extraction.

Here, we integrate medical knowledge on calf-fatigue levels
reported in previous analytical studies [3], [4] into the feature
extraction of an existing recognition method [6]. Previous analytical studies represented body sway as a time-series signalof the center of pressure (CoP) measured using a force plate.
They reported that the left-right and front-back componentsof the CoP signals show different traits depending on thecalf-fatigue level. In the proposed method, we use the center
position of a participant’s head region instead of the CoP. We
hypothesize that the left-right and front-back components of
the head center position also show different traits dependingon the calf-fatigue level. We expect that extracting featuresfrom the left-right and front-back movements of the time-series
signals of the head center position separately will improve theaccuracy of calf fatigue recognition.
An overview of the proposed method for calf fatigue recognition is shown in Fig. 2. First, we estimate the participant’shead region in each frame of the video sequence of body sway.
Second, we compute the center position from the head region
in each frame. Next, we separately generate time-series signals
from the center position along the x and y axes of the image.
The x and y axes represent body movement in the left-rightand front-back directions, respectively. Note that the origin
is set to the average point of the center position. Then, we
obtain the power spectral density (PSD) of each time-series
signal. Here, we apply a low-pass ﬁlter (from DC to 3 Hz)
to each PSD to extract the principal frequency components
of body sway, as done in previous analytical studies [7].

Viewing Marker
direction
Participant(Upright posture)

0.9 m

(a)

First matching rate (%)

2.4 m

Overheadcamera(b)

Fig. 3. (a) Photograph of participant and (b) observation settings used for
obtaining dataset of video sequences of body sway for various calf-fatigue
levels.

B. Evaluation of Calf Fatigue Recognition Method
We performed two sets of calf fatigue recognition experiments to evaluate the effectiveness of our method. In the ﬁrst
set, we compared the accuracy obtained with and without the
integration of medical knowledge into the feature extraction. In
the second set, we assessed the accuracy differences among
several classiﬁers in recognizing calf-fatigue levels. For theﬁrst set of experiments, we set the conditions of the medical
knowledge integration as follows.
• With medical knowledge integration: This is the proposed
method. We used the feature described in Section II-C.
• Without medical knowledge integration: We utilized an
existing feature [6] extracted from a time-series signal
using frame differences to represent a participant’s head

**

*
**

0

With

Without

SVM

RF
(b)

GBDT

Fig. 4. Accuracy of calf fatigue recognition. (a) Results obtained with andwithout medical knowledge integration and (b) results obtained for various
classiﬁers. ∗ denotes a signiﬁcance level of 5% and ∗∗ denotes a signiﬁcancelevel of 1%.

movement for comparison. Even though this feature wasdesigned to distinguish the weight of baggage held by aperson, we directly applied it to recognize the calf-fatigue
level without any modiﬁcation.

III. E XPERIMENTS
To evaluate the accuracy of the proposed calf fatigue recognition method, we created a dataset of the video sequencesof body sway for various calf-fatigue levels. We recruited 20
participants (age: 22.4 ± 0.8 years, height: 167.4 ± 5.9 cm,
weight: 60.7 ± 12.0 kg, ethnicity: Japanese) and instructed
them to wear the designated light-blue shirt (see Fig. 3(a)). The
experimental settings for observing participants are shown in
Fig. 3(b). We instructed all participants to maintain an upright
posture and focus on a marker placed in front of them while
they were being observed. We set their standing position to be
directly below an overhead camera.
We conﬁgured the overhead camera to record videos at aresolution of 1920 × 1080 pixels and 30 frames per second.
To remove unnecessary background regions from the video
sequences, we cropped all frames to 200 × 200 pixels. We
observed each participant once per calf-fatigue level and set
the observation time to 120 seconds. Finally, we acquireda dataset of 100 video sequences (20 participants × 5 calffatigue levels). Note that we observed the participants under
the conditions described in Section II-B.

*: 𝑝<0.05 **: 𝑝<0.01(a)

We extract a feature by concatenating the PSDs in the leftright and front-back directions. Finally, we perform the calf
fatigue recognition by inputting the feature into a classiﬁer todetermine the calf-fatigue level (L0 to L100 ).

A. Dataset

50

For the second set of experiments, we set the conditions of
the classiﬁers as follows.
Support vector machine (SVM) [8]: We used a linear
kernel and set the regularization value to 10.
• Random forest (RF) [9]: We set the number of decision
trees to 300 and one of the feature selections to 30.
• Gradient boosting decision tree (GBDT) [10]: We set the
number of weak classiﬁers to 300 and one of the feature
selections to 30.
•

We split our dataset into testing and training sets using theleave-one-participant-out approach. Speciﬁcally, from the 20
participants, we assigned 1 participant to the testing set andthe remaining 19 participants to the training set. We repeatedthis process 20 times (i.e., each participant was assigned to thetesting set). We predicted the calf-fatigue level for each testing
set and computed the ﬁrst matching rate by aggregating all 20
predictions. Because of random sampling in RF and GBDT,
we performed the prediction per participant by averaging 20
prediction trials.
We evaluated the accuracy of the calf fatigue recognition
method using features with and without medical knowledge
integration. We combined each feature with SVM, RF, and
GBDT, respectively. Then, we calculated the average andstandard deviation of the ﬁrst matching rate from all combinations for each feature. Figure 4(a) shows the results of thecalf fatigue recognition with and without medical knowledge
integration. The average accuracy of the existing feature(without medical knowledge integration) was 22.5 ± 3.1% andthat of our feature (with medical knowledge integration) was
32.3 ± 7.8%. We examined whether there was a signiﬁcantdifference between these accuracies by applying the Wilcoxon
signed-rank sum test. The test showed a signiﬁcant differencein accuracy between the features with and without medical
knowledge integration. These results show that integrating
medical knowledge associated with calf fatigue into the feature
extraction of body sway improves calf fatigue recognition.

Next, we evaluated the accuracy of calf fatigue recognition
for each classiﬁer. We combined SVM with the featureswith and without medical knowledge integration, respectively.
Then, we calculated the average and standard deviation of
the ﬁrst matching rate for all combinations. For comparison,
we also performed the same procedure for RF and GBDT.
Figure 4(b) shows the results of calf fatigue recognition foreach classiﬁer. The average accuracy for SVM, RF, and GBDT
was 22.5±0.5%, 30.3±5.2%, and 29.3±10.7%, respectively.
We conﬁrmed signiﬁcation differences in the accuracy among
these classiﬁers by applying the Wilcoxon signed-rank sum
test and the Bonferroni correction for multiple comparisons.
The tests showed signiﬁcant differences in accuracy between
SVM and RF and between SVM and GBDT. These results
show that applying RF or GBDT as the classiﬁer to combinewith features extracted from body sway improves calf fatigue
recognition compared with that obtained with SVM.
C. Comparison with Existing Methods Designed for Human
Action Recognition
We attempted to recognize calf-fatigue levels from video
sequences of body sway using existing methods [11]–[13] designed for human action recognition. Although these methods
were not designed for calf fatigue recognition, we directly
applied them for calf fatigue recognition since they were
designed for recognizing class labels from video sequences.
We expected these methods to have low accuracy in calf
fatigue recognition. To conﬁrm this, we evaluated the methods
with the following settings.
• DI [11]: This dynamic image algorithm extracts a feature
vector from a video sequence of body sway. A 40,000dimensional weight feature vector was used to represent the temporal appearance changes using RankSVM.
GBDT was used for classiﬁcation.
• C3D [12]: This three-dimensional convolutional neuralnetwork extracts a feature map from a video sequenceof body sway. A network structure with four 3D convolutional layers and four 3D pooling layers was used.
Categorical cross entropy was used for classiﬁcation.
• LSTM [13]: This long short-term memory network extracts a feature vector from a video sequence of bodysway. A 128-dimensional feature vector calculated fromthe cell corresponding to the current time was used.
Categorical cross entropy was used for classiﬁcation.
For our method, the feature with medical knowledge integration and GBDT was used (see Section III-B). Note that theexperimental conditions other than those given above were thesame as those in Section III-B.
Table I shows the accuracy of the existing methods incalf fatigue recognition. The results conﬁrm that our method
achieved the highest accuracy. Therefore, our method extracts
more informative features for calf fatigue recognition compared with those extracted by existing human action recognition methods. The results also conﬁrm that integrating medical
knowledge on calf fatigue into feature extraction improves theaccuracy of calf fatigue recognition.

TABLE IC ALF FATIGUE RECOGNITION ACCURACY OF PROPOSED METHOD AND
EXISTING HUMAN ACTION RECOGNITION METHODS .
MethodAccuracy(%)

Ours

DI

C3D

LSTM

40.0 ± 1.8

20.1 ± 2.1

16.8 ± 0.3

19.1 ± 2.3

IV. C ONCLUSIONS
We validated calf fatigue recognition based on video sequences of body sway acquired after the heel-lift exercise. We
extracted an informative feature for calf fatigue recognition by
integrating medical knowledge, reported in previous analyticalstudies [3], [4], into the feature extraction of body sway. The
experimental results show that our method can recognize ﬁvelevels of calf fatigue with an accuracy of 40.0 ± 1.8%. For
calf fatigue recognition, the accuracy of our method is higher
than those of existing methods [11]–[13] designed for humanaction recognition. Our method has potential applications inphysical therapy, sports science, and personal ﬁtness.
In future work, video sequences of participants of various
ages will be collected. Furthermore, to expand the use case of
our method, we plan to recognize calf-fatigue levels after exercises other than heel lifts. We appreciate Professor Yoshio Iwai
for his valuable advice during this research.
R EFERENCES[1] D. E. Houghton, A. Ashrani, D. Liedl, R. A. Mehta, D. O. Hodge,
T. Rooke, P. Wennberg, W. Wysokinski, and R. McBane, “Reduced calfmuscle pump function is a risk factor for venous thromboembolism: apopulation-based cohort study,” Blood, vol. 137, no. 23, pp. 3284–3290,
2021.
[2] M. H. Laughlin and B. Roseguini, “Mechanisms for exercise traininginduced increases in skeletal muscle blood ﬂow capacity : Differenceswith interval sprint training versus aerobic endurance training,” Journalof Physiology and Pharmacology, vol. 59, no. 7, pp. 71–88, 2008.
[3] N. Vuillerme, N. Forestier, and V. Nougier, “Attentional demands andpostural sway: the effect of the calf muscles fatigue,” Medicine andScience in Sports and Exercise, vol. 34, no. 12, pp. 1907–1912, 2002.
[4] E. J. Bisson, D. McEwen, Y. Lajoie, and M. Bilodeau, “Effects of ankle
and hip muscle fatigue on postural sway and attentional demands during
unipedal stance,” Gait Posture, vol. 34, no. 1, pp. 83–87, 2011.
[5] R. B. Armstrong, “Mechanisms of exercise-induced delayed onset
muscular soreness a brief review,” Medicine and Science in Sports andExercise, vol. 16, no. 6, pp. 529–538, 1984.
[6] Y. Yamaguchi, T. Kamitani, M. Nishiyama, Y. Iwai, and D. Kushida,“Extracting features of body sway for baggage weight classiﬁcation,” In
Proceedings of IEEE 9th Global Conference on Consumer Electronics,
pp. 345–348, 2020.
[7] K. Taguchi, “Spectral analysis of body sway,” Journal for Oto-RhinoLaryngology, Head and Neck Surgery, vol. 39, no. 6, pp. 330–337, 1977.
[8] C. Cortes and V. Vapnik, “Support-vector networks,” Machine learning,
vol. 20, pp. 273–297, 1995.
[9] L. Breiman, “Random forests,” Machine learning, vol. 45, pp. 5–32,
2001.
[10] J. H. Friedman, “Greedy function approximation: a gradient boostingmachine,” Annals of statistics, vol. 29, no. 5, pp. 1189–1232, 2001.
[11] H. Bilen, B. Fernando, E. Gavves, and A. Vedaldi, “Action recognition
with dynamic image networks,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, vol. 40, no. 12, pp. 2799–2813, 2018.
[12] D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning
spatiotemporal features with 3d convolutional networks,” In Proceedingsof the IEEE International Conference on Computer Vision, pp. 4489–
4497, 2015.
[13] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” NeuralComputation, vol. 9, no. 8, pp. 1735–1780, 1997.