Calf Fatigue Recognition in Heel-lift Exercise
Using Video Sequences of Body Sway
T. Kamitani, M. Kojima, and M. Nishiyama, Tottori University, JapanIntroduction
We aim to develop a system that provides feedback on the level of calf muscle fatigue after heel-lift exercise.
CameraFront-backCalf fatigue
accumulation

feedback
Your calf fatigue level is75%, which is the perfectamount of exercise!

Body sway
measurement

Observation

Feature extraction& classification

Related works

Proposed method (Calf fatigue recognition)

Body sway measurement

Fatiguelevel isunknown.

TimeAcquisition of video sequences ofbody sway from a standing person

Center position

Classifier

Distance

Leftright

FrontbackPrevious analytical studies show different traits
between the front-back and left-rightcomponents depending on calf fatigue. ※2

Force
plate

Frontback(y-axis)FrontbackLeftright

Person standing
with calf fatigue

Time

Generation of PSD foreach time-series signal

With fatigue
Left-right

Front-backCenter of pressure◼ 𝑳𝟎 : Participants showed no calf fatigue.
◼ 𝑳𝟏𝟎𝟎 : Participants reached maximum heel lifts.
◼ 𝑳𝟐𝟓 ~𝑳𝟕𝟓 : Participants reached 25% to 75% ofmaximum heel lifts.

Dataset ※ A total ofImpact of integrating medical knowledge
𝐿0

Medical knowledge Ave. accuracy

Height +30% of foot length

𝐿50

Observing 𝐿75
10 days

𝐿75

Observing 𝐿50

Heel-lift exercise

10 daysObserving 𝐿25
Order for observing⚫ Calf fatigue initialization:
No vigorous exercise for 10 days.
⚫ Heel lift height:
30% of each participant’s foot length.
⚫ Heel lift speed: 100 beats per minute.

32.3±7.8 %Without ※1

22.5±3.1 %Comparison with human action recognition

10 days
10 daysWith (Ours)

 Using three classifier for average accuracy

Observing 𝐿0
Observing 𝐿100

30% offoot length

Analysis of bodysway components

Results

20 individuals.

𝐿25

Foot length

Affected components
by fatigue※3 Bilen et al., TPAMI’18※4 Tran et al., ICCV’15※5 Hochreiter et al., Neural Computation’97

Five calf-fatigue level & heel-lift exercise

Metronome

Without fatigue

We integrated their idea of separating bodysway into front-back and left-right components.

Generation of time-series signals
from center positions in x and y axes

Experiments

Head-board

Baggage weight recognition

We incorporated their technique formeasuring body sway from video sequences.

Leftright(x-axis)

Frequency

Classification of calf fatigue
level based on PSD feature

Body swayfrom camera

Medical knowledge integration

Distance

PSD feature forcalf-fatigue recognition

PSD

Frontback

PSD

Leftright

Concatenation

Calf fatigue level

Time

Person standing
with baggage

Estimation of head regions
Calculation of centerin video sequences
positions from head regions

Feature extraction and classification
𝑳𝟎 𝑳𝟐𝟓 𝑳𝟓𝟎
𝑳𝟕𝟓 𝑳𝟏𝟎𝟎z

Left-right (x-axis)

Black pixels represent
head region.

Gazedirection

The baggage
weight is 3.0kg

Front-back (y-axis)

Upright
postureOverhead camera

Centerposition

Overhead camera※1 Yamaguchi et al., GCCE’20※2 Vuillerme et al., MSSE’02

An existing method measures body sway
to recognize the weight of held baggage. ※1

We integrate medical knowledge on calf fatigue levels from previous
analytical studies into the existing body sway feature extraction.
zTo provide valuable feedback,it is necessary to accurately
recognize the calf fatigue level.

Centerposition

Heel-lift exercises canload the calf muscles andimprove poor blood circulation.

Calf fatigue
recognition

Left-right

𝐿100Method

Best accuracy

Ours

40.0±1.8 %DI ※3

20.1±2.1 %C3D ※416.8±0.3 %LSTM ※519.1±2.3 % Classifier (ours/DI): Gradient boosting decision tree

PersonA

PersonB

PersonC

PersonD

PersonE

Calf fatigue differences were noteasily visible in appearance.

 Dataset splitting: Leave-one-participants-out
 Evaluation metrics: First matching rate

Our method integrating medical
knowledge achieved top accuracy.