論文

Considering the privacy-protected detecting algorithm forperiodic motions with visual IoT
Shin SUZUKI (Aichi University)
Keiichiro FUKAZAWA (Kyoto University)
Takako MURAI (Junshin Gakuen Univerity)
Abstract
In the ﬁelds of nursing and caregiving, there is an increasing demand to monitor the condition
and movements of patients due to chronic manpower shortages and the anticipated decreasein the future workforce. ICT technologies are expected to address this need. In this study,
we considered a detection algorithm for various periodic motions through video analysis. Wecalculate the magnitude of movement by comparing video frames at diﬀerent intervals. The
detection results include only the magnitude and period of movements, reducing the risk ofinvading the subject’s privacy compared to capturing full video data. By using video as a motion
sensor, the system can monitor while protecting the subject’s privacy.

Keywords: IoT, Video analysis, Privacy protection

advancement and widespread adoption

1. Introductionof the Internet of Things (IoT) in recent
Currently, many countries areyears, its application has extended

grappling with the challenges posed bybeyond industrial sectors to encompass

an aging population. Japan, in particular,caregiving, childcare, and healthcare ﬁelds.

is facing an unprecedented level of

By harnessing the power of IoT, a vast

population aging, having entered a super-

amount of data can be collected fromaged society in 2007, with over 21% ofvarious sensors and cameras, connected4, 6).

through networks, making it an invaluable

As society continues to age, the shortage

approach for “monitoring” purposes.

of caregivers and the increasing burden

For example, there is a high demandplaced on them have become signiﬁcantin nursing and caregiving for detecting

social issues. However, with the rapid

excretory behavior in persons who require

愛知大学情報メディアセンター― 37―

the population being 65 years or older

Vol.33, No.1, 2024

nursing care, particularly at night whentechniques to exclude raw video data.

caregivers need to assist with excretion,

In this study, we present preliminary

handle bed-changing, and perform cleaning

results on a privacy-protected detection

tasks due to incontinence or unclean acts,

algorithm aimed at identifying the periodic

which impose a signiﬁcant burden on

movements associated with various

them. However, the use of camera data

actions performed by the subjects.

often raises concerns due to its highly
sensitive and personal nature, leading

2. Methodology

to instances where camera recording is
Recent studies in computer vision

declined.
We have developed a monitoring

enable the easy detection of humans,

system for elderly individuals using small

moving objects, and characteristic

sensors to monitor indoor environmental

motions1, 3, 7, 11). In this study, we employedfactors such as temperature, light, noise,a simple motion detection method, asand more

2, 5, 8, 9). This system enables

depicted in Figure 1, to detect motion

remote estimation of residents’ activities

in the video. The input consists of aand monitors their daily lives by issuing

30-second video of a metronome set atalerts when unusual situations area tempo of 120 beats per minute (bpm),detected. It has been used as a non-recorded in Full HD with a resolution ofinvasive monitoring system in various

1920×1080 pixels and a frame rate of 30

settings, including homes with families

frames per second (FPS).

dealing with dementia, homes of elderly

The process is as follows: (1) theindividuals living alone, and nursing

original frames are read as grayscale

homes. However, to achieve more accurate

images; (2) frame-diﬀerence images areactivity estimation and detect subtle

generated by skipping a predetermined

behaviors, video monitoring is highly

number of frames (F); (3) the resulting

advantageous.

frame-diﬀerence images are combined

We are currently in the process ofusing a logical AND operation to

designing a new monitoring system that

identify the moving objects in the video.

safeguards individuals’ privacy by storing

We consider the resulting image as

monitoring data without identiﬁcation. In

representing the moving objects at thethis system, we will employ preprocessing

exact moment of the second frame in the― 38―

愛知大学情報メディアセンター

Vol.33, No.1, 2024

Fig 1: Diagram of the method overview in this study. The left three panels show thegrayscale frames of the original input video with skipping predetermined frames.
The right two panels show the frame-diﬀerences.
original video. In this study, we obtainedConsequently, the cumulative intensity ofprocessed images at a time resolution of

the frame-diﬀerence image theoretically

10 frames, resulting in the total number

becomes zero with a F of 30 frames (1of frames being one-tenth of the originalsecond).

video.
Figure 1 illustrates an example of the3. Results and discussion

process with F set to 5 frames. Notethat a metronome set at a tempo of 120

Figure 2 shows the the total intensity

bpm produces metallic sounds at bothof the processed image captured withends twice per second. This means that

diﬀerent F values: (a) 20, (b) 25, (c) 30,

the metronome’s pendulum completesand (d) 35 frames.

one round trip at the ends t times within

was determined by summing the pixelt seconds, and it passes through theintensities within each processed image.

same spot 2t times in other locations.

The intensity when F was set to 30― 39―

愛知大学情報メディアセンター

The total intensity

Vol.33, No.1, 2024

Fig 2: Time series of the total intensity in the processed video image of themetronome set to a tempo of 120 bpm with varying (a) F=20, (b)F=25, (c) F=30, and (d) F=35.
frames (Figure 2c) reveals a distinct

distinguish intervals of 1 frame (1/30

decrease in intensity compared to thes). It is worth noting that the intensity

other.

reaches its minimum even at F＝59,

Figure 3a displays the variation of

approximately to twice the period, and atmean intensity, obtained by averaging theF＝1, where the motion of the pendulum

total intensity of the processed images

is extremely small, resulting in negligible

throughout the entire 30-s video duration.

frame diﬀerence intensity.
Figure 3b illustrates the areas where

This clearly shows a decrease in intensity
around F＝30.

To be more precise,motion was detected in the input video’s

the minimum intensity occurred atﬁeld of view. The contour colors indicate

F＝29.

This slight shift in F at thethe corresponding value of F when thepoint of minimum intensity can bemean intensity reaches its minimum. The

attributed to the precision limitations ofregion of the metronome’s pendulum

the analog metronome used in this study,

motion in the images of input video closely

which lacks the necessary accuracy tocorresponded to the region with F≃30.

― 40―

愛知大学情報メディアセンター

Vol.33, No.1, 2024

Fig 3: (a) Mean intensity variations with diﬀerent frame gaps. (b) Contour plots
illustrating the detected areas of periodic motion in the 9×16 resolution images
with a 120×120 binning of the original FHD image.

These results suggest that the4. Conclusions

characteristics of periodic motion in avideo can be recognized by calculating theWe reported a procedure for extracting

frame difference while exploring a range

periodic motion from the test inputof F values that encompass the target

video (1920×1080 pixels, 30 FPS). The

period of the motion. This recognition

motion and its corresponding areas in themethod is signiﬁcantly faster thanimages can be recognized by comparing

performing frequency analysis using thethe intensities of frame diﬀerences with

Fast Fourier Transform (FFT) technique

varying frame intervals.

on all frames of the video. Additionally,

This method will be adopted for theit can be executed on edge devices as it

new monitoring system, where the input

does not heavily rely on machine power.

video will undergo a conversion process

By storing only the images shown into generate privacy-protected frames,

Figure 3, it is possible to monitor speciﬁc

as illustrated in Figure 3b, before being

movements without collecting anystored. The system is designed to function

personally identiﬁable information, such as

as a personalized monitoring systemfacial data.

capable of detecting speciﬁc motions
that we aim to identify, such as ﬁdgety
movements associated with excretion or
scratching caused by atopic dermatitis,― 41―

愛知大学情報メディアセンター

Vol.33, No.1, 2024

by deﬁning their respective periods inInterest Group Technical Reports of IPSJ

advance. Consequently, it has the potential(IS) 2022(7), 1―6 (2022)

to substantially alleviate the physical and3 ）Husein, A. M., Halim, D., Leo, R.: Motion

psychological burdens experienced bydetect application with frame diﬀerence

caregivers.

method on a surveillance camera. Journalof Physics: Conference Series, Vol. 1230,Acknowledgements

doi:10.1088/1742-6596/1230/1/012017

This work was supported by Grantsin-(2019)

Aid for Scientiﬁc Research (18K03728,4 ）L e r o i , I . , K i t a g a w a , K . , V a t t e r , S . ,20K21739, and 23K10394) from MEXT,Sugihara, T.: Dementia in ‘superaged’

Japan, and the joint research programs ofJapan: challenges and solutions.

Junshin Gakuen University and Center for

Neurodegenerative disease management

Regional Policy Studies, Aichi University.

8(4), 257―266 (2018)

Additionally, this research received partial5 ）Murai, T., Fukazawa, K., Suzuki, S.:

support from AMED under Grant Number

A Study on Predictive Detection ofJP22ym0126814.

Excretion in Elderly Requiring Care

In typesetting this paper, we utilized
theclass ﬁles10)

Using Video Internet of Things. the 25th

generously provided

East Asia Forum of Nursing Scholars

by A. Umegaki.

(EAFONS) Conference (21―22 April 2022)
6 ）Nakamura, K.: Locomotive syndrome:

References

disability-free life expectancy and1 ）Arefeen, M. A., Nimi, S. T., Ud-din, M. Y. S.:

locomotive organ health in a “super-aged”

Framehopper: Frame-Hopper: Selectivesociety. Journal of Orthopaedic Science

Processing of Video Frames in Detection-14, 1―2 (2009)

driven Real-Time Video Analytics. In:7 ）Singla, N.: Motion Detection Based on

2022 18th International Conference on

Frame Diﬀerence Method. International

Distributed Computing in Sensor Systems

Journal of Information & Computation(DCOSS), pp. 125―132. IEEE (2022)

Technology 4(15), 1559―1565 (2014)

2 ）Fukazawa, K., Suzuki, S., Murai, T.:

8 ）Suzuki, S., Fukazawa, K., Murai, T.:

Development of remote autonomousActivity recognition system based onwatching system with non-contact

the indoor environment sensing (insensor and IoT (in Japanese). The SpecialJapanese) (2022). https://ipsjbti.github.io/

― 42

―

愛知大学情報メディアセンター

Vol.33, No.1, 2024

proceedings/202303/pdf/ bti0337.pdf
9 ）S u z u k i , S . , F u k a z a w a , K . , M u r a i ,T.: Remote sensing of the indoor
environment based on IoT (in Japanese).
COM: Journal of Aichi University Media
Center 32(1), 47―55 (2023)
10）Umegaki, A.:

Class File for COMof Aichi University (in Japanese). COM:
Journal of Aichi University Media Center
33(1), (2024), in press11）Wren, C. R., Azarbayejani, A., Darrell,T., Pentland, A. P.: Pﬁnder: Real-timetracking of the human body. IEEE
Transactions on pattern analysis andmachine intelligence 19(7), 780―785 (1997)― 43―

愛知大学情報メディアセンター

Vol.33, No.1, 2024