ビデオと音声の分析により睡眠時無呼吸症候群の検出
Apnea Detection via Depth Video & Audio Analysis$ National Institute of Informatics$##$
Gene CHEUNG , Cheng YANG , Vladimir STANKOVIC , Kevin CHAN*, Nobutaka ONO# University of Strathclyde* University of Western Sydney

Non-intrusive respiratory and heart rates
TMestimation using Microsoft Kinect
Motivation: Non-intrusive bio-signal estimation via RGB
camerasis sensitive to ambient lighting conditions.

wearing a maskside view
back view
front view
Heart rate estimation with different views

Solution: Estimating bio-signal using DEPTH sensingTM
devices, e.g., MS Kinect, must overcome
the low bit-depth and acquisition noise.
Head region
motion tracking

１．Respiratory rate estimation.
video recording

Noticeable
respiratoryrelated
chestandabdominal
movementgraph-based video denoisingnoisy
temporaldata

body part modelingmotion tracking
feature extraction
and classification

denoised
temporaldata

２．Heart rate estimation [1,2].
Subtleheart beatrelated
headmovement

video recording andgraph-based denoisingfrequency analysis

motion tracking
frequency analysis

estimateheart rate

Estimatedheart rate

peak detection

estimateheart beat regularity

peak detection
 Estimating heart beat regularity:
Compute the standard deviation of
the distance between neighbouring
detected peaks.

Example application of respiratory
rate estimation: Sleep monitoring
Solution: Record depth video
Motivation: detect sleep
and audio, extract relevant
apnea events (temporaryfeatures, train classifiers torespiratory cessation) nondetect obstructive / central
intrusively and reliably.
apnea, hypopnea [3,4,5].
Sampledepth
image

Changes of the fitted abdomen ellipse over time
(3 minutes out of 8-hour sleeping data)

4-class classification: central apnea; obstructive/mixedapnea; hypopnea; normal breathing.
TMSleep monitoring with Kinect Audio When the captured depth video is obstructed, one
could still use the audio signal to detect sleep events.
 Audio features can be extracted using, e.g., nonnegative matrix factorization.

Cables of a sleep
monitoring device

References:

Two cross-sections of the human
Fitted ellipse of each cross-section
torso: chest and abdomen[1] C. Yang, G. Cheung, V. Stankovic, “Estimating heart rate via depth video motion tracking,” in IEEE
International Conference on Multimedia & Expo, Turin, Italy, June 2015.
[2] C. Yang, G. Cheung, V. Stankovic, “Estimating heart rate and rhythm via 3D motion tracking in depthvideo,” submitted to IEEE Transactions on Multimedia, Apr. 2016.
[3] C. Yang, G. Cheung, K. Chan, and V. Stankovic, “Sleep monitoring via depth video recording &analysis,” in IEEE International Workshop on Hot Topics in 3D, Chengdu, China, July 2014.
[4] C. Yang, Y. Mao, G. Cheung, V. Stankovic, and K. Chan, “Graph-based depth video denoising and event
detection for sleep monitoring,” in IEEE International Workshop on Multimedia Signal Processing, Jakarta,
Indonesia, Sept. 2014.
[5] C. Yang, G. Cheung, V. Stankovic, K. Chan, and N. Ono, “Sleep apnea detection via depth video & audio
feature learning,” submitted to IEEE Transactions on Multimedia: Special Issue on Multimedia-BasedHealthcare, Jan. 2016.

contact： Gene CHEUNG / Associate Professor Digital Content and Media Sciences Research Division National Institute of Informatics
TEL : 03-4212-2567

FAX : 03-4212-2120

Email : cheung@nii.ac.jp