Gene Cheung
National Institute of Informatics
6th July, 2015

Depth Image Coding & Processing
Part 1: Introduction.
COST Training School 7/06/2015

1

Biography• MS from UC Berkeley in EECS in 1998.
2D video• Thesis: Joint source / channel coding for wireless video.
Communication• PhD from UC Berkeley in EECS in 2000.
(12 years)• Thesis: Computation / memory / distortion tradeoff in signal compression.

• Senior researcher in HP Labs Japan from 2000 ~ 2009.
• Topic 1: 2D video coding & streaming (2000~2007).
• Topic 2: Multiview video, w/ Prof. Ortega (2007~).

• Faculty in NII from 11/2009 to now.
3D video
Communication(8 yrs)• Topic 1: Image & video representation.
• Topic 2: Immersive visual communication.
• Topic 3: Graph signal processing.

• Adjunct associate professor in HKUST from 1/2015.

COST Training School 7/06/2015

2

Acknowledgement
Collaborators:• Y. Mao, X. Liu, Y. Ji (NII, Japan)• W. Hu, P. Wan, W. Dai, J. Pang, J. Zeng, A. Zheng,
O. Au (HKUST, HK)• Y.-H. Chao, A. Ortega (USC, USA)• D. Florencio, C. Zhang, P. Chou (MSR, USA)
• Y. Gao, J. Liang (SFU, Canada)• T. Maugey, L. Toni, B. Motz, A. De Abreu, P. Frossard(EPFL, Switzerland)• C. Yang, V. Stankovic (U of Strathclyde, UK)• X. Wu (McMaster U, Canada)• P. Le Callet (U of Nantes, France)• H. Zheng, L. Fang (USTC, China)• B. Machiavello, C. Dorea, M. Hung (University ofBrasilia, Brazil)
COST Training School 7/06/2015

3

NII Overview• National Institute of Informatics• Chiyoda-ku, Tokyo, Japan.
• Government-funded research lab.

• Offers graduate courses & degrees
through The Graduate University for
Advanced Studies (Sokendai).
• 60+ faculty in “informatics”:
quantum computing, discretealgorithms, database, machinelearning, computer vision, speech &audio, image & video processing.

• Get involved!
• 2-6 month Internships.
• Short-term visits via
MOU grant.
• Lecture series,
Sabbatical.
COST Training School 7/06/2015

4

Outline• Introduction• What is 3D imaging?
• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate estimation• Fundamental of Graph Signal Processing (GSP)• Spectral Graph Theory• Graph-based Transform
COST Training School 7/06/2015

5Outline• Introduction• What is 3D imaging?
• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate estimation• Fundamental of Graph Signal Processing (GSP)• Spectral Graph Theory• Graph-based Transform
COST Training School 7/06/2015

6Video Communication:
2D to 2.5D to 3D

Microsoft Kinect

Mesa'sSwissRanger• 2D Video• 1 capturing camera @ sender.
• 1 2D display @ receiver (non-interactive).

senderreceiversenderreceiverMultiview Camera Setup @ Nagoya U.

• 2.5D Video (stereoscopic)• 2 capturing cameras @ sender.
• 1 stereoscopic display @ receiver (non-interactive).
• 3D Video (multiview, free viewpoint)• Multiple capturing cameras @ sender.
receiver• 1 2D / stereoscopic display @ receiver (interactive):• Receiver observes subset of high dimension
media available @ sender!

sender FB channel

COST Training School 7/06/2015

7

texture map

depth map

Multiview Video Streaming• Interactive view-switches among captured camera viewpoints.

COST Training School 7/06/2015

8

Free Viewpoint Video Streaming• Interactive view-switches to any virtual camera viewpoints.

*Courtesy of KDDI Laboratories, Japan

COST Training School 7/06/20159texture map

depth map

3D Imaging:
Overview

Microsoft Kinect

Mesa'sSwissRanger• 3D Imaging? Given depth sensors capturing texture &geometry per pixel, process multi-d image data for 3D apps.
• App 1: Immersive visual communication:• video conferencing (gaze correction,
motion parallax)• free viewpoint TV (user view selection)xz• mixed reality (virtual objs + real view)App 2: 3D data
sensing & analysis:• 3D event detection• Non-intrusive
health / sleep
monitoring*Courtesy of KDDI Laboratories, Japan

COST Training School 7/06/2015

10

Potential Impact• Commercial Immersive Communication• Immersive entertainment (Disney, Samsung).
• Immersive conferencing (KDDI, Cisco).
• 3D visualization (Microsoft, Google).
xz• Enhance Virtual Reality is 1 of 14 grand challenges chosen
by National Academy of Engineering for 21st century.
• Growing Market for Health Monitoring• 1 / 4.5 Japanese have sleep disorders.
• Sleep clinic is 7 billion $ US market.
• Imaging means “Non-intrusiveness”.

COST Training School 7/06/2015

11

3D Video Representation• Texture + depth maps from 1 or morecamera viewpoints [1].
• Texture map: color image like RGB.
• Depth map: per-pixel distance bt’n captured
objects in 3D scene & capturing camera.

• Synthesis of intermediate views via depth-imagebased rendering (DIBR) [2].
• Computation-efficient.
• Unlike model-based approach, complexity notscene-dependent.
texture map

depth map

[1] P. Merkle, et al., “Multi-view Video Plus Depth Representation and Coding,” IEEE International Conference on
Image Processing, San Antonio, TX, October 2007,.
[2] D. Tian, et al., “View synthesis techniques for 3D Video,” Applications
DigitalImage Processing XXXII,COST Trainingof
School
7/06/2015
Proceedings of the SPIE, vol. 7443, (2009), San Diego, CA, February 2009,.

12

Outline• Introduction• What is 3D imaging?
• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate estimation• Fundamental of Graph Signal Processing (GSP)• Spectral Graph Theory• Graph-based Transform
COST Training School 7/06/2015

13

edge

Depth Image Compressionnode• Goal: reduce coding rate w/o distortion.
• Graph Signal Processing:• Graphs describe correlations bet’n neighboring pixels.
DCT• Transforms defined on graphs replace DCT.

• Depth Image Compression:• Graph Fourier Transform compactly describes PWS signals.
• up to 68% bitrate reduction compared to HR-DCT [3].

• Geometry Coding:
GFT• Project 3D geometry
to 2D image tiles.

• 35% rate saving for opt
tiles w/ GFT over naïve
tiles w/ DCT [4].
[3] W. Hu, G. Cheung, A. Ortega, O. Au, "Multi-resolution Graph Fourier Transform for
Compression of Piecewise Smooth Images," IEEE Trans on Image Proc, Jan 2015.
[4] Y. Gao, G. Cheung, T. Maugey, P. Frossard, J. Liang, “3D Geometry Representation using
Multiview Coding of Image Tiles,” IEEE ICASSP, May, 2014.

14

Background to Interactive Multiview Video StreamingInteractive Multiview Video Streaming (IMVS) A client can periodically request one of many captured views, as
video is played back in time.
 To reduce transmission BW, transmit only views interactively
selected by client.
 The encoding is done once at the server for a possibly large groupof clients.

COST Training School 7/06/2015

15

Background to Interactive Multiview Video Streaming
Multiview Video Coding (MVC) [5]•

Code frames of all views in RD manner.

Disadvantage: MVC not suitable for IMVS Insufficient decoding flexibility forinteractive view-switching of 1 view.
Redundant frame structure for IMVS [6] Multiple decoding paths to facilitateview-switching.
 Distributed source coding (DSC) frames
to merge decoding paths.

view
same instantsame viewM[5] P. Merkle et al., “Efficient Prediction Structures for Multiview Video Coding," IEEE Transactions on Circuits
and System for Video Technology, vol.17, no.11, pp.1461-1473, November 2007.

time[6] Cheung et al., "Interactive Streaming of Stored Multiview Video using Redundant Frame Structures," IEEE
Transactions on Image Processing, vol.20, no.3, pp.744-761, March 2011.

16

Advanced Interactive 3D Video
Streaming• 3D Video Compression with Flexible Decoding:• Prob: diff. coding in video has dependency.
• Solution: Interactive view-switching using Merge
frame built from piecewise constant func.
• Distributed Source Coding w/o channel codes! [7]same instantsame view

M[7] W. Dai et al., “Rate-distortion Optimized Merge Frame using Piecewise Constant Functions," IEEE International
Conference on Image Processing, Melbourne, Australia, September, 2013. (Best student paper award)• Interactive Navigation of High-dimen. Data:
1. View selection for observer group [8].
2. Synthesize new reference views at proxies
for delay-free navigation at client [9].
[8] D. Ren et al., “Anchor View Allocation for Collaborative Free Viewpoint Video Streaming,” IEEE Transactions
on Multimedia, vol.17, no.3, pp.307-322, March, 2015.
[9] L. Toni et al., "In-Network View Re-Sampling for Interactive Free Viewpoint Video Streaming," accepted
to IEEE International Conference on Image Processing, Quebec City, September, 2015.

17

Outline• Introduction• What is 3D imaging?
• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate estimation• Fundamental of Graph Signal Processing (GSP)• Spectral Graph Theory• Graph-based Transform
COST Training School 7/06/201518

Video Enhancement for Depth Camera• Problem: Depth images fromToF camera are low-resolution,
blurred, noisy

 Setting: Given a noisy, lowresolution depth map DL anda registered noise-free,high-solution color image I Estimate DH*Courtesy of Prof. M. Do, UIUC, USA

COST Training School 7/06/2015

19

Proposed Method: Weighted Mode Filtering• Generating joint histogram [10]:• g(p): color value at pixel p• f(p): depth value at pixel p• fG(p): enhanced depth value at pixel p• GI, GS, Gr: Gaussian function
color Gaussian

pixel pspatial Gaussianerr Gaussian

dth binneighbors of pixel p[10] D. Min, J. Lu, and M. N. Do, “Depth video enhancement based on weighted mode filtering,” IEEE Trans.
on Image Processing, 2012.
*Courtesy of Prof. M. Do, UIUC, USA

COST Training School 7/06/201520

Result Comparison

Up-sampling results for low-quality depth image (from ‘Mesa
Imaging SR4000’, 176x144) with corresponding color image(from ‘Point Grey Flea’, 1024x768).
*Courtesy of Prof. M. Do, UIUC, USA

COST Training School 7/06/2015

21

Depth Image Denoising (w/o color)• Denoising problem is ill-defined:

desired signal

y xvobservation

noise• Interpret graph-signal as instance of MRF, then x is graph sparsity prior [11]:GFT0bilateral filter

nonlocal GBT

min y  x 2    x 0

L0-norm

 ,xfidelity termsparsity prior• Interpret graph-signal as samples onhigh-dimen. manifold, then xT L x isgraph-signal smoothness prior [12]:min y  x 2  xT Lx

L=D -AL ,xfidelity termsmoothness prior[11] W. Hu, X. Li, G. Cheung, O. Au, “Depth Map Denoising using Graph-based Transform and Group Sparsity,” IEEE
International Workshop on Multimedia Signal Processing, Pula (Sardinia), Italy, October, 2013. (Top 10% paper award.)[13]J.W.
Sun,G.
G.Cheung,
Cheung,A.
P. Ortega,
Chou, D.O.Florencio,
C. Zhang,
O. Au,Laplacian“Rate-constrained
3D Surface
Estimation
Noise- IEEE
[12]
Pang,
C. Au, “Optimal
Graph
Regularization
for Natural
Imagefrom
Denoising,,”
corrupted Multiview
DepthVideos,” SpeechIEEE Transactions
on Image Processing,vol.23, no.7,
pp.3138-3151,
July 2014.
International
Conferenceon Acoustics,
and Signal Processing(ICASSP), Brisbane,
Australia,
April, 2015.

Joint denoising / compression of multiview depth images.

22Depth Image Denoising (w/o color)• Experimental Setup:• Test Middleburry depth maps: Sawtooth• Additive White Gaussian Noise (AWGN)• Compare to: Bilateral Filtering (BF), Non-Local Means Denoising, (NLM),
Block-Matching 3D (BM3D).

• Results: 2.28dB improvement over BM3D.

NLGBT

BM3D
COST Training School 7/06/2015

NLM

BF
23

Depth Image Interpolation (local)• Problem:• Fill holes in sparsely sampled depth images.

• Idea:*1. Find right graph for missing pixels.
Depth image missing 85% pixels•

Adaptive kernel

2. Compute edge weights using initial values.
 y  y 2ij 
w i, j  exp 

2



3. Find smooth graph-signal given observations.

min  u x  yi  x t Lx
x

t
i

i

fidelity term22smoothness prior[14] Yu Mao, Gene Cheung, Yusheng Ji, "Image Interpolation During DIBR View Synthesis Using Graph Fourier Transform," 3DTV-Conference
2014, Budapest, Hungary, July, 2014.

24

Denoising Results

COST Training School 7/06/2015

Original

Noised(theta = 20)

LARK(26.99dB)

Our Proposal(27.36dB)

25

Interpolation Results

COST Training School 7/06/2015

Original

Partial sample

LARK(34.82dB)

Our Proposal(35.31dB)

26

Image Bit-depth Enhancement
Problem:an estimate of theoriginal HBD image

low bit-depth (LBD) image y—a quantizedversion of underlying HBD image

Objective: find x̂ that minimizes mean-squared-error (MSE),

x̂

MMSE

 arg min  x̂  x
x̂

22

f x | y  dx

squared err
posterior prob of HBD
signal x given LBD signal yPosterior:
Likelihood: equals to 1 iff xi quantizes
to yi

Smoothness prior: HBD signal issmooth[15] Rudin et al, “Nonlinear total variation based noise removal algorithms”, Elsevier, 1992

Conventional smoothness (e.g., total Variation*27
are signal-independent  over-smoothingImage Bit-depth
Enhancement
Graph-signal smoothness prior

L is the graph Laplacian matrix
describing inter-pixel similarities*
Reconstruct smooth signal without blurring edges

Proposed ACDC Algorithm:•••

Input bit-depth

Compute edge weights from quantized signal.
Compute MAP solution of AC signal.
Compute MMSE solution of DC signal given
AC signal.

Ground-truth

Input yOur output[16] P. Wan et al., “Image Bit-depth Enhancement via Maximum-a-Posteriori Estimation of Graph AC Component," IEEE
International Conference on Image Processing, Paris, France, October, 2014. (Top 10% accepted paper recognition)

28

Soft-Decoding of Compressed Images
DCT transform

Problem

Encoder: quantization of DCT coefficients.

8x8 pixel
blockqi  round (Yi / Qi ), Y =Ty
DCT coefficients
quantization parameter
JPEG

Soft decoding

JPEG

Soft decoding

Decoder: 64 quantization bin constraints / block.

qi Qi  Yi  (qi  1) Qi
Soft Decoding• Find most probable signal within quantizationbins with help of pre-determined signal priors.
COST Training School 7/06/2015

29

Priors for Soft-Decoding of Compressed Images
Sparsity priorx  Φα   ,dictionary

noise termsparse vector•

Sparse linear combination of atoms approx. signal.

•

Help restore high frequency content (image texture).

Graph-signal smoothness prior••Promote signal-dependent smoothing (avoid over-smoothing of edges).
Help restore sharp discontinuities (image structure).
Sparsity priorObjective function: combine two priors with inter-patch consistency:

arg min  i xi  Φ α i 2  1 α i 1  2 xTi Li xi
2

Graph-signal
smoothness prior{ xi , α i }s.t.,qk ,i Qk  1(k )T TMxi  (qk ,i  1)Qk , k

Quantization constraint

 jN (i ) Ri, j xi  R j ,i x j

Inter-patch consistency

2
2



i.

COST Training School 7/06/201530

Experimentation[17] Xianming Liu et al., "Inter-Block Soft Decoding of JPEG Images with Sparsity and Graph-Signal Smoothness
Priors," accepted to IEEE International Conference on Image Processing, Quebec City, September, 2015.

31

Outline• Introduction• What is 3D imaging?
• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate estimation• Fundamental of Graph Signal Processing (GSP)• Spectral Graph Theory• Graph-based Transform
COST Training School 7/06/2015

32

Gaze Correction / Face
Beautification in Conference Video
Gaze mismatch problem: video conference
subjects cannot see eye-to-eye.

Solution

captured view

1. Learn offline frontal faces of beautiful people fromlarge databases of images (big data).
2. During conference, re-draw gaze-corrected faces& beautify facial components (eye, eyebrows).

desiredview• Handles “blinking”.

Enhanced Reality: looks real, but overcomes limitations of reality.
Results: natural looking facial video, w/o explicit 3D geometric info.
Note: 1 joint patent application w/ KDDI (submitted).
COST Training School 7/06/2015*In collaboration with KDDI Research Labs, Saitama, Japan, and Harbin Institute of Technology, China.

33

System Overview
Microsoft Kinect

3a. Repair face
using 1st patchbased dictionary.
3b. Beautify facial components
using 2nd component-based
dictionary (eyes, eyebrows).

1. Capture texture & depth images.

2. Synthesize virtual image fromscreen center viewpoint.

Dual sparse coding
formulation

min x   2  1  0   S    S W ( ,   2  2  0
2

2

w

 ,

[18] X. Liu, G. Cheung, D. Zhai, D. Zhao, H. Sankoh, S. Naito, "Joint Gaze-Correction and Beautification of DIBR-Synthesized Human
Face via Dual Sparse Coding," IEEE International Conference on Image Processing, Paris, France, October, 2014.

34

Gaze Correction / Face Beautification
Image Results: no expression

COST Training School 7/06/2015

35

Gaze Correction / Face Beautification
Image Results: with expression

COST Training School 7/06/2015

36

Gaze Correction / Face Beautification
Video Results: talking gene[19] X. Liu, G. Cheung, D. Zhai, D. Zhao, “Sparsity-based Joing Gaze Correction & Face Beautification for Conference Video,"
submitted to IEEE International Conference on Visual Communication and Image Processing (VCIP), Singapore, December, 2015.

37

Sleep Monitoring via Depth Video
Problem: Detect sleep apnea non-intrusively.
Solution: Sleep monitoring via depth video analysis
1. Efficient 8-bit depth video coding via H.264.
2. Temporal video denoising to remove flicker via
graph signal smoothing.
3. Sleep event classification via GSP:

Our system:
Jawbone
UP: inexpensive,
minimal intrusive,
accurate,
non-intrusive
inaccurate
PhilipsPDx:
accurate, intrusive

1. Derive relevant video feature via ellipse model
modeling cross-section of chest & abdomen.
2. Classify data points into normal / abnormal via
graph signal interpolation.

Side view of a sleeping patient

Results:• 100% classification rate for enough training data
(same as SVM-l, SVM-rbf).
• More robust to training data errs than SVM.

Ellipse model
30 training quadruples[20] C. Yang, Y. Mao, G. Cheung, V. Stankovic, K. Chan, “Graph-based Depth Video Denoising and Event Detection for Sleep
Monitoring," IEEE International Workshop on Multimedia Signal Processing, Jakarta, Indonesia, September, 2014.

38

Heart Rate Detection from Depth Video
Heart rate detection from depth video:
Head oscillates slightly as blood circulates.

•

Methodology: joint bit-depth enhancement,
denoising via GSP, motion tracking, PCA,
0.4
identify largest frequency.
Amplitude

•

0.2

0
0

noisy temporal data

79.10 beats/min;groundtruth: 80.61 beats/min.

1

4

5

Single-sided amplitude spectrum of (a).
Amplitude

0.4

60.06 beats/min;groundtruth: 80.61 beats/min.

0.2

0
0

denoised temporal data

2
3Frequency (Hz)123Frequency (Hz)45Single-sided amplitude spectrum of (b).

[21] C. Yang, G. Cheung, V. Stankovic, “Estimating Heart Rate via Depth Video Motion Tracking,” IEEE International Conference on
Multimedia and Expo, Torino, Italy, June, 2015 (best paper candidate).

39

Outline• Introduction• What is 3D imaging?
• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate estimation• Fundamental of Graph Signal Processing (GSP)• Spectral Graph Theory• Graph-based Transform
COST Training School 7/06/201540

Traditional Signal Processingx(t)• Traditional discrete signals live on regular
data kernels (unstructured).

t• Ex.1: audio / music / speech on regularly
sampled timeline.
• Ex.2: image on 2D grid.
• Ex.3: video on 3D grid.

• Wealth of SP tools (transforms, wavelets,
dictionaries, etc) for tasks such as:• compression, denoising, restoration, classification.

COST Training School 7/06/2015

41

Smoothness of Signals• Many known signals are smooth.
• Notion of frequency, bandlimited.
N 1

1 
• Ex.1: DCT: X k   xn cos  n  k n 0

2D DCT basis is set of outer-product of
1D DCT basis in x- and y-dimension.

N

2 

a x

desired signaltransform

transform coeff.

 a0 
a 
 1
a0
 
 

0


COST Training School 7/06/2015

Typical pixel blocks havealmost no high frequency
components.

42

Sparsity of Signal Representations• “Everything should be made as simple as possible, but no simpler.”paraphrase of Albert Einstein• “Among competing hypotheses, the hypothesis with the fewest
assumptions should be selected (simplest explanation is usually thecorrect one).” Occam’s razor(over-complete) dictionary• Many known signals are sparse.

a x(sparse) code vector• KLT: decorrelate input components.

desired signal

 

E xx  C• Eigen-decomposition of covariance matrix.

• DCT approximates KLT*.
COST Training School 7/06/2015*Assuming simple AR process with same innovation variance.

covariance matrixTeigen-matrix

CQ  Q

diagonal matrixof eigen-values

43Graph Signal Processing

node
edge

• Signals live on graph.
• Graph is sets of nodes and edges.
• Edges reveals node-to-node relationships.

• Data kernel itself is structured.
1. Data domain is naturally a graph.
• Ex.1: posts on social networks.
• Ex.2: temperatures on sensor networks.

2. Embed signal structure in graph.
•Ex.1: images: 2D grid → structured graph.

Graph Signal Processing (GSP) addresses the problem ofprocessing signals that live on graphs.
[22] D. I. Shuman et al.,”The Emerging Field of Signal Processing on Graphs: Extending High-dimensional Data Analysis to Networks
and other Irregular Domains,” IEEE Signal Processing Magazine, vol.30, no.3, pp.83-98, 2013.

44

Graph Signal Processing
Research questions:• Sampling: how to efficiently acquire /
sense a graph-signal?

node
edge

• Graph sampling theorems.

• Representation: Given graph signal, how
to compactly represent it?
• Transforms, wavelets, dictionaries.

• Signal restoration: Given noisy and/or
partial graph-signal, how to recover it?
• Graph-signal priors.

COST Training School 7/06/2015

45

Graph Fourier Transform (GFT)
for Graph-signals
Graph Fourier Transform:• Signal-adaptive transform:
1. If two connected pixels are “similar”, thenedge weight is large → adjacency matrix A.
2. Compute graph Laplacian L = D-A.
3. Perform eigen-decomposition on L for GFT.

x   ai ii• Intuition: Embed geometric structure ofsignal as edge weights in graph.
COST Training School 7/06/2015

46

1

1

1

Facts of Graph Laplacian & GFT• L is a high-pass filter.

0 1 1 0
 1 2  1 0 L 0  1 2  1
0 1 1 
0• x T Lx is one measure of variation in signal.
x T Lx 

1
2
2


w
x

x




i, j
i
j
i
2 i, ji

L for 4-node unweighted line graph• L is positive semi-definite; eigenvalues λ’s ≥ 0 →eigenvalues are graph frequencies.
• L = D – A; λ = 0 must be eigenvalue w/ vector [1 … 1]T.
• Use eigenvectors for spectral decomposition of signal.
1
1

1• GFT defaults to DCT for un-weighted connected line.

1• GFT defaults to DFT for un-weighted connected circle.

1

Usage Example: first non-zero eigenvalue → spectral clustering (Shi& Malik’00).

COST Training School 7/06/2015

47

Summary• 3D Imaging: texture-plus-depth image capture.
• Immersive Visual Communication: video conf., mixed reality• 3D Data sensing & analysis: health monitoring• Key problems: 3D video coding & streaming• Other problems: denoising, interpolation, bit-depth
enhancement, de-quantization• Applications: joint gaze correction / face beautification,
sleep monitoring, heart rate detection• Fundamentals of GSP:
• Graph Fourier Transform

COST Training School 7/06/2015

48