Taking advantage of virtual optics
in computer holography

Kyoji Matsushima, Daichi Fujita,
Yutaka Yoshizaki, Sumio Nakahara

Kansai University, JAPAN
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Introduction
Background••••Resolution of high-definition CGH are being comparable with that inclassical holography.
Computer holography is the technique for creating CGHs.
Digitized holography is the technique to digitize the whole process ofholography; real-existing object waves are recorded by an image sensor
and the object waves are reconstructed by CGHs.
In computer holography and digitized holography, object waves arehandled as digital wave-fields, and thus both can take advantage ofvirtual optics.

Outline••Source material of 3D scene in computer holography
Some techniques used in our computer holography– Polygon-based method and Silhouette method•••

Creation of holographic stereograms through virtual optics withnumerical methods
Techniques to realize digitized holography
Digital editing of 3D scene and resizing of 3D object in digitizedholography

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Some of early works

The Venus, 2009
The first work
4 G Pix (64K×64K)

Aqua 2, 2009
Hidden-surface removal
8 G Pix (128K×64K)

The Moon, 2010
Texture-mapping
8 G Pix (128K×64K)

1K = 1024
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Some of early works

The Metal Venus I
The Metal Venus II2010
Specular flat shading 2011
Specular smooth shading

1K = 1024

iShion V2, 2011
Image hologramWhite light reconst.

Bear II, 2010
Digitized holography

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Computer Holography: A Concept
Source materials
Multi-viewpoint
image

Picture image
2D object

Field synthesis

 Multiple source materials areused to build the 3D scene.
 Synthesized or captured
fields are integrated in a
virtual 3D scene using fieldbased digital editing.

Polygon-mesh
3D object

Real-existing
objectField capturing

Digital editing
Printing fringe pattern

Computer hologram

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)3D scene in computer holography
Multi-viewpoint image (Background)
Captured field (Real-existing object)
CG-modeled 3D object(Virtual object)

Photographor illustration

 Conceptual illustration of atypical 3D scene of computer
holography.

Hologram

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Computer holography for polygon-mesh objects
Multi-viewpoint
imagePicture image
2D object

Field synthesisPolygon-mesh
3D object

Real-existingobject

Field capturing

Digital editing
Printing fringe pattern

Computer hologram
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Polygon-based method
Point sourceof light

Spherical
wave

Ray-oriented
renderingSurface sourceof light(polygon source)

Polygon field

Field-oriented
rendering
Sampling
grid

Polygon-based method is usually faster than the point-based method in
rendering surface objects, because the number of the polygons composing asurface object is much smaller than that of point sources.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Amplitude

Surface function

y1Phase^y1 y1x1

^

x1
y3

^

x1

y3y2

Quasi random
phase plays
role of diffuser

y2

^

y2
P1x2

^

x3
x3

P3

P2x2

^

x2

^

yy3
^

x^

Hologramzhn ( x n , y n )  a n ( x n , y n ) exp[ i n ( x n , y n )]Amplitude

Phasex3

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)^y1

Surface function^x1^

y1y2x1^x2y2x2

^

y3y3
^

x3x3

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Texture mapping in polygon-based method
SphereSurface function
Texturean ( xn , yn ) I n ( xn , yn )In(xn, yn) : Texture image
Amplitude

Phase

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Brothers exhibited in MIT museum
3D laserscanner

The shape of live faces are measured by 3D laser scanner,
Konica Minolta Vivid 910.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Optical reconstruction
Exhibition inMIT museum

Brothers, 2012
Diffuse smooth shading, Live faces
Big hologram, 25 G pix (196,608×131,072)
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Exhibition in MIT museum

Exhibition of BrothersHODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Exhibition in MIT museum
Brothers

Description

LED light box

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Digitized holography:
Computer holography for real-existing objects
Multi-viewpoint
image

Picture image
2D object

Polygon-mesh
3D object

Real-existingobject
Interference
Digital holography

Field synthesis
 The object field is
captured by the techniqueof digital holography.
 The captured field isdigitally edited and mixed
with virtual 3D scene.
 The mixed field is opticallyreconstructed bycomputer-hologram.

Field capturing

Digitized
Digital editingHolography
Printing fringe pattern

Diffraction

Computer hologram

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Problems to realize digitized holography
1. Expansion of captured area (Increase of number of samplings)
2. Reduction of sampling intervals

 The captured area ofcurrent image
sensors is not
sufficient for highdefinition computerholograms.
 The samplinginterval of thecaptured field is also
not sufficient forhigh-definition
computer holograms.

Image sensor
3,000 × 2,200 pixels(3.5 m ×3.5 m)

Shion, 2010
131,072×65,536 pixels (0.8 m ×0.8 m)
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Techniques for capturing large fields at highsampling density
1. Lensless-Fourier setup for reducing sampling interval
2. Synthetic aperture digital holography for extending captured area

Interference
fringeImage sensor As the sensor moves, a part of field is captured.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)3D scene including real-existing fields : Penguin
Wallpaper of 2D
digital image

160

CG-modeled 3D object

80

2D digital images
Captured fieldsof real-existing objects

10050

Hologram

50
Units : mm10065131z

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Optical reconstruction

Penguin, 2012
Digitized holography8 G pix

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Field-based digital editing of 3D scene
Component 0

Component 1

Component 2
Component 3
Component 4
 The 3D scene in computer
holography includes many
components.
 The mutual occlusion among
components must be reproducedin optical reconstruction.
 We use silhouette method toachieve this.

Hologram

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)The silhouette method: light-shielding
by opaque object
Silhouette mask
Wave-fieldObject

Real object

Wave field rendering A real opaque object shields light behind the object.
 In computer holography, to prevent the object being a phantom
image, the field of incident light is masked by the object' silhouette.
 We calculate the background field at the center of object and thenmask it by the object’ silhouette.

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Example of Silhouette Masking
Background

Object planeHologram The background field,

1stpropagation

2ndpropagation+Wave-fieldof backgroundpropagated to theobject plane, ismasked with thesilhouette of theVenus stature.
 After superimposing
Venus field, the wave
field is propagated tothe hologram.

=

Superimposed waveWave-field
field in object planeof object
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Silhouette mask in digitized holographybinarize
andreverse

Numerical
Reconstruction

Small part ofcaptured field

Amplitude image

Silhouette mask In digitized holography, we have no information about the object shape.
 Numerical reconstruction of real fields is used for produce the silhouette mask.

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Problems of self-occluded objects
Masking by silhouette of object

Self-occluded
objectMasking by silhouette of polygon

Self-occluded
object Masking technique by object silhouettes does not work well in selfoccluded objects.
 To properly shield light of self-occluded object, fields must be masked by
the silhouette of every polygon.
 However, masking by the polygon’s silhouettes is very time-consuming.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Procedure of switch-back method

Hologram

Polygon P2
Polygon P4

Polygon P1

Polygon P3

Polygon P5

 The switch-back method is developed to speed-up masking by
the polygon’s silhouettes.
 In this technique the field of object is numerically propagatedback and forth and masked by the polygon’s silhouette.
 Many formulas are necessary for the explanation of theHODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)mechanism.

Hologram for self-occluded object

Rose in Ring, 2012
Self-occluded object (Switch-back method)
Specular smooth shading
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)USE OF VIRTUAL OPTICS

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Resizing objects in digitized holography
Magnification
M baW𝑓𝑓a𝑊𝑏byyx

22 ( x  y )Lens function exp   if
yx𝑓axz𝑓b

Numerical propagation
Numerical propagation
Objectplane
Image plane
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Computer hologram for resizing objects: Hamsters
Background
Fields of resizedobjects
M = 1.5
Original object field

M = 0.5

M = 1.5M = 0.25

Hologram Hamsters was created for testing
the technique.
 The 3D scene include 3 resized
object obtained from the originalfield.

Original
M = 0.5 M = 0.25
Hamsters, 2012
Resizing objects

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Using multi-viewpoint image as backgroundSource materials
Multi-viewpoint
image

Picture image
2D object

Field synthesis
 2D digital illustration orpictures were used forbackground so far.
 Multi-viewpoint image isdesirable for the backgroundof the 3D scene.
 A holographic stereogram isnumerically produced andarranged in the 3D scene.

Polygon-mesh
3D object

Real-existingobject

Field capturing

Digital editing
Printing fringe pattern

Computer hologram

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)3D scene in computer holography
Multi-viewpoint image (Background)CG-modeled 3D object(Virtual object)
Numerical
back-propagation
Hologram

Numerical propagation

View pane
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Hologram using multi-viewpoint image

Parked Car 1, 2012
Multi-viewpoint image as background

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Software• Software can be downloaded from our website:– http://www.laser.ee.kansai-u.ac.jp/WaveFieldTools– Or search keyword WaveFieldTools– All contents are written in Japanese.

• The current software is provided as Library of C++.

Windows software
will be available in afew months.

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)Conclusion• Computer holography can use many source

materials and create brilliant CGHs whose
reconstructions are comparable with that bytraditional holography.
• Computer holography has the advantage thatthe 3D images are digitally archivable andtransmittable.
• Real-existing objects and 3D scenes can beincorporated into CGHs by using digitized
holography or as holographic stereograms
through virtual optics with numerical methods.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)References
•••
•

•••

•
•

K. Matsushima, S. Nakahara, et al.: Computer holography: 3D digital art based on high-definition
CGH, International Symposium on Display Holography 2012 (ISDH2012), MIT Media Lab, (2012).
http://river-valley.tv/computer-holography-3d-digital-art-based-on-high-definition-cgh/
K. Matsushima, H. Nishi, S. Nakahara: Simple wave-field rendering for photorealisticreconstruction in polygon-based high-definition computer holography, J. Electron. Imaging 21,023002(2012). DOI: 10.1117/1.JEI.21.2.023002
H. Yamashita, K. Matsushima, S. Nakahara: Image-type high-definition CGHs encoded byoptimized error diffusion, SPIE Proc. #8281, 82810Z(2012.1.25). DOI: 10.1117/12.908353
H. Nishi, K. Matsushima, S. Nakahara: Advanced rendering techniques for producing specularsmooth surfaces in polygon-based high-definition computer holography, SPIE Proc. #8281,828110(2012). DOI: 10.1117/12.908871
K. Matsushima, Y. Arima, S. Nakahara: Digitized holography: modern holography for 3D imagingof virtual and real objects, Appl. Opt. 50, H278-H284 (2011). DOI: 10.1364/AO.50.00H278
H. Nishi, K. Matsushima, S. Nakahara: Rendering of specular surfaces in polygon-based
computer-generated holograms, Appl. Opt. 50, H245-H252 (2011). DOI: 10.1364/AO.50.00H245
K. Matsushima, S. Nakahara: Extremely High-Definition Full-Parallax Computer-Generated
Hologram Created by the Polygon-Based Method, Appl. Opt. 48, H54-H63 (2009). DOI:
10.1364/AO.48.000H54
K. Matsushima: Computer-Generated Holograms for Three-Dimensional Surface Objects withShade and Texture, Appl. Opt. 44, 4607-4614(2005). DOI: 10.1364/AO.44.004607
K. Matsushima, A. Kondoh: A Wave Optical Algorithm for Hidden-Surface Removal in Digitally
Synthetic Full-Parallax Holograms for Three-Dimensional Objects, SPIE Proc. #5290, 90-97(2004).
DOI: 10.1117/12.526747

HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)