Proc. of 17th Int. Display Workshop (IDW '10), vol. 2, Fukuoka, pp.1237-1240(2010.12.1), Invited paper.

Spatial Imaging Based on Extremely High-Definition
Computational Holography— Wave-Field Oriented 3D imaging —Kyoji Matsushima
Department of Electrical and Electronic Engineering, Kansai University,
3-3-35 Yamate-cho, Suita, Osaka 564-8680, Japan
ABSTRACT
Extremely high-definition displays beyond Super Hi-Vision
give us a new horizon in the field of 3D imaging. This paper
introduces spatial 3D imaging brought by computationalholography and techniques for numerical propagation ofwave-fields that make it possible to create the spatial 3Dimages.
1. INTRODUCTION
The resolution of display devices increases year by year. Thepixel density of the devices also increases with increase of thedisplay resolution. In current display devices, we can treat light
as a ray. However, we will have a difficulty in handling light byusing conventional ray-optics beyond the Super Hi-Vision.
The phenomenon of diffraction governs the behavior of lightof display devices in the pixel resolution less than 1 m at arough estimate. In this case, we need techniques for simulating
field propagation based on wave optics to predict what is seen in
the display screen. Many researchers and suppliers of thedisplay devices may believe that such kind of high-definition isunnecessary. However, let me emphasize that the high definition
gives us a new horizon; that is spatial imaging. The spatial
imaging is one of 3D imaging technologies, but the principle
and the reconstructed 3D image are fundamentally different
from conventional 3D images.
In this article, some techniques for field propagation in a freespace are presented for numerical simulation of diffraction.
Furthermore, the spatial imaging by computational holography
based on the field propagation is introduced as a novel
technology of 3D imaging.

numerical techniques for simulating field propagation play animportant role in the spatial imaging.
2.1 Propagation between Parallel Planes
The common category of free-space propagation ispropagation between parallel planes [1]. In this category,various methods such as the single Fourier-transform-based
Fresnel method (SFT-FR) and the convolution-based Fresnel
method are continually being proposed. However, the angular
spectrum-based method (AS) [2] is potentially the mostpowerful, because it is rigorously derived from theRayleigh-Sommerfeld integral. However, the traditional AS
cannot serve as an all-round method in a numerical
implementation due to sampling problems. The author
recently proposed the band-limited AS (BL-AS) to avoid thesampling problems [3]. This is a simple, yet effective,improvement of the AS that magnifies the range of theeffective propagation distance of the AS, as shown in Fig. 1.
2.2 Propagation between non-Parallel Planes
Another category of free space propagation is propagation
between non-parallel planes. The AS also plays an importantrole in this category. The rotational transformation of wave

Planar objectŷx̂ 70

Image sensory

2. WAVE-FIELD PROPAGATION
Wave-fields diffracted by display devices propagate in a freespace. Then, we observe the field intensity by our eye. Thus,zx(a)(b)(c)(a)(b)

Fig. 1 Diffraction by a rectangular aperture [3]Amplitude images calculated by:(a) The conventional AS [2] and (b) improved BL-AS [3].

Fig. 2 Clear imaging of deeply tilted surface [5](a) Capturing the wave-field by digital holography(b) A close-up photograph of the planar object(c) The amplitude image obtained by using the rotationaltransformation

x

Output sampling window
yz0



m
6mz



xx

Amp.

Planewave

Output sampling window
yPhaseAmp.

Circular(a) Input field(b) Output field
aperture
Fig. 3 Simulation of diffraction by a circular aperture with the incident plane wave travelling in an off-axis direction [7](a) Input field of simulation, (b) Diffraction image numerically calculated by using the Shift-AS

fields [4,5], formulated as an expansion of the AS, makes it
possible to calculate wave fields in arbitrarily tilted planes froma given source field. This method is used for clear imaging ofdeeply tilted surfaces [5], as shown in Fig. 2.
2.3 Off-Axis Numerical Propagation
Recently, a new category of free space propagation, off-axis
numerical propagation, was added to the field. This involves
propagation between parallel planes, but with the sampling
window of the output destination field shifted from that of theinput source field. This is very useful in cases where a field is
not paraxial and travels in an off-axis direction. The mostnotable method for off-axis numerical propagation is the shifted
Fresnel method (Shift-FR) [6]. This excellent technique isderived from the SFT-FR using a scaled FFT. However, theShift-FR has a serious problem of strong aliasing in shortdistance propagation.
A novel method for resolving the problem, called the shifted
angular spectrum method (Shift-AS) [7], has been proposed as ageneralization of the BL-AS. Figure 3 (b) shows amplitude
distribution of a wave-field simulated by the Shift-AS. The
wave-field is a plane wave travelling at the incident angle of  =1.5, diffracted by a circular aperture of 6 mm in diameter, as in(a). In conventional propagation methods, calculation of thediffracted fields in planes far from the aperture requires muchcomputational effort (computation time and memory), because
the diffracted field becomes more distant from the optical axis in
proportion to propagation distance. In such cases, the sampling
window needs to be expanded in conventional methods. This
problem is solved by using the Shift-AS, as in (b).

3. COMPUTATIONAL HOLOGRAPHY
3.1 Introduction to Computer-Generated Hologram
As is well-known in the principle of holography, thewave-field emitted from an object can be recorded byinterference of a reference field. The recorded object field isreconstructed by diffraction by the interference fringe pattern.
Therefore, if we can generate the fringe pattern, any object
field, even if the object is not real-existent, can bereconstructed. This fringe pattern is called ComputerGenerated Hologram (CGH).
Since the reference field is generally given by simple
mathematical expression, only synthesis of the object fieldfrom its numerical model is necessary for generating the fringe
pattern. The point-based method has been used for numericalsynthesis of object fields for a long time. However, thepoint-based methods are much time-consuming especially increation of full-parallax CGHs for surface-modeled objects.
Therefore, the polygon-based method was recently presented
for overcoming the problem [8].
3.2 The Polygon-Based Method for Surface Object
3.2.1 Theoretical model of a polygonal surface
We can see real objects illuminated by a light source,
because the object surfaces scatter the light, as shown in Fig. 4(a). Suppose that an object is composed of many polygonalplanes (polygons) and each polygon emits the wave-field. This
field is similar to that diffracted by the aperture irradiated by aplane wave, as in (b). The aperture has the same shape andslant as the polygon. However, a simple polygonal aperture

Diffuser plate

Light sourcex
Human eyesx̂
Human eyesz
Diffused lightObjectSurface

Incident wave field

Diffusedwave field(b)Aperture
Fig. 4 The theoretical model of a polygonal surface(a) The visible surface of a real object, (b) Diffuser-mounted aperture for imitating the visible surface(a)C u b ic o b je c t

3y^

2
1

G lo b a l
c o o rd in a te s

L o c a lc o o rd in a te s
y 2x 2O 2z2x^z^(a )y^N 2
A m p litu d e

P h a seA m p litu d ex
^(b )

(c )

Fig. 5 The principle for synthesizing object field by the polygon-based method(a) The numerical model of a surface object, (b) The surface function for the polygon #2, (c) The polygon field of the polygon #2
may not behave as if it is a surface source of light, because theaperture size is too large to diffract incident light, and thus light
passing through the aperture is not diffused. Therefore, thepolygonal surface source is imitated by a virtual (numerical)
diffuser that is mounted in the aperture whose shape and tilt
angle correspond with the polygon.
3.2.2 Surface functions
To compute the wave-field diffracted by the diffuser with apolygonal shape, a surface function is provided for eachindividual polygon in the local coordinate system that is also
specific to the polygon. An example of the surface function isshown in Fig. 5 (b).
The surface function h( x, y ) for a polygon is generally
given in the following form:h( x, y )  a( x, y )exp[i ( x, y )] ,where a ( x, y ) and  ( x, y ) are the real-valued amplitude andphase distribution in the local coordinates. The phase pattern ( x, y ) is not visible in principle, because all image sensors
including human retinas can detect only the intensity of light,
whereas the amplitude pattern a ( x, y ) directly determines theappearance of the polygon. Therefore, while the properties
required for the diffuser should be provided by the phase pattern,
Mesh data of object
Processing of each polygon
Generation of surface function

the shape, shade and texture of the polygon are given by theamplitude pattern.
3.2.2 Calculation of object fields
The procedure for calculation of object fields is shown inFig. 6. The surface function of a polygon is yielded from thevertex data of the polygon. Since the surface function is given
in a plane not parallel to the hologram, the rotationaltransformation is used in order to calculate the polygon field in
the parallel plane, as shown in Fig.5 (c). Then, the polygon
field is propagated shortly by using AS or BL-AS in order togather all polygon fields in a plane and accumulated in a single
frame buffer. This plane is called the object plane.
The polygon fields gathered in the object plane should be
propagated to the hologram or the plane of a silhouette mask,
as described in the following section. However, the famebuffer for the whole object field is commonly too large tosimultaneously store in memory. Therefore the field issegmented and propagated by using off-axis numericalpropagation such as the Shift-FR or Shift-AS [9].
3.3 The Silhouette Method for Light-Shielding
Occlusion is one of the most important mechanisms in theperception of 3D scenes. To reconstruct occluded 3D scenes,
light behind an obstacle must be shielded by the obstacle.
Supposing that a background wave-field emitted fromother objects behind the obstacle is given in the object plane.
In the silhouette method [9, 10], the background wave-field isBackground

Object plane

Rotational transformationof surface function
Propagation of polygon fieldby AS or BL-AS
Addition of polygon field
to frame buffer

Hologram

Object(Obstacle)

1st.
Propagation

2nd
Propagation

Propagation of whole object fieldby Shift-FR or Shift-AS
Object field

Fig. 6 The procedure for calculating the object field byusing the methods for wave-field propagation

(a)

(b)

Fig. 7 Silhouette masking of the background field [9](a) The background field masked by the object silhouette(b) The object field superimposed on the masked fieldmasked with the shape of the silhouette of the obstacle, as shownin Fig. 7 (a). Then, the wave-field of the obstacle itself issuperimposed on the masked background wave-field in theobject plane, as in (b). This combined wave-field is again
propagated onto the next object or the hologram plane.
4. HIGH-DEFINITION CGH
4.1 Calculation
The size and viewing-angle of CGHs are the most importantparameters for reconstructing high-quality 3D images. Since alarge viewing-angle requires high spatial resolution in CGHs,
both lead to an extremely large number of pixels for CGHs inthat the frame buffer usually can not be stored simultaneously inmain memory. Therefore, the frame buffer must be segmented
and the wave-field must be numerically propagated by using
methods for off-axis numerical propagation such as Shift-AS or
Shift-FR [9].
4.2 Optical Reconstruction
Figures 7 and 8 are photographs of optical reconstruction ofhigh-definition CGHs as examples. Number of pixels of theseCGHs reaches to 8 G pixels and the viewing-angles areapproximately 46 in horizontal and 37 in vertical. These
extremely high-definition CGHs give viewers a strong sensationof depth and reality that are never provided by conventional 3D
systems [11].
5. CONCLUSION
Extremely high-definition displays make it possible toproduce the light of the object itself. It is a true 3D image basedon wave-field propagation. This spatial image never has beencreated by current 3D systems that give only binoculardisparity [12].
ACKNOWLEDGEMENT
I would like to thank Prof. Nakahara for fabrication of CGHsand Mr. Nakamura for assistance in formulation and calculation.
This work was supported by the JSPS.KAKENHI (21500114).
REFERENCES[1] T. M. Kreis, M. Adams, and W. P. O. Jüptner, “Methods ofdigital holography: A comparison,” SPIE Proc. 3098,224–233 (1997).
[2] J. W. Goodman, Introduction to Fourier Optics, 2nd ed.
(McGraw-Hill, 1996), chap. 3.10.
[3] K. Matsushima and T. Shimobaba, “Band-limited angularspectrum method for numerical simulation of free-space(a) left(b) center

Fig. 7 Optical reconstruction of “The Moon”
Sizes: 131,072  65,536 pixels, Pitches: 0.8 m  1.0 m
Viewing-angle: 46  37

propagation in far and near fields,” Opt. Express 17,19662-19673 (2009).
[4] K. Matsushima, H. Schimmel, and F. Wyrowski, “Fastcalculation method for optical diffraction on tilted planes
by use of the angular spectrum of plane waves,” J. Opt.
Soc. Am. A20, 1755–1762 (2003)[5] K. Matsushima, “Formulation of the rotationaltransformation of wave fields and their application todigital holography,” Appl. Opt. 47, D110–D116 (2008).
[6] R. P. Muffoletto, J. M. Tyler, and J. E. Tohline, “ShiftedFresnel diffraction for computational holography,” Opt.
Express 15, 5631–5640 (2007).
[7] K. Matsushima, “Shifted angular spectrum method foroff-axis numerical propagation,” Opt. Express 18,18453-18463 (2010).
[8] K. Matsushima, “Computer-generated holograms forthree-dimensional surface objects with shade andtexture,” Appl. Opt. 44, 4607–4614 (2005).
[9] K. Matsushima and S. Nakahara, “Extremelyhigh-definition
full-parallax
computer-generated
hologram created by the polygon-based method,” Appl.
Opt. 48, H54-H63 (2009)[10] A. Kondoh and K. Matsushima, “Hidden surface
removal in full-parallax CGHs by silhouetteapproximation,” Systems and Computers in Japan 38,
53–61 (2004).
[11] K. Matsushima, M. Nakamura, S. Nakahara, I. Kanaya,“Computational Holography: Real 3D by Fast
Wave-Field Rendering in Ultra-High Resolution,”
SIGGRAPH 2010, Los Angeles, 77B (2010).
[12] http://www.youtube.com/watch?v=DdOveIue3sc(c) rightFig. 8 Optical reconstruction of “Aqua 2”
Photographs are taken from different angles.
Sizes: 131,072  65,536 pixels, Pitches: 0.8 m  1.0 m, Viewing-angle: 46  37