Ultra-shallow DoF imaging
using faced paraboloidal mirrors
Ryoichiro Nishi, Takahito Aoto, Norihiko Kawai, Tomokazu Sato,
Yasuhiro Mukaigawa, Naokazu Yokoya
Graduate School of Information Science, Nara Institute of Science and Technology{nishi.ryoichiro.ne6, takahito-a, norihi-k, tomoka-s,
mukaigawa, yokoya}@is.naist.jp

Abstract. We propose a new imaging method that achieves an ultrashallow depth of field (DoF) to clearly visualize a particular depth in a
3-D scene. The key optical device consists of a pair of faced paraboloidal
mirrors with holes around their vertexes. In the device, a lens-less image
sensor is set at one side of their holes and an object is set at the opposite
side. The characteristic of the device is that the shape of the point spreadfunction varies depending on both the positions of the target 3-D point
and the image sensor. By leveraging this characteristic, we reconstructa clear image for a particular depth by solving a linear system involving
position-dependent point spread functions. In experiments, we demonstrate the effectiveness of the proposed method using both simulation
and an actually developed prototype imaging system.

1

Introduction

Shallow DoF (depth-of-field) imaging highlights a target in a photograph byde-focusing undesired objects that exist outside of a certain depth range. As an
extreme condition, a microscope achieves ultra-shallow DoF imaging by putting
a target object very close to the lens. In this case, the objects except for thetiny target, e.g. like a cell, are extremely blurred and we can see what we want
to see by precisely adjusting the focus. Here, the range of DoF depends on the
combination of the distance to a target and the aperture size. One problem isthat the aperture size cannot be larger than the physical lens size. Althoughultra large lenses are required to construct ultra shallow DoF imaging systems
for standard size objects, it is almost impossible to produce such large lenses.
To solve this problem, a variety of synthetic aperture methods have been investigated and are classified into two categories: One physically captures images
from multiple viewpoints using only standard cameras and the other virtually
generates multi-view point images using cameras and some optical components.
The former methods use a moving camera [1] or a multiple camera array system[2, 3]. Although they can widen the aperture using multi-view images taken fromdifferent viewpoints and synthesize full resolution images, they require time consuming camera calibration, or complex multi-camera devices. In the latter category, a micro lens array [4–6], mask [7, 8], and a micro mirror array [9–11] have2

R. Nishi et al.

been employed to adjust the DoF. Since the aperture size in the systems cannot
exceed that of the original camera lens equipped in their systems, it is practically difficult for these methods to achieve shallower DoF. As another methodin the latter category, Tagawa et al. [12] proposed a specially designed polyhedral mirror called “turtleback reflector”, which can possibly achieve infinite-size
aperture by reflecting light rays on mirrors arranged on a hemisphere placed in
front of a camera. Although this optical system achieves the ultra-shallow DoF
imaging, the resolution of the synthesized image is quite low because all images
captured by multiple virtual cameras are recorded as one image in reality.
Another related approach is a confocal imaging one which highlights a specific
3-D point on a target by setting both the optical center of a camera and a point
light source at the focus of a lens [13–15]. In the systems based on this approach,a half-mirror enables to set them at the same focus position. Since the systems
cannot highlight all the positions on the target at the same time, they physicallyhas to scan the target while changing the highlighted positions to reconstruct
the complete image. In addition, since the systems can highlight only the target
point within the DoF which are determined by the physical lens size, the size ofthe target object is still limited by available lenses.
In this paper, we propose a novel imaging device that consists of a pairof faced paraboloidal mirrors for achieving ultra-shallow DoF imaging. Such a
device has first been developed for displaying 3D objects and is called “Mirage”[16]. This device is, for example, used for an interactive display [17]. In thisstudy, we leverage this device to capture a cross-sectional image for a specificdepth of a 3-D object. To the best of our knowledge, this study is the first
one to use the paraboloidal mirrors-based device as an imaging device. The
proposed system achieves much larger Numerical Aperture (NA) than those ofexisting lens-based camera systems, and has capability to handle larger sizeof objects than conventional microscope systems, while preserving the original
image resolution of an image sensor.

2

Device for ultra shallow DoF imaging

This section introduces the proposed ultra shallow DoF imaging device that cancapture a specific layer of an object that consists of multiple layers. Figures 1
and 2 show the developed prototype system of the proposed imaging device andits internal structure. The proposed imaging device consists of a pair of same
shaped paraboloidal mirrors whose vertex and focal point correspond with eachother and has holes at the vertexes of the paraboloidal mirrors for setting animage sensor without a lens at one side and a target object at the other side.
It should be noted that paraboloidal mirrors have a feature that light rays fromthe focal point of a paraboloidal mirror become parallel after reflecting at themirror and parallel light rays gather at the focal point after reflecting at themirror as shown in Figs. 3(a) and (b). Therefore, light rays from an object at
the focal point of an upper paraboloidal mirror gather at an image sensor at the
focal point of the lower one. Here, since the direct light rays from the object toUltra-shallow DoF imaging using faced paraboloidal mirrors

3

Imaging sensor(Lens-less camera)

Translation stage

Paraboloidal
mirrors

Light source

Fig. 1. Our prototype system.

Translation stage

Mask

Imaging sensor
Paraboloidal mirrors

Target object
Light source

Fig. 2. Internal structure of prototype system.

the image sensor disturb the visualization of an internal layer, a thin mask isplaced at the center of the proposed device for light rays from the object not
to directly reach the image sensor as shown in Fig. 2. In addition, if an object
moves from the focal point in the direction perpendicular to the image sensor
plane (referred to as depth direction), light rays from the object do not gather

4

R. Nishi et al.

Upper paraboloidal mirror

Parallel rays

Parallel rays

Lower paraboloidal mirror(a) Path of rays from focal point(b) Path of rays to focal point

Fig. 3. Path of light rays in paraboloidal mirrors.

𝑤 : width of paraboloidal mirror𝑙 : Focal length𝜃 : Aperture angle
Light path
Focal point
Fig. 4. Aperture angle in our system.

at the image sensor, resulting in generating a blurred image. Therefore, we canvisualize only a specific layer that exists at the focal depth.
Here, we discuss the Numerical Aperture (NA) of the proposed imaging device. The range of the NA is [0, 1] and the DoF gets shallower as the NA getshigher. The NA is defined in general as follows:
N A = n sin θ,(1)

where n is the index of refraction of the medium between a target object andan image sensor. In our case, n = 1.0 because the medium is the air. θ is theaperture angle which means the maximum angle between the optical axis andavailable light rays as shown in Fig. 4. We can calculate sin θ using the width w

Ultra-shallow DoF imaging using faced paraboloidal mirrors

U-axis5

V-axisZ-axisY-axisX-axisFig. 5. Definition of coordinate systems.

and focal length l of the paraboloidal mirror as follows.
sin θ = q

w
2

w 2
+
2


l 2
2

.

(2)

Here, since the expression of the paraboloidal mirror where y = 0 in the coordinate system as shown in Fig. 5 is expressed as x2 = 4lz, the relationshipbetween the width w and the focul length l can be calculated from the following
equation.
 
 w 2
l
= 4l
.
(3)
2
2
By replacing w with l, the NA of the proposed device is calculated as follows:
√
√
2l
2 2
=
N A = sin θ = q' 0.94.
(4)2
3
2l2 + l4
It should be noted here that the NA of the proposed device is a constant even
if the scale of the paraboloidal mirror changes because the NA does not dependon the focal length and width of the paraboloidal mirror as indicated in Eq. (4).
Table 1 shows the comparison of the NAs of the proposed device and various
commercial lenses. In this table, we calculated approximate NAs from F numbers
using the following equation
1
NA =
(5)
2F
From the table, we can confirm that the NA of the proposed device is muchlarger than those of various camera lenses and has competitive performancewith objective lenses of microscopes. It should be noted that our system canhandle relatively larger objects without constructing an ultra-large lens system
which is practically almost impossible.

6

R. Nishi et al.
Table 1. Comparison of Numerical Aperture of various lenses.
NA
0.94
0.85

Lens type
Mirror lensObjective lens(microsocpe)
Large aperture lensTrade nameOur approach
WRAYMERGLF-ACH60X
0.59
HandeVision
IBELUX 0.85/40MM
0.53 Fixed focal length lens
SCHNEIDER
F0.95 Fast C-Mount Lens
0.36 Large aperture lensCanon
EF35mm F1.4L II USM

For the proposed device, depth of field d can be determined geometrically as
follows:d = df ar − dnear ,(6)

where dnear and df ar are distances from the image sensor to the nearest andfarthest points that are in focus, respectively, and are calculated as follows:l2,
l + 2F cl2
df ar =,l − 2F cdnear =

(7)
(8)

where c is the size of a circle of confusion, which is the size of a pixel of an image
sensor. For example, when c is 0.01 mm, l is 100 mm, and the F number is 0.53
(i.e., N A is 0.94), depth of field d becomes 0.0212 mm.

3Experiments

In experiments, first, in order to check the raw performance of the proposed
imaging device, the effect of geometric aberration is evaluated by measuring
PSFs (point spread functions) which vary depending on the 3-D position ofa target point, using a simulation environment. Layered real-images are thencaptured using a prototype imaging device and compared with images captured
by a conventional lens-based camera device which has large NA in order to show
the feasibility and the advantage of the proposed device. In addition to these
two basic experiments for the proposed device, we further show the possibility
to remove blurs on captured images using measured PSFs.
3.1

Characteristic of faced paraboloidal mirror-based imaging
deviceThe shape of the PSF, which is a response of an impulse input from a point lightsource, varies depending on the 3-D position of the point light source placed

Ultra-shallow DoF imaging using faced paraboloidal mirrors

7in the proposed device. In order to analyze the characteristic of the proposed
device, shapes of PSFs for different light source positions in the proposed imaging
device are measured in a simulation environment. Experimental setting in thissimulation is as follows: Focal length p is set to 65 mm. Width w of the mirror
device is determined as 184 mm from Eq. (3). An imaging device (20 mm × 20mm, 201 × 201 pixels) is fixed at one of the two vertex positions. While moving
the position of a point light source, we observed shapes of PSFs by this imaging
device.
Figures 6 and 7 show the PSFs captured on the image plane (U, V ) whilemoving a light source along the Z axis and X axis shown in Fig. 5, respectively.
Since the proposed imaging device is rotationally symmetric, we can know thecharacteristic of the device using these two axes. From these figures, we can seethat the shape of PSF drastically changes when the light source moves along Z
axis as shown in Fig. 6, while the change of the PSF along X axis is comparatively
moderate as shown in Fig. 7. This indicates that the object moving along Z
axis from the vertex position immediately blurs, in contrast to the case for X
direction. From this simulation, we can confirm the desirable characteristic of
the proposed device for achieving ultra-shallow DoF imaging.
3.2

Ultra shallow DoF imaging using prototype

We have constructed the prototype device shown in Fig. 1. In this device, Point
Grey Grasshopper2 (1,384 × 1,036 pixels, CCD) without a lens is employed as
the image sensor and fixed at the vertex of the upper paraboloidal mirror. A
target object is set at the vertex of the lower paraboloidal mirror and its depth
can be adjusted by using a translation stage as shown in Fig. 2.
Figure 8 shows target objects 1 to 3 in this experiment which consist of twolayered flat surfaces which are transparent films with 0.1mm thickness where
different images are printed. The sizes of surfaces are 20 mm × 20 mm and twolayers are separated with 1.2mm empty gap.
Target 1 and 2 shown in Fig. 8(a),(b) are layered objects where a grid mask
texture which contains high frequency component is commonly used as upper
layer images and low and high frequency textures are used for lower layer images
respectively. Target 3 (Fig. 8(c)) has low frequency texture image for upper layer
and high frequency texture image for lower layer where lower layer is almost
completely blinded with upper layer in a standard camera image.
Figures 9 to 11(a) show images captured by the camera (Grasshopper2) witha small DoF lens (Schneider Fast C-Mount Lens, 17mm FL, F=0.95 (NA=0.53))
for different height positions Z of the target objects. As we can see in thesefigures, the lower layer images are partially blinded by the grid patterns for thetarget 1 and 2, and the lower image is completely blinded for the target 3 evenwhen we have employed relatively shallow DoF lens.
In contrast to this, one of two layer images is largely blurred by the proposed
device not depending on the combination of low and high frequency textures, andthe other layer image is focused as shown in Figs. 9 to 11(b). Even for the target
3, the characters behind the upper layer image are readable as shown in Fig.

R. Nishi et al.

Position of lightpoint along Z-axis

Normalized photometric intensity

8

0.00mm0.10mmz = 0.00mm

0.20mm
0.30mm
0.40mmz = 0.20mm( brightness×10)

z = 0.40mm( brightness×40)

U-axis[mm]Fig. 6. Observed PSFs for different Z positions of a point light source with X = 0.
(left: slice of PSF with V = 0, right: cropped PSF image)

Normalized photometric intensity

Position of lightpoint along X-axis
0.00mm
1.00mmx = 0.00mm

2.00mm
3.00mm

4.00mmx = 2.00mm

U-axis[mm]x = 4.00mm( brightness×3)

Fig. 7. Observed PSFs for different X positions of a point light source with Z = 0.
(left: slice of PSF with V = 0, right: cropped PSF image)11(b). By this comparison, we can conclude that our system can achieve muchshallower DoF imaging than the conventional lens-based system. However, we
also confirmed that the proposed device still has two problems: (1) The images
captured by the proposed device blur in the peripheral regions more than theconventional lens, which means that the proposed device has worse geometricaberration than conventional lenses, and (2) textures from the other non-focusedlayer still remain a little in the captured images.

Ultra-shallow DoF imaging using faced paraboloidal mirrors

9

20mm
20mm

Target object

Upper layer(a) Target 1

Lower layerAB

Z= -1.8mm

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm C D
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(a) Images captured by NA=0.53 lensTarget object

Z= -1.8mm

Upper layer(b) Target 2

Lower layer

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)

AB(b) Images captured by prototype device
Z= -1.8mm
Z= -1.2mm
Z = -0.6mm
Z= 0.0mm C D
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(a) Images captured by NA=0.53 lensTarget object

Upper layer(c) Target 3

Lower layer

Fig. 8. Target objects and texture images for layers.

Z= -1.8mm

3.3

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)

by prototype
deviceReconstruction(b)ofImages
layercaptured
imagesusing PSFs

Z= -1.8mm

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(a) Images
capturedby NA=0.53
lens has the weakness about
As described in the previous
section,the proposed
devicethe blurring effect caused by the aberration and the influences from other layers. In order to confirm the future possibility to overcome this weakness, here
we simply deblur the observed images using measured PSFs by the following
manner.

In -1.8mm
the proposedZ=system,
vectorizedZ=observedo can beZ=
-1.2mm theoretically,Z = -0.6mm
0.0mm image
Z = +0.6mm
on upperlayer)(Focused of
on intensity
lower layer)
represented as(Focused
the weighted
sumof the multiplication
wk of the k-th
Images captured
devicepoint light source and (b)vectorized
PSF pbyfor the light-source
position k in the
k prototype

20mm
20mm

10

R. Nishial.
Targetetobject

Z= -1.8mm

Upper layer(a) Target 1

Lower layer

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(a) Images captured by NA=0.53 lensZ= -1.8mm

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(b) Images captured by prototype device
Fig. 9. Experimental results for target 1.

device as follows:o=X

wk pk ,

(9)k

where we ignore occlusion effects for simplicity. Here, if pk is given by the calibration as shown in the first experiment and we have multiple observed images owith different depths, we can easily estimate wk , which means that the aberration
is suppressed and the captured images are decomposed
into ones for respectivePlayers, by minimizing the sum of error ko − k wk pk k2 subject to wk ≥ 0 withconvex optimization [18].
In this experiment, we have tested this method using the same device configuration with the second experiment in the simulated environment. The target
object here consists of two layered films with 0.5 mm gap where different images
are printed as in Fig. 12, which is more severe situation with narrower gap thanthat in the previous experiment. For this target, we have captured five images bymoving the height of the object (focus point of the device) with 0.25mm interval
as shown in this figure.
Figure 13 shows the effect of decomposition. In this figure, (a) shows originallayer images, (b) shows the images captured by focusing on upper and lower
layers, (c) shows the decomposed results from the two images of (b), and (d)shows the decomposed results from all the five images. As shown in (b), even
if the focus point is precisely adjusted to the position where target layer exists,
the effect from the other layer image cannot be avoided, resulting blur effectin this severe situation as similar to the results in the previous experiment. As
shown in (c) and (d) in Fig. 13, using this comparatively simple decomposition
algorithm, the blurs were successfully reduced even by the two images, and were
almost completely removed by the five images.

ABCD

DoF imaging
TargetUltra-shallow
objectUpper using
layer faced paraboloidal
Lower mirrors
layer(b) Target 2

Z= -1.8mm

11Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(a) Images captured by NA=0.53 lensABCD
Z= -1.8mm

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(b) Images captured by prototype device
Target object
Upper layer
Lower layer(c) Target
3
Fig. 10. Experimental
results
for target 2.

Z= -1.8mm

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(a) Images captured by NA=0.53 lensZ= -1.8mm

Z= -1.2mm
Z = -0.6mm
Z= 0.0mm
Z = +0.6mm(Focused on upper layer)(Focused on lower layer)(b) Images captured by prototype device
Fig. 11. Experimental results for target 3.

On the other hand, in the real world, to decompose images captured by the
prototype system, we need to know a PSF for each spatial position in a target
scene. A straightforward way to measure the PSFs is to prepare a hole whose
size is smaller than that of a circle of confusion and align it with each spatial
point. However, it is almost infeasible to create such a small hole and align itwith each position accurately without special and expensive devices. Therefore,
we should develop a method to measure PSFs that is an alternative to the way
above in the future.

12

R. Nishi et al.

Photographed images

Upper layer

Lower layer

Focused plane

Layer gap
0.5mm

Interval
0.25mmFig. 12. Texture images for layers and captured input images for different height positions in simulation.

(a) Original layer images (b) Captured images (c) Decomposed result (d) Decomposed result
from two images
from five images
Fig. 13. Effect of decomposition. Top and bottom row show upper and lower layer
images, respectively.

4

Conclusion

This paper has proposed an ultra-shallow DoF imaging method using faced
paraboloidal mirrors that can visualize a specific depth. We constructed a prototype system and confirmed the proposed device can capture a specific depth
using objects with two layers clearer than a small DoF lens. In the experimentusing a simulation environment, we analyzed the characteristic of the proposed

Ultra-shallow DoF imaging using faced paraboloidal mirrors

13

device, and we showed that the proposed system also can suppress the aberrationand decompose layered images into clear ones using measured PSFs. In future
work, we will develop a decomposition method considering occlusions and apply
it to images captured by the developed prototype system.
Acknowledgement. This work was supported by JSPS Grant-in-Aid for Research Activity Start-up Grant Number 16H06982.

References
1. Levoy, M., and Hanrahan, P., Light field rendering. Proc. SIGGRAPH, pp. 31-42,
1996.
2. Vaish, V., Wilburn, B., Joshi, N., and Levoy, M., Using plane + parallax for calibrating dense camera arrays. Proc. CVPR, Vol. 1, pp. I-2-I-9, 2004.
3. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams,A., Horowitz, M., and Levoy, M., High performance imaging using large camera
array. ACM Trans. on Graph., Vol. 24, No. 3, pp. 765-776, 2005.
4. Adelson, E. H., and Wang, J. Y. A., Single lens stereo with a plenoptic camera.
IEEE Trans. on PAMI, pp. 99-106, 1992.
5. Ng, R., Levoy, M., Bredif, M., Duval, G., Horowitz, M., Hanrahan, P., Light field
photography with a hand-held plenoptic camera. Proc CTSR, Vol. 2, No. 11, pp.
111, 2005.
6. Cossairt, O., Nayar, S., and Ramamoorthi, R., Light field transfer: global illumination between real and synthetic objects. ACM Trans. on Graph., Vol. 27, No. 3,
pp. 57:1-57:6, 2008.
7. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J., Dappled photography: mask enhanced cameras for heterodyned light fields and coded
aperture refocusing. ACM Trans. on Graph., Vol. 26, No. 3, pp 69-76, 2007.
8. Liang, C., Lin, T., Wong, B., Liu, C., and Chen, H. H., Programmable aperture
photography: multiplexed light field acquisition. ACM Trans. on Graph., Vol.27,
No.5, pp. 55-1-55-10, 2008.
9. Unger, J., Wenger, A., Hawkins, T., Gardner, A., and Debevec, P., Capturing andrendering with incident light Fields. Proc. EGSR, pp. 141-149, 2003.
10. Lanman, D., Crispell, D., Wachs, M., and Taubin, G., Spherical catadioptric arrays:
construction, multi-view geometry, and calibration. Proc. 3DPVT, pp. 81-88, 2006.
11. Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., and Bolas, M., synthetic aperture confocal imaging. Proc. SIGGRAPH, pp. 825-834, 2004.
12. Tagawa, S., Mukaigawa, Y., Kim, J., Raskar, R., Matsushita, Y., and Yagi, Y.,
Hemispherical confocal imaging. IPSJ Trans. on CVA, Vol. 3, pp 222-235, 2011.
13. Minsky, M., Microscopy apparatus. US Patent 3013467, 1961.
14. White, J., G., Amos, W. B., and Fordham, M., An evaluation of confocal versus
conventional imaging of biological structures by fluorescence light microscopy. JCB,
Vol. 501, No. 1, pp. 41-48, 1987.
15. Tanaami, T., Otsuki, S., Tomosada, N., Kosugi, Y., Shimizu, M., and Ishida, H.,
High-speed 1-frame/ms scanning confocal microscope with a microlens and nipkow
disks. APPLIED OPTICS, Vol. 41, No. 22, pp. 4704-4708, 2002.
16. Adhya, S., and Noé, J., A complete ray-trace analysis of the ’Mirage’ toy. Proc.
SPIE ETOP, pp. 966518-1-966518-7, 2007.

14

R. Nishi et al.

17. Butler, A., Hilliges, O., Izadi, S., Hodges, S., Molyneaux, D., Kim, D., and Kong,
D., Vermeer: direct interaction with a 360◦ viewable 3D display. Proc. UIST, pp569-576, 2011.
18. Gabay, D., and Mercier, B., A dual algorithm for the solution of nonlinear variational problems via finite-element approximations, Comput. Math. Appl., vol. 2,
pp. 17-40, 1976.