Making Your Bokeh Fascinating
Real-time Rendering of Physically Based Optical Effect in Theory and Practice
SIGGRAPH 2015 Course

Masaki Kawase
Silicon Studio, Corp.
masa@siliconstudio.co.jp

Introduction• Basic idea and theory [Kawase08]– Only circular aperture• Practical implementation and optimization [Kawase12]– Any kind of aperture shapes

2

Contents• Creating the Pencil Map• Creating the Bundle-of-Light-Ray Map (Pencil Map)
– “Bundle of Light Rays” or “Pencil Rays”(referred to as “Pencil” here onwards)• Application to Arbitrary Aperture Shapes• Scattering or Gathering?
• Results• Conclusion
3

Creating the Pencil Map

Creating the Pencil Map• Precompute light paths from the aberration diagram– Takes spherical and axial chromatic aberrations into account

Incident heightAmount of aberration

Longitudinal aberration
diagramLight paths make up
pencil map(small number of rays)

Pencil map

5Circular Bokeh Rendering• V coordinate represents the distance from the optical axis
– Mapping each slice to a circle produces a circular ‘bokeh’
V: Distance fromoptical axis

Pencil map

U: Distance from lensCircular bokeh6

Discretized Result…
• Chromatic aberration is an issue• Three wavelengths (R/G/B) are insufficient to represent thedispersion

7

Increasing Wavelength Samplings• Calculate the map with more wavelengths• Convert into the RGB space8

3-Wavelength Samplings9Sufficient Wavelength Samplings

10Bokeh with Spherical and Chromatic Aberration• Imperfect focus• Front bokeh with red sharp edge
• Back bokeh with blue soft edge

Zoomed-in view around the focal point
11

Comparison with photographs

Generated from pencil map

Captured in real photographs
12Creating the Pencil Map of Doublet• Calculate the map with the longitudinal aberration diagram• Using actual lens parameters (if they exist)– Only ray paths of each wavelength are required

Longitudinal aberration
diagramLight paths make up
pencil map

Pencil map

13

Pencil Map of Doublet
Zoomed-in view around the focal point

Pencil maps and bokeh
14Comparison with photographs• Typical correction– Front bokeh has soft purple edge and the center is darker– Back bokeh has sharp green edge and the center is brighter

Front and back Bokeh with Pencil map

Front bokeh in photographs

Back bokeh in photographs

15

Different Type of Doublet• Residual chromatic aberration is more visible thanresidual spherical aberration

Longitudinal aberration
diagramLight paths make up
pencil map

Pencil map

16

Pencil Map of Doublet (Different Type)
Zoomed-in view around the focal point

Pencil maps and bokeh
17Pencil Map of Doublet (Previous Type)
Zoomed-in view around the focal point

Pencil maps and bokeh
18Comparison with photographs

Front and back Bokeh with Pencil map

Front bokeh in photographs

Back bokeh in photographs
19

Optimization of Pencil Map• Wasteful parts in the texture– Sparse, many texels are empty– There is not enough precision around the more important‘focusing’ texels

Wasteful pencil map
20

Optimization of Pencil Map (cont’d)• Normalizing height of bundle at every distance(u-axis)
by the maximum height(bokeh size)• Less empty texels, and great improvement in precision
around focusing texels

Wasteful pencil map

Normalized pencil map
21

Spherical lens (with no corrections)

Aspherical lens (chromatic aberration is not corrected)

Achromatic doublet lens22

Achromatic doublet lens (different type)

Apochromatic (APO) lensPerfect lens (does not actually exist)

23

Application to Arbitrary Aperture Shapes

Various Aperture Shapes• Aperture shape is an important artistic factor– Typically 5~9 diaphragm blades– Changes from rounded to n-gon• How to map pencil onto the polygonal aperture shape?
– 3D Textures?
• Too large, not practical?

Pencil mapAperture shape25

Indirect Reference of Pencil Map• Precompute an LUT texture that stores V coordinates ofpencil map
00V00001

Vertical slice

10000000

LUT texture
26

Indirect Reference of Pencil Map (cont’d)• Sample pencil map indirectly
at run-time
Sampling0V
1Pencil map
Look up

LUT27

Indirect Reference of Pencil Map (cont’d)• LUT determines the aperture shape– Independent of pencil map– Can reproduce curved shapes of a diaphragm blade• Prepare a set of LUTs for various diaphragm conditions– Other shapes such as stars, hearts, … can be used28

Various LUTs
• For the number of diaphragm blades and opening levels• Smooth deformation is possible by interpolating between two adjacent LUTs

Fully opened

Closing

Closed4 blades5 blades6 blades7 blades29

Silhouette LUTs for debug

Fully opened

Closing

Closed4 blades5 blades6 blades7 blades30

Scattering or Gathering?

Both can be Implemented• Better quality by scattering– Heavy processing load

• Hybrid method is recommended– Both scattering and gathering32Hybrid Method• To determine which pixels will be scattered or gathered,use:– The CoC size– Difference in luminance between neighboring pixels

Original

Result (green: gathering pixels)
33

Gathering34Hybrid of Scattering and Gathering

35

Optimization• Use a half resolution buffer for scattering– Scattering process can be 16x faster• Split the process into several passes withhierarchical resolution buffers– Use lower resolution for larger bokeh– The process at the 1/4 x 1/4 resolution can be 256x faster• Scatter a pixel every 2x2 pixels
for relatively larger bokeh in each resolution– Pixels that have an especially heavy processing load will be 4x faster36

Results

Bokeh Simulation in Real Time

Front bokeh with red/purple fringes
Back bokeh with cyan fringes

Diaphragm Simulation
5-blade ApertureDiaphragm Simulation
6-blade ApertureDiaphragm Simulation
7-blade ApertureDiaphragm Simulation
8-blade Aperture

Curved Diaphragm and Optical Vignetting• Opening / Closing– Deformation• Circular aperture• Polygonal aperture– Rotation– Optical Vignetting• Cat’s Eye Effect43

5-blade Circular Aperture (with Optical Vignetting)f/1.4 (Fully Opened)5-blade Circular Aperture (with Optical Vignetting)f/2 (1 Stop Closed)

5-blade Circular Aperture (with Less Optical Vignetting)f/2.8 (2 Stops Closed)

5-blade Circular Aperture (with No Optical Vignetting)f/5.6 (4 Stops Closed)

5-blade Circular Aperture (with Fake Diffraction Spikes)
f/11 (6 Stops Closed)Various Aberrations and Corrections• Correction of SA and axial CA mostly affect bokeh

Back bokeh
Focus plane
Front bokeh

Differences between front and back bokeh
49

Spherical Lens (with No Corrections)50

Aspherical Lens (with Chromatic Aberrations)
Correction of Spherical Aberration

51

Achromatic Lens
Corrected for Red and Blue Wavelengths

52

Achromatic Lens (Different Type)
Corrected for Red and Blue Wavelengths

53

APO (Apochromatic) Lens
Corrected for 3 Wavelengths (More Expensive Lens)

54

Almost Perfect Lens
Without any Spherical and Longitudinal Chromatic Aberrations

55

STF (Smooth Transition Focus) Lens
Soft Edged Bokeh by Apodization Optical Element

Radial gradient ND filter56

Residual Aberration Visibility

Out of focus: Small
Aberrations: Strongly Visible

Out of focus: Large
Aberrations: Less visible

Residual Aberration Visibility

Out of focus: Large
Aberrations: Less visible

Out of focus: Small
Aberrations: Strongly Visible

Out of focus: Large
Aberrations: Less visible

Out of focus: Small
Aberrations: Strongly Visible

Comparison with Photographs (Achromat)

Real-time simulation results

Real photographs with achromatic lens59Comparison with Photographs (APO)

Real-time simulation results

Real photographs with apochromatic lens60Conclusion

Conclusion• Reproduce photorealistic bokeh with pencil map and LUT– Pencil map defines bokeh characteristics– LUT defines bokeh shapes• Optimization– Various options available– Combinations can be used to improve performance62

References••••••[Kawase08]
Kawase, M. “Bokeh Expressions Based on Optics.” Computer Entertainment Developers
Conference, 2008.
[Kawase12]
Kawase, M. “Practical Implementation of Cinematic Lens Effects.” ComputerEntertainment Developers Conference, 2012.
Kawase, M. “Reduce Artifacts Generated by Mipmapped Buffers.” Computer Entertainment Developers
Conference, 2009.
Trávník, J. “On Bokeh.” Jakub Trávník's resources. http://jtra.cz/stuff/essays/bokeh/index.html
安藤幸司 『光と光の記録「レンズ編」』 AnfoWorld http://www.anfoworld.com/LensMF.html
吉田正太郎（1997）『カメラマンのための写真レンズの科学』地人書館.

63