Recipes for Optical Effect System Design
Real-time Rendering of Physically Based Optical Effect in Theory and Practice
SIGGRAPH 2015 Course

Masaki Kawase
Silicon Studio, Corp.
masa@siliconstudio.co.jp

Contents• Parameters– NOT Based on Optics– Based on Optics but NOT based on Photography– Based on Both Optics and Photography• Optical Animation– Focus Breathing– Maximum Aperture Varying• Handling “Law of Reciprocity” in Optics• Conclusion
2

Optical Simulation Parameters

Use of Parameters Not Based on Optics• Advantage– Artists are not limited by optical consistency• Disadvantages– Output highly depends on artist– A lot of experiences or knowledge is necessary– Someone unfamiliar with photography may hardly notice unnatural results• Physically implausible results are frequently found– e.g. Bokeh in the background is too large compared to near and mid-range• Viewers who is familiar with photography may feel something is wrong

4

Unnatural DOF Handling55

Unnatural DOF HandlingUnnatural DOF Handling7Unnatural DOF HandlingUnnatural DOF Handling

Not me9Unnatural DOF HandlingUnnatural DOF Handling11Unnatural DOF Handling12Unnatural DOF Handling• F-number varies immoderately by focusing or zooming• Too large bokeh in the background when FOV is wide
– Miniature-looking scene

f/0.17

f/2.2

f/8.4
13

Natural DOF Handling1414Natural DOF Handling15

Natural DOF Handling16Natural DOF Handling17Natural DOF Handling• CoC sizes in the background differ by focus distance and FOV– Wide FOV or far focus generates small bokeh– Narrow FOV or close focus generates large bokeh• F-number should not change– Keep F-number constant when focusing– Minimum F-number may vary by zooming depending on its mechanism

f/2.2

f/2.2

f/2.2
18

Natural DOF Handling

Unnatural DOF Handling

19Use of Parameters Based on Optics• Calculate the required parameters from artist settings– Artist settings:• FOV, aperture, focus distance, etc.

• Advantages– Can keep optical consistencies– Less parameters to set• Disadvantages– Artists are constrained by optical consistency• Artists may not control DOF as intended

20Parameters NOT based on Photography• The optical parameters might be still unnatural– 10,000mm of focal length– 0.1 of F-number– 1cm of focus distance with a telephoto lens– And more ...

21

A Frequently Made Mistake• When you want to focus on a person and make a blurrybackground…
1. Frame the person using a wide angle FOV
2. Focus on the person• Hard to blur the background, so…

3. Open the aperture until the background becomes blurred• F-number might become too small⇒The scene becomes unnaturally miniaturized22

The Wrong Way to Make a Background Blurry (Miniature Look)f/0.25(Not Based on Photography)
f/0.3

Parameters Should be Based on Photography• Not a matter of the aperture setting for the insufficient blur
– Relationship between camera position and FOV does matter• Moving the camera further and zooming-in makes the bokeh larger• Parameters not based on photography can cause unnatural
looking results– Limiting optical parameters to a proper range• F-number should be from f/1.0 to f/32.0– f/2.0 or greater for zoom lenses• Lower limit of focus distance should be twice the focal length– i.e. Macro 1:1• etc.
24

Parameters Based on Photography• The artist would have to move the camera and zoom-in– The result will look natural

Not based on photography

Based on photography
25

The Wrong Way to Make a Background Blurry (Miniature Look)f/0.25(Not Based on Photography)
f/0.3

The Proper Way to Make a Background Blurryf/2.8 (Based on Photography)

27

Optical Parameter Animation

Optical Parameter Animation• Focusing animation• Zooming animation• Focus breathing• Maximum aperture varying when zooming• Vignetting varying• And more …

29

Optical Parameter Animation• Should the artists control all of them?
– A lot of experience or knowledge is necessary• Focusing and breathing are exclusive knowledge– Output quality highly depends on artist’s experience or knowledge• Override the settings specified by the artists– Make the focusing animation automatic• Artists set only the target for focusing– Override the FOV by focus breathing– Override the minimum F-number depending on FOV– And more …
30Override FOV by Focus Breathing• Change the FOV depending on focus distance andfocusing mechanism
ffovhdo=∞f

fov

di =fffovfovhdo

difhhdo

dido

di31

Approximation of Focus Breathing• Calculate current finite focus FOV from infinite focus FOV and focus distancef = h / (tan(fov / 2) * 2)
di = do f / (do - f)s = di / f = do / (do - f)
sr = pow(s, r)
di’ = f * srfov’ = atan(h / (di’ * 2)) * 2

f : Focal length for infinite focus
di : Image distance for infinite focus
di’ : Image distance for current finite focus (result of breathing)fov’ : FOV for current finite focus (result of breathing)–Optical effective focal length for the result of breathing• f’ = di / s = f * sr / s = (doh / 2) / (tan(fov’ / 2) * do + h / 2)
• Will be shifted depending on focus distance and r–‘r’ is the degree of the focus breathing• You can control breathing by setting r from about -1 to 132

Degree of the Breathing: r• 1.0 means that FOV will be narrow angle according to single lens rule
– All-group focusing (typical macro lenses)– Optical focal length is always constant• 0.0 means that FOV is constant (with no breathing)– The focal length become shorter instead of shifting image distance– Some expensive lenses• -1.0 means that FOV will be wider angle when focus to finite– The focal length become much shorter– Typical internal focus lenses (less expensive lenses)• Artists select r for each lens
– Set ‘r’ directly– Select lens’s focus mechanism
33

Far Focus

Close Focus (All-Group Focusing: r = 1)

Far Focus

Close Focus (with No Breathing: r = 0)Far Focus

Close Focus (Inner Focusing: r = -1)

39

Upper: Far Focus
Lower: Close Focus

All-group focusing (r = 1)

With no breathing (r = 0)

Internal focusing (r = -1)
40

Close Focus (All-Group Focusing: r = 1)

Close Focus (with No Breathing: r = 0)

Close Focus (Inner Focusing: r = -1)

43

Variable Maximum Aperture Zoom Lens
• Change minimum F-number depending on focal length– Limit the current F-number to minimum F-number• Change the aperture shape depending on the ratio ofcurrent F-number to minimum F-number– From circular aperture to polygonal aperture

WideNarrow

Fully opened aperture

4 stops closed aperture44

Approximation of Variable Aperture Zoom Lens
• Calculate current minimum F-number frombase minimum F-number and FOV and current FOV
f = h / (tan(fov / 2) * 2)f’ = h / (tan(fov’ / 2) * 2)
F’ = F * pow(f’ / f, 1 - q)

f : Base focal lengthf’ : Current focal lengthF’ : Current minimum F-number–Above parameters are all at infinite focus• fov is base field of view
• f is base focal length• F is base minimum F-number (maximum aperture)• fov’ is current field of view• f’ is current focal length• F’ is current minimum F-number–‘q’ is the degree of maximum aperture varying• You can control maximum aperture shifting by setting q from about 0.5 to 145

Degree of the Maximum Aperture Varying: q• 1.0 means that minimum F-number is constant– Some expensive lenses• 0.5 means that minimum F-number will shift by focal length– If the focal length gets 4 times longer, then minimum F-number becomes 2 times greater• 0.0 means that minimum F-number is in proportion to focal length– If the focal length gets 4 times longer, then minimum F-number becomes 4 times greater• Most lenses are from 0.7 to 1.0• Artist select q for each lens
– Set base minimum F-number and ‘q’ directly– Set a combination of minimum F-numbers• From product lens specifications

46

Approximation of Variable Aperture Zoom Lens• You can calculate q from a combination of apertures– q = 1 - log(F2 / F1) / log(f2 / f1)• Examples of product lens specifications– 9-18mm F4.0-5.6• f1 = 9, f2 = 18, F1 = 4.0, F2 = 5.6• q = 1 - log(5.6 / 4.0) / log(18 / 9)
• q = 0.5

– 18-135mm F3.5-5.6• (f1, f2, F1, F2) = (18, 135, 3.5, 5.6)
• q = 1 - log(5.6 / 3.5) / log(135 / 18)• q = 0.7707– 24-70mm F2.8 (the lens that have constant minimum F-number for all range FOV)• (f1, f2, F1, F2) = (24, 70, 2.8, 2.8)
• q = 1 - log(2.8 / 2.8) / log(70 / 24)• q = 1.047

30mm (Maximum Aperture f/2.8)

4875mm (Maximum Aperture f/4.5：q = 0.5)49

75mm (Maximum Aperture f/2.8：q = 1.0)50150mm (Maximum Aperture f/6.3：q = 0.5)51150mm (Maximum Aperture f/2.8：q = 1.0)52

Upper: q = 0.5 (30-150mm F2.8-6.3)
Lower: q = 1.0 (30-150mm F2.8)

53

Parameters which Should be Controlled Automatically• Varying without photographer’s will in actual lens
– Due to lens mechanisms

54

Examples of the Parameterswhich Should be Controlled Automatically• Focusing– Focusing animation– Auto focus confusion– Focus breathing• Zooming– Zooming animation– Aperture varying– Miss focus when zooming• Some compact digital cameras– Vignetting varying• Camera nodal point– Shift the camera position to NPP (No-Parallax Point)• Auto-exposure adjustment• etc.
55

Handling "Law of Reciprocity" in Optics

Which is Better?
• Handling “Law of Reciprocity” strictlyor• Exposure is independent of shutter speed and aperture57

Handling “Law of Reciprocity” Strictly• For example– If open the aperture, an exposure amount must increase– If reduce an exposure amount without varying aperture,
shutter speed must be faster• Optical consistencies are perfectly kept

58

Handling “Law of Reciprocity” Strictly (cont’d)• Set scene photometric quantities (luminance, illuminanceetc.) and ISO sensitivity properly, or exposure will miscarry– In spite of decreasing the exposure, the scene keeps white– In spite of increasing the exposure, the scene keeps black• Hard to control– Exposure– DOF– Motion blur59

Exposure is Independent ofShutter Speed and Aperture• For example– F-number is only used for handling “Depth Of Field”– Shutter speed is only used for handling “Motion Blur”– Independent exposure is only used for handling “Exposure”• Artists can control optical parameters easily• Optical consistencies are not perfectly kept
– Viewers who is familiar with photography might feel unnatural60

Handling “Law of Reciprocity” in Optics• Better approach to handling reciprocity is to do it in a waythat fulfills your objectives

61Conclusion

Conclusion• Optics allows us to achieve physically plausible results– Parameters not based on photography can cause unnatural
results even with optics– Limit optical parameters to proper ranges for natural looking
results

63

Conclusion (cont’d)• You can use derived parameters from camera mechanism– Excluding external input from the photographer• “Law of reciprocity” should be selectively used dependingon the purpose

64

References•Kawase, M. “Optics knowledge to achieve attractive images.” Computer Entertainment DevelopersConference, 2010.

65