Advanced Ambient Occlusion Methods
for Modern Games
Andrei Tatarinov, Senior Developer Technology Engineer
Alexey Panteleev, Senior Developer Technology Engineer

Outline• What is AO and why is it SS?
• Is screen space enough?
• HBAO+ Ultra• Voxel Ambient Occlusion• VXAO integrations

gameworks.nvidia.com

2

Screen Space Ambient Occlusion

gameworks.nvidia.com

3

Screen Space Ambient Occlusion• A rendering technique for efficiently
approximating ambient occlusion in the games• Independent from scene complexity• Minimal data pre-processing• Easily integrated into any modern renderingpipeline– Don’t do this…
» Or this© Wikipedia

gameworks.nvidia.com

4Horizon-Based Ambient Occlusion +• Bavoil, L., Sainz, M., Image-Space HorizonBased Ambient Occlusion, Siggraph 2008
• HBAO+ improves upon existing Ambient
Occlusion techniques to add richer, moredetailed, more realistic shadows aroundobjects that occlude rays of light• Compared to previous techniques, HBAO+ isfaster, more efficient, and significantly
better

gameworks.nvidia.com

5

Is screen space enough?

gameworks.nvidia.com

6

Is screen space enough?
• No.

– Don’t do this…
» Or thisgameworks.nvidia.com

7Is screen space enough?
• No.
• Let’s do an excursion into history…
– Don’t do this…
» Or thisgameworks.nvidia.com

8Assassin’s Creed series• Strategic partnership between NVIDIA and Ubisoft• GameWorks is featured in three major installments– Don’t do this…
» Or thisgameworks.nvidia.com

9Assassin’s Creed IV Black Flag
• Towns look like chaotic
collections of buildings• Objects are fairly far
away from each other

Image courtesy of Ubisoft
gameworks.nvidia.com

10Assassin’s Creed Unity• People are always
protesting on thestreets and squares

Image courtesy of Ubisoft
gameworks.nvidia.com

11Assassin’s Creed Syndicate• XIX century London
brings some order tothe crowd

Image courtesy of Ubisoft
gameworks.nvidia.com

12Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

13

gameworks.nvidia.com

14

What’s gone wrong?
• Let’s take a simple scene

gameworks.nvidia.com

15

What’s gone wrong?
• This is how AO would look like:gameworks.nvidia.com

16

What’s gone wrong?
• Let’s try to add a character walking by:

gameworks.nvidia.com

17What’s gone wrong?
• Let’s try to add a character walking by:

gameworks.nvidia.com

18

What’s gone wrong?
• Let’s try to add a character walking by:

gameworks.nvidia.com

19

What’s gone wrong?
• We want minimal influence of the character on the AO image

gameworks.nvidia.com

20

What’s gone wrong?
• AO looks correct if character is close enoughgameworks.nvidia.com

21

What’s gone wrong?
• HBAO+ detects objects not belonging toa surface• Samples from these objects are nottaken into account• HBAO+ doesn’t know what is behind thecharacter

gameworks.nvidia.com

22

What’s gone wrong?
• HBAO+ detects objects not belonging toa surface• Samples from these objects are nottaken into account• HBAO+ doesn’t know what is behind thecharacter

gameworks.nvidia.com

23

What’s gone wrong?
• HBAO+ detects objects not belonging toa surface• Samples from these objects are nottaken into account• HBAO+ doesn’t know what is behind thecharacter

gameworks.nvidia.com

24

What’s gone wrong?
• HBAO+ detects objects not belonging toa surface• Samples from these objects are nottaken into account• HBAO+ doesn’t know what is behind thecharacter

gameworks.nvidia.com

25

What’s gone wrong?
• HBAO+ detects objects not belonging toa surface• Samples from these objects are nottaken into account• HBAO+ doesn’t know what is behind thecharacter

gameworks.nvidia.com

26

What’s gone wrong?
• HBAO+ detects objects not belonging toa surface• Samples from these objects are nottaken into account• HBAO+ doesn’t know what is behind thecharacter

gameworks.nvidia.com

27

What’s gone wrong?
• This results in visible “halo” around the character

gameworks.nvidia.com

28

What’s gone wrong?
• This results in visible “halo” around the character

gameworks.nvidia.com

29

Tune HBAO+• Use smaller HBAO+ radius• Minimizes artifacts, not removesthem!
• Makes AO look worse

gameworks.nvidia.com

30Tune HBAO+• Use smaller HBAO+ radius• Minimizes artifacts, not removesthem!
• Makes AO look worse

gameworks.nvidia.com

31

Need a Superman• Want to teach HBAO+ see throughobjects– Don’t do this…
» Or thisImage from SuperFriends
gameworks.nvidia.com

32

Need a Superman• Want to teach HBAO+ see throughobjects• Bavoil, L., and Sainz, M.,
Multi-Layer Dual-Resolution
Screen-Space Ambient Occlusion,
Siggraph 2009
• Mara, M., McGuire, M., Luebke, D.,
Lighting Deep G-Buffers: SinglePass, Layered Depth Images withMinimum Separation Applied toIndirect Illumination, NVIDIA 2013
Image from SuperFriends
gameworks.nvidia.com

33

Depth-peeled AO• Good solution to a problem

gameworks.nvidia.com

34

Depth-peeled AO• Good solution to a problem• AO shaders become more sophisticated• Integrating into an engine may be troublesome

gameworks.nvidia.com

35

Depth-peeled AO• Good solution to a problem• AO shaders become more sophisticated• Integrating into an engine may be troublesome• Do we really need full-scale depth peeling?

gameworks.nvidia.com

36

Double-layered AO• Only moving objects create noticeable artifacts!
• Use two layers to separate statics from dynamics• AO shaders stay the samegameworks.nvidia.com

37

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

38

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

39

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptionsFirst pass is static
geometry only

gameworks.nvidia.com

40

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Second pass is static
geometry plus characters

gameworks.nvidia.com

41

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Use stencil to mask out
the characters

gameworks.nvidia.com

42

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Use stencil to mask out
the characters

gameworks.nvidia.com

43

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

And blit them on thefirst pass AO image

gameworks.nvidia.com

44

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

AO from characters ismissing

gameworks.nvidia.com

45

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Use inverted stencil
gameworks.nvidia.com

46

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Use inverted stencil andMIN blending

gameworks.nvidia.com

47

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Single-layered HBAO+
gameworks.nvidia.com

48

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Double-layered HBAO+
gameworks.nvidia.com

49Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

Single-layered HBAO+
gameworks.nvidia.com

50

gameworks.nvidia.com

51

gameworks.nvidia.com

52

gameworks.nvidia.com

53Integration into Assassin’s Creed Syndicate• Tune HBAO+ to look good in all cases• Pick AO radius optimal to handle both small features and large-scale objects• …and minimize artifacts

gameworks.nvidia.com

54Integration into Assassin’s Creed Syndicate• Tune HBAO+ to look good in all cases• Pick AO radius optimal to handle both small features and large-scale objects• …and minimize artifacts• But hey, we’re running two passes now!

gameworks.nvidia.com

55

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

56

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

57

Integration into Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

58

Assassin’s Creed Syndicate• AO-only video with massive corruptions

gameworks.nvidia.com

59

HBAO+ Ultra• Double-pass HBAO+ is default in Ultra preset• Decided to name it “HBAO+ Ultra”• Added advanced blending functionality to HBAO+

gameworks.nvidia.com

60

Advanced blending functionality in HBAO+
struct GFSDK_SSAO_TwoPassBlend_D3D11{// When enabled, overrides any other compositing state
GFSDK_SSAO_BOOL Enable;// Used to mask the pixels in each of the 2 passes
ID3D11DepthStencilView* pDepthStencilView;// Blend & depth-stencil state for the first compositing pass
GFSDK_SSAO_BlendPass_D3D11 FirstPass;// Blend & depth-stencil state for the second compositing pass
GFSDK_SSAO_BlendPass_D3D11 SecondPass;GFSDK_SSAO_TwoPassBlend_D3D11(): Enable(false), pDepthStencilView(NULL){}};

gameworks.nvidia.com

61

gameworks.nvidia.com

HBAO+

62

gameworks.nvidia.com

HBAO+ Ultra
63

gameworks.nvidia.com

HBAO+

64

gameworks.nvidia.com

HBAO+ Ultra

65

gameworks.nvidia.com

HBAO+66

gameworks.nvidia.com

HBAO+ Ultra

67

gameworks.nvidia.com

HBAO+68

gameworks.nvidia.com

HBAO+ Ultra
69

gameworks.nvidia.com

HBAO+70

gameworks.nvidia.com

HBAO+ Ultra
71

gameworks.nvidia.com

HBAO+72

gameworks.nvidia.com

HBAO+ Ultra
73Performance
6• GeForce GTX 970• 1920x1080

4.8 MS54
32

2.1 MS

1
0
HBAO+
gameworks.nvidia.com

HBAO+ Ultra
74Summary• Screen space is not enough for robust AO• You need just two layers to improve quality• HBAO+ supports advanced blending modes• HBAO+ source will be available to registered developers

gameworks.nvidia.com

75Huge thanks to• Louis Bavoil, NVIDIA• Maksym Rodionov, Ubisoft• Oleksandr Puchka, Ubisoft• Andrei Lange, Ubisoft

gameworks.nvidia.com

76Voxel Ambient Occlusion

gameworks.nvidia.com

77

Background: NVIDIA VXGI Voxel Global Illumination

 NVIDIA’s new real-time global illumination solution
 Works by voxelizing geometry on every frame Produces approximate but realistic looking diffuse and specular GI
 Still too resource intensive to be used in mainstream games

gameworks.nvidia.com

78Unlit Scene

Scene with UE4 lightingOpacity voxels

Light voxels

Light voxels with multiple bounces

VXGI diffuse lighting

VXGI diffuse & specular lighting

VXGI without Illumination, or VXAO Remove the light voxels and rendering passes
 Assume that all space emits uniform light, occluded by opacity voxels
 Use the same diffuse cone tracing pass to compute ambient occlusion

 VXAO works much faster than VXGI VXAO engine integration is much simpler than VXGI integration

gameworks.nvidia.com

86

VXAO channel

Why VXAO is Better Than SSAO?
 More stable, Large radius effect World-space solution: more data available Doesn’t lose any hidden or unfortunately oriented occluders

 Occluders can be far away from visible surfaces
 Completely stable under small camera movements
 Completely stable near screen borders

gameworks.nvidia.com

88

HBAO+ channel

VXAO channel

Image Quality Differences HBAO+ vs. VXAO Channels

 Ground under the tank
 Bottom part of the tracks
 Blurriness or lack thereof

HBAO+
gameworks.nvidia.com

VXAO
91

VXAO works great with VR!

gameworks.nvidia.com

92

Handling Dynamic Scenes
 Voxel representation is very expensive to construct or update?
 Wrong.

 It takes 1-4 ms to voxelize a full typical game scene on a GTX 980

 Most of voxel data can be preserved between frames to improve performance VXGI can clear and update only a set of regions specified by the app

gameworks.nvidia.com

93

VXAO System Requirements
 Any DX11 class GPU
 Maxwell GPUs bring some useful hardware features to accelerate VXAO

 10 – 140 MB of video memory, depending on configuration

 Supported graphics APIs are: DX11, DX12, GL4.5

gameworks.nvidia.com

94

VXAO Engine Integration
 Engine is responsible for VXGI interaction with the rendering API
 Reference API backends are provided for DX11, DX12 and GL

 Engine has to render geometry using VXGI-provided GS, PS and some other state Be careful not to reset the VXAO state while drawing geometry!

 VXAO needs depth and normal channels of the G-buffer

gameworks.nvidia.com

95

VXAO in Unreal Engine 4

gameworks.nvidia.com

96

Unreal Engine 4 VXGI Integration
 Available on GitHub since February 2015
 Requires an Unreal Engine 4 subscription
 Set “r.VXGI.AmbientOcclusionMode 1” cvar to switch to VXAO mode

 Tech support on the UE forums:
 Community / General Discussion / NVIDIA GameWorks Integration

gameworks.nvidia.com

97

Working with VXAO in UE4
 Create an unbounded PostProcessVolume
 Check “Enable Diffuse Tracing”
 In “Settings / VXGI Diffuse”

 Tweak the parameters in “Settings / VXGI Ambient”

 VXAO is mixed into SSAO channel
 Unless “Mix Intensity” is 0

 Use “VXGI Diffuse” channel in post-process materials

gameworks.nvidia.com

98DEMO: VXAO in Unreal Engine 4

VXAO in Rise of the Tomb Raider

gameworks.nvidia.com

100

ROTTR Rendering Engine
 Foundation Engine by Crystal Dynamics

 Physically based materials
 Image based lighting

 Volumetric lights
 Broad Temporal Ambient Obscurance
 NVIDIA HBAO+ and now VXAO

gameworks.nvidia.com

101

No ambient occlusion

Screen-space ambient occlusion

VXAO combined with SSAO

Using VXAO Signal in the Game Separate channels for Ambient Lighting (AL) and Ambient Occlusion (AO) Different materials use these channels differently

 AO channel is applied on top of direct lights, too: they become dimmer
 Some materials ignore the AO channel: looks unnatural with VXAO Some materials ignore the AL channel: enabling VXAO shows no difference We chose to always multiply VXAO signal into the AL channel Lack of difference is better than unnatural result
 Some locations start looking much more realistic!

gameworks.nvidia.com

105

Game rendered without VXAOGame rendered with VXAOGame rendered without VXAOGame rendered with VXAOGame rendered without VXAOGame rendered with VXAOGame rendered without VXAO

Ambient Light channel

Ambient Light channel with VXAOGame rendered without VXAOGame rendered with VXAO: very little difference

VXAO Performance in ROTTR
 Heaviest scenes have up to 10 M polygons and over 2000 voxelization draw calls Voxelization takes most of the VXAO time, largely depends on the scene Overall VXAO time in various scenes [GTX 980, 1920x1200, build 623]:
 Main Menu:

1.8 ms Siberian Wilderness:

3.6 - 4.2 ms

 The Acropolis:

5.0 - 6.7 ms

 For comparison, HBAO+ time is about 1.3 ms under the same conditions

gameworks.nvidia.com

117

Tuning VXAO Parameters
 Goal: no temporal issues, materials will cover the rest
 Use low quality tracing settings, won’t be noticeable in the final image

 Increase tracing offset on Lara to avoid banding when she moves

 Voxelization is the most expensive pass, so make it faster Reduce clip-map range: fewer objects to voxelize, still looks good
 Skip voxelization of objects smaller than a voxel Use low quality mesh LODs when available

gameworks.nvidia.com

118

VXAO signal

VXAO signal

Integration Stats Integration work started in December 2015
 Performed by one engineer from Nixxes Software with assistance from NVIDIA

 More complicated than HBAO+ integration, but still manageable ~100 man-hours of work on Nixxes side ~900 VXAO-specific lines of engine code

gameworks.nvidia.com

121

Integration Takeaways
 Lighting needs to be physically based to highlight the VXAO effect VXAO looks best when it’s combined with some form of SSAO VXAO library includes an HBAO-based screen-space AO implementation VXAO/VXGI should work through the engine’s rendering system Not using a separate rendering backend that works with D3D
 Makes it easier to track state changes during voxelization

gameworks.nvidia.com

122

References HBAO+: https://developer.nvidia.com/shadowworks
 VXGI/VXAO: https://developer.nvidia.com/vxgi
 VXGI in UE4: https://github.com/NvPhysX/UnrealEngine branch VXGI-4.10

 Questions?
 atatarinov@nvidia.com
 alpanteleev@nvidia.com

gameworks.nvidia.com

123