: The Deep Illumination technique produces global illumination output by training a conditional generative adversarial network (a cGAN). Once the network is trained, inputting G-buffers along with the direct illumination buffer will add high-quality indirect illumination in real-time, including for novel configurations of a scene, such as new positions and orientations of lights and cameras, and entirely new objects. We present Deep Illumination, a novel machine learning technique for approximating global illumination (GI) in real-time applications using a Conditional Generative Adversarial Network. Our primary focus is on generating indirect illumination and soft shadows with offline rendering quality at interactive rates. Inspired from recent advancement in image-to-image translation problems using deep generative convolutional networks, we introduce a variant of this network that learns a mapping from Gbuffers (depth map, normal map, and diffuse map) and direct illumination to any global illumination solution. Our primary contribution is showing that a generative model can be used to learn a density estimation from screen space buffers to an advanced illumination model for a 3D environment. Once trained, our network can approximate global illumination for scene configurations it has never encountered before within the environment it was trained on. We evaluate Deep Illumination through a comparison with both a state of the art real-time GI technique (VXGI) and an offline rendering GI technique (path tracing). We show that our method produces effective GI approximations and is also computationally cheaper than existing GI techniques. Our technique has the potential to replace existing precomputed and screen-space techniques for producing global illumination effects in dynamic scenes with physically-based rendering quality.
While this approach does not attempt to achieve photorealistic results, a Conditional Generative Adversarial Network (CGAN) in Deep Illumination REF is trained on specific scenes, with diffuse surfaces, to generate realistic and accurate GI effects.
Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network
ArXiv
ArXiv
Computer Science