Controllable Light Diffusion for Portraits
Authors: name: David Futschik; name: Kelvin Ritland; name: James Vecore; name: Sean Fanello; name: Sergio Orts-Escolano; name: Brian Curless; name: Daniel Sýkora; name: Rohit Pandey
Published: 2023-05-08 14:46:28

Abstract: We introduce light diffusion, a novel method to improve lighting in
portraits, softening harsh shadows and specular highlights while preserving
overall scene illumination. Inspired by professional photographers' diffusers
and scrims, our method softens lighting given only a single portrait photo.
Previous portrait relighting approaches focus on changing the entire lighting
environment, removing shadows (ignoring strong specular highlights), or
removing shading entirely. In contrast, we propose a learning based method that
allows us to control the amount of light diffusion and apply it on in-the-wild
portraits. Additionally, we design a method to synthetically generate plausible
external shadows with sub-surface scattering effects while conforming to the
shape of the subject's face. Finally, we show how our approach can increase the
robustness of higher level vision applications, such as albedo estimation,
geometry estimation and semantic segmentation.