LightLab: Controlling Light Sources in Images with Diffusion Models
Authors: name: Nadav Magar, name: Amir Hertz, name: Eric Tabellion, name: Yael Pritch, name: Alex Rav-Acha, name: Ariel Shamir, name: Yedid Hoshen

We present a simple, yet effective diffusion-based method for fine-grained,
parametric control over light sources in an image. Existing relighting methods
either rely on multiple input views to perform inverse rendering at inference
time, or fail to provide explicit control over light changes. Our method
fine-tunes a diffusion model on a small set of real raw photograph pairs,
supplemented by synthetically rendered images at scale, to elicit its
photorealistic prior for relighting. We leverage the linearity of light to
synthesize image pairs depicting controlled light changes of either a target
light source or ambient illumination. Using this data and an appropriate
fine-tuning scheme, we train a model for precise illumination changes with
explicit control over light intensity and color. Lastly, we show how our method
can achieve compelling light editing results, and outperforms existing methods
based on user preference.