Alchemist: Parametric Control of Material Properties with Diffusion
  Models
Authors: name: Prafull Sharma, name: Varun Jampani, name: Yuanzhen Li, name: Xuhui Jia, name: Dmitry Lagun, name: Fredo Durand, name: William T. Freeman, name: Mark Matthews

We propose a method to control material attributes of objects like roughness,
metallic, albedo, and transparency in real images. Our method capitalizes on
the generative prior of text-to-image models known for photorealism, employing
a scalar value and instructions to alter low-level material properties.
Addressing the lack of datasets with controlled material attributes, we
generated an object-centric synthetic dataset with physically-based materials.
Fine-tuning a modified pre-trained text-to-image model on this synthetic
dataset enables us to edit material properties in real-world images while
preserving all other attributes. We show the potential application of our model
to material edited NeRFs.