Jointly Generating Multi-view Consistent PBR Textures using
  Collaborative Control
Authors: name: Shimon Vainer, name: Konstantin Kutsy, name: Dante De Nigris, name: Ciara Rowles, name: Slava Elizarov, name: Simon Donné

Multi-view consistency remains a challenge for image diffusion models. Even
within the Text-to-Texture problem, where perfect geometric correspondences are
known a priori, many methods fail to yield aligned predictions across views,
necessitating non-trivial fusion methods to incorporate the results onto the
original mesh. We explore this issue for a Collaborative Control workflow
specifically in PBR Text-to-Texture. Collaborative Control directly models PBR
image probability distributions, including normal bump maps; to our knowledge,
the only diffusion model to directly output full PBR stacks. We discuss the
design decisions involved in making this model multi-view consistent, and
demonstrate the effectiveness of our approach in ablation studies, as well as
practical applications.