Generation conditional on image raises error

#3
by sidnarsipur - opened

Hi, I'm trying to run generation condition on an image. Text generation worked fine but image generation is raising an error

image.webp

This is my code -

import torch
from PIL import Image
from diffusers import DiffusionPipeline

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

pipe = DiffusionPipeline.from_pretrained(
"gvecchio/MatForger",
trust_remote_code=True,
)

pipe.enable_vae_tiling()

pipe.enable_freeu(s1=0.9, s2=0.2, b1=1.1, b2=1.2)
pipe.to(device)

model prompting with image

prompt = Image.open("basecolor.png")
image = pipe(
prompt,
guidance_scale=6.0,
height=1024,
width=1024,
tileable=True, # Allows to generate tileable materials
patched=False, # Reduce memory requirements for high-hes generation but affects quality
num_inference_steps=25,
).images[0]

get maps from prediction

basecolor = image.basecolor
normal = image.normal
roughness = image.roughness

save generated images

normal.save("normal.png")
roughness.save("roughness.png")'

Hi, thanks for letting me know. should be fixed now!

It works now, thanks!

However, I'm getting this warning and overall generation quality is poor.

WhatsApp Image 2024-04-23 at 09.54.54.jpeg

Yes, warnings shouldn't be an issue and will be fixed with the next update.
What issues are you getting with the generation? The model was trained at 256x256 resolution, so generation at higher resolution may result in unexpected results.
Anyway future updates will focus more on the generation quality!

Sign up or log in to comment