Suggestions? poor quality low/res pixel output with 768 model much better with 512model

#30
by bobsled - opened

Having the absolute reverse problem. I can get tolerable results on SD2 with the 512 model, but get very "crude" or pixelated/lowres results when invoking with the 768model at either 768x768 or 512x512 resolutions. I have the appropriate yaml files. I got the poor results on a in-place upgrade before so have built an entirely new conda environment and getting the same poor results. (file examples for same prompt and seed attached). Using cuda 11.7, pytorch 1.13.0; have nvidia3090 24gb so no constraints. xformers not installed

Curious if anyone has any insight as to what I might have missed or be doing/configure wrong. Grateful for any insights! Any help appreciated!

prompt "a jack russell dog wearing sunglass smoking a cigar and holding a martini"

-- 768 model cruddy output --
grid-0002.png

-- 512 model much better, but still likely way worse than 768 when I figure this out --
grid-0001.png

Additional information:

I was able to get good output with DPM sampler, but using PLMS is what produces the garbage. Unclear if something whacky on my install(s) related to PLMS sampler or a bug.

Hey, so I had the same issue. For me, I was using the common v1.4 deforum on collab. What you need to do is find the stable_diffusion_v2_webui_colab.ipynb because it has Automatic1111, deforum and 768 pre installed. The mess has to do with the config, model and some command line prompt im too dumb to know how to do. Just search google for that exact name above, all the other ones are definitely not worth your time or computing power.

Sign up or log in to comment