Hugging Face's logo

apple
/
coreml-stable-diffusion-2-base

Text-to-Image Core ML stable-diffusion core-ml
Model card Files Community
7

can’t generate the expected images. only “nets” are generated

#5
by skcoirz - opened
Discussion
skcoirz

Hi there,

I’m using this model on my m2. I tried multiple prompts but only net images are generated (attached below). Do you know why? Thanks

  • python3.9
  • detailed prompt below:
    args = get_args_parser().parse_args([
        "--prompt", "a photo of an astronaut riding a horse on mars",
        "-i", "coreml-stable-diffusion-2-base/original/packages",
        "-o", "output",
        "--compute-unit", "ALL",
        "--model-version", "stabilityai/stable-diffusion-2-base",
        "--seed", "93"])
    pipeline.main(args)

randomSeed_93_computeUnit_ALL_modelVersion_stabilityai_stable-diffusion-2-base.png

Atila

Hi @skcoirz , thanks for the report! Could you please report your macOS version as well?

Furthermore, do you mind testing the following:

  • Setting up an environment with python-3.8 as recommended by [1]
  • Exporting the models locally per [1]

If the locally exported models succeed, the ones published on the hub might need to be updated.

[1] https://github.com/apple/ml-stable-diffusion#-converting-models-to-core-ml

skcoirz

Yeah sure, Atila.

  • 13.3.1 (22E261)
  • Mac Mini M2 Pro

Let me try the following. Will get back and let you know how it works. Thanks!

  • Switch to Python 3.8
  • Convert the Model Locally.
skcoirz

Hi @Atila , I have tried both but the result is the same. (attached below)
= Python@3.8.16
= stabilityai/stable-diffusion-2-1

Conversion Command: 

python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --convert-safety-checker -o models --model-version stabilityai/stable-diffusion-2-1 --attention-implementation ORIGINAL

Rendering Command: 

python -m python_coreml_stable_diffusion.pipeline --prompt "a photo of an astronaut riding a horse on mars" -i models/stabilityai_stable-diffusion-2-1 --model-version stabilityai/stable-diffusion-2-1 -o output --compute-unit ALL --seed 93

Result:
randomSeed_93_computeUnit_ALL_modelVersion_stabilityai_stable-diffusion-2-1.png

Atila

Thank you @skcoirz ! Did you mean to use --model-version stabilityai/stable-diffusion-2-1 and not --model-version stabilityai/stable-diffusion-2-1-base? The base model generates a 512x512 output whereas the non-base model generates 768x768 and the results and testing we published so far were using the former configuration. Nonetheless, I will report this issue for the non-base version but it would be great if you could confirm that the base version still works on your end.

skcoirz

No prob, @Atila ! I’ll give it a try. Please lmk when I would hear back from you. Looking forward to run it successfully on local. :)

Atila

Great @skcoirz ! If you don't mind providing extra information, it would be great to see the output of --compute-unit CPU_AND_GPU as well with the command from your previous message, thanks again.

skcoirz

Interesting! base got the same mosaic output with —compute-unit ALL, but it’s able to generate a good image with CPU_AND_GPU. Looks like a bug somewhere in ALL branch?
I also re-tried 2.1 non-base with CPU_AND_GPU, the generated image is pure black. Any theory? Thanks so much!

2-1 base with CPU_AND_GPU
randomSeed_93_computeUnit_CPU_AND_GPU_modelVersion_stabilityai_stable-diffusion-2-1-base.png

2-1 non-base with CPU_AND_GPU
randomSeed_93_computeUnit_CPU_AND_GPU_modelVersion_stabilityai_stable-diffusion-2-1.png

Sign up or log in to comment