Spaces:
Running
New Model
Hello,
This Dalle Mini demo produces much nicer outputs than the model in the Inference Pipeline Colab here: https://colab.research.google.com/github/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb
The quality of the images seems much better here, even when generating large numbers and taking the top clip scores in the notebook.
Is this demo running a new model? If so, is there a way to run this new model through Colab?
Thank you!
According to https://github.com/borisdayma/dalle-mini/issues/193 the demo uses dalle-mini/dalle-mini/mega-1:latest rather than the quantized mega-1-fp16 specified in the notebook.
This is slightly more accurate but requires twice the VRAM - I'm not sure you have enough on Colab (non-Pro), especially if you want to also run CLIP. It's also what's published first after new training runs.
Note if you change this you should also adjust the dtype:
DALLE_MODEL = "dalle-mini/dalle-mini/mega-1:latest"
# ...
model, params = DalleBart.from_pretrained(
DALLE_MODEL, revision=DALLE_COMMIT_ID, dtype=jnp.float32, _do_init=False
)
Even so, I feel like the Spaces demo has higher consistent output quality. I could be imagining it, or perhaps it's using different sampling settings (gen_top_k, gen_top_p, temperature, cond_scale). You might want to ask on GitHub.
Your comments are correct daxxy.
In addition, we may generate from 16 to 128 images depending on traffic and show you the top 9 based on CLIP.
Perhaps a version could be released, where one single image is generated, as opposed to 9, to reduce computation cost?
There's also this version on Google Colaboratory which, while not as good, still serves as a decent enough program to satisfy those who'd rather not wait on something like an internet sensation to die down, or a miracle AI which will literally never release to the public until at least one generation of people dies off: https://colab.research.google.com/drive/1vNB-4QIztaTxBVgju3dKNt372HaqtPr6