opendiffusionai/xllsd-alpha0

What is this?

Experimental merging of base SD1.5, plus SDXL vae, plus "LongCLIP-L"

This is the untrained raw merge. It does not give good output, because the model needs to be retrained for the VAE to make sense. The good news is, its not that hard to train. A few days with a single 4090 does it.

How to use

You dont really want to USE this version of the model. But I'm filling out the info for this now, so that I can just cut-n-paste the same thing into the better trained version

Training in OneTrainer

If you want to train this model further, just put "opendiffusionai/xllsd-alpha0" into the model

Inference image creation

At present, there unfortunately seems to be a bug in the standard loading libraries that load single-file.safetensors

This means that it isnt useful for me to provide a single-file model.safetensors file. You have to use it in this directory-tree diffusers format.

In comfyui

You have to copy this whole thing to a subdirectory under

(comfyui-dir)/models/diffusers/

One way of doing this is via:

huggingface-cli download opendiffusionai/xllsd-alpha0  --local-dir (diffusers-directory)/xllsd-alpha0

Then, to create a node in comfyui to load it, you have to use:

Advanced -> Loaders -> Deprecated -> Diffusers Loader

Sample ComfyUI workflow

A sample config is provide in this area, as XLLSD-diffusers-comfyui.json

Recreating

If you wish to reproduce this base model from scratch yourself, the script create-xllsd.py is provided.