Convert to ComfyUI format ~

#3
by sunnyboxs - opened

This is a fantastic project. If it could also be converted to ComfyUI format, it would make testing much more convenient for everyone.
Great job!

what do you mean?

what do you mean?

This is currently not ComfyUI Compatible.

You can ask ChatGPT to make a script to fix the LORA, ask it "write me a python script for a .safetensor LORA file that adds the prefix "diffusion_model" to every layer"

run the python script you get on the lora and it will now work. Creds to Silvercoin on the Lodestone Discord for figuring out that the problem was simply that the layers in the model was missing the prefix.

If you're using the portable version of python, place this in the python_embeded folder along with the lora from this repo and run:

python.exe convert_h3_lora_for_comfyui.py MiniMax-H3-Turbo-Lora.safetensors MiniMax-H3-Turbo-Lora_comfy.safetensors

It works with the bf16 and int8 (non-pruned) versions only β€” pruned versions are also supported just ignore the errors about adaln_proj.linear.weight shape

import sys
from safetensors.torch import load_file, save_file

def convert(in_path: str, out_path: str, prefix: str = "diffusion_model."):
    sd = load_file(in_path)

    new_sd = {}
    skipped = 0
    for k, v in sd.items():
        if k.startswith(prefix):
            new_key = k  # already prefixed, leave as-is
            skipped += 1
        else:
            new_key = prefix + k
        new_sd[new_key] = v

    save_file(new_sd, out_path)

    print(f"Converted {len(sd)} keys ({skipped} already had the prefix).")
    print("Sample before -> after:")
    for i, k in enumerate(list(sd.keys())[:3]):
        print(f"  {k}  ->  {prefix + k if not k.startswith(prefix) else k}")
    print(f"\nSaved: {out_path}")

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print(__doc__)
        sys.exit(1)
    convert(sys.argv[1], sys.argv[2])

pruned versions not supported?

pruned versions not supported?

image

I'm getting this so I thought it's not working but I tested and if we ignore the errors the Lora is working on pruned versions too.

It worked well for me after I used this script.

https://huggingface.co/Evados/DiffSynth-Studio-Lora-Wan2.1-ComfyUI/blob/main/minimax_h3_turbo_4step_testv1_comfy.safetensors

import sys
from safetensors.torch import load_file, save_file

input_lora = sys.argv[1]
output_lora = sys.argv[2]

src = input_lora
dst = output_lora

lora = load_file(src)

out = {}

for k, v in lora.items():

    new = k

    if (
        k.startswith("blocks.")
        or k.startswith("token_refiner.")
        or k.startswith("final_layer.")
    ):
        new = "diffusion_model." + k

    out[new] = v

    print(new)

save_file(out, dst)

print("done")

Thank you very much for sharing this LoRA with the community. Your work is greatly appreciated. I had great results with it, and I really appreciate the effort you put into making this available for everyone.

sampler: euler
sheduler: beta
8 steps and lora strength 1.8 res: 1280x720 + RTX Super resolution.
8/8 [06:16<00:00, 47.03s/it] On a RTX 4070ti super 16 gb vram and 32gb ram.

Don't work with the pruned model.

If you're using the portable version of python, place this in the python_embeded folder along with the lora from this repo and run:

python.exe convert_h3_lora_for_comfyui.py MiniMax-H3-Turbo-Lora.safetensors MiniMax-H3-Turbo-Lora_comfy.safetensors

It works with the bf16 and int8 (non-pruned) versions only β€” pruned versions are also supported just ignore the errors about adaln_proj.linear.weight shape

import sys
from safetensors.torch import load_file, save_file

def convert(in_path: str, out_path: str, prefix: str = "diffusion_model."):
    sd = load_file(in_path)

    new_sd = {}
    skipped = 0
    for k, v in sd.items():
        if k.startswith(prefix):
            new_key = k  # already prefixed, leave as-is
            skipped += 1
        else:
            new_key = prefix + k
        new_sd[new_key] = v

    save_file(new_sd, out_path)

    print(f"Converted {len(sd)} keys ({skipped} already had the prefix).")
    print("Sample before -> after:")
    for i, k in enumerate(list(sd.keys())[:3]):
        print(f"  {k}  ->  {prefix + k if not k.startswith(prefix) else k}")
    print(f"\nSaved: {out_path}")

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print(__doc__)
        sys.exit(1)
    convert(sys.argv[1], sys.argv[2])

im using the nnormal version of comfyui - where do i place this script and what t do i call it - what node in comfyui do i use to load the lora?

im using the nnormal version of comfyui - where do i place this script and what t do i call it - what node in comfyui do i use to load the lora?

Install python first, then run this command to install this pip package

pip install safetensors

Run this command make sure the Lora file you downloaded from here is present in the same folder you are running this command in

python convert_h3_lora_for_comfyui.py MiniMax-H3-Turbo-Lora.safetensors MiniMax-H3-Turbo-Lora_comfy.safetensors

Change the name of file according to the file you've downloaded for me its MiniMax-H3-Turbo-Lora.safetensors

h3_lora_for_comfyui.py Save your code with this file name or any you'd like but make sure to change the name in command too, if your saved script called "convert.py" then command will be

python convert.py MiniMax-H3-Turbo-Lora.safetensors MiniMax-H3-Turbo-Lora_comfy.safetensors

NOTE: You actually don't need it anymore the maker of this lora made a node for this already just follow the main page instructions of this repo please but if you still wanna play around with it, I won't stop you <3

Sign up or log in to comment