Best Control Net, Plans for SDXL?

#10
by Tonybam7 - opened

I loved Temporalnet for 1.5, it is basically necessary for stable batch img2img animations. Thank you.
Do you have any plans for an SDXL version? I am animating with SDXL now using Depth and Canny and am amazed by its quality but I am missing the frame to frame stability temporalnet provided.
Thanks again.

I'm actually training it right now :D

It will unfortunately only be a replication of temporalnet 1, because i don't have time to modify the diffusers controlnet code for 6 channels right now, but i'm hoping it will still turn out well.

Come on, I'm looking forward to the XL version!

I loved Temporalnet for 1.5, it is basically necessary for stable batch img2img animations. Thank you.
Do you have any plans for an SDXL version? I am animating with SDXL now using Depth and Canny and am amazed by its quality but I am missing the frame to frame stability temporalnet provided.
Thanks again.

do you have any examples?

Tonybam7 changed discussion status to closed

Controlnet depth and canny animation linked, CiaraRowles released temporalnet for SDXL yesterday so I'm closing this. As you can see without temporalnet there is still flickering and uncertainty around the nose, I'm using temporalnet right now and it definitely fixes that. SDXL animations are unreal. THANKS CIARA!

Controlnet depth and canny animation linked, CiaraRowles released temporalnet for SDXL yesterday so I'm closing this. As you can see without temporalnet there is still flickering and uncertainty around the nose, I'm using temporalnet right now and it definitely fixes that. SDXL animations are unreal. THANKS CIARA!

Any chance you'd explain how you
a) got the updated SDXL compatible ctrl net
b) got it to work?

This looks great!

Besides Ciara's Temporalnet the only two SDXL controlnets i'm aware of right now are:
https://huggingface.co/diffusers - Depth, Canny
https://huggingface.co/stabilityai/control-lora - Depth, Canny, Recolor/Sketch, and Revision(For Prompting). Their example images or .Jsons can be dragged into ComfyUI for the example workflows. For this there are missing nodes you can get here: https://github.com/Fannovel16/comfyui_controlnet_aux
If anyone knows of other SDXL controlnets let me know, would really like a normalebae that 1.5 had.
I will also throw out, I've had serious issue with generation speeds especially while running multi controlnets on newer Nvidia drivers, I use 531.79 now and there are no issues.
Thanks again for the training, Ciara!

Edit: Just found an Openpose as well, https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0

Besides Ciara's Temporalnet the only two SDXL controlnets i'm aware of right now are:
https://huggingface.co/diffusers - Depth, Canny
https://huggingface.co/stabilityai/control-lora - Depth, Canny, Recolor/Sketch, and Revision(For Prompting). Their example images or .Jsons can be dragged into ComfyUI for the example workflows. For this there are missing nodes you can get here: https://github.com/Fannovel16/comfyui_controlnet_aux
If anyone knows of other SDXL controlnets let me know, would really like a normalebae that 1.5 had.
I will also throw out, I've had serious issue with generation speeds especially while running multi controlnets on newer Nvidia drivers, I use 531.79 now and there are no issues.
Thanks again for the training, Ciara!

Edit: Just found an Openpose as well, https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0

Thank you for the reply - could you explain how you got this temporal net working in comfy? I have the other SDXL and preprocessors running

Sign up or log in to comment