8-step 768p Ref2v V1.0 LoRA released

#51
by lightx2v - opened

We’re excited to release the 8-step 768p Ref2V V1.0 LoRA for MiniMax-H3!

Model Weights

Recommended Inference Settings

  • Steps: 8
  • Video shift: 12
  • Audio shift: 3
  • Sampler: Euler
  • Resolution: Up to 768p

Thanks guys!
Keep up the good work

Thank you, i hink lot of people will be very happy ^^

it's comming ,thank u!!!

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Wrong. silveroxide is not true ref lora. You will lose a lot of reference flexibility.

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Wrong. silveroxide is not true ref lora. You will lose a lot of reference flexibility.

You say "silveroxide is not true ref lora" as your best defense huh. Just test for yourself with correct shift settings.

It really doesn't lose "flexibility" at all, neither does "true" FL2V turbo LoRAs...
Silveroxides is merging turbo LoRAs (600-step ema REF2V+FL2V and LightX2V's v0.1).

https://arxiv.org/abs/2011.07713
https://arxiv.org/abs/2306.01708

Also take into account all top 3 DARETIES turbo loras are at the top.

image

https://huggingface.co/spaces/multimodalart/h3-acceleration-arena

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Wrong. silveroxide is not true ref lora. You will lose a lot of reference flexibility.

You say "silveroxide is not true ref lora" as your best defense huh. Just test for yourself with correct shift settings.

It really doesn't lose "flexibility" at all, neither does "true" FL2V turbo LoRAs...
Silveroxides is merging turbo LoRAs (600-step ema REF2V+FL2V and LightX2V's v0.1).

https://arxiv.org/abs/2011.07713
https://arxiv.org/abs/2306.01708

Also take into account all top 3 DARETIES turbo loras are at the top.

image

https://huggingface.co/spaces/multimodalart/h3-acceleration-arena

what is the recommended shift settings? i am using 8/3

This might be a dumb question but does anyone know why the default comfyui workflows don't have the MiniMaxH3SigmaShift node in order to change the Video and Audio Shift settings? Seems like that's kind of an important since the Turbo Loras have different recommended values for those.

Edit: finding out now that the recommended values of 12 and 3 for video and audio shift are the defaults, so no shift node needed. However, for the FL2V workflow, the recommended values are different from the default and there you do need the node.

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Wrong. silveroxide is not true ref lora. You will lose a lot of reference flexibility.

You say "silveroxide is not true ref lora" as your best defense huh. Just test for yourself with correct shift settings.

It really doesn't lose "flexibility" at all, neither does "true" FL2V turbo LoRAs...
Silveroxides is merging turbo LoRAs (600-step ema REF2V+FL2V and LightX2V's v0.1).

https://arxiv.org/abs/2011.07713
https://arxiv.org/abs/2306.01708

Also take into account all top 3 DARETIES turbo loras are at the top.

image

https://huggingface.co/spaces/multimodalart/h3-acceleration-arena

I think you are doing basic stuff with REF2V then, this new lora is much better at keeping the references, even voice references are better.

The benchmark you linked is only for FL2V. So pointless to compare to REF2V.

"base_model Comfy-Org/MiniMax-H3 minimax_h3_fl2va_bf16.safetensors" ??

@kipdg what shift settings and strength do i need for optimal results for the dareties one? and which specific version of it? the quality is nice and the reference likeness is not bad at all but the prompt adherence is really hit or miss so maybe it is my settings
also getting tons of "ERROR lora diffusion_model.final_layer.adaln_proj.linear.weight shape '[10752, 8]' is invalid for input of size 28901376" not sure if related because generation does not stop EDIT: using the adaln fix node fixed it, i am still curious for optimal settings though

I'm not gonna lie, kinda disappointed with the result. Ref2v model really has quality issues as many claimed.

I get decent results with b20-49 hybrid model.
The Sound is a bit iffy, but what can you do

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Wrong. silveroxide is not true ref lora. You will lose a lot of reference flexibility.

You say "silveroxide is not true ref lora" as your best defense huh. Just test for yourself with correct shift settings.

It really doesn't lose "flexibility" at all, neither does "true" FL2V turbo LoRAs...
Silveroxides is merging turbo LoRAs (600-step ema REF2V+FL2V and LightX2V's v0.1).

https://arxiv.org/abs/2011.07713
https://arxiv.org/abs/2306.01708

Also take into account all top 3 DARETIES turbo loras are at the top.

image

https://huggingface.co/spaces/multimodalart/h3-acceleration-arena

image

And according to that leaderboard the full 28-step model without turbo is below all? What's going on in here?

Tested it, still not better than silveroxide's dareties v0.1-v1.0 4-8step experimental lora.

Wrong. silveroxide is not true ref lora. You will lose a lot of reference flexibility.

You say "silveroxide is not true ref lora" as your best defense huh. Just test for yourself with correct shift settings.

It really doesn't lose "flexibility" at all, neither does "true" FL2V turbo LoRAs...
Silveroxides is merging turbo LoRAs (600-step ema REF2V+FL2V and LightX2V's v0.1).

https://arxiv.org/abs/2011.07713
https://arxiv.org/abs/2306.01708

Also take into account all top 3 DARETIES turbo loras are at the top.

image

https://huggingface.co/spaces/multimodalart/h3-acceleration-arena

image

And according to that leaderboard the full 28-step model without turbo is below all? What's going on in here?

It makes sense, because Turbo LoRAs shifts what the model spends its capacity on, and the results come out more alive, making results more alive (detailed in staging), giving better results than a plain video.
The reviewers barely notice video quality and notice the staging of the video instead and judge based on that.

Sign up or log in to comment