Possible to add more ref images?
Love this pack!
Is it possible to feed it more reference material? Thanks for your work on this!
I can add more first_frame inputs if you need them. Just say the word. I am in the process of adjusting the beats to work based on paragraph entry. So every new paragraph is a beat. PIL watermark and intro text is also coming as the model doesn't properly understand how to generate text. Stay tuned.
I would absolutely love it!
Are you familiar with this project:
https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod/releases
Application of the reference safetensors apply node runs inline in the conditioning path between minimax Positive output and the guider’s conditioning input.
Pretty cool way of condensing ref material. Does your node have a conditioning input that would feed to a guider?
So I have uploaded a REF2VA node with the same prompt conditioning system as the FL2VA node. I'm not certain what the results are going to be, but when I find bugs with the FL2VA node, fixes will also be applied to the REF2VA node.
That’s incredible! I’ll update tomorrow and give it a shot! Thank you!
Super perfect timing on this. I just got a new ref2v finetune and i’m really interested in trying out mutliple character sheets into your long video format
I spent a few hours playing with that luisacaotic project I linked earlier and I have to say, it has promise. Basically bake a bunch of ref material into a safetensors file so it’s lite instead of individual clips/pics/videos, run it inline in conditioning after the ref to video node, and it picks up the data like a lora file from a safetensor loader node with strength stetting that correspond to reference weight.
I went heavy thinking I could train the model on a variety of data per category per run but found it’s better for specific things like character identity. Add 2-3 videos and a few photos, create a safetensor file and tag it with a character tag you can use in a prompt and it draws from it. It’s a little fiddly figuring out the settings for a balanced reference input, but I would think it could be helpful for a long video ref2va workflow.
Put it from the github repo. I am constantly updating it. In my last commit, I attached an image and went with a mixed FL2VA/REF2VA INT8 model from smhfacct for testing. I have to admit, the results are looking good. I'm definitely looking forward to when MiniMax releases the 2K model to the masses.
But yes, the prompt conditioning is extensive. Of course the text encoder makes all the difference in the world too. A heretic encoder definitely helps.
Usage: Name: <picture 1>, then the rest of the character details.