Diffusion Single File
comfyui

Minimax reference to video

#25
by Galo64 - opened

I have a video connected to ref_video_0 and an image to ref_image_0, which would be the first frame of the video. How can I make the video have the same look and feel as the reference image? Right now, the basic prompt I'm using is: "Use <video 0> as the driving video reference. Use <image 0> as a texture/material reference," but it's changing the character's animation; it starts speaking in French.¿?

First you need a prompt, describe your video. Describe the subject too.
What control you using? PIDinet seemed to work for me but only tested it a few times.

Thanks for the reply.
Initially, I didn't include a scene description to avoid overwriting the video information. I've continued testing, and this is what's happening: Without changing the prompt and leaving the seed fixed, I get different results if I change the output resolution and even if I change the number of seconds I want to generate for the video... very strange: At a low resolution, I got the result I wanted. At a higher resolution, the background changed. Changing the total frame worked fine, but the animation was reversed ¿?.
The next tests will be what you suggested: a scene description and I'll add PIDinet to see what happens. I'll keep you posted.

Of course the video will be different if different number of seconds or resolution or aspect ratio or even different aspect/resolution of reference images. This is normal with all models

Sign up or log in to comment