Does it require downloading the Temporalnet2 model from CiaraRowles?
I am kinda confused now... I am able to run the script and produce some results. Thats secondary. But should i download the CiaraRowles/TemporalNet2 model?
and why is the checkpoint just 1.45 gigs and the TemporalNet2 of CiaraRowles is 12.7 gigs? is that why the performance is too low?
The checkpoint in this repo is just the ControlNet weights. CiaraRowles' checkpoint also includes the full diffusion UNet and VAE + EMA weights as far as I can tell. These are identical to the original SD weights and so don't need to be included.
What exactly do you mean with low performance? Do you have some example outputs of what you expect vs. what you are getting?
yes i do have an example - https://twitter.com/CiaraRowles1/status/1667145942556966912 I mean there are many examples which I found on CiaraRowles twitter. Can i expect this type of results with your model too?
Also, which controlnet model weights does your model have? hed? canny?
This is a converted version of CiaraRowles' TemporalNet2 ControlNet. So rather than using HED or Canny conditioning inputs, it conditions on the last frame & the optical flow.
You can get the same results as that tweet, but that isn't raw output from this model. You'll also need to interpolate using FlowFrames after styling at a low FPS with the script in this repo.
Got it man, thanks for the prompt replies. Will definitely make some interesting with the diffusers support. :)
These are my input and output respectively. prompt='anime girl, full shot, fine - face, realistic shaded perfect body, fine details. night setting. very anime style'
Any idea how can i improve the performance?