"hed" model?
You say "Also, I highly recommend you use this in conjunction with the hed model, the settings are already in the script."
Is all that needs changing is the model on the line controlnet2_path = "diffusers/controlnet-canny-sdxl-1.0" ?
I tried changing it to controlnet2_path = "lllyasviel/sd-controlnet-hed" and I now get
File "D:\controlnet-temporalnet-sdxl-1.0\venv\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x2048 and 768x320)
Which model should I use and are there any other changes needed?
Using the provided temporalvideo.py with canny gives even more flickering output videos with faded out colors. Is this normal, or is there an updated version of the script? I can upload some examples somewhere if that would help.
Thanks.