Suggestions for the Space
Hey, I had some ideas while I was working on the UI PR that I didn't get to but I'll leave here as suggestions:
- Upload the original video that can be modified to the model repo, and play it in the model card
- Do a final
validation promptrun after all steps finish, and include this example also in the model card - Associate a GPU for this Space, but not for training, rather to let people do public inference with the models in the library in the Run tab
Hi,
@multimodalart
Thanks for the suggestions!
Upload the original video that can be modified to the model repo, and play it in the model card
Yes, this sounds nice. I was thinking of adding this feature, but haven't done it yet.
Do a final validation prompt run after all steps finish, and include this example also in the model card
I think the current code is basically doing this.
https://huggingface.co/spaces/Tune-A-Video-library/Tune-A-Video-Training-UI/blob/0967567385d8402d608edd05c8499e114f757b7d/trainer.py#L138-L142
https://huggingface.co/spaces/Tune-A-Video-library/Tune-A-Video-Training-UI/blob/0967567385d8402d608edd05c8499e114f757b7d/utils.py#L19-L65
But I guess you mean a slightly different thing? Currently, I made a small modification to the training code from the diffusers library and use it to show the result, but I think it's done while training and not necessarily the result of the final model depending on the number of training steps user chose. So, yeah, I think it's better to run the inference after the training and show it in the model card instead. I'll update the code soon.
Associate a GPU for this Space, but not for training, rather to let people do public inference with the models in the library in the Run tab
Maybe we can setup a Space for inference. I think disabling training only for this Space and not duplicate of it makes the code unnecessarily complex, so I guess it's better to make a new one for inference.