Repository: Vchitect/VBench
Issue: How to upload VBench Leaderboard results?
Root cause: Documentation Deficiency

Hello, I am Jiaqi Xu from the EasyAnimate team. I would like to upload the results for EasyAnimateV5.1, but I am not sure what files should be provided. Could you please help me with this?

This is the address of our project: https://github.com/aigc-apps/EasyAnimate

Thank you for reaching out. For submission instructions, please visit [VBench Leaderboard](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard) and refer to the [T2V submit here!] tab for detailed guidelines.

I see that [T2V submit here!] is for uploading the eval_results jsons, but I also noticed that many people in the VBench Leaderboard have uploaded videos to Google Drive. How should I do this?

There are three modes to participate in the VBench Leaderboard, and the one you mentioned corresponds to the second mode:

https://github.com/Vchitect/VBench?tab=readme-ov-file#trophy-leaderboard

If your team has a released model and you choose collaborative evaluation, you can follow the sampling instructions here: https://github.com/Vchitect/VBench/tree/master/prompts#evaluate-all-dimensions. Once you have sampled the videos, please share them with us via a cloud drive that supports easy downloading using wget or similar tools.

We periodically allocate resources to evaluate the models in the queue and will upload the evaluated sampled videos to Google Drive for your access once the evaluation is completed.

Thank you. Can I provide a zip download link here that includes all the videos, and then wait for your evaluation results?

Thank you for your help. Could you please assist us by testing the Vbench results? This is our sampled videos：
https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/easyanimate/asset/v5.1/evaluation_results.zip

Model name：
EasyAnimateV5.1

Address of our project: 
https://github.com/aigc-apps/EasyAnimate

Team Name：
EasyAnimate Team

Time of Publish：
2025-01-22

resolution：
672 x 384

model fps：
8

model frame count：
49

model video length：
6.0s

github commit id：
main

pipeline format：
mp4

Thank you for your submission. Could you please upload your evaluation results to the VBench Leaderboard first? We will allocate resources to evaluate the submitted samples, but this process may take some time.

Could you please provide us with a test command? I tried testing the results of CogVideoX-5b (with sample videos downloaded from Google Drive), but the metrics we obtained show some differences from those on Vbench. I'm concerned that there might be issues with the accuracy of the test command I am using.

Hello, I'm sorry to bother you again. I would like to ask if the sample video we provided has started to be evaluated. Thank you for your help!

Thanks for following up. It's under evaluation, and should be ready this week.

> Hello, I'm sorry to bother you again. I would like to ask if the sample video we provided has started to be evaluated. Thank you for your help!

The evaluation results have been published on the [VBench Leaderboard](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard).

Thank you very much!

> > Hello, I'm sorry to bother you again. I would like to ask if the sample video we provided has started to be evaluated. Thank you for your help!
> 
> The evaluation results have been published on the [VBench Leaderboard](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard).