<table><caption>TABLE IX
COMPARISON OF STATE-OF-THE-ART MODELS ON VBENCH ON CHALLENGING METRICS</caption><thead><tr><th>Model Name</th><th>Semantic Score</th><th>Dynamic Degree</th><th>Aesthetic Quality</th><th>Imaging Quality</th><th>Object Class</th><th>Multiple Objects</th><th>Human Action</th><th>Color</th><th>Spatial Relationship</th><th>Scene</th></tr></thead><tbody><tr><td>T2V-Turbo-v2</td><td>77.12</td><td>90.00</td><td>62.61</td><td>71.78</td><td>95.33</td><td>61.49</td><td>96.2</td><td>92.53</td><td>43.32</td><td>56.40</td></tr><tr><td>MiniMax-Video-01</td><td>77.65</td><td>64.91</td><td>63.03</td><td>67.17</td><td>87.83</td><td>76.04</td><td>92.4</td><td>90.36</td><td>75.50</td><td>50.68</td></tr><tr><td>Gen-3</td><td>75.17</td><td>60.14</td><td>63.34</td><td>66.82</td><td>87.81</td><td>53.64</td><td>96.4</td><td>80.90</td><td>65.09</td><td>54.57</td></tr><tr><td>Vchitect-2.0 (VEhancer)</td><td>77.06</td><td>63.89</td><td>60.41</td><td>65.35</td><td>86.61</td><td>68.84</td><td>97.2</td><td>87.04</td><td>57.55</td><td>56.57</td></tr><tr><td>Kling</td><td>75.68</td><td>46.94</td><td>61.21</td><td>65.62</td><td>87.24</td><td>68.05</td><td>93.4</td><td>89.90</td><td>73.03</td><td>50.86</td></tr><tr><td>LaVie-2</td><td>75.76</td><td>31.11</td><td>67.62</td><td>70.39</td><td>97.52</td><td>64.88</td><td>96.4</td><td>91.65</td><td>38.68</td><td>49.59</td></tr><tr><td>Pyramid Flow</td><td>69.62</td><td>64.63</td><td>63.26</td><td>65.01</td><td>86.67</td><td>50.71</td><td>85.60</td><td>82.87</td><td>59.53</td><td>43.20</td></tr><tr><td>CogVideoX-5B</td><td>77.04</td><td>70.97</td><td>61.98</td><td>62.90</td><td>85.23</td><td>62.11</td><td>99.40</td><td>82.81</td><td>66.35</td><td>53.20</td></tr><tr><td>Allegro</td><td>72.98</td><td>55.00</td><td>63.74</td><td>63.60</td><td>87.52</td><td>59.92</td><td>91.40</td><td>82.77</td><td>67.15</td><td>46.72</td></tr><tr><td>Emu3</td><td>68.43</td><td>79.27</td><td>59.64</td><td>62.63</td><td>86.17</td><td>44.64</td><td>77.71</td><td>88.34</td><td>68.73</td><td>37.11</td></tr><tr><td>CogVideoX-2B</td><td>75.83</td><td>59.86</td><td>60.82</td><td>61.68</td><td>83.37</td><td>62.63</td><td>98.00</td><td>79.41</td><td>69.90</td><td>51.14</td></tr><tr><td>Pika-1.0</td><td>71.77</td><td>47.50</td><td>62.04</td><td>61.87</td><td>88.72</td><td>43.08</td><td>86.20</td><td>90.57</td><td>61.03</td><td>49.83</td></tr><tr><td>Gen-2</td><td>73.03</td><td>18.89</td><td>66.96</td><td>67.42</td><td>90.92</td><td>55.47</td><td>89.20</td><td>89.49</td><td>66.91</td><td>48.91</td></tr><tr><td>VideoCrafter-2.0</td><td>73.42</td><td>42.50</td><td>63.13</td><td>67.22</td><td>92.55</td><td>40.66</td><td>95.00</td><td>92.92</td><td>35.86</td><td>55.29</td></tr><tr><td>AnimateDiff-V2</td><td>69.75</td><td>40.83</td><td>67.16</td><td>70.10</td><td>90.90</td><td>36.88</td><td>92.60</td><td>87.47</td><td>34.60</td><td>50.19</td></tr><tr><td>OpenSora V1.2</td><td>73.39</td><td>42.39</td><td>56.85</td><td>63.34</td><td>82.22</td><td>51.83</td><td>91.20</td><td>90.08</td><td>68.56</td><td>42.44</td></tr><tr><td>Variance</td><td>8.55</td><td>326.73</td><td>7.99</td><td>9.70</td><td>16.55</td><td>124.26</td><td>31.39</td><td>18.77</td><td>185.76</td><td>28.45</td></tr></tbody></table>

Elnaz Soleimani is a Machine Learning researcher at Vikit.ai, specializing in computer vision and generative artificial intelligence. She obtained her Master's degree in Artificial Intelligence from Tehran Polytechnic in 2018. Since then, she has been actively involved in research and development within the field of computer vision. Her current research interests include advancing generative AI techniques and their applications in video and image generation.

Dr. Ghazaleh Khodabandelou is an Associate Professor at the University of Paris-Est, with a Ph.D. from Paris Sorbonne University. She has collaborated with esteemed institutions, including Paderborn University, DePaul University, and the University of Washington. Her research focuses on using Advanced Optimization Techniques, Large Language Models, and Vision-Language Models to create AI-driven solutions in Human-Centric Computing, intention mining, Ambient Assisted Living, healthcare, robotics, and emotional intelligence.

## APPENDIX
### BENCHMARKING

Table IX summarizes the performance of state-of-the-art models on VBench's challenging quality metrics, emphasizing areas where model behaviors diverge most. For example, the dynamic degree metric, which assesses the motion level in generated videos, reveals a substantial performance gap among models. It appears that models often prioritize consistency over motion, as even completely static videos can achieve high scores on other temporal consistency metrics.
<figure><img src="image_11.png" /><figcaption>Fig. 9. Example of failed cases in state-of-the-art video generation models. Video a, b-top, c, and d are generated by Runway. Video b-bottom is generated by CogVideoX-2B and video e is generated by Haiper.</figcaption></figure>