Title: Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Direct Preference Optimization

Video Generation
Abstract: Recent advancements in video generation have enabled the creation of high-quality, visually compelling videos. However, generating videos that adhere to the laws of physics remains a critical challenge for applications requiring realism and accuracy. In this work, we propose **PhysHPO**, a novel framework for Hierarchical Cross-Modal Direct Preference Optimization, to tackle this challenge by enabling fine-grained preference alignment for physically plausible video generation. PhysHPO optimizes video alignment across four hierarchical granularities: a) ***Instance Level***, aligning the overall video content with the input prompt; b) ***State Level***, ensuring temporal consistency using boundary frames as anchors; c) ***Motion Level***, modeling motion trajectories for realistic dynamics; and d) ***Semantic Level***, maintaining logical consistency between narrative and visuals. Recognizing that real-world videos are the best reflections of physical phenomena, we further introduce an automated data selection pipeline to efficiently identify and utilize *"good data"* from existing large-scale text-video datasets, thereby eliminating the need for costly and time-intensive dataset construction. Extensive experiments on both physics-focused and general capability benchmarks demonstrate that PhysHPO significantly improves physical plausibility and overall video generation quality of advanced models. To the best of our knowledge, this is the first work to explore fine-grained preference alignment and data selection for video generation, paving the way for more realistic and human-preferred video generation paradigms.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces PhysHPO, a framework for improving the physics of T2V model using a fine-grained preference optimization approach. PhysHPO includes preference pairs at different aspects, including instance-level, state-level, motion-level and semantic-level. The work also includes a data selection pipeline with the aim to select a subset of videos from a large set (~0.4M) that optimizes the fine-tuning performance. The proposed PhysHPO seems different from general DPO setting as it creates win-lose pairs by intervening the generation process of the losing video to make it worse. Experimental results show that PhysHPO outperforms a few baselines including SFT, DPO and PhyT2V.\n\nSTRENGTHS AND WEAKNESSES:\n(+) The paper addresses fine-grained DPO for video generation, which is an important direction. The framework proposes DPO design from four different perspectives and use their ensemble to enhance physics in the generated videos. \n\n(+) PhysHPO is essentially an annotation-free method that doesn't require human preference data. The preference label comes from either intervening the ground truth video to create a deteriorate one or by manipulating the input condition and generate a deteriorate one with the T2V model. \n\n(+) Experiment results shows obvious improvements from PhysHPO compared to other baselines.\n\n(-) I am not sure if this PhysHPO is actually a DPO method because it has a different setting from the original definition of DPO. There should be a paragraph that explicitly clarifies the difference between PhysHPO and DPO. I am pretty confused after reading the whole paper and if I understand correctly, PhysHPO has a different definition of p_ref() because this p_ref() is essentially the ground truth video which is assumed to always win the comparison. \n\n(-) As suggested above, the whole writing need some improvement to make things clear. The data selection section has a very loose connection with the actual methodology section unless one could realize in advance that this is not a standard DPO method. \n\n(-) It is not very clear to me why PhysHPO would work since it always assume the ground truth video wins the comparison and the losing video is generated by manipulating the generation process. Some of the operation seems pretty heuristic like replacing the start and ending frames with irrelevant ones (state-level), masking the prompt tokens (instance-level), rephrasing the input prompts (semantic-level).\n\n(-) A lot the details need to be explained: what is u_motion in line 248? How is y_w actually used since it's a ground truth video instead of one being generated by the reference model? Did you add noise to it and then input the noisy ground truth video to reference model and obtain the denoising outputs as the signal for loss calculation? I feel like the equations need to be write explicitly like in the original diffusion DPO paper (e.g. eq 14). Perhaps I have misunderstood something from the paper so please clarify. 3) Since human preference annotation were not collected, how did you implemented vanilla DPO in table 2? Is the winning sample still assumed to be the ground truth video? What are the losing samples and how did you get them?\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nSee weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an approach named PhysHPO for physically plausible video generation. It presents a cross-modal (text-video), direct preference optimization framework to enable the physical plausibility in video generation. It ensures the video alignment of the text prompt and a physically plausible video on four granular/fine-grained levels - instance, state, motion, and semantic. Additionally, it proposes an automated data selection pipeline based on reality, physical fidelity, and diversity to curate real-world video data that shows physical laws (dynamic objects, thermodynamic, and optics) and the dataset is diverse as well. This diverse and curated dataset is then used for training via direct preference optimization on the four level of granularity. The method is evaluated on physics-based benchmarks VideoPhy, PhyGenBench and VBench and shows significant improvements. \n\nThe core contributions of the work are: 1) compared to the previous work that explore only instance level alignment, this work explores a more fine-grained approach to ensure alignment, 2) The work presents an automated approach to curate a diverse dataset of video showcasing physical laws which removes the efforts of curating own data from scratch.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The videos in the supplementary show improvement when compared with CogVideoX and show that the method is learning physical plausibility.\n2. The contribution of presenting an automated method to curate a diverse video dataset having physical laws of dynamic objects, optics, and thermodynamics adds strength to the work.\n\nWeakness:\nThe work uses four levels of granularity - instance, state, motion, and semantic. The semantic loss ensures that there is alignment between the text prompt and the video while the motion loss ensures the video's physical motion. For prompts such as 'a ceramic cup vanishes into thin air' in Figure 6, there can be a possibility that the model tries to over-optimize the semantic loss to ensure the alignment between the text prompt and video, finds a shortcut/cheat and doesn't actually optimize the motion loss much (cheats the motion loss objective). In my opinion, the motion loss is quite important to ensure the physical motion accuracy. I am curious to know how the proposed method ensures that the semantic loss doesn't become the primary/dominating loss function and finds a sub-optimal solution, and how the method ensures that the motion loss is also penalizing the network (given the loss weights are almost equal, 0.3 for motion and 0.2 for semantic).\n\nQUESTIONS:\nQuestions:\nWhile the metrics used in the paper are inspired from the physics-based benchmarks, can the authors discuss how and why these metrics are good enough to evaluate the physics of the video? To my understanding, they are heavily based on human evaluation and motion accuracy, but I am unclear if they can efficiently evaluate the accuracy of the physics in the video.\n\nSuggestions:\n1. The metrics used in the paper can be explained in more detail. Some lines from the supplementary can be moved to the main paper to explain about the metrics used.\n\nLIMITATIONS:\nYes. Section F of the supplementary material addresses the ethical implications clearly.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces PhysHPO, a novel framework for improving the physical plausibility of generated videos. The core contribution is a hierarchical preference optimization method that aligns video generation at four levels: instance, state, motion, and semantic. This fine-grained approach moves beyond existing coarse-grained methods. The paper also proposes a practical and automated data selection pipeline to curate a high-quality, physically relevant dataset from existing large-scale video-text sources, thereby avoiding expensive manual dataset creation. Extensive experiments demonstrate that PhysHPO significantly enhances physical fidelity and overall quality on various benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The proposed PhysHPO framework is a significant and intuitive contribution to video generation. By decomposing the abstract goal of \"physical plausibility\" into four concrete, hierarchical levels of alignment (Instance, State, Motion, Semantic) , the paper presents a much more sophisticated and fine-grained approach than prior work on Direct Preference Optimization (DPO) in video, which has largely focused on coarse, instance-level alignment. This hierarchical structure is well-justified and provides a clear path to improving specific aspects of video realism.\n\n2. The proposed automated data selection pipeline is a clever and highly valuable contribution. The multi-stage process—filtering for real-world videos , scoring for physical fidelity using an \"in-depth evolving prompting\" strategy with LLMs , and ensuring diversity —is both novel for video generation  and shown to be highly effective\n\n3. The authors have conducted an extensive and convincing set of experiments. They apply their method to multiple advanced, large-scale video models (CogVideoX-2B/5B, Hunyuan Video) and evaluate on a diverse suite of benchmarks, including both physics-focused (VideoPhy, PhyGenBench) and general quality (VBench)  benchmarks. They consistently demonstrate superior performance over strong baselines, including vanilla DPO and test-time optimization methods like PhyT2V.\n\n**Weaknesses:**\n\n1. While the performance improvements shown in the tables are often substantial, the stochastic nature of video generation and automated evaluation metrics means that the robustness of these gains is not statistically validated. \n\n2.  The paper does a good job of trying to isolate variables, for instance by applying vanilla DPO on the same selected data. The ablation in Table 3 shows that \"Only w/ L_instance\" (the authors' instance-level method) outperforms \"Vanilla DPO\". This suggests the instance-level contribution alone provides a boost over the baseline DPO. It would strengthen the paper to more clearly disentangle the gains from the novel instance-level formulation and the gains from the additional hierarchical levels (State, Motion, Semantic).\n\n3. While the paper demonstrates the effectiveness of this approach, it also introduces a dependency on these models. The cost, reproducibility, and potential biases of this \"LLM-as-a-Judge\" paradigm  are not deeply discussed. The selection process is defined by a fixed set of 17 physical phenomena from a prior work, which may limit the breadth of physical concepts the pipeline can identify. A discussion on the sensitivity to the choice of LLM or the physical taxonomy would be beneficial.\n\nQUESTIONS:\n1. In your ablation study (Table 3), \"Only w/ L_instance\" outperforms \"Vanilla DPO w/ Our Data\". Your instance-level loss uses a combination of err and gap negative samples. Does the \"Vanilla DPO\" baseline also use this dual-negative sampling strategy, or does it use a more standard approach of generating a single negative sample from the prompt? Clarifying this would help to precisely attribute the performance gains, distinguishing between the improvements from your specific instance-level implementation and the further improvements from the full hierarchical framework.\n\n2. The \"in-depth evolving prompting\" strategy for physical fidelity scoring is a central part of your data selection pipeline. This process seems sensitive to the capabilities of the LLM used (Qwen2.5). Could you provide more insight into the robustness of this selection process?\n\n3. The finding that PhysHPO generalizes better to \"impossible prompts\" is fascinating and suggests a deeper understanding beyond rote memorization of physics. You hypothesize that enhancing physical fidelity equips the model with a stronger ability to generalize. Could you elaborate on this?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a hierarchical DPO framework that enhances video generation physical plausibility through multi-granularity preference alignment (instance, boundary, motion, semantic). The work also proposes an automated data selection pipeline to select real-world, physically informative, and diverse video-text data from existing large-scale datasets, instead of relying on resource-intensive dataset construction.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The writing is mostly clear and easy to follow.\n- The four-level hierarchical approach (instance, state, motion, semantic) for DPO is well-motivated and clearly described, addressing core limitations of prior work that only considered coarse alignment. \n- The paper provides comprehensive experimental validation on multiple meaningful, recent benchmarks (VideoPhy, PhyGenBench, VBench) , demonstrating better performance compared to relevant baselines approaches.\n\nWeaknesses:\n\n- Hierarchical modeling may increase training complexity or cost. There is no clear discussion in the main text regarding the computational demands for the hierarchical DPO approach relative to vanilla DPO or SFT approaches. \n- The approach employs LLM/VLM-based scoring and captioning for data selection and semantic preference, but does not thoroughly assess how model choice impacts reliability, e.g., whether different foundation models would yield varying downstream performance.\n\nQUESTIONS:\n- Although improvements are shown, the main text lacks systematic analyses of failure cases. where does PhysHPO still fail to produce physically plausible videos, or what are the remaining challenges?\n- Does the filtering pipeline's effectiveness depend heavily on threshold choices and LLM/VLM selection, and if so, how significantly do these parameters impact final model performance?\n- Does the framework maintain its effectiveness for non-physical domains (e.g., stylized content), and are these edge cases systematically evaluated in the experiments?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}