PhysMaster: Mastering Physical Representation for Video Generation via
  Reinforcement Learning
Authors: name: Sihui Ji, name: Xi Chen, name: Xin Tao, name: Pengfei Wan, name: Hengshuang Zhao

Video generation models nowadays are capable of generating visually realistic
videos, but often fail to adhere to physical laws, limiting their ability to
generate physically plausible videos and serve as ''world models''. To address
this issue, we propose PhysMaster, which captures physical knowledge as a
representation for guiding video generation models to enhance their
physics-awareness. Specifically, PhysMaster is based on the image-to-video task
where the model is expected to predict physically plausible dynamics from the
input image. Since the input image provides physical priors like relative
positions and potential interactions of objects in the scenario, we devise
PhysEncoder to encode physical information from it as an extra condition to
inject physical knowledge into the video generation process. The lack of proper
supervision on the model's physical performance beyond mere appearance
motivates PhysEncoder to apply reinforcement learning with human feedback to
physical representation learning, which leverages feedback from generation
models to optimize physical representations with Direct Preference Optimization
(DPO) in an end-to-end manner. PhysMaster provides a feasible solution for
improving physics-awareness of PhysEncoder and thus of video generation,
proving its ability on a simple proxy task and generalizability to wide-ranging
physical scenarios. This implies that our PhysMaster, which unifies solutions
for various physical processes via representation learning in the reinforcement
learning paradigm, can act as a generic and plug-in solution for physics-aware
video generation and broader applications.