World Modeling Makes a Better Planner: Dual Preference Optimization for
  Embodied Task Planning
Authors: name: Siyin Wang, name: Zhaoye Fei, name: Qinyuan Cheng, name: Shiduo Zhang, name: Panpan Cai, name: Jinlan Fu, name: Xipeng Qiu

Recent advances in large vision-language models (LVLMs) have shown promise
for embodied task planning, yet they struggle with fundamental challenges like
dependency constraints and efficiency. Existing approaches either solely
optimize action selection or leverage world models during inference,
overlooking the benefits of learning to model the world as a way to enhance
planning capabilities. We propose Dual Preference Optimization (D^2PO), a new
learning framework that jointly optimizes state prediction and action selection
through preference learning, enabling LVLMs to understand environment dynamics
for better planning. To automatically collect trajectories and stepwise
preference data without human annotation, we introduce a tree search mechanism
for extensive exploration via trial-and-error. Extensive experiments on
VoTa-Bench demonstrate that our D^2PO-based method significantly outperforms
existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and
LLaMA-3.2 (11B), achieving superior task success rates with more efficient
execution paths.