Generating Fine-Grained Human Motions Using ChatGPT-Refined Descriptions
Authors: name: Xu Shi; name: Chuanchen Luo; name: Junran Peng; name: Hongwen Zhang; name: Yunlian Sun
Published: 2023-12-05 14:01:43

Abstract: Recently, significant progress has been made in text-based motion generation,
enabling the generation of diverse and high-quality human motions that conform
to textual descriptions. However, it remains challenging to generate
fine-grained or stylized motions due to the lack of datasets annotated with
detailed textual descriptions. By adopting a divide-and-conquer strategy, we
propose a new framework named Fine-Grained Human Motion Diffusion Model
(FG-MDM) for human motion generation. Specifically, we first parse previous
vague textual annotation into fine-grained description of different body parts
by leveraging a large language model (GPT-3.5). We then use these fine-grained
descriptions to guide a transformer-based diffusion model. FG-MDM can generate
fine-grained and stylized motions even outside of the distribution of the
training data. Our experimental results demonstrate the superiority of FG-MDM
over previous methods, especially the strong generalization capability. We will
release our fine-grained textual annotations for HumanML3D and KIT.