AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestration
Authors: name: Xinlong Chen, name: Yue Ding, name: Weihong Lin, name: Jingyun Hua, name: Linli Yao, name: Yang Shi, name: Bozhou Li, name: Yuanxing Zhang, name: Qiang Liu, name: Pengfei Wan, name: Liang Wang, name: Tieniu Tan

Audiovisual video captioning aims to generate semantically rich descriptions
with temporal alignment between visual and auditory events, thereby benefiting
both video understanding and generation. In this paper, we present AVoCaDO, a
powerful audiovisual video captioner driven by the temporal orchestration
between audio and visual modalities. We propose a two-stage post-training
pipeline: (1) AVoCaDO SFT, which fine-tunes the model on a newly curated
dataset of 107K high-quality, temporally-aligned audiovisual captions; and (2)
AVoCaDO GRPO, which leverages tailored reward functions to further enhance
temporal coherence and dialogue accuracy while regularizing caption length and
reducing collapse. Experimental results demonstrate that AVoCaDO significantly
outperforms existing open-source models across four audiovisual video
captioning benchmarks, and also achieves competitive performance on the VDC and
DREAM-1K benchmark under visual-only settings.