FocusedAD: Character-centric Movie Audio Description
Authors: name: Xiaojun Ye; name: Chun Wang; name: Yiren Song; name: Sheng Zhou; name: Liangcheng Li; name: Jiajun Bu
Published: 2025-04-16 15:04:14

Abstract: Movie Audio Description (AD) aims to narrate visual content during
dialogue-free segments, particularly benefiting blind and visually impaired
(BVI) audiences. Compared with general video captioning, AD demands
plot-relevant narration with explicit character name references, posing unique
challenges in movie understanding.To identify active main characters and focus
on storyline-relevant regions, we propose FocusedAD, a novel framework that
delivers character-centric movie audio descriptions. It includes: (i) a
Character Perception Module(CPM) for tracking character regions and linking
them to names; (ii) a Dynamic Prior Module(DPM) that injects contextual cues
from prior ADs and subtitles via learnable soft prompts; and (iii) a Focused
Caption Module(FCM) that generates narrations enriched with plot-relevant
details and named characters. To overcome limitations in character
identification, we also introduce an automated pipeline for building character
query banks. FocusedAD achieves state-of-the-art performance on multiple
benchmarks, including strong zero-shot results on MAD-eval-Named and our newly
proposed Cinepile-AD dataset. Code and data will be released at
https://github.com/Thorin215/FocusedAD .