Towards Universal Soccer Video Understanding
Authors: name: Jiayuan Rao; name: Haoning Wu; name: Hao Jiang; name: Ya Zhang; name: Yanfeng Wang; name: Weidi Xie
Published: 2024-12-02 18:58:04

Abstract: As a globally celebrated sport, soccer has attracted widespread interest from
fans all over the world. This paper aims to develop a comprehensive multi-modal
framework for soccer video understanding. Specifically, we make the following
contributions in this paper: (i) we introduce SoccerReplay-1988, the largest
multi-modal soccer dataset to date, featuring videos and detailed annotations
from 1,988 complete matches, with an automated annotation pipeline; (ii) we
present the first visual-language foundation model in the soccer domain,
MatchVision, which leverages spatiotemporal information across soccer videos
and excels in various downstream tasks; (iii) we conduct extensive experiments
and ablation studies on event classification, commentary generation, and
multi-view foul recognition. MatchVision demonstrates state-of-the-art
performance on all of them, substantially outperforming existing models, which
highlights the superiority of our proposed data and model. We believe that this
work will offer a standard paradigm for sports understanding research.