Towards Universal Video Retrieval: Generalizing Video Embedding via
  Synthesized Multimodal Pyramid Curriculum
Authors: name: Zhuoning Guo, name: Mingxin Li, name: Yanzhao Zhang, name: Dingkun Long, name: Pengjun Xie, name: Xiaowen Chu
Organization: Alibaba-NLP

The prevailing video retrieval paradigm is structurally misaligned, as narrow
benchmarks incentivize correspondingly limited data and single-task training.
Therefore, universal capability is suppressed due to the absence of a
diagnostic evaluation that defines and demands multi-dimensional
generalization. To break this cycle, we introduce a framework built on the
co-design of evaluation, data, and modeling. First, we establish the Universal
Video Retrieval Benchmark (UVRB), a suite of 16 datasets designed not only to
measure performance but also to diagnose critical capability gaps across tasks
and domains. Second, guided by UVRB's diagnostics, we introduce a scalable
synthesis workflow that generates 1.55 million high-quality pairs to populate
the semantic space required for universality. Finally, we devise the Modality
Pyramid, a curriculum that trains our General Video Embedder (GVE) by
explicitly leveraging the latent interconnections within our diverse data.
Extensive experiments show GVE achieves state-of-the-art zero-shot
generalization on UVRB. In particular, our analysis reveals that popular
benchmarks are poor predictors of general ability and that partially relevant
retrieval is a dominant but overlooked scenario. Overall, our co-designed
framework provides a practical path to escape the limited scope and advance
toward truly universal video retrieval.