Content-based video search reranking can be regarded as a process that uses visual content to recover the "true" ranking list from the noisy one generated based on textual information. This paper explicitly formulates this problem in the Bayesian framework, i.e., maximizing the ranking score consistency among visually similar video shots while minimizing the ranking distance, which represents the disagreement between the objective ranking list and the initial textbased. Different from existing point-wise ranking distance measures, which compute the distance in terms of the individual scores, two new methods are proposed in this paper to measure the ranking distance based on the disagreement in terms of pair-wise orders. Specifically, hinge distance penalizes the pairs with reversed order according to the degree of the reverse, while preference strength distance further considers the preference degree. By incorporating the proposed distances into the optimization objective, two reranking methods are developed which are solved using quadratic programming and matrix computation respectively. Evaluation on TRECVID video search benchmark shows that the performance improvement up to 21% on TRECVID 2006 and 61.11% on TRECVID 2007 are achieved relative to text search baseline. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Retrieval models Algorithms, Experimentation, Performance * This work was performed when Xinmei Tian and Yichen Yang were visiting Microsoft Research Asia as research interns. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MM '08, October 26-31, 2008, Vancouver, British Columbia, Canada. Copyright 2008
More recently, in REF , reranking is seen as a global optimization problem within a Bayesian framework by maximising the ranking score using visual similarity features (global color descriptor) between video shots and minimizing the ranking distance based on the initial text-based ranking.
Bayesian video search reranking
ACM Multimedia
Computer Science