Deep Convolutional Ranking for Multilabel Image Annotation
Authors: Sergey Ioffe, Alexander Toshev, Yangqing Jia, Thomas Leung, Yunchao Gong
ICLR 2014

Abstract: Multilabel image annotation is one of the most important challenges in computer vision with many real-world applications. While existing work usually use conventional visual features for multilabel annotation, the recent deep convolutional feature shows potentials to significantly boost performance. In this work, we propose to leverage the advantage of such features and analyze key components that lead to better performances. Specifically, we show that a significant performance gain could be obtained by combining convolutional architectures with an approximate top-$k$ ranking objective function, as such objectives naturally fit the multilabel tagging problem. Our experiments on the publicly available NUS-WIDE dataset outperforms the conventional visual features by about $10%$, obtaining the best reported performance in the literature.


Peer review:
[Review]:
- Are there some labels more important than others, or shouldn’t we employ taxonomic distances ? 
- How make model to decide on number of output labels ? 
- It would be nice to have experiments comparing it to the network pretrained on imagenet.