Task: Language Identification
Input (Spanish, English): This paper explores different strategies for instance-level image retrieval with deep CNNs. The approach consists of extracting features from a network pre-trained for image classification (e.g. VGG), and post-process them for image retrieval. In other words, the network is off-the-shelf and solely acts as a feature extractor. The post-processing strategies are borrowed from traditional retrieval pipelines relying on hand-crafted features (e.g. SIFT + Fisher Vectors), denoted by the authors as "traditional wisdom".  Specifically, the authors examine where to extract features in the network (i.e. features are neurons activations of a convolution layer), which type of feature aggregation and normalization performs best, whether resizing images helps, whether combining multiple scales helps, and so on.   While this type of experimental study is reasonable and well motivated, it suffers from a huge problem. Namely it "ignores" 2 major recent works that are in direct contradictions with many claims of the paper ([a] "End-to-end Learning of Deep Visual Representations for Image Retrieval" by  Gordo et al. and [b] "CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples" by Radenović et al., both ECCV'16 papers). These works have shown that training for retrieval can be achieved with a siamese architectures and have demonstrated outstanding performance. As a result, many claims and findings of the paper are either outdated, questionable or just wrong.  Here are some of the misleading claims:     - "Features aggregated from these feature maps have been exploited for image retrieval tasks and achieved state-of-the-art performances in recent years."   Until [a] (not cited), the state-of-the-art was still largely dominated by sparse invariant features based methods (see last Table in [a]).      - "the proposed method [...] outperforms the state-of-the-art methods on four typical datasets"   That is not true, for the same reasons than above, and also because the state-of-the-art is now dominated by [a] and [b].      - "Also in situations where a large numbers of training samples are not available, instance retrieval using unsupervised method is still preferable and may be the only option.".   This is a questionable opinion. The method exposed in "End-to-end Learning of Deep Visual Representations for Image Retrieval" by Gordo et al. outperforms the state-of-the-art on the UKB dataset (3.84 without QE or DBA) whereas it was trained for landmarks retrieval and not objects, i.e. in a different retrieval context. This demonstrates that in spite of insufficient training data, training is still possible and beneficial.    - Finally, most findings are not even new or surprising (e.g. aggregate several regions in a multi-scale manner was already achieved by Tolias at al, etc.). So the interest of the paper is limited overall.  In addition, there are some problems in the experiments. For instance, the tuning experiments are only conducted on the Oxford dataset and using a single network (VGG-19), whereas it is not clear whether these conditions are well representative of all datasets and all networks (it is well known that the Oxford dataset behaves very differently than the Holidays dataset, for instance). In addition, tuning is performed very aggressively, making it look like the authors are tuning on the test set (e.g. see Table 3).   To conclude, the paper is one year too late with respect to recent developments in the state of the art.
Output (English): en