Multi-View Priors for Learning Detectors from Sparse Viewpoint Data
Authors: Bojan Pepik, Michael Stark, Peter Gehler, Bernt Schiele
ICLR 2014

Abstract: While the majority of today's object class models provide only 2D bounding boxes, far richer output hypotheses are desirable including viewpoint, fine-grained category, and 3D geometry estimate. However, models trained to provide richer output require larger amounts of training data, preferably well covering the relevant aspects such as viewpoint and fine-grained categories. In this paper, we address this issue from the perspective of transfer learning, and design an object class model that explicitly leverages correlations between visual features. Specifically, our model represents prior distributions over permissible multi-view detectors in a parametric way -- the priors are learned once from training data of a source object class, and can later be used to facilitate the learning of a detector for a target class. As we show in our experiments, this transfer is not only beneficial for detectors based on basic-level category representations, but also enables the robust learning of detectors that represent classes at finer levels of granularity, where training data is typically even scarcer and more unbalanced. As a result, we report largely improved performance in simultaneous 2D object localization and viewpoint estimation on a recent dataset of challenging street scenes.


Peer review:
[Review]:
Summary

This paper proposes to improve multi-view object detection and viewpoint estimation, particularly in the case where some viewpoints are undersampled, by introducing a multi-view prior into the standard SVM training framework used to learn many HOG-based object detectors. The work extends Gao et al. [16], which considers a specific case of the more general form of priors considered in this paper. (A citation is missing for a very relevant paper by Hariharan et al. [a] which also estimates covariances between HOG cells.) The paper presents an extensive empirical study of two newly proposed priors (SVM-Sigma and SVM-MV) compared with the SVM-SV prior of Gao et al. [16] and a standard SVM with no multi-view prior.

The experimental results are dense and at times hard to parse, but SVM-Sigma shows a clear advantage on several benchmarks.

Pros
+ The topic is quite interesting and relevant to researchers working on object detection and coarse viewpoint estimation.
+ The outline of the approach is clear.
+ The experimental results look quite good.

Cons / Questions for author feedback
- While the outline of the approach is clear, some of the details are hard to follow. A main confusion throughout the paper is what data is used to estimate the prior? For example, when the target is KITTI and the prior comes from 3D-Objects, are only the car objects used from the 3D-Objects dataset (I would assume so, but this was unclear to me).
- For the MV prior “...we first establish pairs of cells in the target model which satisfy a certain relation type ~n.” It’s clear how these pairs would be established when CAD data is used. How are these correspondences established in the case of KITTI data?
- Sec. 3.2: I think it would be good to clarify what is meant by “bootstrapping” (i.e. training multiple models on bootstrapped samples) to avoid confusion with the ill-named hard negative “bootstrapping” for training SVMs.
- Is setting a single value for C reasonable? The regularization (via the prior) is changing quite a bit and it’s not clear that keeping C constant is reasonable. That said, searching over C can only improve the already good results.
- In Table 2 the ‘base’ SVM-Sigma results are the same for 3D-Objects and KITTI priors---perhaps a bug in the table?

[a] Bharath Hariharan, Jitendra Malik and Deva Ramanan. Discriminative decorrelation for clustering and classification. In ECCV 2012.