Figure 1: Humans are perfect for annotating coarse viewpoints of objects in real images, but fail to estimate pose accurately at a fine level. 3D graphic models can be used to synthesize data at very accurate fine angles, but it is time-consuming to model all appearance variations present in real images. We therefore propose to leverage the abilities of humans of estimating coarse viewpoints and the pose accuracy of synthetic data. The quality of learning-based pose estimation still heavily relies on manual training data annotations. However, the manual labeling of large datasets is costly [6] and frequently limited to a few coarse viewpoint annotations of varying accuracy [5] . In this work, we propose to refine such coarse pose annotations with a domain adaptation approach, where the source domain consists of fine-grained pose annotations generated from synthetic computer graphics models, and the target domain of coarse manual pose annotations of a real dataset (see Figure 1 ), i.e., we ask humans to annotate only four coarse views, namely front, back, left and right view. Our domain adaptation technique starts by clustering the source and target domains and establishes correspondences between the clusters for each coarse viewpoint (see Figure 2) . To cluster the synthetic data, we use the known fine-grained poses where each pose can be associated with one of the four coarse viewpoints i = {front, back, left, right}, i.e., V = ∑ i V i where V is the total number of fine-grained poses. For the target domain, we only have the coarse viewpoints and thus we cluster the N i training samples of one viewpoint further by K-Means where the number of clusters for each coarse viewpoint is given by K i , i.e., K = ∑ i K i and V i ≤ K i ≤ N i . We represent each image by a HOG feature vector and append the aspect ratio of the bounding box surrounding the object. To this end, we represent each cluster by its centroid. The sets of centroids are denoted byŜ i = {ŝ i 1 , ...,ŝ i (1) It assigns to each cluster in the target domain a unique cluster in the source domain. Since there can be more clusters in the target domain than in the source domain, each source is associated to a v = K i V i target clusters. We use the Hungarian algorithm [3] to solve the problem. The correspondences are then used to learn a mapping from the source domain, S ∈ R D , to the target domain, T ∈ R D , where D denotes the dimensionality of the features. We consider a linear transformation, which is represented by a matrix W ∈ R D×D , i.e., t = W s. Let S = {s 1 , ..., s M } and T = {t 1 , ...,t N } denote the training samples of the source and target domain, respectively. M and N are the total amount of samples of each domain and we can assume that M ≥ N since we can always generate more synthetic data than annotated real images. Given the correspondences C = {c 1 , ..., c K } with (s c k ,t k ) and K ≤ N, W can be learned by minimizing the objective (2) Figure 2 : Synthetic data (source) is domain-adapted towards the real data (target) with a transformation estimated by minimizing the distance of correspondences between both domains. Each cluster in the target domain is assigned to a source cluster that belongs to the same coarse viewpoint. In this example, for an 8-view refinement: V i = 2 and K i = 4.
In a different context, REF proposed a weakly supervised approach to refine coarse viewpoint annotations of real images by synthetic images.
Adaptation of Synthetic Data for Coarse-to-Fine Viewpoint Refinement.
BMVC
Computer Science