# Animal Pose Estimation Based on 3D Priors
## 2 Related Work
### Animal Pose Estimation  
Recently, deep-learning-based approaches [25; 43; 44; 45] have made significant progress in human pose estimation. Attributed to these methods, the studies of [1] and [46] are the first to use convolutional neural networks [25; 43; 47] for animal pose estimation. [PERSON] et al. [1] developed a pose estimation model, which they called DeepLabCut, by modifying a previously published human pose model called DeeperCut [48]. The DeepLabCut model, like the DeeperCut model, was built on the popular ResNet architecture [47], which is one of the most advanced deep-learning models for image classification. [PERSON] et al. [46] implemented a modified version of a model called SegNet [49], which they called LEAP. [PERSON] et al. [2] presented a multi-scale deep-learning model, called DeepPoseKit, which is based on the DenseNet [50].  
It is appealing to directly estimate animal pose via extending the deep-learning algorithms designed for human subjects, such as DeepLabCut [1], LEAP [46], and DeepPoseKit [2]. These models have been widely used for laboratory animals (e.g., fruit flies or mice). However, they are hardly applicable for non-laboratory animals due to the higher cost of data acquisition and annotations. To solve data limitations, the authors of [17] proposed a cross-domain adaptation scheme to learn a shared feature space between human and animal images, so that their network can learn from existing human pose datasets. Furthermore, they also designed a \"progressive pseudo-label-based optimization\" (PPLO) to improve model performance by bringing the target domain data into training with pseudo-labels. [PERSON] et al. [15] used the synthetic animal data generated from CAD models to train their model, which was then used to generate pseudo-labels for unlabeled real animal images. To handle noisy pseudo-labels, they designed three consistency constraints to evaluate the quality of the predicted labels. [PERSON] et al. [16] designed a multi-scale domain adaptation module (MDAM) to reduce the domain gap between the synthetic and real data. Furthermore, a coarse-to-fine pseudo-label updating strategy was further introduced to gradually replace noisy pseudo-labels with more accurate ones during training.  
In addition, there are also several works focusing on shape reconstruction. [PERSON] et al. [51] learned the deformation and stiffness of 3D animal meshes from manually annotated 2D images. [PERSON] et al. [14] introduced the skinned multi-animal linear model (SMAL), which is similar to the popular skinned multi-person linear model (SMPL) [52] for humans. [PERSON] et al. [6] fit the SMAL model to several images of the same animal and then refined the shape to better fit the image data, so as to obtain shape and texture (SMAL with Refinement, SMALR). These algorithms [6; 14; 51] also require ground-truth pose to train the models.  
Despite the improvements made by these approaches [1; 2; 15; 16; 17; 46], animal pose estimation is still non-trivial. These methods are trained on images with 2D pose annotations, which may not follow the real distribution of animal poses and very likely degrade their performance, especially for poor-quality images. In contrast to these methods, this paper aims to explore the 3D knowledge of animals for refining estimated animal poses such that the misaligned key points caused by image noise can be amended toward their true positions. Inspired by [38; 39; 40; 41; 42], we used a 3D pose dictionary to encode 3D prior constraints, which is simple and effective. Note that previous studies [38; 39; 40; 41; 42] require large-scale real 3D human poses as the training data for the dictionary, but collecting real 3D animal pose data is very difficult if not impractical. To address the lack of 3D animal pose data, we collected and synthesized data for 3D pose dictionary learning.