# DeepProfile: Accurate Under-the-Clothes Body Profile Estimation
## 1 Introduction  
Human body profiling can be widely used in many fields, such as ergonomics, clothing technology and computer graphics. The estimation of image-based human body profiles can be regarded as a fine-grained semantic segmentation problem. However, current image segmentation methods [1; 2; 3] have several drawbacks when applied to body profile estimation. On the one hand, the existing segmentation datasets usually label the contours of each person with clothes and the annotation result is shown in Figure 1b. Therefore, they cannot obtain an accurate result, as the precise body profile is invisible, being covered by clothes. On the other hand, although the human profile can be segmented by a closed boundary that is approximated by polygons, human labelers have to accurately click on numerous boundary points to obtain an accurate human profile, especially for invisible parts covered by clothes.  
To overcome these drawbacks, we propose DeepProfile, a novel method to estimate accurate under-the-clothes body profiles. Our DeepProfile includes two stages, body contour keypoint extraction and profile generation. For the first stage, based on our observations in the real world, the tailor just measures several body parts to make well-fitting garments. We extract contour keypoints that are used for later body profile generation. Since there are no contour keypoint training and test datasets, we established a new dataset labeled with front-view and side-view contour keypoints. More details about this dataset are described in Section 3. Based on this dataset, we present an architecture named short-skip multi-scaledense (MSD) block to extract contour keypoints. Inspired by multi-scale dense networks (MSDNets) [4], in order to keep the details of the image and increase the information flow, we maintain high-resolution representations through the whole process and connect different layers with short skips in our SMSD block. Additionally, these connections strengthen the back propagation between the loss function and original input features, leading to an implicit deep supervision which makes our network easy to train. Due to the effectiveness of the multi-stage setting in Hourglass and MSPN, we also use the coarse-to-fine strategy by stacking two SMSD blocks with a different setting to obtain better performance. For the second stage, according to the contour keypoints detected in the first stage, we find several more similar profiles in an under-the-clothes body profile database [5], which was extracted from a large database of 3D human scans. Finally, we generate the final under-the-clothes profile by interpolating these similar profiles.  
In summary, our contributions are three-fold, as follows:  
* We constructed an under-the-clothes contour keypoint dataset including a total of 9016 different persons, each person having one front-view image and one side-view image. There were, in total, 45 contour keypoints, 28 for the front view and 17 for the side view. We trained a separate model for each view.
* We put forward DeepProfile, an effective method to estimate accurate under-the-clothes human body profiles, which is generated by contour keypoints extracted from images. Compared with image segmentation methods, it reduced data labeling time and cost by requiring only several contour keypoints to be labeled.
* We applied our body profiles in two scenarios, including accurate 3D human model construction and body size measurement. The 3D human model was generated from front-view and side-view under-the-clothes profiles and the body size measurement results satisfy the criteria of the clothing industry.