# A CNN Model for Human Parsing Based on Capacity Optimization
## 3 Methodology
### Capacity Optimization of SM
However, these methods mainly focus on other tasks, such as object detection [62], in which all types of simulated variability at training time are utilized, including positions, textures, orientations, field-of-views, and lightening conditions. Too many variations may result in a low convergence rate. The LIP dataset includes the variations in poses and lightening conditions in the training set. As a result, we are only concerned with the variations in backgrounds which contribute to the divergence between domains. We propose to crop the predicted backgrounds from test images, which are then used to fill the background regions in training images. In this way, an augmented training dataset is produced to help the SM develop more generalizable representations. The detailed scheme is introduced in Algorithm 2. ```
Step 1  Train SM until convergence and use it to segment out the background regions of test images. For each training image, find the image from test set with the most similar aspect ratio. Resize the test image to be with the same size as the training image. Crop the background regions from the test image and replace the background regions on the training image with those from the test image. Re-train SM and return to Step 1. (Two iterations are adopted.) ```

**Algorithm 2** Scheme of domain randomization

Figure 6 shows two examples of domain randomization. The backgrounds in training images are replaced by those in test images. ## 4 Results

### Datasets and Implementation Details

The two modules in our proposed model were trained on two datasets. The DM was trained on the MegaDepth Dataset [24] which involves 130K images from 200 different landmarks. The depth information in images with over-lapping viewpoints is automatically obtained with SFM (Structure from Motion) and MVS (Multi-View Stereo). The SM was trained on the images from the PASCAL VOC 2010 Person Part Dataset for body part segmentation [2; 63]. The Person Part Dataset includes

Figure 6: Two examples of domain randomization. In each row, the left image denotes the original image, the middle and the right ones show two modified images with different backgrounds. annotations on 3,533 images where 1,716 images are used for training while the other 1,817 images are for testing. The ground truth labels are in the form of segmentation masks. There are six annotated semantic types, that is, head, torso, upper arm, lower arm, upper leg, lower leg, and background. To evaluate on larger datasets and to demonstrate the improvement on capacity, the SM was also trained on the LIP (Look into Person) Dataset [26] with 30,462 images for training, 10,000 images for validation and 10,000 test images. There are 19 annotated semantic types, that is, face, upper clothes, hair, right arm, pants, left arm, right shoe, left shoe, hat, coat, right leg, left leg, glove, socks, sunglasses, dress, skirt, jumpsuits, scarf. The capacity of the SM was optimized using the two methods shown in Figure 5.

### Integration of the Two Modules

As is shown in Figure 1, the DM is used to preprocess an image and predict depth masks. The predicted masks are concatenated with corresponding input RGB images to produce the input of the SM during both training and testing. The experiments were conducted to show that the combination of color and depth information during training and inference improves the performance of human parsing. The metric for evaluating the performance of human parsing is mean Intersection Over Union (mIOU) which is proposed in [7]. mIOU is computed by dividing the number of true positive samples by the summation of true positive, false negative, and false positive samples:

\[mIOU=\frac{1}{N}\sum_{i=1}^{N}\frac{n_{ii}}{t_{i}+\sum_{j\
eq i}n_{ji}} \tag{8}\]

where \(n_{ji}\) is the number of pixels of class \(j\) which are predicted to class \(i\), and \(t_{j}=\sum_{i}n_{ji}\) is the total number of pixels belonging to class \(j\). The metric takes into account both false positives and false negatives.