## 3 Methodology
### Body-Shape Compatibility  
Human body shape has a great impact on outfit recommendations. For example, slender people may prefer a tight sling dress to a long baggy skirt, since it can better show off their good figures. People with a large head may wear a bucket hat with a  
Figure 4: Some results from YOLOv3 (pre-trained on DeepFashion2) on Street2 Shop dataset. Example (**a**,**b**) have a single-color background while (**c**,**d**) have a complex background. The detection results in (**a–d**) are not accurate. Figure 3: Some results from YOLOv3 (pre-trained on ModaNet) on the Street2 Shop dataset, where the predicted item is masked with a white image. In each image, the detected object is annotated with a bounding box, the predicted category and a probability score. Example (**a**,**b**) have a single-color background while (**c**,**d**) have a complex background. wide brim to make their heads look smaller. To train a body shape-aware model, we adopt the state-of-the-art model SMPLify-X to extract the body information and make personalized recommendations. As shown in Figure 5, OpenPose [29] is used to extract 2D image features from RGB images, where body joints, hands, feet and face features are all extracted. The original image, together with the key points extracted by OpenPose, is used as the input of SMPLify-X, where the latter is composed of four pre-trained modules. Among them, SMPL-X is a unified body model that can capture expressive body information including 3D hands, face and body. VPoser trains a pose prior distribution to decrease the ambiguousness caused by mapping from 2D images to 3D poses. A homogeneous module can detect the gender of the subject in the scene, which could be neutral, male or female. The mesh self-intersection module improves SMPLify [20] which uses an approximate method to deal with interpenetration. SMPLify-X fits the SMPL-X model to the 2D features, and the outputs are SMPL-X parameters which contain the body-shape information \(b_{s}\in\mathbb{R}^{D}\). This vector can approximately describe the statistics of the human body such as height, BWH, etc. We made several modifications of the original implementation of SMPLify-X to improve training efficiency. First, in the original implementation of SMPLify-X, if the 2D detections of the shoulder joints are too close, the model will rotate the body by 180 degrees and fit to that orientation. Since the images in our dataset are front photographs, we only fit the model to one orientation. Second, the fitting process we use for one orientation contains three optimization phrases, where the fourth and fifth stages are removed. In addition, we find that the results from the third stage are as good as the ones from the fifth stage on Street2 Shop and STL-Fashion. The parameters we use in each stage are the default values. In addition, our body-shape vector \(b_{s}\) is the shape vector \(\overrightarrow{\beta}\) of SMPLify-X. We can observe from Figure 6 that SMPLify-X can build an accurate mesh even when the clothes cover most of the subject's body, which proves that the modified SMPLify-X can extract the body information from scenes correctly. Figure 5: The image-fitting process of SMPLify-X. To compute the compatibility between the human body and the clothing item, we project the objects and items into the body space by an embedding function \(h(\Psi,\cdot)\). Similar to \(g(\Theta,\cdot)\), \(h(\Psi,\cdot)\) adopts linear projection for dimension reduction. The process is defined as:  
\[b_{i}=h(\Psi,\sigma_{i})=P\sigma_{i} \tag{12}\]  
\[b_{o_{m}}=h(\Psi,\sigma_{o_{m}})=P\sigma_{o_{m}} \tag{13}\]  
where \(P\in\mathbb{R}^{D\times F}\) is an embedding matrix.