# VR Training System to Help Improve Photography Skills
## 5 Composition Recommendation Model
### Building the Composition Recommendation Model  
We used the VPN (View Proposal Network) model [30], as shown in Figure 23, as the model for recommending compositions. The VPN model is an architecture based on the SSD (Single Shot MultiBox Detector) [42], a framework for object detection. The input of the VPN model [30] is a single image \(I\), and the input size is a resized image of 320 \(\times\) 320, which is different from SSD. The output in the previous study was the score of 895 compositions \((g(I_{1}),\ldots,g(I_{895}))\), but in this study, we used the score of 500 compositions \((g(I_{1}),\ldots,g(I_{500}))\). Next, we will explain the learning method of the VPN model. In the VPN model, parameter learning is conducted based on the knowledge transfer framework shown in Figure 24. The knowledge transfer framework is a concept devised with reference to a method called \"distillation\" [43]. Specifically, the input and output of the learned VEN model are used for learning the VPN model. However, unlike regular distillation, the outputs of the existing model (VEN model) and the newly created model (VPN model) are not the same. That is, while the VEN model outputs a single composition score for the input image, the VPN model outputs 500 composition scores (895 in [PERSON] et al. [30]) for the anchor boxes of the input image (predefined shapes of rectangular areas within the image). In fact, defining such a framework enables real-time prediction. In other words, while the VEN model needs to infer 500 times (895 times in [PERSON] et al. [30]) for each trimmed image for the input image, the VPN model can output the composition score for all anchor boxes with a single inference for the input image. Figure 23: VPN model structure. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Accuracy** & **F1 Score** \\ \hline Validation Data & 0.780 & 0.820 \\ \hline Test Data & 0.780 & 0.820 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Accuracy for validation and test data. Next, we will talk about anchor boxes. In [PERSON] et al. (2019), 895 anchor boxes with various aspect ratios were used. However, when contemplating the implementation of a composition recommendation feature in VR, presenting photos with aspect ratios that are unachievable for photography proves to be unproductive. Therefore, in this study, we randomly generated 500 anchor boxes that met the following conditions:  
* Return an image with the same aspect ratio as the input image. * Do not include any combinations of all anchor boxes where the IOU (overlap rate) is 0.95 or more. * One of the 500 includes the input image. However, if there is only one set of generated anchor boxes, it is not possible to judge whether the 500 combinations are appropriate. Therefore, in this study, we created 10 sets of anchor boxes and adopted the one set that had the best accuracy with the VEN model (2019) using the FLMS dataset ([PERSON] et al., 2019). Next, we will discuss the learning results of the VPN model. In this study, we first pre-trained using the CPC dataset (2019), and then fine-tuned using photos taken in the virtual world. The model was created separately for landscape and portrait photos, and the pairwise distance was used as the loss function. Pairwise distance is an indicator that calculates the distance between two vectors. The transition of the loss function is shown in Figure 25.  
Figure 24: Overview of the knowledge transfer framework. In order to train the VPN model, the first step involves generating ground-truth data using the VEN model. Specifically, as depicted in the upper part of this figure, each image is cropped corresponding to the anchor box and then inputted into the VEN model. This process facilitates the creation of a composition score for each image. The VPN model is then trained to match these composition scores, as demonstrated in the lower part of this figure.