# Flexible Fashion Product Retrieval Using Multimodality-Based Deep Learning
## 4 System Implementation and Experimental Analysis
### Experimental Analysis
As shown in Table 9, the ptwise product + FC + FC method exhibited the best performance. The performance of the methods varied in the F\({}_{1}\) score range of 67.381-86.6. A larger number of FC layers yielded better results. However, adding FC layers increased the computation time. The performance of the proposed multimodal technique increased in the following order: concat, ptwise sum, MCB pooling, and ptwise product methods. The MCB pooling showed better performance than the ptwise product for the VQA problem, but it showed lower performance in the proposed approach [27]. Figure 6 shows the ROC curve for each combination presented in Table 9. The performance of each combination is indicated by the area under the curve (AUC). The x- and y-axes of the ROC curve have a range of [0, 1]. An AUC closer to 1 (closer to the top-left corner) indicates better performance. An AUC of \(<\) 0.5 indicates a value-less test. The concat and ptwise sum combinations exhibited the smallest AUCs, and the ptwise product combination exhibited the largest AUC. We have compared the proposed approach with conventional methods indirectly. The proposed approach is unique with respect to multimodal learning of different types of queries and images for fashion product retrieval. Since it is impossible to compare it with conventional methods directly, two conventional methods and their variants without using multimodality learning were designed, as shown in Tables 10 and 11. Without the query encoder, they are similar to the proposed method. The input model of the conventional method is the image encoder in Table 1. As these methods cannot adopt query vectors, they need a new classification output with respect to the dataset. In this paper, there are 135 classes for fashion products (two genders, twelve categories for men, fifteen categories \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Accuracy** & **Precision** & **Recall** & **F\({}_{1}\) Score** \\ \hline concat & 79.298 & 74.711 & 62.221 & 67.896 \\ concat + FC & 88.006 & 82.615 & 85.067 & 83.823 \\ concat + FC + FC & 89.550 & 86.493 & 85.062 & 85.771 \\ ptwise sum & 79.222 & 75.241 & 61.008 & 67.381 \\ ptwise sum + FC & 88.220 & 83.943 & 83.843 & 83.893 \\ ptwise sum + FC + FC & 89.864 & 86.185 & 86.597 & 86.390 \\ MCB + FC + FC & 89.975 & 86.736 & 86.238 & 86.486 \\ ptwise product & 86.296 & 81.957 & 79.578 & 80.750 \\ ptwise product + FC & 88.192 & 84.563 & 82.846 & 83.695 \\ ptwise product + FC + FC & 90.000 & 86.348 & 86.853 & 86.600 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance analysis for different combinations of the image feature and query feature in the proposed approach. for women, and five colors). As there is a difference between the gender-related categories, we just merged them into 27 categories. As a result, \(27\times 5=135\) classes were defined for the single-label classification, and 75 and five classes were defined for the multi-label classification. As shown in Table 11, the proposed approach outperformed conventional methods regardless of the number of input models and that of FC out layers, as conventional methods did not utilize multimodal information simultaneously. If the number of type or factor increases, the number of classification increases drastically so that the conventional method requires a lot of resources, and the results deteriorates. In particular, the difference between precision and recall is due to the much higher rate of false negative decision of the models since there are only one true answer and 134 false answers. Nevertheless, a further study is still needed to evaluate the proposed approach compared with other methods with similar architectures directly.