# A Knowledge-Driven Multimedia Retrieval System Based on Semantics and Deep Features
## 4 Experimental Results
Mean Average Precision is expressed as:  
\[AveP=\frac{\sum_{k=1}^{n}(P(k)\cdot rel(k))}{number\ of\ relevant\ documents} \tag{11a}\] \[MAP=\frac{\sum_{q=1}^{Q}AveP(q)}{Q}. \tag{11b}\]  
In this work, we consider the MAP@10 because it is mostly used on web document retrieval considering the first 10 results. As previously stated, the measure of semantic similarity has been performed considering: _path similarity_, _Leacock-Chodorow Similarity_, _Wu-Palmer Similarity_, _Resnik Similarity_ and _Jiang-Conrath Similarity_. We use the _Lesk_ algorithm as disambiguation technique. The query set used consists of twelve text queries, which have polysemic meaning. An example of a polysemic query is \"mars\" which in WordNet has two meanings, \"Mars\" as god and \"Mars\" as a planet. In this context we are interested in text analysis and the used document collection is 20 newsgroups. In the Figure 11 there is shown the Precision-Recall curve, where we can observe that _Jiang-Conrath Similarity_ is the best measure, but in Figure 12 the _path similarity_ where the best measure with respect to MAP@10. \begin{table}
\begin{tabular}{c r} \hline \hline
**Level** & **Num.** \\ \hline I & 2 \\ II & 54 \\ III & 1023 \\ IV & 2353 \\ V & 2485 \\ VI & 1699 \\ VII & 918 \\ VIII & 641 \\ XI & 81 \\ X & 8 \\ \hline Total & 9264 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Number of multimedia web document for each level of DMOZ collection. According to results shown in Figures 11 and 12 we choice path similarity and Resnik similarity, the first one is path-based type. We consider two measures on semantic textual similarity due to the very small difference obtained obtained by experiments. In this way we can investigate more deeply in the use of a combination of these measures and multimedia descriptors. The used visual descriptors are: _PHOG_, _ORB_ and deep descriptors. In particular, the deep descriptors are:  
* VGG-16 with global max pooling and global average pooling;
* ResNet-50 with global max pooling and global average pooling;
* Inception V3 with global max pooling and global average pooling;
* MobileNet V2 with global max pooling and global average pooling. The used query set consists in sixty-five images, divided into ten classes. The used dataset is PASCAL VOC2012. Figure 11: Precision-Recall Curve for Textual Semantic Similarity Measures. Figure 12: MAP@10 for Textual Semantic Similarity Measures. Figure 13 shows the Precision-Recall curve, where we can see that the descriptor extracted from _ResNet-50 with global average pooling_ has the best result. Moreover, in the Figure 14 the _MobileNet V2 with global average pooling_ is the best considering MAP@10. The chosen descriptor, according to Figures 13 and 14, is the one extracted with MobileNet V2 with global average pooling. In this case we consider only one descriptor due to the high difference in accuracy obtained by the results comparison. The evaluation strategy has been defined by different test cases performed on DMOZ data set to have a complete analysis of our framework in a real scenario. The test cases are:  
* _Case A_: text query:
* Path similarity;
* Resnik similarity;
* _Case B_: visual query, using deep descriptor obtained with MobileNet V2 with global average pooling;
* _Case C_: visual and text query:
* Path similarity and deep descriptor;
* Resnik similarity and deep descriptor;  
Figure 13: Precision-Recall Curve for Image Descriptors.