# Evaluation of Multi-Stream Fusion for Multi-View Image Set Comparison
## 5 Results  
For an experimental evaluation, the Stanford Cars dataset [25] was used. We fixed the number of images in every image set to four. The order of views in the set is not relevant. We assumed that each proper set has four views: front, side, front-side, and back-side. Obviously, the approach is view-independent and in other applications any other number of views can be set. This doesn't mean that with different views we can get the same quality of results. The choice of a specific kind of view depends on the user's decision and their experience. From a technical and methodological point of view, our solution is independent of the selection of object views. Our dataset has less accurate data of back view images, so we decided to skip this view. Hence, the dataset is labeled by 4 views, 9 car types, and 197 car models. Both types of object-related label contain the class _other_ for images which are not matched with the basic labels. We conducted several experiments with this dataset and different settings of the system, and produced much improved results compared with the earlier base solution. The assumptions for the experiments are described in Table 4. With the same dataset we used, alternatively, DeepRanking- or SIFT-based features, and turned on or off an additional stream for car model classification. The dataset was split into learning (70%) and test parts (30%), and every experiment was repeated 10 times with different dataset splits. In Table 4, the average accuracy rates of different configurations of the proposed solution and the base solution are presented. By \"accuracy\", we mean the \"genuine acceptance rate\" (GAR) of a recognition system when \"false acceptance rate\" (FAR) is kept at 3%. The summarized accuracy of set-similarity recognition was increased from 50.2%, for the best baseline setting, to 67.7% for the best option of proposed approach. The best architecture appeared to include the (additional) car model classification stream and the DeepRanking net for feature extraction (instead of the SIFT descriptor). As we can clearly see in Table 4, the use of an explicit BSS error index as a measure of permutation-independent distance between two image sets and the additional car model classification stream allow for a significant improvement in the set similarity classification accuracy. For image feature extraction, it is more advantageous to use the DeepRanking approach than the classic SIFT descriptor. At first glance, the accuracy value of 67.7% seems to be a relatively worse result. However, the considered problem is rather hard to solve. Let us illustrate the results in more detail, by considering three different set-versus-set situations (Table 5): \begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **Accuracy** \\ \hline Base solution (DeepRanking + Euclidean distance) [24] & 50.2\% \\ Proposed with DeepRanking and BSS score, without car model stream & 62.1\% \\ Proposed with DeepRanking and BSS score and car model stream & 67.7\% \\ Proposed with SIFT and BSS score, without car model stream & 61.6\% \\ Proposed with SIFT and BSS score and car model stream & 64.4\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average accuracy of the similarity of two image sets. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer Number** & **Type of the Layer** & **Output Size** \\ \hline
1 & Dense & 64 \\
2 & Batch Normalization & 64 \\
3 & Dense & 128 \\
4 & Batch Normalization & 128 \\
5 & Dense & 196 \\
6 & Batch Normalization & 196 \\
7 & Dense & 128 \\
8 & Batch Normalization & 128 \\
9 & Dense & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Architecture of the final classifier for the base solution. is affected by such a situation. An image-pair similarity score of around 0.5 may result from obviously different views or from a visible change of the viewpoint direction, but within the same view type (see Figure 9).