# Music Similarity Detection Through Comparative Imagery Data
## 4 Experimental Results: Model Training, Validation, and Testing
### Testing with Known Plagiarism Data  
The model can now be tested with known plagiarism data. In Section 2.3, we discussed one such case, \"Under Pressure\" vs. \"Ice Ice Baby\". Figure 13a shows the similarity patterns detected by our ensemble model, which compared two songs using a 2D sliding window of \(8\times 8\) bars and created an output heatmap. In other words, each pixel in this heatmap corresponds to an \(8\times 8\) pixel block in the original feature-based heatmap at the resolution of one bar per pixel. For human visualization, this model output is then dilated to recover the locations of the regions of similarity in the songs as shown in Figure 13b, where the similarity mask is thresholded to reduce the visibility of pixels with low similarity values. We can compare this with the original weighted-sum heatmap shown in Figure 2b; Figure 13b, which is referred to as an _ensemble similarity mask_, identifies all the critical regions shown in Figure 2b while allowing humans to focus on these regions more quickly and easily.  
One important benefit of using comparative imagery as the input and output of a model for music similarity detection is that humans can review the input and output more intuitively. Figure 13c-f show four individual feature-based comparative heatmaps that are inputs to the four feature-based sub-models, respectively. We can observe that the ensemble output in Figure 13a does not solely depend on one feature. From the top to bottom of the image, the following can be observed:  
* The first long line of bright dots (\(y\approx 0\)) is related to the corresponding bright patterns in (c, d).
* The second long line of bright dots (\(y\approx 35\)) corresponds to the bright patterns in (c, d, e).
* The short line of dots on the center-left (\(y\approx 60\)) corresponds to the bright patterns in (c), where the bright patterns span across the entire heatmap horizontally. It is interesting that the ensemble model considered only the small part on the left.
* The third long line of bright dots (\(y\approx 70\)) corresponds to (d).
* The fourth long line of bright dots (\(y\approx 120\)) corresponds to (c, d, e).  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Model** & **TP** & **FP** & **TN** & **FN** & **Precision** & **Recall** & **Accuracy** & **F1 Score** \\ \hline MI Histogram & 25 (83.3\%) & 4 (11.1\%) & 32 (88.9\%) & 5 (16.7\%) & 0.862 & 0.833 & 0.864 & 0.847 \\ FFPC Histogram & 21 (70.0\%) & 10 (27.8\%) & 26 (72.2\%) & 9 (30.0\%) & 0.677 & 0.700 & 0.712 & 0.689 \\ WVI Histogram & 27 (90.0\%) & 2 (5.6\%) & 34 (94.4\%) & 3 (10.0\%) & 0.931 & 0.900 & 0.924 & 0.915 \\ DBTMCVI & 19 (63.3\%) & 13 (36.1\%) & 23 (63.9\%) & 11 (36.7\%) & 0.594 & 0.633 & 0.636 & 0.613 \\ \hline Ensemble & 28 (93.3\%) & 3 (8.3\%) & 33 (91.7\%) & 2 (6.2\%) & 0.903 & 0.933 & 0.924 & 0.918 \\ \hline \hline \end{tabular}
\end{table}
Table 9: The results of testing the four sub-models and the ensemble model with independent testing data that differ from the training data significantly. The percentage values are normalized based on the class size, i.e., TP/(TP + FN), FP/(TN + FP), TN/(TN + FP), or FN/(TP + FN). The abbreviations of feature names are MI (melodic interval), FFPC (folded fifths pitch class), WVI (vertical interval), and DBTMCVI (distance between two most common vertical intervals).  
The input heatmap in (f) does not contribute to the horizontal-line pattens shown in (a).
* Input heatmaps (c, d) show four vertical strips, while input heatmaps (e, f) show three vertical strips. For the left part of heatmaps (\(x<20\)), the ensemble model was more influenced by (c, d).
* The bright spot on the bottom-left of (c) did not trigger a positive reaction in (a), suggesting that the ensemble model does not completely rely on the melodic feature. These observations demonstrate that our approach can provide a means to improve the explainability of AI.