# Spectrogram Classification Using Dissimilarity Space
## 3 Siamese Neural Network  
The Siamese Neural Network (SNN) is a class of neural network architectures that contains two or more twins, i.e., sub-networks with the same parameters and weights. SNNs are used in tasks involving similarity or in identifying correlations between different entities. SNN was first proposed by [PERSON] et al. [39] for performing signature verification. SNNs have since been used successfully in other application domains, such as face verification [40], image recognition [41], human fall detection [42], content-based audio representation [34], and sound search by vocal imitation [43]. The SNN architecture used in this work is similar to the one used in [43] and is represented in (Figure 3).  
As shown in Figure 3, the SNN used in this work is composed of five blocks:  
Figure 3: Siamese Neural Network architecture.  
* _Two identical twin subnetworks_ The twin subnetworks in our SNN are two Convolutional Neural Networks composed of 13 layers, as listed in Table 1. \\ \end{tabular}
\begin{tabular}{c c c c} \hline \hline \# & **Layer** & **Filer Size** & **Number of Filters** \\ \hline
1 & Input Layer & 224 \(\times\) 224 images & \\ \hline
2 & 2D Convolution & 10 \(\times\) 10 & 64 \\ \hline
3 & ReLU & & \\ \hline
4 & Max Pooling & 2 \(\times\) 2 & \\ \hline
5 & 2D Convolution & 7 \(\times\) 7 & 128 \\ \hline
6 & ReLU & & \\ \hline
7 & Max Pooling & 2 \(\times\) 2 & \\ \hline
8 & 2D Convolution & 4 \(\times\) 4 & 128 \\ \hline
9 & ReLU & & \\ \hline
10 & Max Pooling & 2 \(\times\) 2 & \\ \hline
11 & 2D Convolution & 5 \(\times\) 5 & 64 \\ \hline
12 & ReLU & & \\ \hline
13 & Fully Connected & Returns a 4096-dimensional vector & \\ \hline \hline \end{tabular} These subnetworks learn the features best representing the spectrograms in the input (\(X1\) and \(X2\)), returning a 4096-dimensional feature vector for each (\(F1\) and \(F2\)). The subnetworks share parameters and weights which are mirrored during the training.
* _Subtract block_ The output vectors of the subnetworks are subtracted, resulting in a feature vector \(Y\) representing the features in which the images differ: \[Y=|F1-F2|\] (3)
* _Fully Connected Layer_ As in [37], the Fully Connected Layer (FCL) learns the distance model to calculate the dissimilarity. The output vector of the subtract block is fed to the FCL which returns a dissimilarity value for the pair of spectrograms in the input.
* _Sigmoid_ The sigmoid function is a class of mathematical real functions having a characteristic S-shaped curve. We apply the sigmoid to the dissimilarity value returned by the FCL to convert it to a probability value in the range [0, 1], using the standard logistic function: \[S(x)=\frac{1}{1+e^{-x}}\] (4)
* _Binary Cross Entropy_ The Binary Cross Entropy (BCE) is a popular loss function, which, given the prediction of the model and the correct observation label (in our case, 1 if the two spectrograms belong to the same class, 0 otherwise) returns a measure of the performance of the model. Loss functions are  
\begin{table} \end{table}
Table 1: Siamese subnetworks layers.  
used by learning algorithms to train the network by adjusting the weights. BCE is applied to the probability obtained from the sigmoid and computes the gradients of the loss function with respect to the weights of the network in order to adjust them. In a two-class problem, BCE can be calculated as:  
\[BCE(y,p)=-(y\log(p)+(1-y)\log(1-p)), \tag{5}\]  
where \(y\) is the binary value that indicates whether the class label \(c\) is correct for the observation \(o\), \(p\) is the predicted probability that observation \(o\) is of class \(c\), and log is the natural logarithm.