# Detecting Selected Instruments in the Sound Signal
## 5 Used Architectures and Experimental Results
### Convolutional Neural Networks  
In the next phase of experiments, research was carried out on the effectiveness of convolutional networks. Although mainly used in image processing, convolutional networks also work well in the audio field. The purpose of the tests was to check whether the _CNN_ network would also be effective in the analyzed problem. The convolutional networks used for testing were based on the architecture presented in Figure 3. The created _MFCCs_ matrix 128 \(\times\) 13 is an input data of evaluated convolutional neural networks. Such a matrix is processed by consecutive layers of three convolutional blocks, and then the data are flattened into one dimension so that it can be fed to the dense layers. The last layer contains 8 neurons according to the number of recognized instruments, and, as in the previous model, the softmax activation function is used. The first convolutional network that was built was marked as Model CNN1. Compared to the Figure 3 architecture, Model CNN1 does not have drop-out layers. It is worth paying attention to the _batch normalization_. This is a technique that allows to speed up the calculation process and improves the quality of the network. Knowing that data normalization brought the expected results in dense networks, it was decided to check whether it would be similar in the case of _CNN_ networks using this technique. Unlike the normalization performed in the previous cases, _batch normalization_ does not involve transforming the raw input data but normalizes the signals sent between the network layers. The model was trained using parameters with values similar to those used in the case of dense networks. The network with the given configuration was able to achieve an accuracy of 66%. The initial configuration of the _CNN_ network, as in the case of dense networks, is characterized by overfitting. The overfitting phenomenon was again partially eliminated by implementing _drop-out_ (Figure 3). This model with the _drop-out rate_ parameter set to 0.1 is marked as Model CNN2. However, it should be noted that this technique proved to be more effective for dense networks. The effectiveness of the network increased to 68%, and although precision for some instruments actually increased slightly, there was a significant drop in sensitivity for the clarinet. The next set of tests was performed for different values of the drop-out parameter. The Table 4 confirms a relationship analogous to that occurring in dense networks: too high a _drop-out rate_ value causes deterioration of results, although this time, the drop in effectiveness is not so drastic. Moreover, it should be noted that the optimal value of this parameter for _CNN_ networks is slightly larger than for dense networks (0.2 and 0.1, respectively). Therefore, the network with a drop-out of 0.2 was marked as Model CNN3, for which the accuracy is 69%. Figure 3: Diagram of a convolutional network. Guided by the experience gained from previous tests, it was decided to use normalization to reduce overfitting and improve the results. There are already normalizing elements in the network structure, but they operate based on individual batches of data, not the entire set. The test was to see whether normalizing the entire batch along with _batch normalization_ would improve the results or if it would lose some information by scaling too frequently. This network architecture with a drop-out rate equal to 0.1 is marked as Model CNN4 in Table 5. The network again made 68% accurate predictions, overfitting was slightly reduced, and based on the confusion matrix, it can be concluded that it managed to modestly increase sensitivity in relation to the clarinet and saxophone (by about 20%). A network analogous to CNN4 was also tested, but this time with a dropout of 0.2. This network architecture is marked as Model CNN5 in Table 5. The network made 64% of its predictions accurate. The next step was to check whether, as in the previous tests, the _LeakyReLU_ activation function would improve the results.