# Respiratory Sound Classification by Applying Deep Neural Network with a Blocking Variable
## 4 Experiment Result
### Accuracy and Confusion Matrix  
We compared our results with the state-of-the-art model result on both the official train-test split and the 8:2 train-test split. The experiment results are shown in Table 2. For the 8:2 train-test split, our final proposed model achieved a 72.72% for the average score and 66.31% for sensitivity, improving the state-of-the-art average score by 4.22%. The model also raised the state-of-the-art sensitivity by 12.61%, which is a massive improvement because the sensitivity is considered a more challenging task than the specificity due to the data imbalance problem. For the official train-test split, our model achieved 51.98% average score, which is the state-of-the-art result if the model is not pre-trained on any other data set. Our model achieved 42.63% sensitivity on the official train-test split, which improves the state-of-the-art result by 2.53%. The details of the confusion matrix are shown in Figure 5. The low sensitivity in the official train-test split can primarily be attributed to the model's inability to accurately identify Wheeze data. This is due to the scarcity of training Wheeze data available within the official train-test split. There are only 491 Wheeze data and277 Both data in the train set. However, the results are much better in the 8:2 train-test split because of more sufficient data. Table 2 shows that the accuracy improves from 71.35% to 72.72% using two-stage training on the 8:2 train-test split. It proves that adding the blocking variable in a deep learning model improves the model's performance. The effectiveness of the blocking variable grows with the size of the data, as the improvement of the model in the second stage is more significant in the 8:2 train-test split than in the official train-test split. However, we found that the blocking variable was only effective when the distributions between the blocks were heavily skewed. If it is not the case, adding the blocking variable in the fully connected layer will introduce noise, which may even lead to an opposite effect. We also conducted the experiment by adding other possible blocking variables, e.g., Acquisition mode and Recording equipment. However, no significant improvement in the prediction result could be found. It is worth mentioning that different recording equipment has different sampling frequencies, which do have an effect on the distribution of the input picture. However, this problem has already been addressed in the pre-processing stage. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Method** & \(S_{e}\) & \(S_{p}\) & **Score** \\ \hline \multirow{6}{*}{Official Split} & [PERSON] et al. [9] & - & - & 39.56 \% \\  & [PERSON] et al. [10] & 20.81\% & 78.05\% & 49.43\% \\  & [PERSON] et al. [11] & - & - & 49.86\% \\  & [PERSON] et al. [14] & 31.12\% & 69.20\% & 50.16\% \\  & [PERSON] et al. [15]. & 41.32\% & 63.20\% & 52.26\% \\  & [PERSON] et al. [16] (Pre-trained) & 40.1\% & 72.3\% & 56.2\% \\  & Proposed model stage 1 & 42.88\% & 60.05\% & 51.47\% \\  & Proposed model stage 2 with blocking & 42.63\% & 61.33\% & 51.98\% \\ \hline \multirow{6}{*}{8:2 Split} & [PERSON] et al. [12] & 58.43\% & 73.00\% & 65.70\% \\  & [PERSON] et al. [13] & 48.63\% & 84.14\% & 66.38\% \\  & [PERSON] et al. [15] & 63.69\% & 64.73\% & 64.21 \% \\  & [PERSON] et al. [16] (Pre-trained) & 53.7\% & 83.3\% & 68.5 \% \\  & Our proposed model stage 1 & 63.99\% & 78.72\% & 71.35 \% \\  & Our proposed model stage 2 with blocking & 66.31\% & 79.13\% & 72.72 \% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of our model with other methods (\(S_{e}\) indicates the sensitivity, and \(S_{p}\) indicates the specificity). Figure 5: Confusion matrices for two splits of the data. Other techniques that were found to be effective can be summarized as follows. We found that adopting the different kernel sizes and average pooling increased the accuracy of the model in the first stage of training by 1.21% in the 8:2 train-test split.