# An Initial Machine Learning-Based Victim's Scream Detection Analysis for Burning Sites
## 4 Results and Discussion
### Analysis of LSTM  
Experimentation was performed to evaluate the classification performance of the LSTM model on the custom dataset. The model was trained on the training data transformed in a three-dimensional array (batch size, time steps, features). We calculated the median time steps for this custom dataset to be 110 frames by setting the sampling rate at 16 kHz, frame length of 512 samples (32 ms), and 50% overlap (256 samples or 16 ms)--the input size of (110, 16), where sixteen represents the number of features. We set the data split ratios to be 60%, 10%, and 30%, respectively, for training, validation, and testing. Several experiments were performed by changing the number of LSTM layers, and we obtained the best result with two LSTM layers. Figure 9 shows the multi-layered LSTM architecture model that was used for classification. The hyperparameters of the LSTM were tuned with the validation set with the configuration set to the batch size of 24, a learning rate of 0.1, the activation function of 'tanh', 'Adam' optimizer; input units for both LSTM layer to be sixteen, and loss function of 'categorical_crossentropy'.  
Table 6 reports the overall accuracy indicated for the LSTM model is 95%, 2% lower than the SVM model. LSTM samples files with the 'alarm' audio sound presented more  
Figure 8: Feature contribution in the classification of the SVM classifier.  
Figure 9: Network architecture summary with two LSTM layers and a dense layer.  
misclassification and having the lowest rate scores that mirrors the same recall performance from the 'conversation' target.  
We can conclude that the model performs well with test accuracy of 95%. Table 7 shows the results of the'scream' target compared with the other targets in precision, recall, and F1 scores. The observation of individual targets' performances shows high comparative accuracies with'scream', but it does present more variation of results than the SVM model. Our class of interest'scream' shows a recall value of 0.95, 3% lower than SVM classifier. These performance metrics are close to the once we observed in the case of the SVM.  
The confusion matrix in Figure 10 reveals that 199 samples have been misclassified out of 4545 samples, where 40 scream files are misclassified as glass breaking, and only two scream files are misclassified as alarms. SVM outperforms LSTM by a minimal margin of 55 misclassified samples. Furthermore, similar to the SVM, the LSTM provided a stable model with minimal misclassifications and symmetrical misclassifications of edge cases presented during the test phase.  
We investigated the accuracy and loss curves to check if the model faces any overfitting or underfitting issues. The accuracy curve in Figure 11a looks a good fit indicating no issues  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Result Summary** \\ \hline Data samples in custom dataset & 15,147 \\ Training set:Validation set:Test set & 60\%:10\%:30\% \\ Training accuracy & 97\% \\ Validation accuracy & 95\% \\ Test accuracy & 95\% \\ Recall & 0.96 \\ Misclassified samples & 199 out of 4545 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance metrics of LSTM network.  
Figure 10: Confusion Matrix for the LSTM network.  
with overfitting or underfitting. However, when we looked into the loss curve shown in Figure 11b, we found that the loss in the validation curve started to increase and make the model unstable. We added a dropout layer [63] of 50% in both LSTM layers to resolve this issue, keeping all the other parameters the same to regularize the training and validation weight updates. This approach resolved our overfitting issue, and the curves indicated a convergence of high accuracy and minimal loss.