# Application of Machine Learning to Include Honking Effect in Vehicular Traffic Noise Prediction
## 3 Results and Model Comparison  
The results obtained by using the four machine learning models are compared on the basis of r, R\({}^{2}\), MSE and accuracy. The above relation has been used for an accuracy of \(\pm 1\) dBA. The values of the different criteria for the four models are shown in Table 3. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Machine Learning Method** & **r** & **R\({}^{2}\)** & **MSE** & **Accuracy (\(\pm 1\) dBA)** & **[\%]** \\ \hline Decision Trees & & & & \\ (DT) & 0.935 & 0.876 & 0.884 & 68 \\ Random Forests & & & & \\ (RF) & 0.987 & 0.975 & 0.413 & 94 \\ Generalized Linear Model & & & & \\ (GLM) & 0.974 & 0.949 & 0.616 & 82 \\ Neural Networks & & & & \\ (ANN) & 0.974 & 0.949 & 0.616 & 82 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of results for the prediction models in the training phase. It is seen that random forests performs better than other models in the training phase, in the criteria of r, R2, MSE and accuracy. The value of MSE is 0.413 for rf which is the lowest and the value of accuracy is 94% (for \(\pm\)1 dBA) which is the highest among all other models. However, for a proper comparison, a testing dataset check was performed, in order to avoid overfitting. In order to meet this objective, the dataset was split into 70/30 as training and testing data, respectively. Following the training, using the 70% data, the models were tested on the testing dataset (30%). This was done 10 times (ten-fold cross validation) for the best performing model in order to check the stability and robustness of the model. The results obtained from the testing dataset show that the generalized linear model (GLM) performs better on the considered criteria, as seen in the Table 4.  
As seen in Table 4, the value of mean square error is the lowest (0.666) for the generalized linear model (GLM) and the accuracy is highest (80.0%). The values of MSE and accuracy obtained with neural networks are 0.717 and 73.3% respectively. Random forests and neural networks might give better results for bigger datasets but for the present case, GLM has outperformed the other models. Results of the models' predictions plotted versus measured equivalent levels in the validation and training datasets are shown in Figure 4. It is easy to see that apart from decision trees (DT) that in 4 points is very far from the measured \(L_{eq}\), all the models' results are gathered almost close to the bisector, meaning a general effectiveness of the proposed models. When the measured \(L_{eq}\) are around 70 dBA (site 2), the models have the best performances. This site is characterized by absence of heavy vehicles and honking frequencies lower than in the other sites. Data above 78 dBA are related to site 4, in which the traffic flows are higher than in the other sites. A scatter plot between the measured and predicted values of \(L_{eq}\) for the four methods is shown in Figure 5.  
A ten-fold cross validation was also performed for the best performing model, GLM and the results are shown in Figure 6. A general stability of the results is observed, confirming the goodness of the model and the suitability to be used in such applications, on noise data that includes a relevant contribution from honking. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Machine Learning Method** & **r** & **R2** & \
\begin{tabular}{c} **MSE** \\ **[dBA]** \\ \end{tabular} &
\begin{tabular}{c} **Accuracy (\(\pm\)1.0 dBA)** \\ **[\%]** \\ \end{tabular} \\ \hline Decision Trees & 0.797 & 0.636 & 1.780 & 40.0 \\ (DT) & 0.955 & 0.913 & 0.797 & 60.0 \\ Randomized Forests (RF) & 0.965 & 0.933 & 0.666 & 80.0 \\ Generalized Linear Model (GLM) & 0.959 & 0.921 & 0.717 & 73.3 \\ Neural Networks (ANN) & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of results for the prediction models (testing dataset). Figure 4: Scatter plot of the models’ predictions versus measured equivalent levels in the training dataset.