# Analysing Urban Traffic Patterns with Neural Networks and COVID-19 Response Data
## 3 Methodology and Data
### Methodology of the Study
#### 3.2.3 Methodology of Model Evaluation  
After creating neural network models, evaluating the quality of the obtained predictions is important. Current machine learning models are very sophisticated and capable of memorising input data perfectly, leading to overestimated evaluation statistics obtained from such models because predictions are checked on the same data on which the model was created. To verify the model's true predictive power, we randomly divided the dataset into training and testing sets. The second part of the data, the testing set, is used for model validation. Predictions (estimated numbers of vehicles of various types) are made on these testing cases in data and then compared to the actual values of the vehicle numbers. Since the model did not know the testing data, it was not trained on them; this comparison can be considered as the true predictive ability of the model without overestimation [55; 56].  
The predictive ability of the model can be evaluated using several evaluation statistics [56]:  
* Minimum error, maximum error, and mean error, which means a comparison of the predicted and actual vehicle numbers;
* Mean absolute error, calculated from the absolute differences between predicted and actual vehicle numbers;
* The standard deviation of the differences between predicted and actual values of vehicle numbers;
* Linear correlation is a coefficient expressing the strength of linear dependence between predictions and reality.  
When comparing the accuracy of models, it is advisable to focus on one of these evaluation statistics. The most commonly used are _mean absolute error_ (which should be as small as possible) and the _correlation coefficient_ (the closer the value is to 1, the more accurate predictions the model generates). In this study, we mainly used the correlation coefficient to evaluate the models, although, in the Results section, we provide all the mentioned evaluation statistics of the created partial models for individual types of vehicles and their directions and the comprehensive model for all types and directions.  
Finally, the precision of the predictions made by the neural network ensemble models is compared with that of four models created using other data mining techniques. For this purpose, we chose two decision trees: CART (Classification and Regression Tree) and CHAID (Chi-square Automatic Interaction Detection), as well as the Nearest Neighbours method and Multivariate Linear Regression. To ensure comparability of model performance, all these comparative models were also created as ensembles, following a similar procedure to that used for the neural network models, with up to ten component models combined by averaging their predictions into a final meta-model. After the learning procedure, the predictions for the cases in the training sample were compared with the actual values of the number of vehicles of individual types. Finally, the performance of these models was evaluated using the correlation coefficient mentioned above.  
For brevity, we do not provide a detailed explanation of the methodology used for the four comparative models here, but it can be found, for example, in [55; 57; 58; 59].