## 4 Experiments and Results
### Performance Evaluation  
Figures 9 and 10 illustrate the loss function convergence curve and accuracy curve of the neural network training and testing phases for beach occupancy prediction, respectively. The decreasing loss and the increasing accuracy curves in Figures 9 and 10, respectively, generally suggest that the neural network model is learning to generalize on the target problem. To figure out the performance of the developed DNN, Figure 11 shows the confusion matrices on the test set (1) only for June, (2) only for July, (3) only for August, and (4) for the whole dataset (June, July, and August). The classification error for June (see Figure 11a) came mainly from the prediction of classes 150-199, 200-249, 250-299, and 300+. The reason is that there are fewer training samples for these classes, since the prime bathing season starts in June, and during this month, fewer people come to the beach. In July (Figure 11b) and August (Figure 11c), beach visitors of all classes are well predicted, with the exception of class 200-249 for July, since there are very few training samples for this class. No sample of class 300+ can be predicted for August, because during this month, the residents in Spain have holidays; they tend to travel far away from the cities, and Castelldefels beach is located very close to the city of Barcelona. All classes are well predicted for the whole dataset (Figure 11d). Figure 10: Accuracy for training and testing phases. Figure 9: Loss curve for training and testing phases. Figure 11: _Cont._  
The accuracy, precision, recall, and F1-score for all the classes and the whole data set were computed using the confusion matrix. The results as well as the number of instances per class for the test set are shown in Table 4. Accuracy is not a very good measure of performance when dealing with unbalanced datasets (like our case) because it counts the number of correct predictions regardless of the type of class. For this reason, it is biased towards the majority classes. As we can see in Table 4, the accuracy is extremely high (0.99 or 1), even when the model fails to identify some samples of minority classes (such as classes 200-249, 250-299, and 300+). Since accuracy gives biased results with unbalanced data, it is not a good metric to use. Therefore, we selected precision and recall, together with F1-score, as more reliable measures. Table 5 reports the results of our analysis for every category in the test set using the MAE, accuracy, precision, recall, and F1-score. The results of 10 independent runs are shown. We present the results (1) only for June, (2) only for July, (3) only for August, and (4) for the whole dataset. The DNN achieves the highest F1-score, with a value of 94.3866%, for August and a high F1-score of 92.7114% for the whole data set. The lowest results in terms of F1-score are obtained when the DNN is trained just for June, since the video cameras were unavailable during several days, and these days were essential to train the DNN. This fact suggests that training with more images would improve the classification. Furthermore, from the confusion matrixes, it can be observed that in June (when the beach attendance is lower) there are fewer samples to train categories with a high number of beach visitors, and \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Classes** & & & & & **Number of Samples** \\ (**Beach Visitors**) & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **per Class for the Test Set** \\ \hline
**0–49** & 0.99 & 0.98 & 0.99 & 0.99 & 121 \\
**50–99** & 0.94 & 0.94 & 0.90 & 0.92 & 302 \\
**100–149** & 0.96 & 0.88 & 0.92 & 0.90 & 159 \\
**150–199** & 0.98 & 0.93 & 0.93 & 0.93 & 110 \\
**200–249** & 1 & 1 & 0.90 & 0.95 & 10 \\
**250–299** & 0.99 & 0.83 & 0.97 & 0.90 & 36 \\
**300+** & 1 & 0.92 & 1 & 0.96 & 12 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of accuracy and F1-score for all classes. Figure 11: Confusion matrixes for (**a**) June, (**b**) July, (**c**) August, and (**d**) the whole dataset.