# Using Computer Vision to Collect Information on Cycling and Hiking Trails Users
## 4 Performance Evaluation
### Results and Discussion
#### 4.4.2 Tests  
To validate the results obtained, several comparative tests were carried out on the three trained models, using various videos recorded in real context. For these tests, the error/accuracy of the different models was taken into account as well as the consumption of computational resources using the FPS comparison. According to [71], video input is more demanding in terms of computing power than direct input from a webcam or similar device. The confidence parameter was set to 93%, i.e., only bounding boxes appear for the object classes that the model has identified with a confidence equal to or greater than this threshold. In a future prototype, it is intended to place the camera in strategic locations, such as on a pole about a meter off the ground (a topic that will be studied in the second part of this work). In addition, cameras with a high field of view (FOV) will be used, allowing objects to appear in the scene for longer. The initial option of carrying out tests with the camera positioned on the ground aims to provide an effective preliminary evaluation of the algorithms under controlled conditions, allowing for initial optimizations before transitioning to the final configuration. For the first test, a video containing a bicycle moving toward the camera was used to see how each of the models identified this element. In Figure 38, we can see that all the models correctly identify the bicycle class. Interestingly, ResNet-50 also classifies a dog in the scenario as a person, which is an error. Figure 39 shows that both YOLOv3-Tiny and ResNet-50 models correctly identify a bicycle even when it moves in the opposite direction to the camera. In contrast, MobileNet-SSD V2 identifies a bicycle as a person. It can also be seen that ResNet-50 wrongly identifies a dog as a person. For this test, it can be concluded that YOLOv3-Tiny performs better than the other models. Figure 37: ResNet-50 validation results. Figure 38: Examples of bicycle detection results for the three models (moving towards the camera). For the second test, videos containing people were used to see how each of the models identified this element. Figure 40 shows that all the models correctly identified the people present, classifying them as a person. It is worth noting that ResNet-50 again identified wrongly a dog as a person. So, YOLOv3-Tiny and MobileNet-SSD V2 models perform better in this test. For the last test, videos containing motorcycles were used to assess their identification by each of the models. Figure 41 shows motorcycles moving in the opposite direction to the camera. YOLOv3-Tiny and ResNet-50 models correctly detect and classify motorcycles, whereas MobileNet-SSD V2 incorrectly mistakes the motorcycle with a person. Therefore, for this test, it can be concluded that YOLOv3-Tiny and ResNet-50 perform better. Figure 42 shows a motorcycle moving towards the camera. Both the YOLOv3-Tiny and MobileNet-SSD V2 models fail to classify the motorcycle, wrongly classifying it as a person. ResNet-50 correctly identifies the motorcycle. Therefore, it performs better in this case. Figure 43 shows that all the models were able to correctly classify the motorcycle when it was near the camera. As far as computing resources are concerned, after the video test was completed, it was observed that the most demanding model was the ResNet-50, reaching only 4.15 FPS. On the other hand, the model that proved to be the least demanding was the MobileNet-SSD V2, reaching 27.85 FPS. Finally, the YOLOv3-Tiny achieved a rate of 16.58 FPS. From these results, it was concluded that ResNet-50 could be too demanding for the scenario considered in the context of this work. Table 3 summarizes the results of the performance metrics of the models under evaluation. Figure 40: Examples of person detection results for the three models. Figure 39: Examples of bicycle detection results for the three models (moving in the opposite direction to the camera). It was concluded that YOLOv3-Tiny achieved a mAP of 68.7%, while MobileNet-SSD V2 only achieved 44.52%.