# Developing Occupancy Grid with Automotive Simulation Environment
## 3 Results  
Experimental evaluation was performed using the configurations of the methods and the architectures. The results of this evaluation are presented in Table 1. The presented methods were run in both of the test scenarios (Scenario 1--without moving obstacles--and Scenario 2--including two moving targets). Reprocessing was carried out through the complete processing chain and resulted in the list of performance indicators presented in the bar plots in Figure 11.  
Figure 11: Colored bar plots of KPI results per scenario. TPR—True Positive Rate, PPV—Positive Predicted Value, F1 Score, FPR—False Positive Rate, FDR—False Detection Rate, NOMS—Normalized Occupancy Map Score.  
Figure 11 presents the results for Scenario 1 and Scenario 2. The relation between a higher sensitivity of a method (having a higher True Positive Rate and Positive Predicted Value) and its susceptibility to false classification when subjected to clutter (having higher False Detection Rate and True Positive Rate) was observed. This is reflected in small differences of the F1 score. This relation can also be observed in the raw probability of occupancy presented in a form of snapshots in Figures 12 and 13.  
Overall, most of the methods have lower performance in Scenario 1 (visible in blue in Figure 11) compared to Scenario 2 (visible in red in Figure 11). This observation relates to the inability of the performance assessment method to explicitly handle occlusions and dynamic objects.  
Looking at the results presented in the Figure 12, observations on practical aspects of the methods can be made. An important practical factor for all methods is that the obstacles, such as impact attenuating barrels, are properly reported by all the tested configurations, so this means that methods should satisfy simple object avoidance perception needs. In the cases of High Level Bayesian, Fused High Level Bayesian, Fused High Level Logarithmic Independent Opinion Pool and Fused High Level Independent Opinion Pool, we can see a tendency in the suppression of false occupancy in the right emergency lane, which is considered positive for the methods' performances in applications, such as emergency trajectory planning. The mentioned methods, however, present much lower maximum range at which the occupancy from guardrails are accumulated, which can have a negative effect in case of application--i.e., for map matching.  
During the course of the analysis of the probability representation presented in Figure 13, inconsistency in the occupied areas was identified in the right lane. Further investigation of the observed behavior led to the conclusion that deficiencies in the sensor simulation process resulted in false ground detection. Those deficiencies result in a reduction in reliability of the KPIs and create an opportunity for further development. The second observation related to the influence of sensor model \"spreading\" of probability influences the overall performance, depending on the method used for fusion. This indicates that the performance of given method could be adjusted by fine tuning or modifying the ISM.