# A Geographical Heuristic Routing Protocol for VANETs
## 5 Performance Evaluation
### Statistical Procedure  
The performance analysis of our heuristic protocol GHR is based on four different metrics. These metrics are:  
* Packet Delivery Ratio (_PDR_): This is the total percentage of packets (sent from the vehicles) that reach the AP. PDR does not increase due to the reception of copies of a packet. Hence, this metric measures the effectiveness of the routing protocol in terms of different delivered packets.
* Average end-to-end delay: This is the average time elapsed from the transmission of a packet until it arrives at the destination (computed either for the original packet or for a copy).
* Average number of hops: This is the average number of hops that a packet needs to reach the AP. This average includes the hops performed by the original packet and by its copies.
* Percentage of idle time: This is the average of the idle time sensed by a node measured in 1 s. A node senses the channel as idle when it is not transmitting nor receiving a packet and given that the interference level is below the antenna sensitivity. Notice that a higher idle time sensed by the nodes leads to more bandwidth available in the channel to transmit more information. An estimation of the available bandwidth derived from the idle time measure could be done using the models proposed by [35] or [36].  
We are interested in knowing if the differences in the performance metrics listed above are statistically significant or not. Furthermore, we want to determine if the presence of differences depends on the vehicle density and/or on other factor, for instance the use of the Tabu list in routing operations. To do this, we carry out MANOVA [37] (Multivariate Analysis of Variance ) tests over the data using the statistical software SPSS [38]. We use MANOVA to consider the inherent correlation among the performance metrics when they are not independent from each other.  
For the MANOVA tests, we report the value of the statistics Wilk's \(\Lambda\) and F, which allow us to obtain a probability called the \(p\)-value. A \(p\)-value is compared with a threshold named the significance level to determine if the simulation findings are statistically relevant (i.e., the \(p\)-value is lower than the significance level). We use for our test a \(p\)-value \(=0.05\) for the significance level. We divided the statistical procedure analysis into three steps:  
* Tests to determinate interactions among the identified factors for each analysis over the four performance metrics. If an interaction is detected, then the performance differences in a factor will depend on the combination of the factors involved.
* Tests to determine if there is statistical difference in metrics for each one of the groups in which the dataset was divided because of the presence of interactions. If there is not a statistical difference in a metric, then this means that this metric behaves similarly under the different levels of the studied factor (i.e., heuristic technique). Otherwise, the test tells us that there is a difference, but it does not indicate between which levels of a factor this difference is present.
* Pairwise comparisons for a metric. If the previous tests determine a significant difference in a metric with a certain factor, then we run a \(t\)-test pairwise comparison among the different levels of the factor under analysis. The objective of this step is to establish the performance relation between levels.  
By following the above test order, we are able to provide a detailed and accurate analysis of the advantages and costs of our proposals. Next, we present our analyses and statistical results for each routing protocol separately.