# Lightweight CoAP-Based Bootstrapping Service for the Internet of Things
## 4 Experimental Results
### Performance Evaluation
#### 4.2.2 Bootstrapping Time  
We have defined a scenario with different numbers of hops between the Smart Object and the border router. This allow us to evaluate the performance of each protocol in a more realistic scenario than a simple node connected to a border router.  
The tests have been performed with different numbers of hops (\(n\)), from 1 to 7 hops between the Smart Object and the border router. We have observed though that 7-hop case has proved to be a scenario without connectivity between the Border Router and the Smart Object being bootstrapped. Three different packet loss ratios (0, 0.1 and 0.2) are used to evaluate the response of both solutions.  
From the different tests we have gathered the following information: (1) the bootstrapping median time that both alternatives take to complete a bootstrapping for each number of hops and packet loss ratio (Figure 9a-c); and (2) the number of bootstrapping processes that are able to finish (success percentage (%)) in each case (Figure 10a-c).  
We can observe in Figure 9a-c that there is a statistically significant difference between the bootstrapping times in PANATIKI and CoAP-EAP. To obtain these values we performed 200 simulations for each value of \(n\) and for each packet loss ratio.  
On analyzing the distribution of the data, we noticed that, a skewness test on the data, returned a value greater than 1. According to Jain [104] a proper index of central tendency is the median rather than the mean, since it provides a more significant description.  
In the case of 0 loss ratio and 1 hop, the median values do not differ greatly. In this case CoAP-EAP takes \(\approx\)1.5 s and PANATIKI \(\approx\)1.6 s. As expected, this difference, partially due to the reduction in message size of our solution (see Section 4.2.1), increases with the number of hops and packet loss ratio since each intermediate smart object has to forward a message and handle fragmentation of these messages. Since PANA has longer messages, PANATIKI provides a longer bootstrapping time than CoAP-EAP. For example, when packet loss is set to 0.2 loss ratio with 1 hop, the median time to accomplish a bootstrapping in CoAP-EAP \(\approx\)5 s and \(\approx\)6 s in PANATIKI. As expected, this time clearly increases with the increment of number of hops.  
Moreover, when the packet loss ratio increases the possibility of completing a bootstrapping decreases. This is implicitly shown in Figure 9a-c. For example, we were not able to complete a bootstrapping process with PANATIKI beyond 5 hops with 0 packet loss ratio; 4 hops when 0.1 packet loss ratio and 3 hops with 0.2 packet loss ratio. However, CoAP-EAP is able to complete the bootstrapping up to 6 hops. This is mainly due to it has a shorter message length and, therefore, it generates less fragmentation. Additionally CoAP has a less aggressive retransmission policy than PANA, which promotes sending less traffic over a constrained network.  
Figure 9: Bootstrapping median Time. (**a**) Bootstrapping Median Time with 0.0 loss ratio; (**b**) Bootstrapping Median Time with 0.1 loss ratio; (**c**) Bootstrapping Median Time with 0.2 loss ratio.  
Figure 10: Bootstrapping success percentage. (**a**) Success percentage with 0.0 loss ratio; (**b**) Success percentage with 0.1 loss ratio; (**c**) Success percentage with 0.2 loss ratio.  
To corroborate this, Figure 10a-c show how the success percentage, the bootstrapping processes really finished from those that were started, evolves as the packet loss ratio increases. CoAP-EAP demonstrates a better performance in every packet loss ratio in comparison with PANATIKI. In the worst case scenario with 0.2 packet loss ratio, PANATIKI with 2 hops (1 intermediary node) already has a success of \(\approx\)9% which makes the bootstrapping service impractical, whilst CoAP-EAP in the same conditions is able to finish \(\approx\)85% of the initiated bootstrapping processes and keeps an acceptable success percentage as the number of hops increases.