# SURROGATES: Virtual OBUs to Foster 5G Vehicular Services
## 5 Evaluation
### Evaluation Methodology  
A number of trials have been carried out with the aim of obtaining significant results, above all for the performance evaluation. The test location has been the north area of the Espinardo Campus at the University of Murcia, depicted in Figure 4. For each individual trial, the vehicle starts moving at the upper green mark in the map and stops at the lower red one. The length of the circular path is 1 km.  
Figure 4: Test-site and path followed by the car.  
Although OBD data is periodically reported by the vehicle at a rate of 1 Hz, we have used the Android application to regularly ask the platform for car parameters. The way these requests are organized depends on the three experiments carried out. A first one is focused on evaluating the operation of the platform when different cache success rates are obtained in the vOBU. A cache hit represents a data request that is solved by the vOBU instead of asking the physical OBU. This behavior has been emulated by a success rate setting included in the app, varying from 0% of cache hit rate to 100% of success. Upon receiving a request from the app with a particular hit rate, the vOBU asks its physical counterpart or locally resolve the request with the given probability. Five percentage values have been used and, for each one, five trials have been carried out. Each data request involves a TCP transaction, and the reply is marked with a round-trip delay time (RTT) as it traverses the different nodes of the data path. This way, we can analyze the request RTT in different network segments.  
A second experiment is focused on studying the ratio of request losses. Since the network rarely loses packets due the the usage of a cellular connection and the TCP protocol, we have forced the system to be more rigorous regarding the time the vOBU waits for a response from the OBU. This behavior is more realistic because it is expected that packet losses appear when involving crowded urban areas, rural locations with poor coverage, or when communication technologies with a limited deployment are used. Hence, we have developed the vOBU software to provide the cache data when a certain delay threshold is reached. This way, when the OBU is accessed, the communication is not blocked waiting for TCP timeouts. This delay threshold is considered with values of 300 ms, 400 ms and 500 ms, given that, in our testbed, the time measured at the vOBU to receive a response from the OBU is around these values after a set of initial tests. Again, for each configuration, we have carried out five trials.  
The third experiment has been carried out to analyze the system scalability and the time a new vehicle needs to register and obtain a vOBU instance. In this case, it is important to note that our solution hides the instantiation time by using a pool of vOBUs that is dynamically scaled by OSM through monitoring the free VNFs. Although the number of free vOBUs to be maintained in the pool is static at the moment, it is actually dependent on various external factors. This opens up future work to make an estimation of the number of cars that might be traversing a certain VNF infrastructure based on location or time (day, month) by means of machine learning and open data sources. For the tests, 30 registrations have been carried out, measuring the whole time needed by the registration process.  
As can be noted, a total of 45 real trials for the first two experiments, plus the registration trials, are considered in the results presented in the next section. For each trial configuration, the delay times are averaged and, specifically for the second experiment, the ratio of \"lost\" requests is obtained. Moreover, for each configuration, the 95% confidence interval has been calculated for these two metrics and included in the plots.