# Semi-Automatic Cloud-Native Video Annotation for Autonomous Driving
## 5 AaaS Performance Evaluation
### Low to Medium Scale Video Analysis
#### 5.1.1 CPU-Intensive Job Scaling (CNN Object Detector)  
Different concurrences of batch jobs were tested for each of the clusters. Specifically batches of 16, 32, 64, 128, and 256 concurrent jobs. The analysis was carried out using a replica of the same video of a real driving scene on the road, so that the execution of all jobs could be compared. From the point of view of this study, the video content is not relevant since what we are looking for is the study of the management of the workload generated in its processing and analysis. For transcoding a video of 25 GB in size was used, so the studied concurrences were equivalent to video processing from 400 GB to approximately 6.5 TB. For CNN detection, a 6.25 GB mp4 video was used. Each of the different simultaneous batch execution experiments were deployed sequentially that is, a new concurrency test did not start until after 5 min from the time the previous test was completed, and thus guarantee that the system had released the resources from the previous test and was in a standby state. Figure 3 shows the average CPU load (average per minute) for each worker node in the Kubernetes cluster compared to the time of the entire experiment. The results in plots [a] and [b] are for the base cluster (4 working nodes and 64 cores in total) and for the horizontally scaled cluster (8 working nodes and 128 cores in total) respectively. The hill-like elevations in the graphs represent the workload in each of the different batch concurrency experiments run sequentially (16, 32, 64, 128, and 256). In both graphs [a] and [b], an average CPU load value greater than the number of cores per worker node, which means that traffic in the cluster is queued. The scheduling of a Pod is based on the request values defined for each job. Therefore, the execution of a Pod will be scheduled towards a worker node, only if it has enough available CPU resources to satisfy this requested CPU value. Thus the results show how the system is actually capable of handling (orchestrating) compute requests that are greater than the cluster's capacity and successfully complete all jobs. The plots [c] and [d] in Figure 3 show, for the two clusters compared, the same analysis of load per worker node, now stacked at 100%. In this way, it is shown how the system balances (orchestrates) the load between the worker nodes throughout the duration of the experiment. Figure 4 shows the total duration of each complete batch experiment vs. concurrency per worker node (concurrency job/cluster job nodes) for the three deployed clusters (the reference base, horizontally scaled, and vertically scaled). As expected, qualitatively from the graph we can confirm that the duration of the experiment is shorter for scaled clusters, with more resources. More interestingly, we can also observe that the relationship between the cluster resources and the duration of batch processing is not linear. It is observed that doubling the scale of the cluster does not provide half of the processing time. Nor is there a similar behavior between the two cluster scaling strategies. These non-linearities suggest a strong dependence on the efficiency of Kubernetes to manage the orchestration based on the architecture of the cluster, which translates into the different processing times for each experiment. To clarify this last analysis, the average duration of the jobs deployed in each batch experiment is analyzed in relation to the concurrency per core in the cluster (illustrated in Figure 5). It is observed that the end-to-end execution of the average job in the base cluster was always faster than in the scaled clusters. This difference decreased with increasing concurrency. On the other hand, the horizontally scaled cluster showed better performance than the vertically scaled.