# FlockAI: A Testing Suite for ML-Driven Drone Applications
## 4 Modeling and Implementation
### Drone Configuration
This is due to the data preparation tasks of the sensory data and the \"firing\" of the data transmission. In regards to the effect on the total flight time, with inference performed remotely the drone can stay in the air for 25 min and 51 s, which is a bit more than 1 min when compared to on-board inference. Still, since the data payload is actually small, images in the DroneFace dataset and not FHD quality usually found in today's drones, performing remote inference seems to \"save\" a bit of flight time, but only 1 min more. However, Figure 9 provides a deeper overview of the cost of performing remote inference. In this,

Figure 8: Energy decomposed into its sub-components (Remote ML with CNN). Figure 7: Processing energy decomposed into its sub-components (On-board ML with CNN). one immediately observes the high latency for remote inference (median 3.15 s) compared to on-board inference (median 1.36 s), that can be disastrous depending on the use-case. Thus, an important insight is derived; deciding to \"offload\" the ML task from the drone may save a bit of flight time, however this actually can hamper overall flight performance in the end due to the delay in performing the inference task. Hence, having performed this experiment, a ML practitioner can answer Q3. ### Scenario 3: Testing Different ML Algorithms

This experiment scenario examines the trade-off between accuracy and drone performance when using different ML algorithms to achieve the same end goal (e.g., human detection and counting). This is of high interest to ML practitioners as one would like to be able to compare algorithms, based on different metrics, to understand advantages and pains of the algorithms before deploying to production. Hence, Scenario 3 adopts the same blueprint as Scenario 1, but for each experiment run a different algorithm is used for on-board inference. The ML algorithms that are examined are, the CNN used in Scenario 1, and an ML algorithm for human face recognition adopting Frozen Inference Graphs (FIGs) with the use of Tensorflow [32]. All of the sequences part of the DroneFaces dataset were used again as test data. Table 3 depicts the results of the experiments after being extracted from the monitoring of FlockAI. From the results, we observe that the CNN algorithm achieves the best accuracy with the accuracy standing at 95%. This translates to being able to correctly detect the crowd in 19 out of the 20 sequences. On the other hand, the FIGs algorithm achieves only a 60% accuracy which translates to 12 correctly identified out of the 20 sequences. In regards to inference time, we observe that the median inference time to handle an image in the camera feed is far from considered as comparable for the two algorithms with the CNN featuring a 10\(\times\) higher inference time. To further investigate inference, a custom monitoring metric was created in FlockAI to also keep track of how fast the crowd was detected in the image sequence. We reiterate that each sequence contains 30 images and the metric created (inference delay), measures when in the 30 image sequence was the crowd detected. Hence, if the inference delay is 26, this denotes that the crowd was detected from image 26 and onward. Table 4 depicts the inference delay for the two algorithms. In this table, a dash denotes that the algorithm could not detect the crowd. Figure 9: Inference latency. Clearly, the CNN algorithm is more accurate but this accuracy comes with a penalty in the speed, and compute resources consumed, at which the ML task can output a decision for the captured images. To examine the impact of the additional processing overhead imposed by the CNN when compared to FIGs, we examine the total flight time for the drone. Hence, Table 2, depicts the time the drone is in the air when employing either of the two algorithms.