# FlockAI: A Testing Suite for ML-Driven Drone Applications
## 4 Modeling and Implementation
### Drone Configuration
for the to-be collected metrics and the implementation of the collect() method, periodically invoked during runtime. Hence, with the use of the probing interface, users can solely focus on \"pushing\" updates to the monitoring system rather than having to deal with time-consuming tasks, such as thread synchronization, logging, error reporting, that are requirements of the monitoring process. Moreover, users can attach to the FlockAI monitoring a queue to receive real-time updates of new measurements and/or utilize a key-value storage so that experiment data are persistently stored. Finally, monitoring data can be provided in raw form, json, and/or a pandas dataframe so that data can be used by ML practitioners in their native (data science) environment. \begin{table}
\begin{tabular}{l l l} \hline \hline
**ComputeProbe** & \\ \hline
**Metric** & **Unit** & **Desc** \\ cpuUsage & \% & Current drone CPU utilization averaged across all cores \\ cpuTime & ms & Total time CPU in processing state for the current flight \\ ioTime & ms & Total time CPU blocked waiting for IO to complete \\ memUsage & \% & Current drone memory utilization \\ \hline
**CommProbe** & \\ \hline
**Metric** & **Unit** & **Desc** \\ kbytesIN & KB & Total incoming traffic (in KB) across selected network interfaces (i.e., wifi) \\ kbytesOut & KB & Total outgoing traffic (in KB) across selected network interfaces \\ pctsIN & \# & Total number of incoming message packets \\ pcOut & \# & Total number of outgoing message packets \\ \hline
**FlightProbe** & \\ \hline
**Metric** & **Unit** & **Desc** \\ flightTime & s & Timespan of the current flight \\ hoveringTime & s & Timespan drone in hovering state \\ takeoffTime & s & Timespan drone in takeoff state \\ landingTime & s & Timespan drone in landing state \\ inMoveTime & s & Timespan drone in moving state \\ inAccTime & s & Timespan drone is in moving state and velocity is fluctuating \\ \hline
**EnergyProbe** & \\ \hline
**Metric** & **Unit** & **Desc** \\ eProc & J & Energy consumed for compute tasks (when drone in flight \\  & & continuously updated) \\ eComm & J & Energy consumed for communication tasks \\ eMotor & J & Energy consumed for powering drone motors \\ eTotal & J & Total energy consumed by drone for the current flight \\ batteryRem & \% & Percentage of drone battery capacity remaining \\ \hline \hline \end{tabular}
\end{table} 
Table 1: FlockAI monitoring probes and metrics. ## 5 Evaluation

FlockAI is deliberately designed to facilitate the rapid experimentation of ML-driven drone applications to derive analytic insights referring to the impact of ML algorithms to a drone's resources. This section provides a comprehensive study demonstrating the main contributions of our work by introducing a realistic use-case under-going various deployments and tested with different ML algorithms, so that ML practitioners can truly assess that pinpointed KPIs are met before deploying the application to production. ### Use-Case

Suppose an ML practitioner must test both the efficacy and efficiency of a developed ML solution that employs the use of drones for pinpointing potential crowded areas by performing face detection to count the number of humans in an urban environment. Without ML, this process must either be performed through human inspectors on the ground or by manually counting people from the video feed of a drone. Both are timely consuming processes. Towards this, the drone is given a flight plan including a (bounded) area of interest and a starting location. The drone will then fly above this area and scan the bounded area with its camera to capture a detailed image feed. This image feed will be used by the ML algorithm to infer both the locations of the human subjects and count how many are in the drone's viewpoint. Figure 4 depicts an example when applied on a camera feed extracted from the VisDrone dataset [29].