# FlockAI: A Testing Suite for ML-Driven Drone Applications
## 1 Introduction  
We are now witnessing the extensive deployment of drones in a diverse set of applications, such as rescue missions [1], infrastructure monitoring [2], urban sensing [3], and precision farming [4]. In rescue missions, due to rough terrain and possible ruins after a disaster (e.g., forest fire), it is difficult to timely deploy emergency monitoring facilities. Moreover, missions in such areas are dangerous for first responders and volunteers. Fortunately, with the rapid progression of robotics, artificial intelligence and edge computing, drones are now equipped with cameras and sensors, and can aid in identifying survivors, pinpoint rescue routes and detect environmental hazards by obtaining a favorable viewpoint for situational assessment [5]. However, drone technology comes with notable limitations. As the on-device AI tasks become more demanding (e.g., face detection), drones are severely impacted by battery autonomy [6]. This key limitation is inhibited by the AI algorithmic process, which imposes vast requirements in the data analysis, a prominent energy drain for drones and mobile systems in general [7, 8]. For example, commercial drones can fly for 30-40 min at best, thus mapping a medium-size area of 5 km\({}^{2}\) requires multiple battery charges that severely impacts mission timeliness [9]. To cover more terrain and increase mission timeliness, multiple drones can be deployed to \"parallelize the workload\". Recently, drone swarms are being employed, where drones are programmed to achieve a common objective by autonomously adapting their behavior based on shared information communicated between drones [10]. This new technology is based on Machine Learning (ML), wherealgorithms embedding trained models, enable--seemingly independent--robotic entities to share data and converge online by combining their insights into a single and emergent intelligence unknown to the individuals [11]. Hence, ML is a significant driver in the new era of drone applications, with ML algorithms limiting the reliance of the human-in-the-loop by tackling our inability to recognize objects from viewpoints far in the sky [12]. However, as the number of drones in a swarm increases, so does the complexity of the data-driven decision-making and the exchange of data between both participants and the ground control, which, in turn, critically affects swarm scalability and mission timeliness [13]. This brings us to the focal point of our work. Evaluating on-board ML algorithms for drone technology is inhibited by the lack of realistic testing frameworks that can be used alongside open and popular robotic simulators. Simulators are used heavily during the design and development phase of drone projects. This unfortunately burdens ML practitioners with the task of developing not only the proposed ML algorithms and training models but also requires vast amounts of time dedicated for the development of the benchmarking infrastructure to evaluate them in consolidated environments. As a result, ML algorithm evaluation for drone technology over key performance indicators, is limited, poorly repeatable and ad-hoc, at best. Consequently, the design, deployment, and evaluation of AI-driven drone applications becomes a complex and costly endeavor for researchers and engineers, since it requires the exploration of numerous conditions and parameters through repeatable and controllable experiments on a combination of physical and virtual testbeds that are hard to configure and measure. Most importantly, ML practitioners require frameworks that can reproduce experiment results so that experiment validation can be performed. Towards this, the main contributions of our work are:  
* A **comprehensive programming paradigm** tailored to the unique characteristics of ML-driven drone applications. The model expressivity enables developers to design, customize, and configure complex drone deployments, including encompassed sensors, compute modules, resource capabilities, operating flight path, and the emulator world along with simulation settings.