# Designing UAV Swarm Experiments: A Simulator Selection and Experiment Design Process
## 2 Background and Motivation  
A structured framework design for important UAV swarm processes, such as experiment design and validation, is of paramount importance for several reasons. Firstly, it provides a systematic and organized approach to tackle complex problems and research questions. By following a well-defined framework, researchers can establish clear objectives, identify variables, and establish a coherent plan, ensuring that their experiments yield reliable and meaningful results. Secondly, a structured framework enhances the reproducibility and comparability of experiments. With a detailed and standardized methodology, other researchers can replicate the study to validate its findings or build upon the existing knowledge, fostering scientific progress and collaboration. Furthermore, a well-designed framework helps in minimizing bias and errors in the experimental process [17]. By carefully considering potential confounding factors and controlling variables, researchers can enhance the accuracy and validity of their conclusions, strengthening the overall credibility of their work. Lastly, a structured approach aids in identifying limitations and potential pitfalls early in the research process. By incorporating rigorous validation procedures and statistical analyses, researchers can gain a deeper understanding of the robustness of their results, acknowledge any shortcomings, and suggest areas for further investigation. Thus, a structured framework design experiment design and validation is a cornerstone of reliable and impactful research. It fosters clarity, reproducibility, objectivity, and efficiency, ultimately advancing knowledge and facilitating evidence-based decision making in various domains. A search for recent articles focused on UAV simulator review yielded only a few results and are outlined in Table 1. A comparison of three simulation tools for UAV use is described in [18]. While this is a more concise approach, the authors in [19] present a broader review of the various platforms available. Article [20] further expands the review process by considering 17 different simulation platforms to review. Article [16] presents a broad view of swarm robotics, simulators, hardware, and behavior. During the course of the literature review, it was observed that multiple tools mentioned in past literature are now deprecated and, thus, were not included in Table 1. As technology rapidly evolves, previous studies may not reflect the current state-of-the-art in terms of available simulators and their capabilities. Tools such as USARSim [22] have pre-2020 development dates and are no longer updated. Additionally, tools such as VREP [23] have changed names, whereas Microsoft AirSim [24] while still having workable versions has been archived in favor of future projects. Additionally, as observed in [25], many researchers create custom scenario-based simulation packages for testing certain components. These lack code reusability and further support. Even currently acceptable broad-range simulation platforms may not satisfy all requirements for the objective study [26]. Therefore, there is a pressing need to create an updated and comprehensive list of UAV simulators that encompasses the latest advancements in the field. This list will serve as a valuable resource for researchers, developers, and practitioners, providing them with accurate and relevant information to choose the most suitable simulator for their specific needs. Researchers can stay informed about the latest options and make informed decisions when it comes to simulation-based studies and the development of UAV systems. Article [27] in its literature review survey concludes that MATLAB was the most widely used simulation platform for UAV swarm experiments. However, their methodology compares items such as Java against Visual Studio, which are a programming language and an IDE, respectively. Our search parameters were more concentrated and considered simulation platforms only.