# Future Wireless Networking Experiments Escaping Simulations
## 1 Introduction  
The main vision of this paper is to pioneer future wireless experiments escaping simulations. Simulation has long been a valuable tool in testing and analysing the behaviour of new protocols and ideas in computer networking. However, when we move from simulation to testbeds we add more realism to the evaluation of the protocol/system [1]. A simulation environment can be seen as an ideal environment in which a protocol/system can behave well. However, when unexpected events and dynamic environmental constraints are induced (such as in a testbed including real physical systems), the behaviour of the protocol/system can be evaluated with higher confidence using testbeds. There are currently several testbeds available in the EU [2] and the US [1]. In the EU, several testbeds located in different locations can be accessed through Fed4 Fire ([[https://www.fed4](https://www.fed4) fire.eu/]([https://www.fed4](https://www.fed4) fire.eu/)) (accessed on 29 March 2022), whereas in the US, there are Cloud Enhanced Open Software Defined Mobile Wireless testbed for City-Scale Deployment ([[https://www.cosmos-lab.org/](https://www.cosmos-lab.org/)]([https://www.cosmos-lab.org/](https://www.cosmos-lab.org/))) (accessed on 29 March 2022) (COSMOS) and Platform for Open Wireless Data-driven Experimental Research ([[https://powderwireless.net/](https://powderwireless.net/)]([https://powderwireless.net/](https://powderwireless.net/))) (accessed on 29 March 2022) (POWDER) testbeds. As part of our EU NIGAtlantic project [3, 4], wireless Internet of Things (IoT) devices send data to an IoT application in the cloud via a gateway that runs network functions to ensure security and latency. In this project, software-defined networking ([5, 6]) and machine learning are used to make intelligent IoT decisions. In addition, several EU and US testbeds--WiLab1.t, W-iLab2.t, CityLab, COSMOS, and POWDER--are considered forexperimentation. The purpose of this paper is to highlight some of the conclusions made in this project. The motivation of our work is to create exemplary knowledge on a number of advanced wireless testbeds, located in the EU and US by providing a theoretical and practical analysis of a network deployed on these testbeds in terms of throughput, latency, jitter, resources available, and failure recovery time. The goal is to find the bottlenecks of the considered testbeds. Numerous papers in the literature focus on a particular scenario on a particular testbed, including [7, 8, 9, 10, 11, 12]. The objective of this paper is to report the results of comparing a similar network scenario deployed on different testbeds. There are also several survey papers in the literature reporting a theoretical comparison of testbeds, such as [13, 14, 15, 16]. However, this paper provides the theoretical results with the results of experiments performed on the testbeds considered. The contributions are as follows:  
1. To report a comparison of all the considered testbeds in the EU and the US. The following attributes of the testbeds are reported: (1) architecture comparison, (2) resources available, (3) IoT capabilities, (4) data that can be collected, (5) limitations, (6) Software-Defined Networking (SDN) capabilities, and (7) machine learning capabilities. This comparison is based both on our own understanding of the testbeds along with any additional information available from the literature. 2. To report the steps to deploy the experiments on the above EU and US testbeds. 3. To present the experiments performed by other researchers on each of the considered testbeds. 4. To design a benchmark and failure recovery wireless networking IoT experiments for all the above EU testbeds with different types of topologies: linear, ring, mesh, and grid. 5. To perform the experiments on all the testbeds and report the comparison with the resources available in each testbed. 6. To report the issues faced during experimentation on each of the testbeds. 7. Finally, we report our future work which is part of our NGIAtlantic project [3].