# A Survey of LoRaWAN for IoT: From Technology to Application
## 4 Tools and Methodology
### System Level Simulators  
System level simulators are used to perform reproducible experiments for different systems, considering the impact of all parameters in the overall performance of the system. Such simulators are beneficial for determining the overall network performance when there is no large scale testbed deployed. Using such an approach, LoRaWAN network level performance under certain heavy condition scenarios, such as large number of end nodes or high traffic loads, can be studied. For this, a number of system level simulators have been presented in the literature [24; 34; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49]. They account for different LoRa physical layer characteristics as well as LoRaWAN MAC behavior. NS3 is one of the most common used network level simulators. Due to this, a number of LoRaWAN simulators are NS3 compatible [40; 42; 44; 48]. On the other hand, there exist other simulators implemented in Python [41] and C++ [24]. However, not all of them are publicly available and open source. The network simulator in [41] is a simple simulator that can be used to assess the performance of LoRaWANs, however, it has some drawbacks. First, the collision property is only based on the average RSSI difference and relative timing between transmissions, not accounting for the SNR values between transmissions. Moreover, signals received with different SFs are considered purely orthogonal, that is not true under certain power level conditions. The end node parameters (SF and transmit power) can be adjusted based on the distance from the gateway and it offers the possibility to run networks with multiple gateways. However, LoRaSim does not implement any ACK or downlink traffic possibility. An extended version of LoRaSim with downlink support has been presented in [50]. The study performed in [51] determined the Signal-to-Interference (SIR) ratio threshold at which a signal can be correctly decoded under inter-SF interference. The study was based on MATLAB simulations as well as real node measurements of SIR. Based on these figures, the authors of [51]extended the LoRaSim simulator [41] to receive both packets under inter-SF interference condition only when the differences between RSSI is below certain threshold (in this case 6 dB). In addition, they also extended the simulator to account for log-normal channel fading. Another simulator implementation offers the possibility to use MAC commands, collision based on interference between different SFs as well as multi-GW support, however, it lacks the downlink traffic as well as confirmed uplink traffic [48]. The physical layer model in [40] is based on a LoRa PHY error model that is determined using an extensive set of physical layer simulations in MATLAB. Interference from other signals using different SFs and the same channel is considered by calculating instant SNR values. Contrary to this, the physical layer model in [42] considers only the received power level of the concurrent transmissions and not the overall interference impact. This approach leads to an underestimation of the interference and collision impact in the network. In [40], multi-gateway networks can be simulated. The implementation also includes downlink traffic, ACKs as well as confirmed downlink/uplink traffic. In addition, different SF assignment techniques can be considered: random, fixed or PER based. A drawback of the implementation in [40] is that no MAC commands are implemented. However, this one will not impact the performance evaluation of LoRaWAN, unless ADR impact will be assessed. Contrarily, in [42], MAC commands are implemented. In [42], there are some small flows in implementation that differ from the standard: for example, the MAC at the network side is implemented in the gateway and not in the network server; the end node will wait a random time before transmission (that does not make a real Aloha); and, at the gateway, the downlink traffic is delayed for the next RX-WINDOW if there is uplink traffic being received.