# Architecting and Deploying IoT Smart Applications: A Performance-Oriented Approach
## 4 Performance Analysis: Design and Methods
### Experimental Design  
Figure 6 extends Figure 5, providing additional technical information for the six deployment configurations introduced in, needed for the clear understanding of both experiments and results.  
* Smart applications: two scenarios of smart agriculture and smart city are considered in our experiments. The critical difference between them is the approach for sensor data generation. In the smart agriculture scenario, we consider many soil moisture sensor probes, continually generating and transmitting one data packet every 10 min. In the smart city scenario, we consider vehicles playing the role of sensors that communicate with a traffic light according to a Poisson arrival rate. In both scenarios, the SenSE sensor simulator synthetically generates sensor data.
* LPWAN Technology: data is generated by SenSE using the Ultralight 2.0 protocol format in both scenarios. Also, for LoRaWAN, sensor data packets are coded according to the LoRa PHY format and encrypted according to the AES algorithm determined by the LoRaWAN standard. For Generic LPWAN, plain sensor data is transmitted directly to the receiving end, which is always an MQTT broker. There is a sizeable difference in the payload generated by SenSE in the configurations with LoRaWAN and generic LPWAN. With LoRaWAN, SenSE sends ChirpStack a JSON over MQTT message with information about the emulated LoRa modulation in addition to the actual encrypted payload, composing a message of 314 bytes. In the configurations without LoRaWAN, SenSE sends an Ultralight 2.0-structured MQTT 65-byte message to the IoT Agent, significantly smaller than the LoRa message
* S1-Thing and S2-Mist: the SenSE simulator abstracts L1-Device in both S1-Thing and S2-Mist for all configurations. In other words, since our experiments are controlled and performed in a testbed, L1-Device is not present, except for the LoRa PHY format that is generated by SenSE, in order to understand the real effect of ChirpStack.
* _ChirpStack_: an open source LoRa server composed of two main components, the ChirpStack Network Server and the ChirpStack application server. Both communicate via MQTT with each other and with sensors and actuators, as well as platform components, e.g., FIWARE IoT Agent. Also, they use Redis and PostgreSQL as databases.
* WAN Emulation: to emulate the network link between the fog and the cloud, we used a WAN emulator (WANem) with 45 ms of latency and 5 ms of jitter. We defined the parameters to input in the WANem through a simple experiment performed the ping command to a VM located in the Amazon cloud in Sao Paulo, sent through 4G.
* Infrastructure: experiments were performed in an OpenStack-powered private cloud. As experiments were in a testbed, they are not influenced by other virtual machines running in the same physical servers. VMs for cloud and fog are based on standard Amazon AWS configurations: cloud runs on a t2.medium instance (2 vCPU-4 GB of RAM) and fog on a t2.small instance (1 vCPU-2 GB RAM). Our testbed was composed of two physical servers with the following configuration: Intel(R) Xeon(R) CPU E3-1240 V2 @ 3.40 GHz-8 cores and 16 GB of RAM.  
Experiments with a large number of sensors-representing soil moisture sensors or cars passing through a traffic light-simultaneously send messages to the platform, in order to verify and understand scalability tradeoffs. We varied all levels of all factors in each scenario, consisting of 36 different possibilities (Table 3).  
Figure 6: Testbed setup for the six deployment configurations.  
Notice that the workload for smart agriculture and smart city scenarios is specified in different units since they are based on different probability distributions for time-driven and event-driven sensors, respectively. Both represent somewhat similar workloads: 500-1000-1500 messages per minute for the smart agriculture scenario, and 480-960-1440 average messages per minute for the smart city scenario. Each experiment took 2 min with 30 replications, totalizing 36 h.