# Architecting and Deploying IoT Smart Applications: A Performance-Oriented Approach
## 5 Performance Analysis: Results  
Figure 7 summarizes the key results of Thing-to-Cloud Elapsed Times for smart agriculture and smart city for the six configurations C1, C2, C3, C4, C5, and C6. The bars filled with horizontal, diagonal and zigzag patterns mean that the experiments did not finish correctly due to higher workloads, even after numerous repetitions. Also, configuration C1 did not run at all for smart agriculture with 1500 messages per minute-15,000 sensor probes-as VMs always failed shortly after the beginning of the experiments. Comparing both scenarios, we can observe that smart city (Figure 7a) had a slightly better result, especially for the highest workload of each scenario. Those results show that the components tested performed better with messages arriving in bursts-event-driven sensors-instead of messages arriving with a constant flow-time-driven sensors. Notably, configurations, including LoRaWAN (C1, C4, and C5), achieved higher scalability than the ones with generic LPWAN. Besides performing its intrinsic function, ChirpStack works as a cushion for sensor data and softens message spikes. Regarding an overall comparison of the experiments using LoRaWAN (configuration C1, C4, and C5) and generic LPWAN (configuration C2, C3, and C6), it can be noted that for lower workloads ChirpStack adds about 200 ms to the processing, since more software components are needed for addressing issues such as security and authorization. \begin{table}
\begin{tabular}{l l} \hline \hline
**Factor** & **Level** \\ \hline Scenario & Smart Agriculture–Smart City \\ \hline Workload & \\ \hline Smart Agriculture: soil sensor probes sending data every & 5000–10,000–15,000 \\
10 min (time-driven sensors following a Constant distribution) & (500–1000–1500 messages per minute) \\ \hline Smart City: car arrival rate given in cars per second & 8–16–24 (480–960–1440 messages per minute) \\ (event-driven sensors following a Poisson Distribution) & \\ \hline Deployment configurations & C1–C2–C3–C4-C5-C6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Factors and levels for the performance analysis study. In the smart agriculture scenario with the highest workload (1500 messages per minute), only two configurations ran up smoothly to the end of the experiments without failing, namely C4 and C5. This result was not expected since C6 failed, and it is the lightest configuration in terms of the number of software components that are placed in the robust VM in the cloud. Nevertheless, both scenarios highlight the stability and scalability of configurations C4 and C5, mainly due to the improved data flow control of ChirpStack. On the other hand, when the workload is placed directly on the IoT Agent, the experiment fails, e.g., C6.Configurations C1 and C2 use a heavy fog, where the Consumer uses data directly in S3-Fog, allowing applications to process part of the data closer to the user and thus to respond to changes in real-time. In Figure 8, we see that for C1, most of the time budget is elapsed up to S3-Fog, whereas for C2, the time to reach and process data in the cloud dominates the path for both scenarios. This behavior comes from the ChirpStack processing time, present in C1 and absent in C2. For applications that require lower response times-like a Smart City scenario-the LoRaWAN 200 ms processing time might be unacceptable, which can make a case for a more straightforward LPWAN solution. Figure 7: Device-to-Cloud elapsed time for IoT smart applications. We also measured CPU and RAM usage for all software components, located both in S3-Fog and S4-Cloud, which helped us to understand the behavior of the elapsed time metric. In all experiments that did not finish correctly, the cause was a failure in IoT Agent. However, it is relevant to notice that we used two different implementations: (a) a specially developed LoRaWAN IoT Agent; (b) the FIWARE official Ultralight 2.0 IoT Agent.