# Architecting and Deploying IoT Smart Applications: A Performance-Oriented Approach
## 5 Performance Analysis: Results
Although both play the same role in converting IoT protocols into NSSI JSON and share the same programing language-JavaScript using Node.js-they differ in terms of performance and scalability. Some findings are worth highlighting:  
* The LoRaWAN IoT Agent located at S3-Fog crashes in configuration C1 with the high workload for the smart city scenario due to spikes in CPU usage as depicted in Figure 9. A few seconds after a certain threshold of messages per second is reached, the CPU usage goes up to 100%, and this component presents a type of instability similar to a phase transition, leading it to crash soon after 341 s of the experiment. * The Ultralight 2.0 IoT Agent located at both S3-Fog and S4-Cloud crashes in configurations C2, C3, and C5 with the high workload for the smart agriculture scenario due to a linear and constant increase in memory usage. Figure 10a shows it keeps allocation memory up to 239 s and crashes, which corresponds to the CPU usage of MongoDB, and also Orion, at S4-Cloud. Figure 10b shows that after the IoT Agent crashes, the CPU usage of MongoDB drops from more than 70% to almost zero after 40 s. In other words, the crash of the IoT Agent in S3-Fog is instead a consequence of another performance bottleneck located at S4-Cloud. The service time of the IoT Agent is shorter than the joint service time of Orion and MongoDB so that the former queues messages waiting for HTTP REST requests to be answered by the latter. In other words, the IoT Agent waits for a TCP ACK packet to free the memory allocated for the messages. As the message flow is constant in the smart agriculture scenario, the memory allocated by the IoT Agent increases boundlessly up to a point the operating system fails to provide more space, and it crashes. The high usage of CPU by MongoDB does not affect the performance of the platform since the IoT Agent crashes before the database starts to present significant problems. * ChirpStack is a well-developed and stable software component that adequately uses system resources. The combined CPU Usage of all individual components that make up ChirpStack was around 40% for high workloads and 6% for low workloads, whereas the sum of memory usage was approximately 30 MB for high workloads and 22 MB for low workloads. Except for C1, all the configurations that crashed during the experiments were not based on LoRaWAN and consequently did not use ChirpStack. We conclude that, in addition to performing his LoRaWAN functions, the ChirpStack also works as a cushion for sensor data that softens message spikes that prevent IoT Agent to queue too many messages. Figure 8: Device-to-Fog elapsed time for IoT smart applications. * The smart agriculture and smart city scenarios presented a similar performance for low and medium workloads. In those cases, the difference in the probability distribution for message generation did not impact the system performance, unlike for high workloads, where all experiments with Generic LPWAN (no ChirpStack) crashed for smart agriculture. On the other hand, in the smart city scenario, configurations C2 and C5 ran to the end of the experiment, and C3 was the only one with LoRaWAN that faced problems. This behavior indicates that the system can better handle traffic in bursts than in a constant flow of messages, because the time between bursts is enough for Orion to dequeue some messages and send ACKs back to the IoT Agent, freeing some memory in the process. Figure 10: Cloud CPU Usage and Fog Memory Usage for Configuration C2 in a smart agriculture scenario in the experiment with 1500 messages/min. Figure 9: CPU usage of S3-Fog components-Configuration C1–Smart City.