# A Distributed Real-Time Monitoring Scheme for Air Pressure Stream Data Based on Kafka
## 4 Experimental Results and Analysis
### Kafka Throughput Results and Analysis  
Kafka demonstrates high throughput and low transmission delay in the real-time transmission of air pressure data streams. Therefore, we have selected Kafka as the messaging middleware for the distributed real-time air pressure data stream monitoring scheme. We conducted tests on the absolute throughput and relative throughput of the data streams to comprehensively evaluate the performance of the middleware. The aim was to identify the bottlenecks in system performance and improve the overall performance and reliability of the system through scheme enhancements. Producers and consumers input and output messages through specified topics. In the topic settings, the partition specifies the practical storage method of the message. To effectively use a topic, multiple partitions must be defined. In Kafka's pub/sub structure, the consumer determines the topic of the received message, and the number of consumers determines the number of messages received. As part of a general distributed system, Kafka provides system fault tolerance through replication settings. When the leader of a partition fails, Kafka can automatically select another leader for compensation to ensure the repeated reception or output of data. Therefore, the number of topic partitions and replication factors in Kafka's broker have a significant impact on system performance. We determined the specific parameter configuration by setting four different topics. The topics created are listed in Table 5. \begin{table}
\begin{tabular}{c c} \hline \hline
**Topic ID** & **Configuration** \\ \hline
1 & 3 partitions, 3 replications \\
2 & 1 partition, 3 replications \\
3 & 3 partitions, 1 replication \\
4 & 1 partition, 1 replication \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Topic list in Kafka. Figure 8: Delay comparison. In our performance analysis experiments, we divided the throughput tests for producers and consumers into two modes: independent and synchronous. In the independent test, we conducted separate performance tests for producers and consumers to ensure that they did not interfere with each other. This allowed us to accurately verify the performance of producers and consumers and provided a reference for the design of the scheme parameters. In the synchronous test, we simultaneously ran performance testing scripts for producers and consumers to evaluate their relative throughput performance across multiple nodes. This approach simulated the scenario wherein producers and consumers work simultaneously in the practical system, providing a better reflection of the overall performance of the scheme. The results of independent and synchronous tests conducted on Topics 1-4 are depicted in Figure 9. The x-axis represents 5 s time intervals, while the y-axis represents the average number of messages over consecutive 5 s periods; for instance, in test A, the value corresponding to marker point 1 on the x-axis is 2,070,549. This indicates that during the initial 5 s of the test, the average rate at which the producer transmitted data to the Kafka broker through Topic 1 was 2,070,549 msg/s. The remaining points in the graph reflect the performances of the producer and consumer during specific time intervals. The independent test results of A, C, E, and G demonstrate that the consumer's throughput exceeded that of the producer. This phenomenon can be attributed to the Kafka controller's intervention when the producer inputs messages into the partition, which subsequently impacts the production speed. Specifically, the controller must perform multiple operations. First, it manages partitions and replicas and sends related information to ZooKeeper. Second, for topics that contain multiple partition groups, the controller distributes messages to each partition. Additionally, the controller must also manage the replication mechanism to ensure that the message replicas of each partition are distributed to the various brokers according to the specified replication quantity.