# A Distributed Real-Time Monitoring Scheme for Air Pressure Stream Data Based on Kafka
## 3 Design of Real-Time Monitoring Method
### Framework Design  
With the advent of the Industrial IoT (IIoT) and industrial analytics, many new application scenarios have emerged for which critical decisions rely on the large-scale analysis of sensor streams [32]. As data volumes grow exponentially, applications must handle larger amounts of unstructured and complex datasets simultaneously. Relational database models (such as MySQL v5.7.44 ) have severe limitations when dealing with such massive data [33]. Non-relational databases, conversely, are well-suited for large-scale data processing due to their flexible structures, high performance, fault tolerance, and adaptability to dynamic schema. Among these, MongoDB, based on dynamic schema JavaScript Object Notation (JSON) documents, outperforms MySQL in insertions, queries, updates, and deletions [34], making it more suitable for the real-time monitoring method proposed in this paper. Therefore, we chose MongoDB v4.4.6 to store the processed data. In real-time data streaming scenarios, a smaller number of brokers can significantly reduce end-to-end delay due to the reduced complexity of cluster management processes [35]. For data packets of the same size, more partitions lead to an incremental increase in transmission intervals. This is because Kafka producers need more time to arrange the metadata for each partition [28]. Therefore, we configured the Kafka cluster with three brokers and divided the message transmission topic into three partitions. The overall framework of the proposed real-time pressure monitoring scheme consists of a sensor network module, a producer module, a cluster module, consumer modules, and a database module (Figure 4). The sensor network module consists of multiple sensor devices distributed in the working environment. Each device consists of an Arduino Uno microcontroller combined with an MPX5700 AP pressure sensor that gathers real-time environmental data. Using the SIM7000 communication module, the sensor sends real-time data, including the sensor ID, pressure value, battery level, and signal strength, in string format via the UDP protocol to the producer. Then, the producer module splits and integrates the data strings from the sensor network module into JSON format and sends the data to the specified topic in the Kafka cluster according to the arrival time. Next, the cluster module is distributed on a Kafka cluster consisting of three servers: each acting as a Kafka broker. The message topics are distributed among the three Kafka brokers and are divided into three partitions. For each broker, one partition is designated as \begin{table}
\begin{tabular}{c c} \hline \hline
**Content** & **Specification** \\ \hline Working voltage & 3.3 \(\leq\) 5.5 V DC \\ Power dissipation & 0.06 W (5 V) \\ Output signal & I2C output (0 \(\leq\) 3 V) \\ Measurement range & 15–700 kPa \\ Resolution ratio & \(\leq\)1 kPa \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Specifications of MPX5700 AP. Figure 4: Real-time air pressure monitoring method framework. the leader, while the remaining partitions are followers. The JSON data from the producer module are written to the leader partition and then replicated to the follower partitions. Then, the consumer modules receive the JSON data from the cluster module based on the subscribed topics and perform predictive analyses on them according to predefined thresholds. We designed a web and app interface (Figures 5 and 6) to provide users with visualization interfaces for sensor management, abnormal data alerting, historical data querying, and real-time stream data monitoring. Finally, the database module runs on another server and receives data from consumer modules; these data are persistently stored according to predefined storage policies. The consumer modules can retrieve historical data from the database module to formulate and modify alert and alarm strategies. The framework is designed to host data from sensors connected to equipment other than Arduino.