# Stream-DBSCAN: A Streaming Distributed Clustering Model for Water Quality Monitoring
## 2 Method Principle
### Flink  
Flink is a distributed stream processing platform that has emerged in recent years, originating from the Stratosphere project. Its main function is to distribute streaming data, which is characterized by stateful computations of both boundless and bounded streaming data. Flink can adapt to all common cluster environments and compute data of any size in memory storage. Next, Flink will be introduced from several aspects.  
1. Batch and stream processing: In Spark, all data is processed in batches. Offline data is processed in a large batch, while real-time data is processed in an infinite number of smaller batches. In contrast, Flink processes all data in the form of streams. Offline data is bounded, while real-time data is unbound. Flink treats batch processing as bounded flow processing, relying on precise time control and stateful running of applications.
2. Overview of system architecture: Similar to other streaming data processing frameworks, Flink uses a master-slave architecture composed of four parts: JobManager, ResourceManager, TaskManager, and Dispatcher. Figure 1 illustrates the interaction of these components after application submission.  
(1) Layered architecture  
Flink is a distributed platform for stream processing that follows a layered architecture, where each upper layer depends on services provided by the lower layer. Flink can be divided into four main layers:  
Deployment layer: This layer covers the installation methods of Flink, such as local, cluster, and cloud server.  
Core layer: This layer provides all the core implementations that support Flink computing. API layer: This layer implements the DataStream API for unbounded streams and the DataSet API for bounded streams, which are used for stream and batch processing, respectively.  
Library layer: This layer is the application architecture layer of Flink, which builds computing frameworks that meet specific application scenarios on top of different types of API layers. The hierarchical architecture of Flink is depicted in Figure 2.  
(2) Features of Flink  
Flink boasts the following seven key features:  
High throughput and low latency: Flink can process millions of data events per second with a computing speed of milliseconds.  
Event Time: Flink supports window calculations based on event time for unordered stream data, ensuring processing results' accuracy.  
State Management: Flink 1.4 provides state management, which stores intermediate results generated by previous calculations in memory, eliminating the need to count all data each time during stream processing. This approach reduces computational resource consumption and greatly improves computational efficiency.  
Highly Flexible Windows: Flink's windows can flexibly change to cope with complex streaming modes, including time-, count-, session-, and data-driven types.  
High Error Tolerance: Flink supports distributed snapshot technology and can recover tasks from Checkpoints when a parallel node encounters a problem.  
Figure 1: The interaction process of Flink components.  
Figure 2: Flink Layered Architecture Diagram.  
Independent Memory Management: Flink implements a memory self-management mechanism, which reduces the impact of JVM and GC on resource usage. It also serializes and deserializes data into binary and stores it in memory, effectively improving memory utilization.  
Save Point Mechanism: Flink sets the Save Points mechanism to save the execution of the task in a snapshot manner when processing infinite stream data. When the task restarts, it can restore its working state by reading the Save Points.  
Table 1 shows a comparison between Flink, Storm, and Spark Streaming, demonstrating the various advantages of Flink over the other two platforms. These features make Flink the ideal choice for the later work discussed in this article.