# Stream-DBSCAN: A Streaming Distributed Clustering Model for Water Quality Monitoring
## 3 Stream-DBSCAN Water Quality Detection Clustering Model
### Build the Model  
To establish the relationship between water quality and time from the water quality data, the Stream-DBSCAN model employs a clustering method to group the pH, ammonia nitrogen, and turbidity indicators into clusters. The model then combines data points with similar indicators into a single cluster, allowing each cluster to represent a specific class of water conditions. The process of Stream-DBSCAN model construction can be expressed as follows:  
1. The Stream-DBSCAN model utilized experimental data from the Menlou Reservoir between May to August 2019, focusing on three water quality indicators: NH\({}_{4}\)N, pH, and turbidity. Since the magnitude of each dimension in the experimental data varied significantly, the model first standardized and sampled the data to ensure that the results considered the impact of each dimension while avoiding redundancy in similar data operations. 2. To ensure the streaming nature of the dataset, the Stream-DBSCAN model employs Kafka to convert sensor-generated data into a continuous data stream prior to clustering. The stream processor in Kafka can continually collect the data stream, apply built-in processing to adjust it, and output the stream in real-time. Within this framework, Kafka combines data generated by the NH\({}_{4}\)N, pH, and turbidity sensors based on a common timestamp attribute before transmitting the data stream to the Flink framework. By utilizing Kafka, the model benefits from high throughput and low latency, allowing it to process hundreds of thousands of data points per second with only a few microseconds of delay. 3. During the data partitioning stage, it is crucial to consider whether the partitioning results will result in larger clusters during the clustering process and decrease the number of noise points. Once Kafka passes the data stream into Flink, the DataStream must be converted into KeyedStream using keyBy for data partitioning. To ensure the partitioning is reasonable in this model, we implement the K-means algorithm [28] to perform rough clustering of the dataset [29], dividing data with similar values into the same KeyedStream. If the data is directly inputted into each node without rough clustering, numerous small-scale clusters will appear in the clustering results, and some points that should be in the cluster will be misjudged as noise points, resulting in rejection before cluster merging [30]. Considering that the dataset's size is not too significant, this model opts to divide it into three. The rough clustering process of the dataset using the K-means algorithm is as follows: Using K cluster centroid points \(\mu_{1}\), \(\mu_{2}\),......, \(\mu_{K}\), the following clustering process occurs for each 3D data point \(\eta i\): \[C^{(i)}=arg\ \min_{j}\Bigl{|}\Bigl{|}x^{(i)}-\mu_{j}\Bigr{|}\Bigr{|}^{2}\] (3)  
Among them, \(C^{(i)}\) is the class closest to point \(i\) among the K classes, and the value range of \(C^{(i)}\) is 1-K. \(x^{(i)}\) is the data point to be determined, and \(\mu_{j}\) represents each centroid point. For the K generated clusters, their centroids \(\mu_{i}\) have the following iterative formulas: \[\mu_{i}=\frac{\sum_{i=1}^{m}\Bigl{\{}C^{(i)}=j\Bigr{\}}x^{(i)}}{\sum_{i=1}^{m} \{C^{(i)}=j\}}\] (4) The Stream-DBSCAN model utilizes the K-means algorithm to coarsely cluster data, yielding K data streams that exhibit internal similarities and external differences. 4. During the process of distributed clustering, the selection of data parallelism K is crucial in determining whether the processing time is acceptable. Therefore, the K value is adjusted according to the size of the data, and the selection of K should strike a balance between the computing power of the computer and the time-consuming curve of the clustering algorithm. An optimal value of K will control the data processing time of each computer node within a manageable time cost.