## 2 Methods
### Methodology  
#### 2.2.1 Identification Method  
Data-driven methods are used to identify abnormal data and restore them. For the collected data on the elevation of pipeline, some of the data are judged and identified via expert knowledge, and the labels 0 and 1 are assigned to abnormal data and normal data. The classification model is then trained based on the labeled data, which are used to identify other data for the elevation of pipeline. Before model training, additional input features are constructed to assist the training of the classification model. The difference method is used to construct the new feature, i.e., the difference between the data and the last data. The nearby data of the data point are used as input features at the same time to  
Figure 1: The cross-sectional configuration of this project. give the model a better understanding of the sequential data. The support vector machine classifier is used in this model. For water level data, there are many influence factors (e.g., water flow, water level, pump frequency and valve opening at nearby nodes) and various operation patterns, so it is difficult to directly identify abnormal data. Therefore, in the judgment of abnormal water level data, the clustering algorithm is used to cluster the data first, and then the abnormal data are identified by the 3-sigma method according to different operation patterns. The clustering algorithm uses the K-medoids algorithm, which is an improvement on K-means clustering. Instead of using the mean, K-medoids clustering uses the most central object in the cluster, that is, the medoid, as the reference point, and that of each selection must be a sample point. When there are noise and isolated points, K-medoids is more robust than K-means, but the time complexity of calculating the centroid step is O(n2), and the running speed is relatively slow. The basic idea of the algorithm is to divide the observed objects into several subsets, and each subset is regarded as a class, so the objects within the class are similar and the objects between the classes are not similar. The steps are as follows:  
1. Input the data to be clustered and randomly select K points in the data as the centroids. 2. Aggregate all data points in the dataset into the closest cluster based on the distance to the centroid. 3. Compute the distance between each cluster point as a centroid, and then choose the point with the smallest distance as the new centroid. Repeat steps 2 and 3 until the centroids of every cluster no longer change or the sum of squared errors (convergence function) is minimized. The sum of squared errors (SSE) is calculated as follows:  
\[\text{SSE}=\sum_{i=1}^{k}\sum_{\text{p}\in\text{C}_{i}}\abs{\text{p}-\text{m }_{i}}^{2} \tag{1}\]  
where C\({}_{i}\) is the ith cluster, p is sample data points in C\({}_{i}\), and m\({}_{i}\) is the centroid of C\({}_{i}\). The smaller the SSE of each cluster, the higher the overall compactness of the data around the centroid in each cluster under this clustering condition, and the better the clustering effect. It is very important to find the optimal value of K for the k-medoids algorithm. The silhouette coefficient is adopted in this paper, and its basic principle is as follows: first, the intra-cluster similarity a(i) of the sample is calculated, which represents the average distance between the sample and other sample points in the same cluster. The smaller the value, the more correct the classification. Then, the dissimilarity measure b(i) is computed, which is the average distance between the sample and the other clusters. Higher values indicate better classification. Therefore, the calculation method of Silhouette(i) is as follows:  
\[\text{Silhouette}(\text{i})=\left\{ \begin{array}{cc}1-\frac{\text{a}(\text{i })}{\text{b}(\text{i})}&\text{a}(\text{i})<\text{b}(\text{i})\\ 0&\text{a}(\text{i})=\text{b}(\text{i})\\ \frac{\text{b}(\text{i})}{\text{a}(\text{i})}-1&\text{a}(\text{i})>\text{b}( \text{i})\end{array} \right.