# Intelligent Measurement of Coal Moisture Based on Microwave Spectrum via Distance-Weighted kNN
## 2 Materials and Methods
### Data Analysis  
In this paper, the process of data analysis combined with machine learning methods mainly includes outlier detection and elimination, data split, data modeling and model testing. The purpose of outlier detection and elimination is to improve the cleanliness of the data for accurate subsequent modeling. Data split is to randomly divide the data set into training sets and test sets according to a certain proportion. The training set is used for subsequent data modeling, while the test set is used to test the trained model to verify the effect of the model. Next, the methods used in this paper are introduced in detail. In practical applications, experimental spectral data may contain random noise fluctuations in different frequency bands. These fluctuations can be caused by environmental changes or incorrect operation during the spectral data collection process. When the noise fluctuation exceeded a certain threshold range, the signal was considered an outlier signal. The existence of outliers could interfere with the spectral feature extraction by machine learning models, and the prediction results could be adversely affected. Therefore, it was necessary to detect and eliminate abnormal samples. To address this problem, an iterative clipping method based on Mahalanobis distance (MD-ICM) was employed. The Mahalanobis distance (MD) indicates the similarity between a point and a cluster of points [30]. It accounts for the distribution and correlations of the cluster data while keeping the scale-invariant features [31]. Let \(x_{i}\) be the \(i\)th sample in the spectral data \(X_{m\times n}=[x_{1},x_{2},\ldots,x_{m}]^{T}\), where \(m\) denotes the number of samples, and \(n\) denotes the number of frequency points. Then, the MD value between \(x_{i}\) and \(\gamma\) is calculated by:  
\[D(x_{i},\gamma)=\sqrt{(x_{i}-\gamma)^{T}\mathbf{\Phi}^{-1}(x_{i}-\gamma)}, \tag{7}\]  
Figure 3: The original microwave spectrum curves: each spectral curve corresponds to a coal sample with one specific moisture content. where \(\gamma\) and \(\mathbf{\Phi}\) are the mean and the covariance matrix of the spectral data, respectively, and they are expressed as follows:  
\[\gamma^{p}=\frac{1}{m}\sum_{i=1}^{m}x_{i}^{p},v=1\ldots n, \tag{8}\]  
\[\mathbf{\Phi}=\frac{1}{m-1}(X_{m\times n}-\gamma)^{\mathsf{T}}(X_{m\times n}- \gamma), \tag{9}\]  
where \(\gamma^{p}\) and \(x_{i}^{p}\) are the \(v\)th element of \(\gamma\) and the sample \(x_{i}\), respectively. The selection of distance threshold determines whether outliers can be effectively detected and eliminated. According to the Pauta criterion, outliers can be detected under the \(3\sigma\) threshold (i.e., three times the MD standard deviation between a sample and the mean) with a confidence probability of 99.73% if the distance between the target sample and the mean spectrum follows the Gaussian distribution [32]. The specific steps of the MD-ICM diagnostic are as follows:  
1. Calculate the MD, mean, and covariance matrix of the spectral data by Equations (7)-(9), respectively;
2. Calculate the mean and standard deviation of the MDs by Equations (10) and (11), respectively: \[\chi=\sum_{i=1}^{m}D(x_{i},\gamma)/m,\] (10) \[\sigma=\sqrt{\frac{1}{m}\sum_{i=1}^{m}(D(x_{i},\gamma)-\chi)^{2}};\] (11)
3. Compare \(D(x_{i},\gamma)\) with the threshold; if \(|D(x_{i},\gamma)-\chi|>3\sigma\), sample \(x_{i}\) is considered an outlier;
4. After outlier detection, repeat the above steps until no outlier is found. There is a strong nonlinear and complex relationship between the microwave signals and coal MC. Therefore, it is difficult for traditional linear fitting methods to establish an accurate relationship model between them. In response to this problem, a coal MC analysis method based on the DW-kNN algorithm is proposed. The \(k\)-nearest neighbor (kNN) is a machine learning method for predicting continuous data based on sample spacing in the feature space, which has been widely used in the fields of data classification and regression analysis.