## 3 Experiment
### Comparison of Accuracy Assessment Efficiency between Different Sampling Methods  
In this section, we used four different sampling methods to a conduct accuracy assessment of the classification results, i.e., the simple random sampling method (SRS), stratified sampling method (SS), spatial sampling of the gray level co-occurrence matrix method (GLCM) [2], and our proposed method (MNSS). In order to verify the efficiency of our proposed MNSS, a full inspection was performed. Four indexes were used to evaluate the efficiency of different sampling methods: overall classification accuracy (OA), Kappa coefficient (Kappa), producer accuracy (PA), and user accuracy (UA).  
Overall classification accuracy (OA) [22] is the proportion of the correct classification of all categories relative to the total number of samples, reflecting the overall correctness of the classification results [23; 24]. The formula is:  
\[OA=\frac{\sum\limits_{i=1}^{n}m_{ii}}{\sum\limits_{j=1}^{n}\sum\limits_{i=1}^{ n}m_{ij}} \tag{5}\]  
The Kappa coefficient is different from the overall classification accuracy which only uses the diagonal elements of the confusion matrix; it utilizes the entire confusion matrix information and is a comprehensive measure of the classification accuracy [25]. The formula is:  
\[Kappa=\frac{N\sum\limits_{i=1}^{n}m_{ij}-\sum\limits_{i=1}^{n}\left(m_{i}+m+i \right)}{N^{2}-\sum\limits_{i=1}^{n}\left(m_{i}+m+i\right)} \tag{6}\]  
Producer accuracy (PA) is the ratio of the classifier correctly dividing the cells of the entire image into the number of cells in the class A (diagonal value) and the total number of real references in class A (the sum of the class A columns in the confusion matrix). The formula is:  
\[PA=\frac{m_{ii}}{m_{+i}} \tag{7}\]  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Sampling Rate (\%)** & \begin{tabular}{c} **1-Zoneat the** \\ **Original** \\ **Level** \\ \end{tabular} & \begin{tabular}{c} **2-Zoneat the** \\ **Original** \\ **Level** \\ \end{tabular} & \begin{tabular}{c} **3-Zoneat the** \\ **Original** \\ **Level** \\ \end{tabular} & \begin{tabular}{c} **4-Zoneat the** \\ **Original** \\ **Level** \\ \end{tabular} & \
\begin{tabular}{c} **5-Zoneat the** \\ **Original** \\ **Level** \\ \end{tabular} \\ \cline{2-7} \multirow{2}{*}{
\begin{tabular}{c} Zones at the \\ original spatial \\ resolution \\ \end{tabular} } & 1-zone at the & 15.1425 & 8.2901 & 1.2994 & 0.0355 & 0.0000 \\  & 2-zone at the & 6.7153 & 38.5560 & 11.7212 & 0.5592 & 0.0017 \\  & 3-zone at the & 0.1393 & 8.6040 & 7.3503 & 0.7383 & 0.0209 \\  & 4-zone at the & 0.0000 & 0.0814 & 0.5928 & 0.1345 & 0.0035 \\  & 5-zone at the & 0.0000 & 0.0000 & 0.0084 & 0.0042 & 0.0014 \\  & second level & & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 5: Sampling rate of each zone at the original spatial resolution (2-m).  
As shown in Figure 6, compared with random sampling method (SRS), the proposed method overcame the shortcomings that the sample points were randomly distributed in the studied region. Compared with stratified sampling method (SS) and spatial sampling of the gray level co-occurrence matrix method (GLCM), the proposed MNSS reduced the redundant information between different sample points.  
From Figures 7 and 8, we can conclude that the accuracy results of our proposed MNSS were lower than those of other methods. By the proposed method, the sample points were located in the region, which had more mixed pixels and was the change boundary of land covers, and also had a higher probability of classification error. The result showed that our proposed MNSS was stricter than other methods and was more suitable for higher accuracy requirements of remote sensing image applications.