## 2 Materials and Methods
### Experimental Design
#### 2.4.2 Preparation of Training and Reference Datasets and Accuracy Evaluation  
All the crop parcels in the study area were first divided into two exclusive groups including training and reference parcels. For an objective comparison of classification results from the two study regions, the same ratio of training and reference parcels, approximately 1:3, was applied to the two regions. The procedure for selecting training samples shown in Figure 4 was then applied to predetermined training parcels. A predetermined proportion of training patch candidates was selected and used as the final training patches to mimic the case with limited training samples, which is typical in supervised classification. In Anbandegi, many pixels are contained in each crop parcel owing to the ultra-high spatial resolution of the UAV imagery (the total number of crop pixels was 4,531,661). Only 0.1% of the training patch candidates was extracted and used as training patches, based on the suggestion from the previous classification study in Anbandegi [32].  
The area extent of the Illinois region is much larger than that of Anbandegi, but the total number of crop pixels (213,733) was much smaller than it was in Anbandegi (only 4.72% of the total in Anbandegi) owing to the relatively coarser spatial resolution of the Landsat imagery. Consequently, many training patch candidates could not be generated, in contrast to Anbandegi. Using a small training dataset for 2D-CNN might produce poor classification accuracy, and the quantification of variations in classification accuracy with respect to different class purity values might be impossible. By considering the number of pixels in the Illinois region, all the training patch candidates collected for the predetermined patch size and class purity values were used as training patches. More training patch candidates were likely to be extracted for CP100 owing to the primary selection of pixels inside the crop parcels, compared with CP60 and CP80. All the numbers of training samples for different class purity values were initially set to be the same or similar via random sampling.  
Furthermore, the proportions of crop types in the two regions were considered to select the final training samples to mitigate the bias sampling problem caused by the major class in the study area [5]. More specifically, the maximum numbers of training patches were limited according to the proportion of each crop type in the study area [34]. The proportions of highland Kimchi cabbage, cabbage, potato, and fallow in the Abandegi region were 40%, 30%, 20%, and 10%, respectively. Because the proportions of corn, soybeans, and winter wheat were different in the Illinois region (35%, 45%, and 20%, respectively), the total numbers of final training patches differed slightly for all class purity values.  
To quantitatively evaluate the classification performance, several accuracy statistics, including the overall accuracy, producer's accuracy, and user's accuracy, were computed from an error matrix. The overall accuracy is the proportion of correctly classified pixels in the reference dataset, the producer's accuracy measures how well reference pixels of the given land-cover type are classified, and the user's accuracy is a measure of how well the classified pixel actually represents the given land-cover class on the reference data [41]. All the pixels in the reference parcels were used as reference data. The total numbers of reference data for Anbandegi and Illinois regions were 4,522,331 and 164,048,respectively. When using 2D-CNN for classification, the initial weights and a kernel configuration are randomly determined. To reduce the effect of random sampling fluctuations, classification was repeated four times, and the final classification accuracy was obtained by averaging the four classification accuracy statistics.