## 2 Data and Methods
Note: class 2 is shown overlaid on top of part of class 1 and the area of overlap may be inferred. Ground reference data for the purposes of network learning (training) and evaluation were acquired. Following [43; 44], the ground reference dataset was partitioned such that 50% was used for training, 25% for validation, and 25% for testing. The composition of these datasets, however, sometimes varied (Table 4). Conventional guidance on the design of the training set was followed with 110 cases of each class obtained for training purposes; meeting the often stated requirement of a sample of at least 10 times the \begin{table}
\begin{tabular}{c c} \hline
**Dataset** & **Size and Class Composition** \\ \hline Training & 400 cases of each class; total = 1600 cases \\ \hline  & 500 class 1 \\ Validation (random) & 100 class 2 \\  & 100 class 3 \\  & 100 class 4; total = 800 cases \\ \hline Validation (stratified) & 200 cases of each class; total = 800 cases \\ \hline  & 500 class 1 \\ Testing & 100 class 2 \\  & 100 class 3 \\  & 100 class 4; total = 800 cases \\ \hline \end{tabular}
\end{table} 
Table 2: Class composition of the datasets used for analyses of the simulated data. \begin{table}
\begin{tabular}{c c} \hline
**Waveband** & **Wavelength (\(\mu\)m)** \\ \hline
1 & 0.42–0.45 \\
2 & 0.45–0.52 \\
3 & 0.52–0.60 \\
4 & 0.60–0.63 \\
5 & 0.63–0.69 \\
6 & 0.69–0.75 \\
7 & 0.76–0.90 \\
8 & 0.91–1.05 \\
9 & 1.55–1.75 \\
10 & 2.08–2.35 \\
11 & 8.50–13.00 \\ \hline \end{tabular}
\end{table} 
Table 3: The 11 spectral wavebands of the ATM sensor used. Figure 2: Data for Feltwell. (**a**) The location of Feltwell, © OpenStreetMap contributors; (**b**) airborne thematic mapper (ATM) image extract in waveband 4 and (**c**) ATM image extract in waveband 7.  
number of discriminating variables (wavebands) used as input. The training set, therefore, contained 660 cases acquired by a stratified random sampling design. This training set was used throughout. Testing sets should, ideally, be acquired using a probability sampling design [18; 19]. Here, a simple random sample (without replication) was used to acquire 330 cases to use for testing. Note that this sample size exceeds that required for accuracy estimation of a map with an accuracy of 85%, a standard if contentious target accuracy in remote sensing, with an allowable error of 4%. This testing set was used in all analyses of the ATM data. This latter issue impacts on comparisons of accuracy estimates and requires the use of a technique suited for related samples [45]. As with the analyses of the simulated dataset, the search for an optimal neural network was undertaken twice. In each case the training and testing sets were the same, the only difference was the composition of the validation set used to identify the optimal network from a set of candidate networks generated for the task. In one set of analyses, the validation set was generated by simple random sampling and thus the number of cases of each class tended to reflect the actual abundance of the classes in the region to be mapped. Indeed, here, the sample was selected to ensure that the class composition of the validation set equalled that of the testing set. This sample comprised 330 cases. In the other set of analyses, a validation sample of the same size, but acquired following a stratified random sample such that each class was equally represented was used. The nature of the datasets used is summarized in Table 4.  
In the search for an optimal neural network to classify the ATM data, optimality was defined in relation to the maximum overall accuracy of the classification of the validation data. The accuracy of each classification was calculated from the confusion or error matrix generated for it which shows a cross-tabulation predicted and actual class label for each case in the dataset analysed [1; 18].