## 4 Results
### Prediction Results  
Using all of the training datasets described in the previous subsections, we will construct a nearest neighbor, support vector machine, decision tree and random forest classifier for each dataset. We will use each classifier to label all of the remaining data (_i.e._, the test data) and compare the results to ground truth change/no-change labels applied by an experienced imagery analyst. Based on these classification results, we catalog the following:  
* true positive results: relevant change occurred, and it was classified as such;
* false positive results: no change occurred or a change that was not relevant occurred, but was classified by the algorithm as change;
* false negative results: a relevant change occurred, but was not correctly classified;
* true negative results: no change occurred or a change occurred that was not relevant, and the classifier correctly indicated this condition. Based on these four factors, we can calculate traditional assessment metrics of precision and recall as follows:  
\[Precision=\frac{TP}{TP+FP} \tag{4}\]  
\[Recall=\frac{TP}{TP+FN} \tag{5}\]  
An accuracy metric can also be calculated to measure the overall performance of each algorithm, as shown in Equation (6). \[Accuracy=\frac{TP+TN}{TP+FP+FN+TN} \tag{6}\]  
We present four tables that illustrate the precision, recall and accuracy values for each of the types of classifiers described in Section 3. Table 8 provides results for the nearest neighbor classifier, Table 9 for support vector machine, Table 10 for the decision tree and Table 11 for the random forest classifier. Each row in the table corresponds to one of the training datasets described in Tables 3-5, 6 and 7. \begin{table}
\begin{tabular}{c c c c c c c c} \hline  & & \multicolumn{3}{c}{**No. of Cluster Tiles for Training**} & \multicolumn{3}{c}{**Total \% of Tiles**} \\ \cline{3-8}
**Dataset** & **Nos. from Each Cluster** & **Columbia** & **Las Vegas** & **Natanz** & **Columbia** & **Las Vegas** & **Natanz** \\ \hline C1 & 1 & 100 & 90 & 90 & 3.96\% & 17.24\% & 2.46\% \\ C2 & 2 & 292 & 256 & 268 & 11.55\% & 49.04\% & 7.32\% \\ C3 & 3 & 477 & 368 & 441 & 18.87\% & 70.50\% & 12.05\% \\ \hline \end{tabular}
\end{table} 
Table 5: In the Series C datasets, we utilize the most prototypical cluster members from each pair for training data. Each row in this table shows the number of training tiles used as the number of cluster members is increased. Table 8 shows the results of change prediction using a nearest neighbor classifier. Recall rates are highest when only using the high change tiles as training data (Datasets A1-A3); this holds true for all three test sites. However, when using this training data, overall accuracy clearly suffers. Generally, the highest combinations of precision, recall and accuracy values come from the training datasets that \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **\% of High Tiles** & **\% of Low Tiles** & **Nos. from Each Cluster** \\ \hline D1 & 5\% & 0\% & 1 \\ D2 & 5\% & 0\% & 2 \\ D3 & 5\% & 0\% & 3 \\ E1 & 10\% & 0\% & 1 \\ E2 & 10\% & 0\% & 2 \\ E3 & 10\% & 0\% & 3 \\ F1 & 5\% & 5\% & 1 \\ F2 & 5\% & 5\% & 2 \\ F3 & 5\% & 5\% & 3 \\ G1 & 15\% & 15\% & 1 \\ G2 & 15\% & 15\% & 2 \\ G3 & 15\% & 15\% & 3 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The D, E, F and G series datasets combine tiles with high and low amounts of change in them with prototypical cluster members to form diverse training datasets.