# Assessment of Interventions in Fuel Management Zones Using Remote Sensing
## 3 Results
### Static Analysis  
The first analysis consists of comparing different data sets using only Sentinel-2 data from just one date, 12 September, this is the date closest to our ground truth. Figure 9 shows the F1-score values for the used algorithms and the different data sets, note that the data sets are sorted by performance and by model. The RF and XGBoost were the algorithms that achieved better results, with RF reaching slightly better values.  
The data set \(DS\_ALL\) consistently delivered good results for all the algorithms and the best result was achieved using this data set with the RF algorithm. The data set choice, in this case, can have a great impact on the results, in some cases, it has an impact in the F1-score superior to 4%. The lowest result was obtained using the data set \(DS\_INDICES\) with SVM, resulting in an F1-score of 0.83.  
Analyzing in more detail the combination that generated the best results (RF using \(DS\_ALL\) data set) using a confusion matrix (Table 3) the class of \"not treated\" sections was the one with a higher  
Figure 9: F1-score values of the different algorithms and data sets, for the Fuel Management Zones (FMZs) along roads.  
F1-score of 0.98 and with a precision and recall of 0.98 and 0.99, respectively. The class of \"treated\" sections had a lower F1-score value of 0.82.  
Although the best model performs quite well, the previous results are only shown to one date, 12 September, given there is only ground truth for that particular date. Since the static analysis only uses reference satellite images from one date to train the model, the other left out products can be used to check if the behavior of the algorithm is as expected. This model (RF) was trained with a reference image from 12 September and the ground truth until the same date. The model was then applied for each date that had Sentinel-2 data. This shows how the model behaves with data before (start of the year), during (middle of the year), and after the interventions (end of the year). This analysis is especially interesting because the ground truth shows interventions in FMZs up to 12 September, meaning interventions can appear in images at distinct points in time before that date. In Figure 10 the number of sections classified as treated for each date is presented.  
There are no precise metrics for the results of the dates before and during the observed intervention period because the ground truth does not have the date of intervention for each section. As the classification advances throughout the year, the number of detected interventions starts to climb as soon as the first intervention is made. As more FMZs are cleaned throughout the year the model can detect those interventions. After the ground truth date, the number of detected intervention zones starts declining, mainly due to vegetation regrowth and shadows or dark area pixels in the images at the end of the year.  
Figure 10: Number of sections classified as treated using the best model (RF with DS_ALL) along the whole year of 2018. Trained with data of 12 of September.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Predicted \(\backslash\)Reference** & **Treated** & **Not Treated** & **Total** & **Precision** & **Recall** & **F1-Score** \\ \hline
**Treated** & 90 & 17 & 107 & 0.84 & 0.80 & 0.82 \\
**Not Treated** & 23 & 1257 & 1287 & 0.98 & 0.99 & 0.98 \\
**Total** & 112 & 1275 & & & & \\ \hline
**Global F1-Score (macro)** & 0.90 & & & & & \\
**Global Kappa** & 0.80 & & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Confusion Matrix for the algorithm RF using the \(DS\_ALL\) data set.