# Business Intelligence through Machine Learning from Satellite Remote Sensing Data
## 5 Experimentation and Evaluation
### Ground-Truth Comparisons and Neural Network Implementation and Testing  
Even though the One-ClassSVM approach seems promising for tackling the lack of ground-truth data for the city of Volos, we would like to present selected findings from the Chicago dataset. As mentioned, these were retrieved from the city's open-data repository, and as such, we can efectivelly validate our models' performance.  
We performed various tests using different algorithms, including a Convolutional Neural Network and state-of-the-art time series classifiers (e.g., HIVE-COTE 2.0) from the sktime library, keeping their default parameters and calculating their accuracy, precision, and F1 score. Please recall that precision refers to how close measurements of the same  
Figure 8: Experimentation results on all areas (25 \(\times\) 25) using EMBI index (**top three pictures**) and EMBI average radiance (**bottom three**) in One-Class SVM.  
item are to each other and that precision is independent of accuracy. We refer the reader to [[https://arxiv.org/abs/2104.07551](https://arxiv.org/abs/2104.07551)]([https://arxiv.org/abs/2104.07551](https://arxiv.org/abs/2104.07551)) and [[https://inria.hal.science/hal-03558165/document](https://inria.hal.science/hal-03558165/document)]([https://inria.hal.science/hal-03558165/document](https://inria.hal.science/hal-03558165/document)) (accessed on 1 September 2023) for all the details available. Our results are given in Table 2.  
In Figure 9, we present the performance of the two classifiers through their maps resulting from two different experiments for each classifier (the top two for the Convolution Neural Network and the bottom two for the Time Series Forest Classifier. We mention that the general behavior depicted in these maps is indicative of the behavior of almost all the related experiments we performed.  
Specifically, in these maps:  
* Light green or light red dots represent abandoned buildings falsely classified as not-abandoned by our models.
* Dark green or dark red dots represent abandoned buildings correctly classified as abandoned by our models.
* Gray dots represent not-abandoned buildings falsely classified as abandoned by our models.  
Certain issues or observations are worth mentioning. The Chicago dataset does not include polygon shapes for the individual buildings (valued for  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Classifier** & **Accuracy** & **Precision** & **F1** \\ \hline Convolution Neural Network & 0.641148 & 0.563805 & 0.409708 \\ Time Series Forest Classifier & 0.607582 & 0.609809 & 0.594830 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance comparison for the Convolution Neural Network and the Time Series Forest Classifier for the city of Chicago dataset.  
Figure 9: Experimentation with Neural Net (on the two color labeled maps on the **top** of the figure) and with Time Series Forest (on the two color labeled maps on the **bottom** of the figure).  
the x- and y-axis) but only points with their coordinates. This hinders the calculation of accurate time series for each building, since we can calculate only a part of it. A possible solution is to infer the building polygon and perform the computations involved, which is a rather demanding process.  
A problem with the use of more advanced models is that they are resource-intensive and cannot be easily loaded on free cloud-based hosting platforms (e.g., streamlit).