# Multi-Class Strategies for Joint Building Footprint and Road Detection in Remote Sensing
## 4 Experimental Study
### Experiment 2: Improving the Binary Performance Using a Multi-Task Scheme  
The previous experiment has shown the benefits of decomposing a multi-class semantic segmentation problem to reduce its complexity. Moreover, the OVA decomposition strategy has led to better results than its OVO counterpart. In this experiment, we will focus on further improving the results of the OVA model. In this regard, a combination between the OVA model and a multi-task learning scheme will be analyzed. Specifically, it will be studied whether learning not only the main segmentation task, but also a complementary one such as predicting the percentage of annotated pixels can further improve the results. The results obtained in this experiment are presented in Table 3.  
Figure 7: Qualitative results for the benefits of using decomposition strategies. Visual comparison of the results obtained with the direct multi-class approach vs. OVA and OVO binary decomposition strategies for a zone taken from Pamplona city in the test set. TP are presented in green, FP in blue, FN in red and TN in white. Whereas the OVA model performs better in the background and road classes (0.8670 vs. 0.8631, and 0.5828 vs. 0.5698, in terms of averaged IoU, and 0.9287 vs. 0.9264, and 0.7347 vs. 0.7252, in terms of averaged F-score, for both classes, respectively), its combination with a multi-task learning scheme leads to better results in the building class (0.5954 vs. 0.5789, and 0.7450 vs. 0.7325, in terms of averaged IoU and F-score, respectively). Figures 9 and 10 set out a visual comparison between the OVA binary decomposition strategy and its combination with a multi-task learning scheme. Closely looking to these figures, one draws the similar conclusions to the ones extracted from the previous table with extra information. Although both approaches perform similarly overall, the multi-task variant reduces the number of false positives in the background and road classes. However, the number of false positives is increased in the building class. Figure 8: Qualitative results for the benefits of using decomposition strategies. Visual comparison of the results obtained with the direct multi-class approach vs. OVA and OVO binary decomposition strategies for a zone taken from Barcelona North city in the test set. TP are presented in green, FP in blue, FN in red and TN in white. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**IoU** & & & & & \\ \cline{2-6}
**City** & \multicolumn{2}{c}{**OVA**} & \multicolumn{2}{c}{**Multi-Task OVA**} \\ \cline{2-6}  & **Background** & **Building** & **Road** & **Background** & **Building** & **Road** \\ \hline Bilbao & 0.8381 & **0.6367** & 0.5118 & **0.8465** & 0.6319 & **0.5290** \\ Granada & **0.8841** & 0.5660 & **0.5554** & 0.8760 & **0.6554** & 0.5450 \\ Leon & 0.8756 & 0.7020 & 0.5826 & **0.8880** & **0.7070** & **0.5849** \\ Lugo & **0.8501** & 0.5932 & 0.4798 & 0.8428 & **0.6213** & **0.4865** \\ Madrid S. & **0.8676** & **0.5705** & **0.6392** & 0.8452 & 0.5548 & 0.5989 \\ Oviedo & 0.8604 & 0.5430 & 0.4894 & **0.8781** & **0.5732** & **0.5371** \\ \hline Average & **0.8670** & 0.5789 & **0.5828** & 0.8631 & **0.5954** & 0.5698 \\ \hline
**F-Score** & & & & & \\ \hline \hline
**City** & \multicolumn{2}{c}{**OVA**} & \multicolumn{2}{c}{**Multi-Task OVA**} \\ \cline{2-6}  & **Background** & **Building** & **Road** & **Background** & **Building** & **Road** \\ \hline Bilbao & 0.9119 & **0.7771** & 0.6764 & **0.9169** & 0.7736 & **0.6917** \\ Granada & **0.9385** & 0.7228 & **0.7141** & 0.9339 & **0.7918** & 0.7054 \\ Leon & 0.9337 & 0.8248 & 0.7362 & **0.9407** & **0.8282** & **0.7381** \\ Lugo & **0.9189** & 0.7444 & 0.6484 & 0.9147 & **0.7659** & **0.6545** \\ Madrid S. & **0.9291** & **0.7265** & **0.7799** & 0.9161 & 0.7136 & 0.7488 \\ Oviedo & 0.9249 & 0.7037 & 0.6569 & **0.9351** & **0.7284** & **0.6985** \\ \hline Average & **0.9287** & 0.7325 & **0.7347** & 0.9264 & **0.7450** & 0.7252 \\ \hline \hline \end{tabular}
\end{table}