# ExtractEO, a Pipeline for Disaster Extent Mapping in the Context of Emergency Management
## 3 Results  
### Event Delineation Masks  
Both flood and fire detection chains should produce delineation masks that are almost ready to be published. A perfect chain that produces 100% reliable and complete delineation masks that would remove human validation is the final goal. However, the algorithms are  
Figure 5: SAR water extraction workflow.  
Figure 6: Fire Extraction workflow.  
not infallible, and validation by an expert is still mandatory. Moreover, a manual refinement is often necessary to remove wrongly detected features that would not be understood from the user point of view (e.g., 'flooding' located in a high relief area, far from hydrologic features). Indeed, the expected quality of delineation maps from the CEMS users is very high and goes beyond the computed thematic accuracies. A few wrongly classified pixels can discredit an entire rapid-mapping service or activation. The automatic vector in the output is manually refined by an operator expert in a GIS software by overlaying the extracted features, the post-event image used for the detection, the pre-event image, and a DEM (or derived slope). Missing flooded areas are added to the flood mask, and land areas wrongly classified as water are removed.  
The thematic quality of the automatic outputs of both flood/fire chains was assessed through a relative thematic accuracy approach comparing them to the final delineation layers published in CEMS-RM. Indeed, the changes in surface comparing the automatic outputs to the final published vectors was the key performance indicator (KPI) to assess the quality of the algorithms implemented in ExtractEO. The associated processing time was also very important to leave enough time for manual refinement in the production workflow. Beyond the traditional F1 Score measure, the Critical Success Index (CSI) was also computed. Both combined precision and recall, but the CSI was more severe. A CSI above 0.7 is considered as a good basis on which to significantly limit the refinement job [22].  
The results of the quality assessment of the event delineation masks (flood, fire) produced by EEO are presented in Table 3. Among the 13 products validated, only two presented a CSI value below 0.7, but were very close (0.65 and 0.69). Only one had an F1 score below 0.8 (0.79 for the Pleiades processing). Processing VHR images was more challenging, but the scores were not directly correlated to the resolution. The performances of the SAR imagery flood processing chain were satisfactory, as false detection due to topography, permanent low backscatter, or urban areas was limited. The global mean scores were 0.82, 0.90, 0.91 and 0.88 for the CSI, F1 score, precision, and recall, respectively. The classifications' overall accuracies were well balanced with similar precision and recall mean values. The CSI and F1 score mean values were very good and demonstrated that the EEO considerably limited the operator refinement process, which can be long and prone to error through stress and night-time tiredness. Looking deeper into the performance scores, the burnt area algorithm out-performed the flood algorithm, with CSI/F1 score mean values of 0.84/0.91 compared to 0.79/0.88. The robustness, rapidity, and quality of the results characterize the huge added value of the EEO in rapid-mapping production, being fit-to-purpose with respect to demanding CEMS-RM product specifications [23]. Beyond being time-saving (which was difficult to measure, because it depended on the size of the event, the type and resolution of the input data, the thematic complexity, and other factors), the results were harmonized by using the same methodology for whoever was running EEO. Thematic continuity was thus ensured, for example, in two areas of interest processed by different operators.