SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(f)
Figure 3: Scatter plots of reference labels and predictions, (a), (b), (c) are pixel-wise prediction comparisons and (d), (e), (f) are mean (per chip) prediction comparisons of U-TAE, Swin UNETR and U-Net++ ensemble. Scatter plots are prepared on the complete test dataset. The dotted line represents the  $y = x$  axis and the solid line represents the line fitting the scatter plot. The vehicle color labels represent the number of samples for each plot.
results from the three models. Similar to the average RMSE scores, the visualized results from the three models are comparable. However, a few distinguishing features are worth noting (highlighted in Figure 4). For example, in the first sample (see highlighted area in the first row), swim UNETR and U-Net++ produced better outcomes. Upon closer inspection, we observed that the U-TAE and U-Net++ models generated somewhat blurred/smooth results, whereas the segmentation results from
swin UNETR were fine-grained (see highlighted area in rows 2, 3 and 4), resembling the reference label high-frequency details.
(a) Reference Label
(b) U-TAE
(c) Swin UNETR
(d) UNet++
Figure 4: Result samples from the three baseline models. The predictions from the three models are quite similar. Some of the visual differences are highlighted in white boxes.
Over the past decade, several global maps of AGB have been produced such as Baccini 2000, GeoCarbon, GlobBiomass and CCI Biomass. The AGB estimates are usually based on traditional remote sensing modeling approaches [38, 39]. These approaches used for AGB estimation are typically based on the physical interaction between SAR backscatter and canopy density. Although these models are widely excepted and have been extensively used for AGB estimation, they have some limitations: (1) it's more challenging to integrate multi-modal data (e.g. multispectral and SAR fusion) (2) they were developed for lower spatial and temporal resolution SAR images and are not exploiting the more recent satellite constellation capabilities. Furthermore, notable disparities exist in the worldwide biomass products, which limit their usefulness in climate and carbon cycle modeling, as well as in national assessments of forest carbon stocks and fluctuations [5].
(c)
Figure 5: Visual comparison between (a) Google Earth high-resolution base map (b) Reference ALS AGB (c) Best Model Prediction (U-TAE) (d) ESA CCI Biomass 2018
Azara et al. [5] conducted a comprehensive assessment of available biomass products. They reported average RMSE values ranging from  $30\mathrm{t / px}$  for areas with low biomass density  $(< 50~\mathrm{t / ha})$  to  $50\mathrm{t / px}$  for regions with higher biomass density (between 50 and  $250~\mathrm{t / ha}$ ). Compared to average RMSE values obtained in their study, presented DL models gain a significant improvement in both low and high-density areas, with average RMSE values of  $15\mathrm{t / px}$  and  $37\mathrm{t / px}$ , respectively. Moreover, they reported that all global maps tend to overestimate lower biomass and underestimate higher biomass. This phenomenon is a common issue, also observed in the globally available biomass products (such as ESA CCI Biomass, and GeoCarbon). This is due to saturation issues of MSI reflectance and SAR backscatter in densely forested areas. Our reported results also show a decrease in AGB estimation accuracy with an increase in forest density (see Table 2). However, the use of multi-modal data helped in reducing the effect and showing a more balanced behavior.
Another fundamental aspect to consider when comparing the performance of the top models with respect to available biomass products is spatial resolution (pixel size in m). The existing products are generated at a coarse spatial resolution (e.g., 100m for ESA CCI Biomass), which is ten times lower than the resolution achieved using our proposed BioMassters dataset. Figure 5 presents a visual comparison between the reference LiDAR AGB (subfigure 5.b), the prediction of the best model (subfigure 5.c) and the corresponding ESA CCI Biomass map (subfigure 5.d). The ESA CCI Biomass product is generated using a combination of Sentinel-1 and ALOS satellite imagery. From the visual comparison, it is clear that improvement in terms of spatial resolution is achievable using cutting-edge DL models.
However, the ESA CCI biomass is globally available and more studies should be conducted in the future to test the scalability of the presented models. In particular, the dataset is now limited only to boreal forests and it is not easy to transfer the model to other forest types, such as tropical forests. In the future, it could be possible: (1) to extend the dataset to other regions where it's possible to obtain reliable airborne LiDAR data as reference; (2) to use the dataset for pre-trained models and fine-tuned them to other regions or tasks We believe that a fundamental aspect to improve biomass maps at the global scale is the release of more freely and open-access deep-learning-ready datasets such as BioMassters.
In this study, we presented a benchmark dataset 'BioMasster' for forest AGB estimation. The idea is to investigate the potential of dense time series multi-modal (Sentinel-1 SAR and Sentinel-2 MSI) satellite data. The dataset consists of 310,000 patches from a 5-year timespan that covers 8 million hectares of forested areas in Finland. The AGB references are derived from airborne LiDAR measurements conducted by the Finnish Forest Center and the National Land Survey (NLS) of Finland. The dataset has experimented with various DL models by participants from the "BioMasster" data challenge hosted by Driven Data. The top three baseline models achieved an RMSE of  $\approx 27$  tonnes per  $2,560 \times 2,560$  square meters area on an unseen holdout set. The results highlighted a decrease in accuracy with the increase in forest density. The accuracy drop is mainly caused by a rise in the dispersion of the predicted biomass values; the systematic error component remains low showing a good balance (no over or underestimation) of the predictions for higher-density areas. We conducted
a visual comparison Figure 5 between predicted biomass maps and existing global AGB products to illustrate the potential of DL methods to enhance spatial resolution. However, more studies are needed to verify the applicability of the investigated models in different forest types (e.g. tropical regions) and the scalability from national to global scales. In summary, we have shown that the combination of multi-modal time series satellite data, specifically Sentinel-1 and Sentinel-2, with DL algorithms can provide a highly accurate estimation of AGB over vast areas, while also achieving impressive spatial resolution. Our findings suggest that this approach holds significant potential and the release of more freely accessible deep-learning-ready datasets and models would accelerate global-scale biomass mapping.
We would like to thank MathWorks for their generous contribution of prizes for the BioMasster data competitions (more information available at https://www.drivendra.org/competitions/99/biomass-estimation/page/534/) and express their gratitude to all the participants. This research work is also part of the EO-AI4GlobalChange project funded by Digital Futures.