Lake ice, as part of the Essential Climate Variable (ECV) lakes, is an important indicator to monitor climate change and global warming. The spatio-temporal extent of lake ice cover, along with the timings of key phenological events such as freeze-up and break-up, provide important cues about the local and global climate. We present a lake ice monitoring system based on the automatic analysis of Sentinel-1 Synthetic Aperture Radar (SAR) data with a deep neural network. In previous studies that used optical satellite imagery for lake ice monitoring, frequent cloud cover was a main limiting factor, which we overcome thanks to the ability of microwave sensors to penetrate clouds and observe the lakes regardless of the weather and illumination conditions. We cast ice detection as a two class (frozen, non-frozen) semantic segmentation problem and solve it using a state-of-the-art deep convolutional network (CNN). We report results on two winters (2016-17 and 2017-18) and three alpine lakes in Switzerland. The proposed model reaches mean Intersection-over-Union (mIoU) scores >90% on average, and >84% even for the most difficult lake. Additionally, we perform cross-validation tests and show that our algorithm generalises well across unseen lakes and winters. Climate change is one of the main challenges humanity is facing today, calling for new methods to quantify and monitor the rapid change in global and local climatic conditions. Various lake observables are related to those conditions and provide an opportunity for long-term monitoring, among them the duration and extent of lake ice. Remote sensing of lake ice also fits well with the Climate Change Initiative (CCI+, 2017) by the European Space Agency (ESA), where lakes and lake ice were newly included. Additionally, CCI+ promotes long-term trend studies and climate studies, as recognised by the (GCOS). Furthermore, lake ice influences various economic and social activities, such as winter sports and tourism, hydroelectric power, fishing, transportation, and public safety (e.g., winter and spring flooding due to ice jams). In addition, its impact on the regional environment and ecological systems is significant, which further underlines the need for detailed monitoring. Satellites are a secure source for remote sensing of the cryosphere and for sustainable, reliable, and long term trend analysis. Additionally, satellite images are currently the only means to monitor large regions systematically and with short update cycles. This increasing importance of satellite observations has also been recognised by the GCOS. Recently, Tom et al. (2018) proposed a machine learning-based semantic segmentation approach for lake ice detection using low spatial-resolution (250m-1000m) optical satellite data (MODIS and VIIRS). Although the nominal temporal resolution of those sensors is very good (daily coverage), the main drawback of this methodology is frequent data loss due to clouds, which reduces the effective temporal resolution. This is critical, since important phenological variables depend on frequent and reliable observation. In par-ticular, the ice-on date is defined as the first day when the lake surface is (almost) completely frozen and remains frozen on the next day, and ice-off is defined symmetrically as the first day where a significant amount of the surface is liquid water, and remains in that state for another day (Hendricks Franssen and Scherrer, 2008). The GCOS accuracy requirement for these two dates is +/-2 days. Systems based on optical satellite data will fail to determine these key events if they coincide with a cloudy period. Moreover, low spatial resolution of MODIS and VIIRS is also a bottleneck for spatially explicit ice mapping. Higher resolution optical sensors like Landsat-8 or Sentinel-2 do not provide a solution, due to their low temporal resolution and susceptibility to clouds. On the contrary, Sentinel-1 represents a favourable trade-off between spatial and temporal resolution. Additionally, Radar is unaffected by clouds, which in many regions is a considerable advantage. Here we propose to use Sentinel-1 SAR data, which mostly meets the requirements of lake ice monitoring, and additionally comes for free and with a commitment to ensure continuity of the observations. Its spatial and temporal resolution (GSD ca. 10 m / revisit period if 1.5 days) make it possible to derive highresolution ice maps almost on a daily basis. For completeness, we mention that, taking into account estimation uncertainty, the temporal resolution of Sentinel-1 falls just short of the 1-day temporal resolution requirement of lake ice monitoring, it can still provide an excellent "observation backbone" for an operational system that could fill the gaps with optical satellite data (Tom et al., 2018) or webcams (Xiao et al., 2018). Converting a Sentinel-1 image to a lake ice map boils down to 2-class semantic segmentation, i.e., assigning each lake pixel to one of two classes, frozen or non-frozen. We do this with the Deeplab v3+ semantic segmentation network (Chen et al., 2018). Examples of Sentinel-1 SAR composites over the lake St. Moritz is visualised in Fig. 1, showing the VV amplitude in the red channel, and the VH amplitude in the green channel. Target lakes and winters. We analyse three selected lakes in Switzerland (Sils, Silvaplana, St. Moritz, see Table 1) over the period of two winters (2016-17 and 2017-18). These three lakes are located close to each other in the same geographic region, referred to as Region Sils. The lakes are comparatively small and situated in an Alpine environment, and they are known to reliably freeze over completely every year during the winter months. For the two winters 2016-17 and 2017-18, all available images were collected for the nine months between September 1 and May 31. After back-projecting the digitised lake outline polygons from Open Street Map (OSM) on to the SAR images, for each lake, we extract the lake pixels which lie inside the lake outline. In low-spatial resolution satellite images such as MODIS and VIIRS, only few such lake pixels are available (Tom et al., 2018), which makes the analysis of very small lakes such as St. Moritz difficult or even impossible.
Thanks to the higher spatial resolution, the Sentinel-1 time series provides us with millions of lake pixels, which makes it possible to train powerful deep learning models for segmentation, which are extremely data-hungry. Contributions. We address the problem of lake ice detection from Sentinel-1 SAR data, as an alternative to optical satellite data which is impaired by clouds. In the process, we show that a deep learning model pre-trained on an optical RGB dataset can nevertheless be re-used successfully as initialisation for finetuning to Radar data. To our knowledge, our work is the first one that utilises Radar data and deep learning for lake ice detection, though it has been used for sea ice analysis. Many studies discussed the trends in lake ice formation in different parts of the globe. Duguay et al. (2006) presented the trends in lake freeze-up and break-up across Canada for a long period from 1951 until 2000. Later, Hendricks Franssen and Scherrer (2008) reported the decreasing tendency in lake freezing in several Swiss lakes. Then, Brown and Duguay (2010) reviewed the response and role of ice cover in lake-climate interactions. This paper observed that the ability to accurately monitor lake ice will be an important step in the improvement of global circulation models, regional and global climate models and numerical weather forecasting. Brown and Duguay (2011) used the (CLIMo) to simulate lake ice phenology across the North American Arctic from 1961-2100, using two climate scenarios produced by the (CRCM). They projected changes to the ice cover using 30-year mean data between 1961-1990 and 2041-2070, which suggested a probable drift in freeze-up (0-15 days later) and break-up (10-25 days earlier). Duguay et al. (2015) presented an overview of the progress of remote sensing for lake and river ice. For lakes, that work reviewed a number of topics, including ice cover concentration, ice extent and phenology, and ice types, as well as ice thickness, snow on ice, snow/ice surface temperature, and grounded and floating ice cover on shallow Arctic and sub-Arctic lakes. Lake ice monitoring using Radar data. Duguay and Lafleur (2003) proposed to determine the depth and thickness of ice in shallow lakes and ponds using the Landsat Thematic Mapper and European Remote Sensing (ERS)-1 SAR data. Almost a decade later, Surdu et al. (2014) conducted a study of the shallow lakes on the north slope of Alaska to find the response of ice cover on the climate conditions using Radar remote sensing and numerical analysis. A machine learning-based automated ice-vs-water classification was proposed by Leigh et al. (2014) using dual polarisation SAR imagery. Later, Surdu et al. (2015) performed a study on the ice freezing and thawing detection in shallow lakes from Northern Alaska with spaceborne SAR imagery. Antonova et al. (2016) monitored ice phenology in lakes of the Lena river delta using TerraSAR-X backscatter. Du et al. (2017) performed an assessment of lake ice phenology in the Northern Hemisphere from 2002 to 2015. Pointner et al. (2018) studied the effect of the lake size on the accuracy of a thresholdbased classification of ground-fast and floating lake ice from Sentinel-1 SAR data. Duguay and Wang (2019) presented various algorithms such as thresholding, Iterative Region Growing with Semantics (IRGS) and k-means for the generation of a floating lake ice product from Sentinel-1 SAR data for various permafrost regions (Alaska, Canada and Russia). Geldsetzer et al. (2010) used RADARSAT-2 SAR data to monitor ice cover in lakes during the spring melt period in the Yukon area of the Canadian Arctic. They put forward a threshold-based classification methodology and observed that the HH and HV backscatter from the lake ice have significant temporal variability and interlake diversity. Murfitt et al. (2018) used the RADARSAT-2 imagery to develop a threshold-based method to determine lake phenology events for the mid-latitude lakes in Central Ontario from 2008 to 2017. Wang et al. (2018) also used RADARSAT-2 imagery (dual polarised) for developing a lake ice classification system acquired over lake Erie, with the IRGS method. Additionally, Gunn et al. (2018) used the polarimetric RADARSAT-2 (C-Band) to observe the scattering mechanisms of bubbled freshwater lake ice. SAR data analysis is challenging, and deep learning could play a significant role because of its ability to Lake ice monitoring with webcams. Xiao et al. ( 2018) described a system that detects lake ice in webcam data with the help of a deep neural network. Public webcams have two main advantages compared to optical satellite images. Firstly, they are usually not affected by clouds, except for the comparatively rare case of dense fog. Secondly, they have a very high temporal resolution (up to one image per 10 min). Although the approach generated excellent results, it also has disadvantages. Webcams are usually placed arbitrarily (e.g., too far away or covering a small lake area), and often only low above the lake, leading to great scale differences between front and back of the lake surface. Moreover, they are prone to hardware failure, and, being very cheap cameras, they have poor spectral and radiometric quality with significant compression artifacts. Another practical problem with webcams is that it is difficult to operationalise them at country-or even world-scale. Optical data for lake ice monitoring. Tom et al. ( 2018) proposed a machine learning-based methodology for lake ice detection using low resolution optical satellite images. The main problem with optical satellite images is the data loss due to clouds. However, the authors showed that the algorithm produces consistent results when tested on data from multiple winters. In addition, Barbieux et al. (2018) used Landsat-8 multispectral data for extraction of frozen lakes and water-vs-ice classification. Recently, Tom et al. (2019) put forward a feasibility study, which targeted for a unified lake ice monitoring system that combines images from optical satellites, in-situ temperature data and webcam images. Sentinel-1 SAR consists of two identical satellites (S1A and S1B) operational in space with In addition, Sentinel-1 covers the Region Sils in four orbits. See Table 3 for the details.
Footprints of the four orbits are shown in The S1A and S1B satellites have the same SAR system on board which sends out frequency-modulated electromagnetic waves it depends on surface parameters which can be either geometrical factors such as roughness, landscape topography, etc. or physical factors such as the permittivity of the surface material. Significant factors for lakes are also wind speed and direction, and the water content in snow. For smooth and plain water, almost all radiation is scattered away from the sensor making it appear very dark. As the wind speed picks up, waves occur on the water surface and significant scattering can occur. When perfectly plain water is covered by perfectly plain ice, microwaves penetrate the ice without absorption and are reflected at the ice-water interface away from the sensor, and the ice covered lake would in theory appear completely black. In reality, cracks in the ice scatter some microwaves back to the sensor. Therefore, visible and well located cracks are clear indicators for ice cover. Furthermore, in reality, the ice-water interface is never completely smooth, therefore some scattering can occur at these boundaries which, however, can be weak. The older the ice, the more air bubbles are enclosed in it, which increase the backscatter within the ice volume by direct backscattering and also by double reflection of microwaves at the air-bubbles and the ice-water interface. With snow cover, the air-ice interface becomes increasingly rough which further increases the backscatter signal. Finally, with snow melt, the liquid water content of the overlying snow pack increases and significantly reduces the backscatter signal, as the snow-water mixture of wet snow absorbs a significant fraction of the microwave energy. Data collection and pre-processing. GEE is a cloud-based platform for large-scale geo-spatial data analysis (Gorelick et al., 2017). It stores and provides data of various satellite missions, performs data pre-processing and makes them freely available for education and research purposes. The Sentinel-1 backscatter coefficients (in decibels) were downloaded from the GEE platform after several inbuilt pre-processing steps such as GRD border noise removal which corrects the noise at the border of the images, thermal noise removal for correcting the thermal noise between the sub-swaths, radiometric calibration which calculates the backscatter intensity using the GRD metadata, terrain correction to correct the side looking effects using the digital elevation model (SRTM, 30m), and log-scaling to transform the approximate distribution of the SAR responses from Chi-squared to Gaussian . Note that we did not perform any absolute geolocation correction, since the backprojected lake outlines suggested a sufficient accuracy. Transition and non-transition days. All the data from two winters was divided into two categories: non-transition dates where the lake is almost fully frozen or fully non-frozen, and transition dates with partially frozen lake surface. Both freezeup and break-up dates belong to the transition category. The dataset statistics are shown in Ground truth. For each lake, one label (fully frozen / nonfrozen, partially frozen / non-frozen) per day was assigned by a human operator after visual interpretation of the visible part of the lake from freely available webcam data. The ground truth thus generated was further enriched by visual interpretation of the Sentinel-2 images whenever available. However, some remaining noise in the ground truth is likely due to interpretation errors, as a result of overly oblique viewing angles of webcams and compression artefacts in the images. During the transition days, ground truth estimation was very difficult since we had partially frozen and non-frozen states and there was a difficulty to discriminate transparent ice and water. Thus, transition days were not used for quantitative analysis. and fine-tune it on the relatively small Sentinel-1 SAR dataset . Surprisingly, we find that pre-training on RGB amateur images of indoor scenes, vehicles, animals, humans etc. greatly enhances the performance even on a data source as different as interferometric Radar, compared to training from scratch only on the SAR data. Note, all weights were finetuned, we did not freeze any network layers. We use various measures to quantify performance, including recall, precision, overall accuracy, and the IoU score (Jaccard index). In all experiments described in the paper, we used only lake pixels from the non-transition dates to train the network and to compute performance metrics. This is due to the lack of reliable ground truth during the transition dates. Additionally, whenever the ground truth cannot be established for a nontransition date due to foggy webcam images and/or clouds in Sentinel-2, it is exempted from the training set. However, qualitative analysis is done on all the dates. Quantitative results: Semantic segmentation. For developing an ideal operational system for lake ice monitoring, the data from a couple of lakes from a few winters would have to be used to train the model, which can then be deployed on unseen lakes and winters. However, generating ground truth for each lake is a tedious task. Nevertheless, we make sure that the data from at least one lake from one full winter is in the training set for the classifier to learn the proper class decision boundaries. We employ Cross-Validation (CV), i.e., the data is partitioned into k folds, usually of approximately the same size. Then, the evaluation is done k times, each time using one fold as test set and the union of all remaining folds as training set. Leave-oneout cross-validation is the setting where the number of folds equals the number of instances (in our case the number of winters/lakes) in the dataset. The goal of leave-one-winter-out CV is to investigate the generalisation capability of a model trained on one winter when tested on a different winter. The results are shown on The results are shown on Quantitative results: Time-series. The ice-on and ice-off dates are of particular interest for climate monitoring. From the per- day semantic segmentation results, we estimate the daily percentage of frozen surface for each observed lake. Thus, for each available SAR image, we compute the percentage of frozen pixels, throughout the entire winter.
An example time series for lake Silvaplana in winter 2017-18 is shown in Fig. 7a. Although we do not have per-pixel ground truth on the partially frozen transition days, we know whether the lake has more water (shown with a value of 75% in the ground truth) or more ice/snow (shown with a value of 25% in the ground truth). Even though some miss-classifications exist during the transition days, the non-transition days are almost always predicted correctly, likely because the network was trained solely on non-transition days. For a comparison, we also plot the time series of temperature values (sliding window mean of the daily median, window size 7 days) obtained from the nearest meteo station in Fig. 7b. Sub-zero values in this graph correlate (with some time lag) with the period in which the lakes are fully or partially frozen. Qualitative analysis. Exemplary qualitative results are depicted in Miscellaneous experiments. In all the experiments reported so far, we used the data from all four orbits (both ascending and descending) and both polarisations (VV and VH). To study the individual effect of polarisations VV and VH, we drop either of them and report the corresponding results on Table 7 (left). Note that mIoU drops by almost 25% when VV is left out, while it drops by only 3.7% without VH, confirming the significance of polarisation VV for lake ice detection. This finding also aligns with the visual differences in In another experiment, we drop VH but use VV from two temporally adjacent acquisitions (VV and VV_prev), thus simultaneously feeding the network with data from two different days. The mIoU rises by 2%, see Table 7 (left). However, we noted some stability issues especially during fully frozen days. We believe this is due to the fact we removed VH. We did another experiment to check the effect of acquisition time. Here, we used the data from both VV and VH. However, we drop the data from some orbits . Table 7 (right) shows that the mIoU drops by 7.1% and 5.7% respectively when the data from only descending orbits (66 and 168) or only ascending orbits (15 and 117) were used. A final experiment was done to assess the influence of the (rectangular) training patch size, see We have described a system for reliable monitoring of lake ice based on Sentinel-1 SAR imagery, with the potential to retrieve long, consistent time series over many years (assuming continuity of the satellite mission). The proposed method has been demonstrated for three different Swiss lakes over two complete winters, and obtains good results (mIoU 90% on average, and >84% even for the most difficult lake), even when generalising to an unseen winter or lake. Given the main advantage of SAR data for our purposes -its ability to observe with very good spatial and temporal resolution independent of clouds -we see the possibility to extend our method into an operational monitoring system. A logical next step would be to process longer time series, which unfortunately is not yet possible with Sentinel-1. It is quite possible that even a moderate time span, say 20 years, would suffice to reveal trends in lake freezing patterns and perhaps also correlations with climate change. Another future direction is an integrated monitoring concept, using SAR together with optical satellite imagery and optionally images from webcams, to ensure reliable identification of ice-on and ice-off dates within the GCOS specification of +/-2 days. For each lake we show the Sentinel-1 composite image (column 1), the ground truth (column 2), the predicted frozen probability map derived from the network logits (column 3), and the corresponding binary classification map (column 4). Additionally, column 5 shows a corresponding Sentinel-2 image for better visual interpretation. ACKNOWLEDGEMENTS. This work is part of the project Integrated lake ice monitoring and generation of sustainable, reliable, long time series funded by Swiss Federal Office of Meteorology and Climatology MeteoSwiss in the framework of GCOS Switzerland. Also, we thank Anton B. Ivanov from Skoltech for his support. ACKNOWLEDGEMENTS. This work is part of the project Integrated lake ice monitoring and generation of sustainable, reliable, long time series funded by Swiss Federal Office of Meteorology and Climatology MeteoSwiss in the framework of GCOS Switzerland. Also, we thank Anton B. Ivanov from Skoltech for his support.