dataesr/scientific-paragraphs-categorization huggingface dataset:
Retrieval and transformation function 180 CSTools builds on the experience gained from the development of other R packages for climate data analysis, such as s2dverification. Specifically, CSTools has adopted its "s2dv_cube" object as a central data structure to represent and carry data and metadata across function calls. A "s2dv_cube" object is essentially a R list which includes a multi-dimensional array of climate data originating from either observations or forecasts, and metadata, such as time period or region covered, dataset and variable name, and units. The development guidelines define conventions to ensure "s2dv_cube" objects are used 185 in a coherent way throughout the package. CSTools has a single but powerful function to retrieve data from netCDF files called CST_Load. This function is a wrapper of the s2dverification Load function which allows reading monthly or daily data from a set of specified forecast datasets together with specified date-corresponding observations  (Manubens et al.
s2dv_cube function. One of the capabilities of CSTools is to create a new dataset after, for example, the data has been downscaled and/or 210 calibrated. In that case, the user may need to save the new dataset into files to be shared among other users or its community. Therefore, the package comes with a saving function called CST_SaveExp which creates netCDF files in a directory set by the user and which can be loaded again with the CST_Load function. Moreover, the climatological essential steps of computing anomalies can be done with CST_Anomaly which is a wrapper function of s2dverification methods that also allows computing smoothed climatologies.
, 2019; Torralba, 2021). The first function computes a set of weather regimes using a cluster analysis. The dimensionality of 235 this object can also be reduced by using PCs obtained from the application of the EOF analysis to filter the dataset, while the cluster analysis can be performed with the traditional k-means or hierarchical clustering. On the other hand, CST_RegimesAssign matches anomalies to a set of reference maps obtained using CST_WeatherRegimes. The anomalies are assigned to the most similar reference map using either the minimum Euclidean distance or the highest spatial correlation, which can be particularly useful to classify the predictions according to the clusters identified in the observational reference.
Downscaling methods Downscaling is designed to increase the resolution of a dataset. In a climate service chain, downscaling is a fundamental step to transform the climate simulations from their coarse resolution to the finer resolution required by many final users studying 260 regional environmental changes  (Maraun et al. 2010; Rössler et al. , 2019). CSTools contains five different downscaling methodologies based on analog techniques, stochastic simulations or regression. 270 CST_RainFARM implements a stochastic downscaling technique and represents a so-called full-field weather generator. More specifically, this function generates synthetic fine-scale precipitation fields whose statistical properties are consistent with the small-scale statistics of observed precipitation, while preserving the properties of the large-scale precipitation field.
280 CST_RFTemp implements a simple lapse rate correction to a near-surface temperature field to account for changes in orography between a low and high resolution gridded dataset. ADAMONT (ADAptation of RCM outputs to MOuNTain regions;  Verfaillie et al. , 2017 ) is a downscaling method designed 285 to adjust forecasts of daily variables. The method is based on the quantile mapping approach and originally relied on a regional reanalysis of hourly meteorological conditions. Two functions to implement ADAMONT have been included in CSTools. CST_AdamontQQcor computes a quantile mapping based on weather types for forecast data while CST_AdamontAnalog uses these weather types to find analogous data in the reference dataset.
290 The CST_AnalogsPredictors function downscales precipitation or maximum/minimum temperature low resolution forecast output data, in a domain centred over Iberian Peninsula, through the association with an observational high resolution dataset  (Peral García et al. , 2017)  and a collection of predictors and reference synoptic situations similar to the estimated day. As a first step, a partner function AnalogsPredictors_train must be run to compare the large-scale atmospheric circulation to each of the atmospheric configurations from a reference period. The most similar days, defined by the Euclidean distance of 295 winds, are chosen as their analogs.
Visualization Some of the most requested functionalities in climate services are data visualization tools that allow presenting large quantities of information in an intuitive way. All the visualization functions in CSTools can be customized by modifying 355 result in a pop-up window. PlotCombinedMap combines multiple 2-dimensional datasets into a single map based on a decision function. In other words, several "maps" are provided as input, and for each "map" the function creates a colour legend. A decision function is used at 360 each gridpoint to choose the value to be displayed, in the process retaining the information of which "map" it belongs to. For instance, multiple model skills could be compared in a region to visualize which is the best model in each region  (Mishra et al. , 2019; Torralba et al.
410 The use case presented here shows whether the 2018 event had been anticipated by ECMWF System 5 a few months in advance. We note that the code could be adapted to other regions, time periods and variables and a detailed description of the code is provided below for users interested in modifying the necessary parameters. First, the seasonal forecasts initialized in December 2017, January 2018 and February 2018 are bias adjusted and assessed. Two functions from CSTools are used to 415 post-process the wind speed seasonal forecasts: CST_Load and CST_BiasCorrection. The key decisions are the parameters used to retrieve the data from files to achieve a coherent analysis of the March 2018 event (Fig. 2 ). The analysis is repeated for three different start dates (i. e. December, January and February).
Therefore, the var parameter is set to "sfcWind" when retrieving hindcasts and forecasts, while for the reference dataset it is set to "windagl100". Given the difference in spatial resolution, a regridding of the reference dataset is also requested by the parameter grid. The path pointing to the simulations and the reference are also passed to the CST_Load function through parameters exp and obs, 440 respectively. Notice that the labels $STORE_FREQ$, $VAR_NAME$, $START_DATE$, $YEAR$ and $MONTH$ are used when defining the paths. These labels will be interpreted and substituted by the function following the information provided by the other parameters of CST_Load. An index mm indicating the number of preceding months (mm) is introduced to loop over the three start dates in order to 445 simplify the code.
To support this need, a modelling chain driven by seasonal forecasts of meteorological variables from the C3S seasonal forecasting systems was developed, employing the physical 1-dimensional snow model 550 SNOWPACK  (Bartelt and Lehning, 2002) , to estimate snow depth and snow water equivalent at selected high-elevation sites in the North-Western Italian Alps. A general scheme of this application is shown in Fig. 5. Important decisions include the downscaling method, the target region, the simulation and observation datasets as well as the season of interest. In this case, the region is the Alpine mountain range in central Europe  (42-49 °N, 4-11 °E) , including the high-elevation stations for which the SNOWPACK model is run.
Moreover, this domain has to be larger than the target study area to avoid artifacts/border effects within 565 the target area. The target season is winter, so the 1st November start date simulations available for the period 1993-2018 are considered. Daily precipitation data of SEAS5 are downscaled from the original 1° spatial resolution. The reference datasets employed are i) ERA5 daily precipitation reanalysis at 0. 25° (around 30 km) spatial resolution  (Hersbach et al. , 2020)  for the bias correction and for the estimation of the spectral slopes and ii) the WorldClim2 monthly climatology at 1 km spatial resolution  (Fick and Hijmans, 2017)  for generating the precipitation weights to obtain a more realistic distribution. In the scheme (Fig. 5 ), three steps should be carried out before applying the RainFARM method.
In steps 1 and 3, necessary 575 parameters are computed: the spectral slopes and the orographic weights; quantile mapping correction is applied to the seasonal forecast in order to correct the bias of the model in step 2, and the downscaling is computed in step 4. All computations performed in the first step only require the CSTools package. As mentioned above, the spectral slope is calculated using ERA5 at its original resolution and over a larger domain than the target region (37. 5-53. 25 °N, 2. 5-18. 25 580 °E). The path pattern to the data is defined using labels: $STORE_FREQ$, $VAR_NAME$, $YEAR$ and $MONTH$. These labels will be interpreted by CST_Load. For instance, the $VAR_NAME$ will be substituted by the information passed by the parameter var which in this case is "prlr" that stands for precipitation rate and the $YEAR$ and $MONTH$ will be interpreted from the CST_Load sdates parameter which requires a vector of dates in the format "YYYYMM01" where YYYY is the year and MM the month.
Then, CST_Load retrieves the data from files and arranges it with the 585 following dimensions: dataset of length 1 since only ERA5 is being requested, member = 1 since this reanalysis only provides one simulation, sdate dimension is of length 312 which corresponds to the 26 years of 12 months defined in object "years" with an ftime dimension up to 31 corresponding to each day of the month. The remaining dimensions, lat and lon correspond to the squared domain requested in CST_Load. Given that CST_Load splits the time series among sdates and ftime dimension when specifying a forecast dataset, our ERA5 path pattern has been requested through this option. On the 590 other hand, specifying the ERA5 path pattern as an observational dataset (in the obs parameter), the function will return a continuous time series from 1993 to 2018 which is less convenient for our purposes here.
610 In the second step, we load the data, taking advantage of library zeallot  (Teetor, 2018)  that allows us to simplify our code by using an advanced version of the assignment operator (%<-%). Again, the paths to the necessary data must be defined using labels: for the forecast data the path points to the SEAS5 dataset while for the reference data the path points to the ERA5 reanalysis. Thanks to CST_Load, these datasets could be reshaped onto a common grid, which, by default, is the grid of the 615 first dataset provided, i. e. the SEAS5 grid. The vector "StartDates" which defines the period of study for November 1st simulations, is then assigned to the sdates parameter. In order to apply the quantile-mapping correction month by month, the function CST_SplitDim is used to divide the forecast time dimension in two: one for identifying the days of the month and another to store each month separately.
The result of 620 step 2 is a bias-corrected forecast consistent with the reference dataset: the forecast probability density function matches the one for the references, resulting in the same climatology. A simple visual evaluation of the impact of the quantile mapping correction is shown in Fig. 6. 625 library(zeallot)  StartDates <-paste0(1993 StartDates <-paste0( :2018 StartDates <-paste0( , "1101") ")  exp <-list(name = "ecmwfS5", path = "/esarchive/exp/ecmwf/system5c3s/$STORE_FREQ$_mean/$VAR_NAME$_s0-24h/$VAR_NAME$_$START_DATE$. nc") obs <-list(name = "era5", path = 630 "esarchive/recon/ecmwf/era5/$STORE_FREQ$_mean/$VAR_NAME$_f1h-r1440x721cds/$VAR_NAME$_$YEAR$$MONTH$. nc") c(exp, obs) %<-% CST_Load(var = "prlr", exp = list(exp), obs = list(obs), sdates = StartDates, nmember = 25, storefreq = "daily", sampleperiod = 1, latmin = 42, latmax = 49, lonmin = 4, lonmax = 11, 635 output = "lonlat", nprocs = 1) exp <-CST_SplitDim(exp, split_dim = "ftime")) obs <-CST_SplitDim(obs, split_dim = "ftime")) exp.
Code Step 2. Step 3 computes the orographic weights from a fine-scale precipitation climatology. In this case, the WorldClim2 dataset 645 precipitation at 30 seconds resolution is used although other climatologies at high resolution could be used. The WorldClim2 dataset is formatted in tiff files that can be automatically downloaded in the R session thanks to the raster library  (Hijmans, 2020). The piece of code for Step 3 shows how to compute the orographic weights for all individual months at once: getting the data from the remote dataset, subsetting for the Alps region with a small increment to correctly compute interpolation  (3. 5-11. 5 °E, 41. 5-49. 5 °N) , and storing the data in an "s2dv_cube" object to be passed to CST_RFWeights.
720 After the spatial downscaling, seasonal forecast data are interpolated to one hour temporal resolution with different methods depending on the variable  (Terzago et al. , in preparation). Fig. 8  provides the step-by-step structure of the methodology used while Fig. 9  provides a visual representation of the post-processing chain. As shown in Fig. 8 , the overall methodology can be separated into a bias adjustment phase 770 (steps 1-2) and a downscaling phase (steps 3-5) that uses an analog approach. The bias adjustment phase starts by loading (using CST_Load) the daily forecast and observational rainfall data over Greece at 1° resolution. As forecast data, we take the 25 members of SEAS5 from 1993-2019 initialized in May while the reference is the CHIRPS dataset upscaled to 1° resolution for the same time period.
780 The downscaling phase starts in step 3 (see Fig. 8 ) by loading the sea-level pressure (SLP) of the forecast and reference at 1° resolution over Europe. The reference dataset in this case is ERA5  (Hersbach et al. , 2020)  and upscaling is applied to obtain the 1° resolution. In step 4, both the bias-adjusted precipitation fields over Greece and the SLP anomaly fields over Europe of a particular forecast day are selected. These fields are then compared to all fields of a large climatological reference dataset in order to find the best analog (using the Analogs function). This dataset covers the period 1993-2019 but includes.
https://doi. org/10. 5194/gmd-2021-368 Preprint. Discussion started: 6 December 2021 c Author(s) 2021. CC BY 4. 0 License. The CST_Analogs function can be used to downscale any gridded dataset using analogs. The function, based on the method of Yiou et al. (2013), searches for days with similar large-scale conditions to provide high-resolution fields over a specific 265 region. Regions and variables can be defined by the user and three different criteria to select the analogs are available: (1) minimum Euclidean distance in the large-scale pattern, (2) minimum Euclidean distance in a large-scale pattern and in a local-scale pattern, and (3) minimum Euclidean distance in a large-scale pattern and a local scale pattern as well as maximum correlation in a local variable to be downscaled.
WorldClim data to s2dv_cube library(raster) 665 worldclim <-getData("worldclim", var = "prec", res = 0. 5, lon = 5, lat = 45) wc_month <-lapply(1:12, FUN = function(x) { res <-crop(worldclim[[x]], extent(3. 5, 11. 5, 41. 5, 49. 5)) res <-as. array(res) 670 names(dim(res)) <-c("lat", "lon", "month") return(res) }) https://doi. org/10. 5194/gmd-2021-368 Preprint. Discussion started: 6 December 2021 c Author(s) 2021. CC BY 4. 0 License. xy <-xyFromCell(crop(worldclim[[1]], extent(3. 5, 11. 5, 41. 5, 49. 5)), 1:length(crop(worldclim[[1]], extent(3. 5, 11. 5, 41. 5, 49. 5)))) 675 lons <-unique(xy[,1]) lats <-unique(xy[,2]) wc_month <-unlist(wc_month) dim(wc_month) <-c(lat = length(lats), lon = length(lons), month = 12) wc_month <-s2dv_cube(data = wc_month, lon = lons, lat = lats, 680 Datasets = "WorldClim") weight <-CST_RFWeights(wc_month, lon = exp$lon, lat = exp$lat, nf = 4) Code Step 3.
(Muñoz-Sabater et al. , 2021) considered as the end product. This approach ensures the spatio-temporal consistency of both fields. Rather than giving detailed code instructions similar to the previous use cases, an extensive discussion on the data-generation process is provided and the code is made available online. Note again that all steps are 760 being executed using only CSTools functions. As the reference rainfall dataset, the daily data from CHIRPS (Climate Hazards group Infrared Precipitation with Stations;Funk et al. 2015)is taken, available at 0. 05° resolution is used. Moreover, this dataset incorporates corrections for different mountain elevations and slopes, which is necessary in the orographically complex Aliakmon basin with elevations above 765 2000 m. For temperature, we use the ERA5-Land(Muñoz-Sabater et al.
Acknowledgments 865 This work has been performed in the framework of the  MEDSCOPE (MEDiterranean Services Chain based On climate PrEdictions )  ERA4CS  project ( GA 690462 ) with contributions from the  Sub-seasonal to Seasonal climate forecasting for Energy  ( S2S4E ;  GA 776787 ) and the European Climate Prediction system (EUCP;  GA 776613 ) from the  European Union 's  Horizon 2020 research and innovation programme. CDT acknowledges the funding by the  Spanish Ministry for Science and Innovation (FPI   PRE2019-088646 ). We thank other contributors to the CSTools package:  Filippo Calì Quaglia ,  Chihchung 870 Chou ,  Nicola Cortesi ,  Susanna Corti ,  Paolo Davini ,  Federico Fabiano ,  Ignazio Giuntol ,  Raül Marcos ,  Niti Mishra , Jesus Peña,  Francesc Roura-Adserias  and  Danila Volpi.
Code availability CSTools is released under the Apache License version 2. 0. The latest release of CSTools 4. 0. 1 is publicly available on 840 CRAN repository https://CRAN. R-project. org/package=CSTools, it is being developed at BSC-CNS GitLab repository https://earth. bsc. es/gitlab/external/cstools/ and shared in the Zenodo DOI:10. 5281/zenodo. 5549474. The code to reproduce the use cases and plots shown in this manuscript is shared in the three sites and we recommend to find it at the GitLab repository https://earth. bsc. es/gitlab/external/cstools/-/tree/master/inst/doc.
# WorldClim data to s2dv_cube library(raster) 665 worldclim <-getData("worldclim", var = "prec", res = 0. 5, lon = 5, lat = 45) wc_month <-lapply(1:12, FUN = function(x) { res <-crop(worldclim[[x]], extent(3. 5, 11. 5, 41. 5, 49. 5)) res <-as. array(res) 670 names(dim(res)) <-c("lat", "lon", "month") return(res) }) https://doi. org/10. 5194/gmd-2021-368 Preprint. Discussion started: 6 December 2021 c Author(s) 2021. CC BY 4. 0 License. xy <-xyFromCell(crop(worldclim[[1]], extent(3. 5, 11. 5, 41. 5, 49. 5)), 1:length(crop(worldclim[[1]], extent(3. 5, 11. 5, 41. 5, 49. 5)))) 675 lons <-unique(xy[,1]) lats <-unique(xy[,2]) wc_month <-unlist(wc_month) dim(wc_month) <-c(lat = length(lats), lon = length(lons), month = 12) wc_month <-s2dv_cube(data = wc_month, lon = lons, lat = lats, 680 Datasets = "WorldClim") weight <-CST_RFWeights(wc_month, lon = exp$lon, lat = exp$lat, nf = 4) Code Step 3.
195 Although datasets can be retrieved from OPeNDAP URLs with NetCDF files, in general, the datasets have to be downloaded onto a local repository and formatted to comply with the CST_Load requirements. Observational reference datasets are stored in a folder in separate monthly NetCDF files (other formats are also possible; see https://earth. bsc. es/gitlab/es/s2dverification/-/blob/master/vignettes/data_retrieval. md for more information), while seasonal 200 forecasts are stored by start date in distinct folders (see https://cran. r- project. org/web/packages/CSTools/vignettes/Data_Considerations. html). A python code to download and format the seasonal forecast datasets from the CDS is provided in the repository CDS Seasonal Downloader (https://earth. bsc. es/gitlab/es/cds-seasonal-downloader).
Optical coherence tomography (OCT) is a non-invasive, painless and reproducible examination which allows ophthalmologists to visualize retinal layers. This imaging modality is useful to detect diseases such as diabetic macular edema (DME) or age related macular degeneration (AMD), which are associated with fluid accumulations. In this paper, a cascade of deep convolutional neural networks is proposed using ENets for the segmentation of fluid accumulations in OCT B-Scans. After denoising the B-Scans, a first ENet extracts the region of interest (ROI) between the inner limiting membrane (ILM) and the Bruch's membrane (BM), whereas the second ENet segments the fluid in the ROI. A random forest classifier was applied on the segmented fluid regions to reject false positive. Our framework was trained on three different datasets with several diseases such as diabetic retinopathy (DR) and AMD.
Datasets For the present study, three public OCT datasets were employed to train and test our method. Two of them were created by the University of Minnesota (UMN)  [2] [20]. The OCT volumes were acquired with a Heidelberg Spectralis imaging system for 29 subjects with DME and 24 with AMD. Each B-Scan averages 12 to 19 frames with a September 17, 2021 2/12 resolution of (5. 88 x 3. 87) µm/pixel. The accumulations of fluid were manually segmented by two UMN ophthalmologists. The Kermany dataset consists of 530 OCT volumes divided into three categories: DME, DRUSEN and NORMAL  [21]. The images were also acquired with a Heidelberg Spectralis imaging system and manual fluid segmentation was performed by three trained graders and approved by two ophthalmologists. For each volume, the number of B-Scans varied from 1 to 13, with most volumes containing only one or two.
Fluid Segmentation The second model of this pipeline generates a binary fluid segmentation mask. It takes as input two images: the B-Scan preprocessed by the BM3D algorithm and the output of the first model, namely the ROI mask. One of the major advantages of this approach is that the model will be able to focus only on pixels that are likely to contain fluid. For this segmentation task, we also tested the same four architectures to identify the most relevant one. We compared our performances with several approaches encountered in the literature: Rashno et al. [2]  and Ganjee et al. [16]  methods for the UMN DME dataset, Rashno et al. [2]  and Chen et al. [11]  for the UMN AMD, and Ganjee et al. [16]  and Lu et al. [7]  for Kermany.
Preprocessing We preprocessed all the B-Scans from the UMN and Kermany datasets with three algorithms: BM3D, NLM and median filtering. The preprocessing results have been summarized in Tab 1. The results of the feature-based metrics clearly show that the median filter allows a better preservation of the image features associated with a short execution time, with a SSIM and a VIF 8% higher than the other methods. However, the median filter does not remove as much noise as the BM3D and NLM algorithms. Because of its good ability to remove speckle noise according to its PSNR score, while preserving edges, we decided to preprocess our B-Scans with the BM3D algorithm. September 17, 2021 6/12.
Fluid Segmentation Once we finished the preprocessing step of the OCT B-Scans with the denoising algorithm BM3D and the ROI segmentation, we were able to perform fluid segmentation. As we previously did for the ROI, we determined the best possible architecture for segmenting fluid accumulations in OCT images. Therefore, we trained four different architectures using the preprocessed and ROI segmented B-Scans from the three datasets. Results are detailed in Tab 3.