dataesr/scientific-paragraphs-categorization huggingface dataset:
Gridded observation datasets Among the available gridded products for air temperature and precipitation, we used: E-OBS, CRU and GPCC data were downloaded for the ECEM grid. All three datasets only cover land regions, so any bias adjustment using these data sets will not include marine areas. E-OBS covers the period from 1951-2015, so fully encompassing the 1979-2014 period of ERA-Interim. CRU TS and GPCC cover the period from 1901 to the present, but are both monthly averages/totals, so can only provide an assessment at this timescale. They are of potential use in regions of the domain where daily timescale data are sparse and/or often missing in E-OBS (e. g. North Africa and the Middle East). The principal emphases of ECEM are the countries of Western Europe, so North Africa and parts of the Middle East are often missing on some of the subsequent maps.
HadISD No gridded observed product is available for wind speed and dewpoint temperature. Dewpoint temperature is necessary as it can be combined with air temperature to calculate relative humidity, which is needed for energy calculations, such as demand. Station data for wind speed at 10 m height and dewpoint temperature were extracted from HadISD (http://www. metoffice. gov. uk/hadobs/hadisd/) for approximately 1500 stations across Europe. Station data were extracted 10 every 6 h at the SYNOP hours 00, 06, 12 and 18 for the period 1979 to 2014. HadISD has been assessed for long-term homogeneity by  Dunn et al. (2014). Variations in station coverage within HadISD are considerably greater than the coverage achieved for air temperature from E-OBS and precipitation from E-OBS and GPCC. This indicates that it would be unwise to attempt spatial interpolation to a 0.
As only few stations are available for solar irradiance, it was decided to exploit satellite-derived datasets to correct ERA-Interim. HelioClim-3v5 25 (HC3v5) is such a dataset and results from the long-standing effort by MINES ParisTech for providing accurate assessment of surface solar irradiance  (Blanc et al. , 2011). HC3v5 originates from the daily processing of images acquired by the series of satellites Meteosat-MSG and as such does not cover the extreme northern part of the ECEM domain. The first estimates began on 1 st February 2004 and these have been compared satisfactorily with measurements taken at ground stations  (Eissa et al. , 2015; Thomas et al. , 2016a, b). HC3v5 data were downloaded from the SoDa Service web site (www. soda-pro. com) from which one may select the timescale, here daily mean of irradiance I.
Bias-adjustment approaches Bias adjustment and bias correction are widely-used terms for the assessment of climate model output (from both Global and Regional Climate Models, GCMs and RCMs, see e. g. Maraun et al. , 2010 and Maraun, 2012)  generally through comparison with station observational data. In this context, the biases are often much larger than differences with recent Reanalysis 10 products. There are a number of studies where GCMs and RCMs are bias adjusted against Reanalyses, so the assumption is made there that Reanalyses are correct. This happens more in regions where observational datasets are sparse and/or hard to access. Bias adjustment of Reanalyses has been undertaken for a number of years, though. An extensive exercise was carried out by the WATCH project (http://www. eu-watch.
org/, see  Weedon et al. 2011 Weedon et al. , 2014)). These used the CRU TS dataset as the basis for adjusting ERA-40 and ERA-Interim, and the adjustments are based on average monthly differences treating each 15 variable independently from each other. Numerous and more complex (than Weedon et al. , 2014) methods for bias adjusting climate variables derived from climate models have been proposed. A number of review papers have been published (e. g. Maraun et al. , 2010 , Maruan, 2013 and Vrac and Friedrichs, 2015). Among the various possibilities are the cumulative distribution function (CDF) transform method 20 of  Vrac et al. (2012) , the distribution based scaling (DBS) method of Yang et al, 2010, empirical quantile mapping  (Themeßl et al. , 2011 (Themeßl et al. , , 2012;; Wilcke et al.
The 2-parameter Weibull distribution was 25 fitted to 6-h wind speed data from ERA-Interim on a monthly basis, i. e. a separate fit was made for each month, for each grid box using all the 6-h data for 1981-2010, irrespective of the wind direction. The same approach was applied for the wind data from 803 stations in the HadISD dataset that have at least 66. 6% data completeness for this 30-year period. The shape and scale parameters (α, β) for the 803 stations were compared with the same parameters from the nearest ERA-Interim grid box. Earth Syst. Sci. Data Discuss. , doi:10. 5194/essd-2016-67, 2017 (ftp://ecem. climate. copernicus. eu). It is clear from these two examples that the Weibull distributional fit for the stations has moved the adjusted ERA-Interim data series towards the observational distribution, more so for Kirkwall which shows a much greater improvement than for Maribor, where the distribution moves are a clear improvement in winter months but less so for spring and early summer months.
Bias adjustment is less successful than the examples shown for a few stations located in coastal areas and a few sites in mountainous regions. Some observed distributions are a little erratic due to some years in the 20 observed data having wind speeds rounded to integer values. Similarly to Figure  1 , but for bias-adjusted ERA-interim minus observations, Figure  4  shows differences between the scale and shape parameters for January. Most stations across Europe exhibit similar shape and scale parameters between the stations and ERA-Interim. However, a few stations in coastal areas and at high elevation mountain locations still show differences in parameters. The fit is not perfectly attained as estimation of the shape and scale parameters for the ERA-Interim grid boxes from HadISD is influenced by the station distribution.
Surface air temperature, dewpoint temperature and relative humidity Like wind speed, both surface air temperature and dewpoint temperature are produced from ERA-Interim every 6 h. Unlike 10 m-wind, both these variables have a strong diurnal cycle, which is generally slightly stronger in the summer. A normal 30 distribution was fitted using daily averages of temperature, taking the average of the four 6-h data for each day. E-OBS is the (4) 5 * = ′ ()* 2 345 + 0 345  (5)  where 7 is the normalised ERA-Interim temperature anomaly, * is the bias-adjusted ERA-Interim temperature, 0 is the mean temperature and 2 is the standard deviation. Bias adjustment works by transforming the normalized ERA-Interim gridbox time series back to air temperatures using the means and standard deviations from E-OBS and interpolations from station data in HadISD for dewpoints.
Once daily averages are adjusted, the difference between the original ERA daily mean and the 10 adjusted daily mean is added to each of the four 6-h temperatures within each day. Therefore, no alteration is made to each diurnal cycle of air or dewpoint temperature. This yields the final set of bias-adjusted 6-h surface air and dewpoint temperatures. It might seem more logical to use the same temporal resolution dataset for both temperatures (e. g. by using air temperatures from HadISD), but this would involve not taking advantage of E-OBS, which uses far more input station data than HadISD. 15 Figure  5  shows the differences in the mean and standard deviation for air temperature for April as an example. There is good agreement between estimates for ERA-Interim and those calculated from E-OBS.
As these are both gridded datasets, the maps shown are fully coloured for each 0. 5° grid box. Figures  6  and 7  exhibit the normal distribution fits of the E-OBS, original ERA-Interim and bias-adjusted ERA-Interim for the twelve calendar months for the nearest land grid boxes that approximate 20 the locations of Kirkwall and Maribor used for wind speed. For Maribor this is the 0. 5° grid box where the city is located. Kirkwall is on the Orkney Islands, so the nearest grid box within E-OBS is located further south in Northern Scotland. The distributional fits for the Maribor grid box were good for ERA-Interim and bias adjustment brings minor improvement. For the Kirkwall grid box, the adjustments improve the fits in all months, but are less good for the cold tail of air temperature in winter.
This threshold ignores small precipitation totals, more so for ERA-Interim than for E-OBS, but as both datasets are in essence areal averages, more than would be the case for a station rain gauge series. In the adjusted ERA-Interim all precipitation amounts below the threshold are set to zero, further improving the agreement between E-OBS and ERA-Interim in the number of dry days per month (i. e. days with rainfall less than the 1. 0 mm threshold). 25 Adjustment is performed in a similar way to the temperatures, by back transforming the transformed ERA-Interim precipitation total with the scale and shape parameters from the E-OBS dataset. Figure  13  shows the differences in the scale and shape parameters of the gamma distribution for October, by way of example. There is good agreement between estimates for ERA-Interim and those calculated from E-OBS.
As these are both gridded 30 datasets, the maps shown are fully coloured for each 0. 5° grid box. Of the 4520 possibilities, Figures  14  and 15  exhibit the gamma distribution fits of the E-OBS, original ERA-Interim and bias-adjusted ERA-Interim for the twelve calendar months for the nearest land grid boxes that approximate the locations of Kirkwall and Maribor. The fits for the northern Scotland grid box are considerably better than for Maribor. The full set of results for the 4520 grid-box comparisons can be viewed at the website. Although the gamma distribution is widely used for rainfall data, it is not ideal in all climates and across all seasons in Europe. Problems arise when there are too few rainfall days within dry seasons (the Southern Mediterranean and the Middle East during summer).
The size of the triangles increases with increasing absolute values of the bias. For example, a green downward triangle means that the bias has been Earth Syst. Sci. Data Discuss. , doi:10. 5194/essd-2016 -67, 2017  Open Access Earth System Science Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. decreased (downward triangle, i. e. improvement) and that after bias adjustment, the absolute value of the bias is less than 5 W m -2. One may see that there is an improvement or status quo for all stations, i. e. there is no upward triangle, only circles and downward triangles. 22 stations out of 57 exhibit a bias less than 5 W m -2 in their absolute values, which is a strong improvement compared to the 12 for the original ERA-Interim data.
Discussion As stated earlier in the paper, the work reported here is specifically targeting energy sector applications; however the bias 10 adjustment carried out here could be applied to a wide range of potential applications. ECEM and its users plan to use both the adjusted and unadjusted ERA-Interim gridded products through ESCIIs (Energy System Climate Impact Indicators), which will relate the climate variables to energy-relevant indices. Whether the bias adjustments improve agreement between these ESCIIs and the direct measures of energy production (e. g. renewable energy from solar and wind farms) is a simple way of assessing their effectiveness. 15 The WATCH bias-adjusted datasets developed by  Weedon et al. (2011 Weedon et al. ( , 2014) )  have been used extensively, based on citation counts, but they cover a much larger region than our European window.
Our dataset applies adjustments to the distributions of a similar set of variables, providing daily and 6-hourly estimates. Outside the Energy sector, the bias-adjusted datasets could be used for driving hydrological and land-surface models in a similar way to Orth and Seneviratne (2015). Our bias 20 adjustments, therefore, could be assessed beyond the Energy sector. For Europe, they could be compared with WFDEI data (often referred to as forcing data in hydrology, as opposed to bias-adjusted Reanalyses) through comparison of results from hydrologic and/or crop climate models (e. g. using discharge or yield data). Bias adjustment ought to be an improvement, in a similar way to the assessments we will make with ESCIIs.
25 Is there a way of simultaneously bias adjusting all variables, or at least in pairs to start with? Whereas  Weedon et al. (2014)  have not attempted multivariate adjustment, this is being tested in our project. However, as the number of variables increases this becomes more impractical. If a universal method could be found, the usefulness of the approach can be assessed through ESCIIs and discharge/yield data (i. e. using variables external to Reanalysis) which we would expect to be best simulated just as if we had perfect observational data. Within ECEM, we have experimented with multivariate bias adjustment (using wind 30 speed and temperature), but the results are dependent on the availability of adequate station data for variables measured together (Parie, Pers. Comm. ). Access to data is a crucial aspect of all the datasets used in this study.
ERA-Interim would be Earth Syst. Sci. Data Discuss. , doi:10. 5194/essd-2016 -67, 2017  Open Copernicus Climate Data Service. As this CDS is currently being developed, this ftp site (ftp://ecem. climate. copernicus. eu) can be currently used to access all files discussed in this paper. This site currently has no password, but once on the CDS, there will likely be a registration procedure. Piani, C. , Haerter, J. O. , and Coppola, E. : Statistical bias correction for daily precipitation in regional climate models over  Europe, Theoretical and Applied Climatology, 99, 187-192, 2010. Themeβl, M. J. , Gobiet, A. , and Leuprecht, A. : Empirical-statistical downscaling and error correction of daily precipitation 10 from regional climate models, International Journal of  Climatology, 31, 1530 -1544 , DOI: 10.
1002 /joc. 2168 , 2011. Themeβl, M. , Gobiet, A. , and Heinrich, G. : Empirical-statistical downscaling and error correction of regional climate models and its impact on the climate change signal, Climatic  Change, 112, 449-468, doi:10. 1007 Change, 112, 449-468, doi:10. /s10584-011-0224-4, 2012. 15. 15  Thomas, C. , Wey, E. , Blanc, P. , and Wald, L. : Validation of three satellite-derived databases of surface solar radiation using measurements performed at 42 stations in Brazil, Adv. Sci. Res. , 13, 129-136, doi:10. 5194/asr-13-129-2016 , 2016a. Vrac, M. , and Friederichs, P. : Multivariate-intervariable, spatial, and temporal-bias correction, J. Climate, 28, [218] [219] [220] [221] [222] [223] [224] [225] [226] [227] [228] [229] [230] [231] [232] [233] [234] [235] [236] [237] 25  doi:10.
Earth Syst. Sci. Data Discuss. , doi:10. 5194/essd-2016-67for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. dataset on which ERA-Interim is to be adjusted for air temperature, while for dewpoint we use HadISD in a similar way to wind speeds. Means and standard deviations of daily average of temperature are calculated for each month of the year for each 0. 5° grid cell of ERA-Interim and for the nearest E-OBS grid box, and for HadISD stations. The means and standard deviations of HadISD stations are then interpolated as for wind speed. Data are then normalized as in Equations 4 and 5.
for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. improved with greater numbers of station input data, as would E-OBS and the other data products considered in this paper. Improved access, however, is unlikely to reduce the need for bias adjustment. 6 The ECEM dataset: description and how to access itAll the ERA-Interim (original and bias-adjusted) are available as netcdf files from the Climate Data Store (CDS) of the 5.
Datasets are named according to ECEM project. The original or unadjusted filenames have 'noc' in the file name. They are, for air temperature (T2M), dewpoint temperature (DP), solar irradiance (SSR), wind speed (WS) and precipitation (TPare labelled similarly, but include the distribution and 'bc' instead of 'noc'. So for air temperature and dewpoint they include 'nbc', for solar irradiance 'qbc', for wind speed 'wbc' and precipitation 'gbc'. A final file contains the bias adjusted relative humidity file. for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. downscaling under climate change. Recent developments to bridge the gap between dynamical models and the end user, Reviews in Geophysics, 48, RG3003, doi: 10. 1029/2009RG000314, 2010. Orth, R.
Open Access Earth System Science Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. Acknowledgements The authors would like to acknowledge funding for the  European Climatic Energy Mixes (ECEM)  project by the  Copernicus Climate Change Service , a programme being implemented by the  European Centre for Medium-Range Weather Forecasts (ECMWF)  on behalf of the  European Commission. The specific grant number is 2015/C3S_441_Lot2_UEA. The authors also 5 thank  Robert Dunn  from the  UK MetOffice Hadley Center  who kindly extracted the wind speed and wind direction for all stations from the  HadISD data set. The authors thank all ground station operators of the WMO network for their valuable measurements. They additionally thank the  World Radiation Data Centre  for hosting a web site for downloading data.
Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017. CC-BY 3. 0 License. Data Discussions Manuscript under review for journal Earth Syst. Sci. Data Published: 6 January 2017 c Author(s) 2017.
Fortunately, one of our test runs took place just a few days before HST -WFC3 line filter observation or R Aqr. This provides a unique opportunity for improving the ZIMPOL flux measurements and the instrument throughput calibration. For the scientific investigation of the R Aqr system the HST and SPHERE data are very complementary. HST provides a much larger field of view, higher sensitivity, and flux fidelity, while ZIMPOL-SPHERE yields imaging and polarimetric imaging with about three times higher spatial resolution and higher contrast in a small field  (3. 6 × 3. 6 )  centered on the star. The enhanced resolution enables ZIMPOL-SPHERE to resolve the central binary system, the innermost jet clouds, and polarized light produced by the scattering from circumstellar dust particles. In this work we concentrate the scientific investigation on the SPHERE and HST line filter observations for the small central field of R Aqr.
The new data presented in this work provide images of the central jet outflow with much improved resolution and sensitivity, and they resolve clearly the two stellar components in the system. This paper is organized as follows. Section 2 gives an overview on the VLT-SPHERE observations, a description of the used filters, and an assessment of the SPHERE AO performance for the R Aqr observations. In Section 3 we determine the relative astrometric position for the central binary. Section 4 provides the photometry for the mira variable in R Aqr and the flux for the total Hα emission in the ZIMPOL field. The structures of the observed jet clouds are described in Section 5 including cloud position and size, and the derivation of the Hα surface brightness and flux for the individual clouds. Section 6 describes the used HST line filter data and the determination of HST line fluxes.
SPHERE / ZIMPOL data R Aqr was used as a test source for the verification of different instrument configurations and therefore the observations are not optimized for scientific purposes. The data are affected by several technical problems, especially for the July run when no scientifically useful data could be obtained. In August 2014 only imaging observations were possible, but no imaging polarimetry. The data from October 2014 could be taken without technical problems. For the scientific investigation of R Aqr we use only the data from August and October 2014 listed in Table  1. All data were taken in field-stabilized mode without field position angle offset and using the gray beam splitter sending about 21 % of the light to the wave front sensor (WFS) and transmitting 79 % to the ZIMPOL instrument.
Also four frames covering off-axis fields centered about 2 arcsec to the NE, SE, SW, and NW of R Aqr were taken. Observations in polarimetric mode were taken in October 2014. We use in this work only the total intensity images of the polarimetric data. The throughput is about 18 % lower because of the inserted polarimetric components. Also the detector mode is changed for on-chip demodulation of the data (see  Schmid et al. 2012). In particular, the slow modulation modes has a low detector gain of 1. 5 e -/ct (ct = ADU, analog to digital count units), much lower than the 10. 5 e -/ct gain for imaging and fast polarimetry. The read-out noise of the CCD (charge-coupled device) is about 1-2 ct and this translates for the slow polarimetry into a much lower noise level in terms of photo-electrons e -, and therefore a better faint source sensitivity, when compared to imaging and fast polarimetry.
ZIMPOL data reduction The data reduction of ZIMPOL imaging data is for most steps straight forward and follows standard procedures like bias frame subtraction, cosmic ray removal with a median filter, and flat fielding. The data reduction was performed with the (SPHERE-ZIMPOL) SZ-software package, which is written in IDL and was developed at the ETH (Eidgenössische Technische Hochschule) Zurich. The basic procedures are essentially identical to the SPHERE DRH-software provided by ESO. A special characteristic of the ZIMPOL detectors are the row masks which cover every second row of the detector. This feature is implemented in ZIMPOL for high precision imaging polarimetry using a modulation-demodulation technique  (Schmid et al. 2012). A raw frame taken in imaging mode has only every second row illuminated and the useful science data has a format of 512×1024 pixels consisting of the 512 illuminated rows of the 1024 × 1024 pixel detector.
In one of the final steps in the data processing, the 512×1024 pixel images are expanded with a flux-conserving linear interpolation to yield a square 1024 × 1024 pixel image where one pixel represents 3. 6 × 3. 6 mas on sky. The artificial oversampling of the image rows has no significant apparent effect on the resulting images. The detector dark current was found to be variable for our test observations. Therefore, it was not possible to subtract the dark current based on calibration measurements. As an alternative, we determined the dark current from the science frames. In imaging mode one can use, for dark current estimates, the count level in the covered row pixels for detector regions with low illumination, and for polarimetric observations we used the count level in the detector edges (far from the central source) for weakly illuminated frames.