dataesr/scientific-paragraphs-categorization huggingface dataset:
13 Nucleotide transport and metabolism H 112 2. 57 Coenzyme transport and metabolism I 88 2. 02 Lipid transport and metabolism P 197 4. 52 Inorganic ion transport and metabolism Q 77 1. 76 Secondary metabolites biosynthesis, transport and catabolism R 531 12. 20 General function prediction only S 310 7. 12 Function unknown - 1532 35. 20 Not in COGs COGs, Clusters of Orthologous Groups database. a Total is based on total number of protein-coding genes in annotated genome. 138 New Microbes and New Infections, Volume 10 Number C, March 2016 NMNI New Microbes and New Infections © 2016 The Authors. Published by Elsevier Ltd on behalf of European Society of Clinical Microbiology and Infectious Diseases, NMNI, 10, 132-141 This is an open access article under the CC BY-NC-ND license (http://creativecommons.
The method is based on the assumption that the galaxies in the COS-MOS data-set  (Ilbert et al. 2009 ) are representative of the colors and of the redshift distribution of the global galaxy population in the universe and that they allow us to characterize the distribution in the CC space of galaxies in different redshift ranges. Simultaneously, we tune the color cuts on the basis of the amplitude of the shear signal of the background selected galaxies. We use simulations made with the SkyLens software to validate this method assuming different color information and different lens masses and redshifts. We also made comparison with others approach, such as the selections based on the identification of the cluster red sequence and on photometric redshifts. Finally, for each selection method we derive an estimate of the cluster mass, which we compare to the true mass of the simulated cluster.
LENSING SIMULATIONS To test and validate the selection method, we use realistic image simulations produced with SkyLens  (Meneghetti et al. 2008 (Meneghetti et al. , 2010;; Rasia et al. 2012). The code uses real galaxies taken from the Hubble Ultra-Deep-Field to create mock observations of patches of the sky with virtually any telescope. Being coupled with a ray-tracing code, it allows us to include the lensing effects from any mass distribution along the line of sight. As shown in  Meneghetti et al. (2010) and in Rasia et al. (2012) , it is particularly suitable to simulate weak and strong lensing effects by galaxy clusters. The mock galaxy clusters used here are generated with the code MOKA  (Giocoli et al. 2012 ). The cluster mass distributions have multiple components, namely a triaxial halo, modelled with a Navarro-Frenk-White (NFW) profile  (Navarro et al.
More precisely, we simulate images in the BVRi bands with exposure times as listed in Table  4. We reproduce the telescope throughputs, according to informations retrieved from the Subaru and LBT telescope websites. In this case, the cluster simulated with MOKA has the same redshift of Abell 2219 and its virial mass is chosen to be 1 × 10 15 M⊙, consistent with previous mass measurements found in the literature for this cluster. We assume a concentration of 3. 8, consistent with values expected for clusters of this mass on the basis of recent concentration-mass relations  (Meneghetti et al. 2014). Using this set of simulations, we explore the effect of several selection techniques on the mass reconstruction, based on different filter combinations. Hereafter, the analysis of this dataset with our selection method is coded with case 1 and case 2 if the photometry in BVR and BVRi bands is used respectively, while analyses of this dataset with alternative selection methods are coded with case 1-2.
1998). We use the KSB implementation described in  Radovich et al. (2015). The main feature of our pipeline is that the PSF is modelled as a polynomial function of the position, evaluated with the PSFex software  (Bertin 2011) , which extracts models of the PSF from the image. The p and q terms are derived at each galaxy's position using this PSF model. For each galaxy, the output quantity eiso = eanisoP γ -1 is computed and used to derive the average reduced shear, being g = eiso under the assumption that the contribution of the intrinsic ellipticities vanishes when averaging over a large number of galaxies. The main quantities described by the KSB approach are weighted through a gaussian window function of scale length θ to beat down the noise. The size of the window function has a crucial role since, if it is not properly chosen, noise will bias the ellipticity measurements.
Simultaneously, we also check that the tangential shear profile of the foreground and cluster member galaxies is consistent with zero at all radii. Despite the limited size of the COSMOS field, several tests show that using the COSMOS colors as starting point for the identification of distant sources is safe. For example, if we apply the color cuts based on the COSMOS photometry to select high redshift galaxies in the Hubble Ultra-Deep-Field, we find a low contamination by low redshift sources. The method can be also applied to data observed with a different filter set. In the examples discussed later in this paper, we use the software ZEBRA  (Feldmann et al. 2006)  to derive the SEDs of the COSMOS galaxies and we use them to calculate magnitudes in other filters by convolving with the filter transmission curves.
Fig. 2  summarizes the full procedure. The dot-dashed lines in the left panels correspond to the initial values of the a, c coefficients, derived for all the test cases here considered. Obviously, the results depend on the filter combination and on the lens redshift. The coefficients a and c obtained for test cases 1-5 are listed in the Table  2. Test cases 6-7 are omitted because the results are identical to test case 3 (the selection based on COSMOS data does not depend on the lens mass). The second step of our procedure consists in the fine-tuning of the selection based on the amplitude of the shear profile derived from the galaxies which are classified as background sources. Therefore, we consider now the CC diagrams built using all galaxies in the simulated cluster fields. We start by identifying the background sources based on the lines derived in the first step.
84 ± 0. 99. Hoekstra (2007)  performed an independent weak lensing analysis using the same data of  Bardeau et al. (2007). After selecting galaxies in the magnitude range 21 < R < 25, he used the photometry in B and R bands to remove galaxies that lie on the cluster red-sequence. The shear profile was corrected for the residual contamination by increasing the observed shear by the factor 1 + fcg(θ), where fcg(r) is the fraction of cluster galaxies at radius r. The value of fcg as function of radius was determined by stacking several other clusters. Using a value of β = 0. 54 inferred from the redshift distribution of the Hubble Deep Field (HDF)  (Fernández-Soto et al. 1999) , and fitting the tangential shear profile with an NFW model, the resulting virial mass was 11. 3 +3. 2 -2. 7 × h -1 10 14 M⊙.
The fit was done assuming one free parameter (the mass) and using the relation between concentration and mass proposed by  Bullock et al. (2001). Okabe et al. (2010)  also performed a weak lensing analysis of Abell 2219 as part of their study of 30 clusters from the Local Cluster Substructure Survey (LoCuSS) sample. They used only SUBARU data in R and V bands. The exposure times were 24 and 18 minutes, respectively. The selection of background sources was done combining the magnitude cut 22 R 26 with color cuts to identify galaxies redder and bluer than the cluster red sequence. The optimal selection was found by maximizing the mean amplitude of the tangential shear profile with respect to the color offset from the cluster red sequence. The resulting number density of background sources was found to be ∼ 10 gal arcmin -2.
Image reduction The observations of the cluster Abell 2219 were performed with the Suprime-Cam mounted at the prime focus of the SUBARU telescope, an 8. 2 meter telescope located at the summit of the Mauna Kea. The camera is a mosaic of ten 2048 x 4096 CCDs which covers a 34 ′ × 27 ′ field of view with a pixel scale of 0. 20 ′′. We analyze data publicly available in the BVR bands, retrieved from the SMOKA Science Archive. Images are reduced using the VST-Tube imaging pipeline  (Grado et al. 2012) , a software to work on optical astronomical images taken with different instruments and instrumental setups. The data reduction consisted in the following steps: overscan correction, flat fielding, correction of distortions due to optics and sky background subtraction. The astrometric solution describing the distortions produced by the optics and telescope was computed for each exposure using the ASTROMC code  (Radovich et al.
2004 ). This was then used in the SWARP tool, which was used to resample and coadd all exposures. The absolute photometric calibration of the images in each filter was achieved using standard Stetson fields. The SDSS i -band image of Abell 2219 was taken with the Large Binocular Camera mounted on the prime focus of the 8. 4m Large Binocular Telescope. LBC has a field of 23 ′ × 23 ′ and provides images with a sampling of 0. 225 ′′ /pixel. The data were taken in 2010; the data reduction was done by the LBC Data Centre using a pipeline specifically designed for LBC data. Magnitudes in the BVRi filters are in the AB system. The observation nights of the images in each filter, the total exposure time, the photometric zero point of the final coadded images and the average FWHM for point-like sources are given in Table  4.
As done analyzing the simulations, we do not use galaxies with photo-zs > 2 and σz/(1+z) > 0. 1. For the galaxies with spectroscopic redshift given in Boschin et al. (  2004 ) we find z phot -zspec = -0. 017 ± 0. 003. The sample given in Boschin et al. (  2004 ) is  characterized by galaxies with z 0. 4 so it is not possible to verify the accuracy of sources which have higher redshift value. The accuracy of the photometric redshifts derived for z > 0. 4 is checked from the comparison with the Sloan Digital Sky Survey photometric redshifts of galaxies in the Data Release 9, although the latter are obtained from shallower images. We find z phot -z sloan = -0. 066 ± 0. 007. We select as background galaxies those having z phot > 0. 27 (Fig. 12 , both panels, magenta squares), as foreground galaxies those having z phot < 0.
CONCLUSION Dilution of the lensing signal due to unlensed sources can crucially affect the estimation of the cluster mass. We developed a new selection method for the background lensed sources based on the simultaneous analysis of the shear signal and of the colors of the galaxies, with the photometry from the COSMOS used as training set. Our selection method can be applied to the weak lensing analysis of clusters, under different cluster redshifts, masses and filter combinations. Its performance in the selection of the background sources and its effect on the dilution bias are tested using simulations. To this end, we produced realistic SUBARU and LBT images containing lensing signal with the SkyLens code under different cluster redshifts (z = 0. 23 -0. 45), masses (Mvir = 0. 5 -1. 55 × 10 15 M⊙) and filter combinations.
In our work, we used a new KSB implementation based on the PSFex code  (Radovich et al. 2015)  to measure the observed shear signal produced by the gravitational field of the cluster. As shown by simulations the high purity of the selected samples obtained allows us to derive fiducial samples of background sources on which an accurate cluster mass reconstruction can be performed. The virial mass of Abell 2219 was obtained for each kind of selection by fitting the NFW model to data. Our results are in agreement with the previous analysis of Hoekstra  (2007)  and  Okabe et al. (2010)  (for this last one makes exception the selection in the R-i vs B -V space). However differently from our results,  Okabe et al. (2010)  found that the NFW shear model does not fit the observed shear. This can arise from different selection criteria and source redshifts identification used in the analyses, which can origin the other different results found in literature.
ACKNOWLEDGEMENTS We acknowledge the support from the  LBT-Italian Coordination Facility  for the execution of observations, data distribution and reduction. CG's research is part of the project  GLENCO , funded under the  European Seventh Framework Programme ,  Ideas , Grant Agreement n. 259349. CG thanks  CNES  for financial support. We acknowledge support from  PRIN-INAF 2014 1. 05. 01. 94. 02Con   il contributo del Ministero degli Affari Esteri e della Cooperazione Internazionale ,  Direzione Generale per la Promozione del Sistema Paese.
Abstract. High-resolution Earth system model simulations generate enormous data volumes, and retaining the data from these simulations often strains institutional storage resources. Further, these exceedingly large storage requirements negatively impact science objectives, for example, by forcing reductions in data output frequency, simulation length, or ensemble size. To lessen data volumes from the Community Earth System Model (CESM), we advocate the use of lossy data compression techniques. While lossy data compression does not exactly preserve the original data (as lossless compression does), lossy techniques have an advantage in terms of smaller storage requirements. To preserve the integrity of the scientific simulation data, the effects of lossy data compression on the original data should, at a minimum, not be statistically distinguishable from the natural variability of the climate system, and previous preliminary work with data from CESM has shown this goal to be attainable.
However, to ultimately convince climate scientists that it is acceptable to use lossy data compression, we provide climate scientists with access to publicly available climate data that have undergone lossy data compression. In particular, we report on the results of a lossy data compression experiment with output from the CESM Large Ensemble (CESM-LE) Community Project, in which we challenge climate scientists to examine features of the data relevant to their interests, and at-tempt to identify which of the ensemble members have been compressed and reconstructed. We find that while detecting distinguishing features is certainly possible, the compression effects noticeable in these features are often unimportant or disappear in post-processing analyses. In addition, we perform several analyses that directly compare the original data to the reconstructed data to investigate the preservation, or lack thereof, of specific features critical to climate science.
Introduction Earth system models are widely used to study and understand past, present, and future climate states. The tremendous advances in computational power (i. e. , processor speeds) over the last 25 years have allowed Earth system modelers to use finer temporal and spatial model resolutions. While finer resolutions typically produce more accurate and realistic simulations, the resulting datasets are often massive and may severely strain data storage resources. Because supercomputing storage capacities have not increased as rapidly as proces-Published by Copernicus Publications on behalf of the European Geosciences Union. sor speeds over the last 25 years, the cost of storing huge data volumes is becoming increasingly burdensome and consuming larger and unsustainable percentages of computing center budgets (e.
g. ,  Kunkel et al. , 2014). The Community Earth System Model (CESM) is a popular and fully coupled climate simulation code  (Hurrell et al. , 2013) , whose development is led by the National Center for Atmospheric Research (NCAR). The CESM regularly produces large datasets resulting from high-resolution runs and/or long timescales that strain NCAR storage resources. For example, to participate in the Coupled Model Comparison Project Phase 5 (CMIP5, 2013) that led to the Intergovernmental Panel on Climate Change (IPCC, 2016) Assessment Report 5 (AR5)  (IPCC, 2013) , CESM produced nearly 2. 5 PB of raw output data that were post-processed to obtain the 170 TB of data submitted to CMIP5  (Paul et al. , 2015). Current estimates of the raw data requirements for CESM for the upcoming CMIP6 project  (Meehl et al.
, 2014)  are in excess of 10 PB  (Paul et al. , 2015). A second example of a data-intensive CESM project is the CESM-Large Ensemble (LE) project  (Kay et al. , 2015) , a large ensemble climate simulation study. The CESM-LE project is a publicly available collection of 180-year climate simulations at approximately 1 • horizontal resolution for studying internal climate variability. Storage constraints influenced the frequency of data output and necessitated the deletion of the raw monthly output files. In particular, the initial 30 ensemble member simulations generated over 300 TB of raw data, and less than 200 TB of processed and raw data combined could be retained due to disk storage constraints. For large climate modeling projects such as CMIP and CESM-LE, reducing data volumes via data compression would mitigate the data volume challenges by enabling more (or longer) simulations to be retained, and hence allow for more comprehensive scientific investigations.
, 2015). With compression in particular, we must also ensure that nothing systematic (i. e. , over-smoothing) has been introduced. Therefore, to build confidence in data compression techniques and promote acceptance in the climate community, our aim in this work is to investigate whether applying lossy compression impacts science results or conclusions from a large and publicly available CESM dataset. To this end, we provided climate scientists with access to climate data via the CESM-LE project  (Kay et al. , 2015). We contributed three additional ensemble members to the CESM-LE project and compressed and reconstructed an unspecified subset of the additional three members. To determine whether the effects of compression could be detected in the CESM-LE data, we then enlisted several scientists to attempt to identify which of the new members had undergone lossy compression by using an analysis technique of their choosing (i.
e. , we did not specify what analysis technique each should use). In addition, we provided a different group of scientists with both the original and reconstructed datasets and asked them to directly compare features particular to their interests (again, we did not specify how this analysis should be done) and determine whether the effects of compressing and reconstructing the data impacted climate features of interest. Indeed, a significant contribution of our work was enabling scientists to evaluate the effects of compression on any features of the data themselves with their own analysis tools (rather than relying solely on simple error metrics typically used in compression studies). Note that while the three additional CESM-LE ensemble members were generated at NCAR, the scientists participating in the ensemble data evaluations were from both NCAR and external institutions.
This paper is organized as follows. We first discuss background information in Sect. 2. In Sect. 3, we describe our approach to demonstrating the effects of lossy compression on climate science results. Then, in Sects. 4 and 5, we present selected results from data analyses evaluating compression effects in the context on the CESM-LE data. Finally, we summarize the lessons learned from this study in Sect. 6 and offer concluding thoughts in Sect. 7. A. H. Baker et al. :  Evaluating lossy data compression on climate simulation data 4383 2 Background In this section, we further discuss lossy data compression. We then provide additional details on the CESM-LE project datasets.
Data compression Compression techniques are classified as either lossless or lossy. Consider a dataset X that undergoes compression, resulting in the compressed dataset C (X ⇒ C). When the data are reconstructed, then C ⇒ X. If the compression technique is lossless, then the original data are exactly preserved: X = X. Note that the commonly used gzip compression utility is a lossless method. If, on the other hand, the compression technique is lossy, then X ≈ X ; the data are not exactly the same (e. g. ,  Sayood, 2012). Lossy compression methods generally give the user some control over the information loss via parameters that either control the compression rate, precision, or absolute or relative error bounds. The effectiveness of compression is generally measured by a compression ratio (CR), which is the ratio of the size of the compressed file to that of the original file (cf.
Iverson et al. , 2012) : CR(F ) = filesize(C) filesize(X ). (1) While lossless methods are often viewed as "safer" for scientific data, it is well known that lossless data compression of floating-point simulation data is difficult and often yields little benefit (e. g. ,  Lindstrom and Isenburg, 2006; Bicer et al. , 2013; Lakshminarasimhan et al. , 2011). The reason for the relative ineffectiveness of lossless methods on scientific data (in contrast to image or audio data, for example) is that trailing digits of the fixed-precision floating-point output data are often essentially random, depending on the data type and the number of physically significant digits. Random numbers are a liability for compression, thus giving lossy methods a significant advantage. Many recent efforts have focused on effectively applying or adapting lossy techniques for scientific datasets (e.
Nevertheless, the contrast is notable between the meteorological community's widespread use and acceptance of GRIB2 and the climate community's apparent reluctance to adopt lossy methods, even when proven to be safe, flexible and more effective. In this context, when applying lossy compression to scientific datasets, determining appropriate levels of precision or error, which result in only a negligible loss of information, is critical to acceptance. In summary, there are several salient points to recognize in the case for adopting lossy compression for climate simulation data. First, the least few significant bits of data are usually noise resulting from the fixed-precision rounding error and are not physically meaningful. Second, while 32 bit and 64 bit are meaningful data sizes for hardware, those sizes have no inherent relevance to a particular climate simulation.
The CESM Large Ensemble project dataset The CESM-LE project  (Kay et al. , 2015)  is a community project that includes a publicly available ensemble of climate model simulations generated for the purpose of studying internal climate variability. All data are currently available from the Earth System Grid website (http: //www. earthsystemgrid. org). The CESM-LE project is an ideal venue for this evaluation because of its use of climate ensembles, struggle with storage limitations, and availability to the broader climate community. The project began with a set of 30 ensemble members, each of which covers the period from 1920 to 2100. All simulations use the fully coupled 1 • latitude-longitude version of CESM-CAM5 (Com-munity Atmosphere Model version 5). Historical forcing is used for the period 1920-2005 and RCP8.
5 radiative forcing (i. e. , forcing that reflects near-past and future climate change; e. g. ,  Lamarque et al. , 2011)  thereafter. Ensemble spread is generated using small round-off level differences in the initial atmospheric temperature field. Comprehensive details on the experimental setup can be found in  Kay et al. (2015). CESM outputs raw data in NetCDF-formatted time-slice files, referred to as "history" files, for post-processing analysis. Sample rates (daily, monthly, etc. ) are determined for each variable by default, depending on the grid resolution, though a user can specify a custom frequency if desired. When the floating-point data are written to these history files, they are truncated from double precision (64 bits) to single precision (32 bits). For the CESM-LE project, monthly, daily, and 6-hourly history file outputs were converted and saved as a single-variable time series, requiring approximately 1.