dataesr/scientific-paragraphs-categorization huggingface dataset:
As the maximum absolute value in 3 dataset is 145, the pw_relBoundRatio parameter in Sz is set to 0. 00345 and the number of significant digits nsd parameter in the Bit Grooming and in the Digit Rounding algorithm is set to 3 in Table  7. It can be observed in Table  7  that all three algorithms respect the relative error bound specified. However, on this dataset, Sz algorithm is twice too conservative. That is why, results obtained with pw_relBoundRatio = 0. 0069 are also provided in order to obtain a maximum absolute error of 0. 5. The compression ratio obtained with the Digit Rounding algorithm is higher than the one obtained with Sz.  For Sz algorithm, the pw_relBoundRatio parameter is successively set to 6. 9e-6, 6. 9e-5, 6. 9e-4, 6. 9e-3, 6. 9e-2, 6. 9e-1  For the Bit Grooming algorithm, the nsd parameter is successively set to 6, 5, 4, 3, 2, 1  For the Digit Rounding algorithm, the nsd parameter is successively set to 6, 5, 4, 3, 2, 1 As for the results reported in Table  7 , Zstandard with zstd_lvl = 5 has been applied after Sz and Shuffle and Zstandard with zstd_lvl = 5 has been applied after Bit Grooming and Digit Rounding algorithms.
For the dataset 3, the Bit Grooming algorithm provides better compression performance than Sz. Nevertheless, the Digit Rounding algorithms provides compression performance very closed to the one of the Bit Grooming algorithm. Figure  8  (right) compares the compression ratio obtained as a function of the parameter nsd, which is the user specified number of significant digit. As for dataset 1, we made the correspondence between the pw_relBoundRatio and the nsd parameters for the dataset 3 (i. e. pw_relBoundRatio = 6. 9e -nsd ) and plot the compression ratio obtained with Sz algorithm on the same figure. On the dataset 3, and whatever the nsd specified by the user, the compression ratio obtained with the Digit Rounding algorithm are higher than the compression ratio obtained with the Bit Grooming algorithm or Sz.
Application to a CFOSAT dataset The CFOSAT program is carried out through cooperation between French and Chinese Space Agencies  (CNES and CNSA respectively). CFOSAT aims at characterizing the ocean surfaces to better model and predict the ocean states, and improve the knowledge in ocean/atmosphere exchanges. The CFOSAT products will help for marine and weather forecast and for climate monitoring. The CFOSAT satellite will carry two scientific payloads: SCAT, a wind scatterometer, and SWIM, a wave scatterometer to allow a joint characterization of ocean surface winds and waves. The SWIM (Surface Wave Investigation and Monitoring) instrument delivered by CNES is dedicated to the measurement of the directional wave spectrum (density spectrum of wave slopes as a function of direction and wavenumber of the waves).
CFOSAT L1A product contains calibrated and geocoded waveform. Currently, the baseline for the compression of CFOSAT L1A product involves a "clipping" method as a data reduction step, the Shuffle preprocessing and Deflate lossless coding with a compression level dfl_lvl of 3. The compression with "clipping" is liken to a compression in an absolute error bounded mode. It defines the least significant digit (lsd) and "clips" the data to keep only lsd decimal digits. The lsd is defined specifically for each variable of the dataset. We study the performance of the following alternative compression methods:  CFOSAT "clipping" method followed by Shuffle and Zstandard with a compression level zstd_lvl of 1 or 2 to achieve favor compression speeds;  Bit Grooming (in the absolute error bounded compression mode) followed by Shuffle and Deflate or Zstandard.
Bit Grooming has been configured to keep the same number of decimal digits as CFOSAT "clipping" on each variable: nsd = lsd. Unfortunately, Sz crashes on the compression of CFOSAT or SWOT datasets. That is why, no results with Sz are provided in the following tables. The results for the compression of a CFOSAT L1A product of 7. 34GB (uncompressed) are provided in Table  8. Compared to the CFOSAT baseline compression, Zstandard increases the compression speed of about 40% while offering similar compression ratio. The use of Bit Grooming instead of CFOSAT "Clipping" method increases the compression ratio by a factor of 2 but decreases the compression speed by 40%. The decompression speeds are similar for all solutions. Our recommendation is thus to use the Bit Grooming algorithm with Zstandard coding rather than the CFOSAT "Clipping" method with Deflate coding to achieve high compression ratio on this CFOSAT dataset, at the price of a lower compression speed.
Application to SWOT datasets The Surface Water and Ocean Topography Mission (SWOT) is a partnership between NASA and CNES and continue the long history of altimetry missions with an innovative instrument: KaRin, a Ka band synthetic aperture radar. The launch is foreseen for 2021. SWOT addresses both oceanography and hydrology communities, measuring with a high accuracy water level of the ocean, rivers, and lakes. SWOT has two modes of processing and thus two different types of products are generated: the high resolution products, dedicated to hydrology and low resolution products mostly dedicated to oceanography. The Pixel Cloud product (called L2_HR_PIXC) contains data from the high-resolution (HR) mode of the KaRIn instrument. It contains information on the pixels that are detected as being over water.
This product is generated where the HR mask is turned on. The Pixel Cloud product is organized into sub-orbit tiles for each swath and each pass, and this is an intermediate product between the L1 Single Look Complex products and the L2 lake/river ones. The product granularity is a tile of 64km long in along-track and it covers either the left or the right swath (~60km wide). The compression performance is evaluated on two different datasets:  A simplified simulated SWOT L2_HR_PIXC pixel cloud product of 460MB (uncompressed);  A realistic and representative SWOT L2 pixel cloud dataset in which only few attributes may be missing of 199MB (uncompressed). The current baseline for the compression of the simplified simulated SWOT L2 pixel cloud product involves the Shuffle preprocessing and Deflate lossless coding with a compression level dfl_lvl of 4.
However, the compression method for the official SWOT L2 pixel cloud product has not yet been defined. We study the performance of the following compression methods:  Shuffle and Deflate;  Shuffle and Zstandard;  Bit Grooming (in the absolute error bounded compression mode) followed by Shuffle and Deflate or Zstandard.  Bit Grooming (in the relative error bounded compression mode) followed by Shuffle Zstandard;  Bit Grooming (in the relative error bounded compression mode) followed by Shuffle Zstandard;  Digit Rounding followed by Shuffle Zstandard. Bit Grooming and Digit Rounding have been configured on a per variable basis to keep the precision required by the scientists on each variable. It was not possible to evaluate the compression time needed to compress the datasets using the Digit Rounding algorithm because h5repack only allows defining filters parameters for a small number of variables.
The way around to compute the compression ratio has been to process each variable one after the other. Nevertheless, we observed similar speeds for the compression/decompression of the largest variable of this dataset using Bit Grooming algorithm in the relative error bounded mode or the Digit Rounding algorithm. The results for the compression of the simplified simulated SWOT L2 pixel cloud product are provided in Table  9. Compared to the SWOT prototype baseline compression, Zstandard increases more than 5 times the compression speed while offering similar compression ratio. The use of Bit Grooming in the absolute or relative error bounded mode, or the use of the Digit Rounding algorithm, increases the compression ratio by more than 30%, but divides by more than 3 the compression speed. The decompression speeds are similar for all solutions.
Our recommendation for the compression of this dataset is thus to use of Shuffle and Zstandard in lossless mode to achieve very high compression speed, or either the bitgrooming or the Digit Rounding algorithm to achieve slightly higher compression ratio at the price of lower compression speed. The results for the compression of the representative SWOT L2 pixel cloud product are provided in Table  10. Compared to Deflate, Zstandard increases more than 2. 5 times the compression speed while offering similar compression ratio. The use of the Bit Grooming algorithm in the absolute error-bounded mode increases more than 2 times the compression ratio but reduces the compression speed. The compression ratios obtained in the relative error bounded mode, either with the Bit Grooming algorithm or the Digit Rounding algorithms, are not as high.
Conclusions We have studied the performance of lossless and lossy compression algorithms both on synthetic datasets and on realistic simulated datasets of future scientific satellites. The compression methods have been executed using NetCDF-4 and HDF5 tools. It has been shown that for the lossless compression of scientific dataset the preprocessing by Shuffle of Bitshuffle is very helpful to increase the compression performance. The compression level options of Zstandard or Deflate have lower impacts on the compression ratio achieved but can significantly reduce the compression speed. Low compression levels are thus good choices to achieve high compression speed with satisfactory compression ratio. Zstandard can provide similar higher compression speed than Deflate or LZ4 with similar compression ratios.
However, on the three dimensional dataset, we have observed that Zstandard compression and decompression speeds are lower than the one obtained with Deflate. Depending on and Bit Grooming algorithms are competitive. Bit Grooming tends to provide higher SNR than Sz at low compression ratios while Sz tends to provide higher SNR than Bit Grooming at higher compression ratios. In the relative error bounded mode, the Digit Rounding algorithm introduced in this work provides higher compression efficiency than the Bit Grooming algorithm from which it derives. Moreover, it provides higher SNR than Sz in most cases. Extends to this work could be to modify the implementation of the HDF5 filter for Sz to allow configuring the data loss on a per variable basis or to adapt the NetCDF-4 library to allow the activation of other filters, not only Shuffle and deflate.
Code and data availability The Digit Rounding software source code and the data are currently only available upon request to Xavier Delaunay (xavier. delaunay@thalesgroup. com) or to Flavien Gouillon (Flavien. Gouillon@cnes. fr). Author contributions. Xavier Delaunay designed and implemented the Digit Rounding software and wrote most of the manuscript. Aurélie Courtois performed most of the compression experiments and generated the analytical datasets. Flavien Gouillon provided the scientific datasets used in the experiments, supervised the study, contributed to its design and to the writing of the manuscript. number of significant digits preserved in each sample.
Geosci. Model Dev. Discuss. , https://doi. org/10. 5194/gmd-2018-250 Manuscript under review for journal Geosci. Model Dev. Discussion started: 20 November 2018 c Author(s) 2018. CC BY 4. 0 License. The maximum absolute error  is defined by  = max  |̂ -  | where the   are the samples values of the original dataset and the ̂ are the samples values of the compressed dataset. An absolute error bound specifies the maximum absolute error   allowed between any sample of the original and compressed data:  ≤. The maximum relative error mre is defined by  = max  | ̂-    | A relative error bound specifies the maximum relative error   allowed between any sample of the original and compressed data:  ≤.
Geosci. Model Dev. Discuss. , https://doi. org/10. 5194/gmd-2018-250 Manuscript under review for journal Geosci. Model Dev. Discussion started: 20 November 2018 c Author(s) 2018. CC BY 4. 0 License. speeds are lower than the one obtained with Deflate. Deflate and Zstandard are thus both option to consider for the lossless compression of scientific dataset but always with the Shuffle or Bitshuffle preprocessing step.
Geosci. Model Dev. Discuss. , https://doi. org/10. 5194/gmd-2018-250 Manuscript under review for journal Geosci. Model Dev. Discussion started: 20 November 2018 c Author(s) 2018. CC BY 4. 0 License. the dataset, Deflate and Zstandard are thus both reasonable options to consider but always with Shuffle or Bitshuffle preprocessing step. Lossy compression of scientific datasets can be achieved in two different error bounded modes: the absolute and relative error bounded mode. Sz and Bit Grooming algorithms can work in both modes. In the absolute error bounded mode, both Sz.
Design and setting This was a retrospective analysis of routinely collected project data. Healthy Heart Africa (HHA) is an innovative programme created to tackle the rising burden of hypertension and cardiovascular diseases (CVD) in Africa  [20]. The programme aims to reach 10 million patients with hypertension in Africa by 2025. This will be achieved by awareness creation on symptoms and risk factors, supporting local health systems, and by offering screening and reduced-cost treatment. The programme is funded by AstraZeneca and has four local partners in Kenya implementing different models of care; Amref is one of these partners. In Kibera, the project is implemented in collaboration with the Ministry of Health in government-run health facilities and some private clinics. The Kibera Community Health Centre (KCHC), an Amref-sponsored clinic, is an example of a public-private partnership in health where some of the staff at the clinic are supported by Amref, while the rest are Ministry of Health (MOH) staff.
Patient population All patients with hypertension from three clinics, six walkway clinics and a weekend clinic in Kibera who were on antihypertensive medication and considered actively in care for at least 6 months during the study period (April 2015-December 2016) were included. Data variables, sources of data and data collection. The variables collected included: unique patient identifier, date enrolled into care, last visit, point of enrolment and point of care (health facility, walkway and weekend clinic), demographics (age, sex), hypertension risk factors (smoking, drinking, body mass index [BMI], diabetes, diet, physical inactivity, family history of CVDs) and HIV status. Information routinely recorded on each clinic visit included: date of clinic appointment, BP, selfreported adherence, medication regimen and next appointment date.
All patient data were recorded in patient files by clinicians and later keyed into an electronic database, International Quality Care (IQ Care), by Futures group  [22]. For this study, the primary data source was IQ Care. Data retrieval and cleaning took place between April and October 2017. Verification and comparison were carried out from patient files, the patients' master list and clinic appointment books. Data were stratified by health facility, walkway and weekend for analysis. Compliance with scheduled clinic visits was defined as a proportion of visits attended against those scheduled. Compliance was defined as when a patient came for a visit not more than 90 days before the scheduled appointment for an early visit and not later than 7 days after the appointment date. Adherence to medication was selfreported as a "yes" or "no" and only calculated for those who attended a clinic visit.
Analysis To determine attendance to clinic visits and self-reported adherence to medication, we analysed patients who had been in the programme for more than 6 months and were active between April 2015 and December 2016. Data were downloaded to Excel and exported to STATA Version 12 for analysis. Data were summarised using ranges, means, AE standard deviations (SDs) and frequencies (percentages) where appropriate. Compliance with clinic visits and adherence was compared between models using the chi-square test. Univariate and multivariate logistic regression models were used to estimate the odds of overall adherence to medication. Sex and age of the patient at enrolment were a priori included in the multivariable model while other risk factors for hypertension were not included because they were not significant at P < 0.
Acknowledgements This research was conducted through the  Structured Operational Research and Training Initiative (SORT IT ), a global partnership led by the  Special Programme for Research and Training in Tropical Diseases at WHO/ TDR. The training model is based on a course developed jointly by the  International Union Against Tuberculosis and Lung Disease  ( The Union ) and Med ecins sans Fronti eres (MSF). The specific  SORT IT program  which resulted in this publication was implemented by: Med ecins  Sans Fronti eres, Brussels Operational Center, Luxembourg  and the  Centre for Operational Research, The Union, Paris, France. Mentorship and the coordination/facilitation of these SORT IT workshops were provided through the  Centre for Operational Research, The Union, Paris, France ; the  Operational Research Unit (LuxOR) ; AMPATH, Eldoret, Kenya; The  Institute of Tropical Medicine, Antwerp, Belgium; The Centre for International Health, University of Bergen, Norway ; and  The National Institute for Medical Research ,  Muhimbili Medical Research Centre , Dar es Salaam, Tanzania.
In the laboratory The following measures were obtained: -green weights of twigs and leaves measured separately for each sample collected in the field, -area of leaves for each sample by using a scanner and ImageJ (http://rsb. info. nih. gov/ij/) software which gives individual leaf area values and total leaf sample area, -green weights of wood and bark for a sub-sample of branch discs, -ring widths in 4 perpendicular directions for the last 6 years on each branch disc, -dry weights of branch discs (separately for wood and bark for the sub-samples), of twigs and leaves samples (separately) and of beechnuts, after drying at 105°C for 48 hours. 2. 6 Adaptations to stem and branch measurements 2. 6. 1 Large trees Several adaptations to the protocol of stem and branch measurements were made in the case of large trees (samples #3 & #5: see Table  1 ).
• model fitting Two procedures were used for fitting Eq. 2 and then compared. First, the fitting of Eq. 2 was done after a "both sides" logarithmic transformation allowing the use of linear regression; second, Eq. 2 was fitted directly using a non-linear regression procedure. In the first procedure, the log-transformation of both sides of Eq. 2 gives: LogW = Logβ + γ * Log D 2 * Ht ( ) or : LogW = λ + γ * Log D 2 * Ht ( ) (with λ = Logβ ) (3) Eq. 3 was fitted using the software Data Desk 6. 3  (Velleman, 2011)  on a Mac OS 10 system, to total above ground biomass for each age class, after trees were sorted into 4 age classes (Table  3 ). An analysis of the variation of each parameter (λ, γ) with age (Age) and relative crown length (RCL) 9  was then performed, λ and γ being linearly related to RCL and to the inverse of Age.
The following equation, derived from Eq. 3 and including Age and RCL variables, was then fitted to biomass data: LogW = a 0 + a 1 1 Age + a 2 RCL ( ) + b 0 + b 1 1 Age + b 2 RCL ( ) * Log D 2 Ht ( ) (4) Equation (  4 ) was fitted with Datadesk 6. 3 to each above (stem and branches) and belowground (coarse, small and fine roots) tree compartment and to total above and total belowground biomasses -with possible adaptations to Eq. 4 -and the residuals of this equation were examined to detect any bias or remaining tree or stand effect. In the second followed procedure, Equations (  1 ) or (2) were fitted as non-linear models, using the R project for statistical computing (  2009 ), with parameters dependent more specifically on tree age  (Genet & al, 2011). The fitted equation was the following: W = a 0 + a + b * Age + c * exp d* Age ( ) * D 2 * Ht ( ) e+ f * Age ( ) (5) Equation (  5 ) was fitted to each above (stem and branches) and belowground (coarse, small and fine roots) tree compartment and for total above and total belowground biomasses with the statistical package nls of R (2009), and the residuals were examined as for the first procedure.
As height (HT) is correlated with diameter (D) at a given age, a height-diameter equation with age-dependent parameters was fitted to sampled trees data. The following power model was retained after fitting separate equations to data from different age classes and analyzing the variation of equation parameters with age: (10) In this equation, the parameter a appeared to be dependent on forest plot. Then, Eq. 10 was fitted with nlme (R), fixed and random effects appearing in Table  8. Plotting residuals against fitted values did not reveal any bias. HT = a + bAge ( )D cAge d ( ) Biomass distribution in trees was then represented in relation to diameter at breast height and age, replacing tree height (HT) in biomass equations by its estimation obtained from Eq. 10. In Fig. 7 , the contributions of the main above and belowground compartments to tree biomass is represented, while in Fig.
When linear or non-linear models could equally be used, notably after log-transformation, the non-linear ones revealed better suited when looking at the residuals. When fitting the relations, a particular attention was brought to the unexplained variation and to the quality of the fitting in order to obtain an eventually better fit. Variables not introduced in the models but characterizing the status of sampled trees -tree social status, year of tree sampling, forest plot belonging -were retained for the analysis of residuals and eventually added in the models as random effects. In this last case, mixed models were fitted using the package nlme of R. It appeared that such random effects were significant for yearly varying biomass variables (foliar biomass, biomass increment of aboveground tree compartments): in this case, it could be interesting to try to identify these effects, probably related to the yearling varying climate and particularly its consequences on the water availability for trees as it is a limiting factor of the growth of beech (Le  Goff & Ottorini 1999; Granier & al, 2008).
Acknowledgments Our thanks go the  INRA  technicians of the  LERFoB research unit at INRA-Nancy , who proceeded to the tree measurements:  R. Canta ,  F. Bordat ,  G. Maréchal  and  S. Daviller. Special thanks are addressed to  R. Canta  who supervised the field and laboratory measurements and developed a specific apparatus to allow the biomass measurements on the large root systems. The support of the ONF section of Sarrebourg (57), greatly appreciated, made possible the felling of the tree samples used in this study. We would like also to thank warmly  A. Granier  ( EEF, INRA-Nancy ) for his interest in the biomass studies that we conducted in the Hesse forest over several years, in parallel with the ecophysiological studies that he conducted himself, and for his comments on the first draft of the manuscript.
Funding This work was supported by grants from  ONF (French National Forest Service)  and from the  GIP "ECOFOR"  (a French public benefit corporation on  FORest ECOsystems ). The  UMR   Silva , regrouping the previous LERFoB and EEF research units, is supported by the  French National research Agency (ANR)  through the  Laboratory of Excellence ARBRE  ( ANR-11-LABX-0002-01 ).