dataesr/scientific-paragraphs-categorization huggingface dataset:
We tested the effects of applying different combinations of filters to the citizen data and of complementing it with additional standardized fieldwork. Cross-validation with an internal evaluation dataset resulted in higher AUC (Area Under the receiver operating Curve) than external evaluation causing overestimation of model accuracy and did not select the same models; models integrating sampling effort performed better with external validation. AUC, specificity and sensitivity of models calculated with different filtered external datasets differed for some species. However, for most species, complementary fieldwork was not necessary to obtain coherent results, as long as the citizen science data was strongly filtered. Since external validation methods using independent data are considered more robust, filtering data from citizen sciences may make a valuable contribution to the assessment of SDM.
, 2017). However, quality of data from participatory sciences is heterogeneous and different methods have been developed to boost data accuracy and account for bias, including interactive project development, volunteer training, expert data validation and statistical modelling improvement  (Kosmala et al. , 2016). Although researchers have been skeptical about the value of datasets from citizen science, recent publications show that some could be as valid as data collected by professional scientists  (Kosmala et al. , 2016). This is conditional on such data being judged in context (i. e. according to the sampling methods used, program objectives and applications) on the use of rigorous data sorting and analyses  (Isaac et al. , 2014; Steen, Elphick & Tingley, 2019; Robinson et al. , 2020).
Opportunistic presence-only data collected by citizens at large scales have contributed to the expansion of species distribution models (SDM) over the past twenty years, particularly for biological conservation applications  (Guisan & Thuiller, 2005). The validity of presence-only SDM is however increasingly questioned as well as their applicability  (Barve et al. , 2011). Presence-only data come from different source databases reduced to simple species presence records and mostly collected in an unstandardized way by volunteers. In contrast to presence-absence data, they are abundant but have poor quality, few metadata and come from different sources  (Robinson et al. , 2020). This introduces numerous sources of bias that need to be assessed and accounted for in modelling processes  (Phillips et al.
Currently, testing model accuracy with a fully independent dataset is considered the most robust method for assessing SDM  (Araujo et al. , 2005; Guisan, Thuiller & Zimmermann, 2017). However, obtaining an external data set for large-scale studies is often cost prohibitive and exploiting data from standardized citizen science programmes may in some cases provide the solution. For example,  Robinson et al. 2020  have shown that using filtered large-scale citizen science data for SDM calibration can improve model accuracy. Alternatively, detection-nondetection data from more standardized citizen sciences programs which are rarer than opportunistic data but have higher quality could provide presence-absence sets for external validation of presence-only SDM. In addition, using presence-only and presence-absence data at different stages of the modelling process could be a method for combining different data sets with heterogeneous quality which is a current challenge to improve SDM validity  (Zipkin & Saunders, 2018; Robinson et al.
, 2016). Despite abundant literature on amphibian ecology and the availability of several citizen science databases, few studies have attempted predictive amphibian distribution models at large scales  (Graham & Hijmans, 2006; Brown et al. , 2016; Préau et al. , 2019). Therefore, amphibian data could be suitable for testing the capacity of different types of citizen data (presence-absence or opportunist) for calibrating and assessing SDM. Here we compare the predictive performance of presence-only SDM for nine amphibian species using different types of data (internal presence-only or external presence-absence) from citizen science programs for their assessment. We also test the opportunity to use filtered standardized citizen science data to constitute the independent data set for external evaluation.
Biological data We studied habitat suitability of 9 amphibian species: Bufo spinosus, Hyla arborea, Rana dalmatina, Rana temporaria, Triturus cristatus, Triturus marmoratus, Lissotriton helveticus, Salamandra Salamandra, and Pelodytes punctatus. Two types of amphibian data were used: (1) opportunistic data from a citizen database with presence-only records for model calibration and internal evaluation (2) standardized detectionnondetection data from a citizen science programme and complementary field work for external evaluations. A more detailed description of the data sets and complementation strategies is available in Appendix 1.
Opportunistic presence data (calibration and cross-validation dataset) We accessed presence-only occurrences from a regional database for the period 2013-2019. 86% of the dataset was collected by citizens and recorded online (website or associated mobile application) and 14% by various professional organisations involved in nature protection. All data were compiled for the regional Atlas of amphibians by a French non-governmental organisation (French BirdLife partner -LPO). See Appendix 1 Table  1  for data sources. We retained only species with enough data according to number of predictors used (i. e. at least 477 presence cells; see Appendix 1 Table  2 ). We selected only precise GPS records (precision of the observation under 50 meters) and we checked all data for anomalies in geographical location or species identification.
Standardised detection-nondetection data (external validation datasets) For external validation, we firstly extracted detection-nondetection amphibian data for the period 2013-2019 collected as part of a citizen science program called "Un Dragon dans mon Jardin" (Appendix 1 section 1. 2. ). We retained 576 sites which were monitored at least 3 times between February and June during at least one year and following a standard method commonly used for amphibian community surveys  (Boissinot, Besnard & Lourdais, 2019). We called this dataset CS. 0 (see Figure  1 ). Some large areas of the region were not sampled due to lack of observers so that data were clustered near cities, with spatial autocorrelation. Therefore, with help from volunteers, we completed this dataset with some additional fieldwork and applied filters.
To complete and filter CS. 0, different strategies were used. Firstly, we organised complementary fieldwork in 2018 and 2019 to complete two landscape gradients (woody element density and pond density) which are two variables known to strongly affect amphibian distribution and which are relevant in our regional context. All sites were selected randomly but so as to maximise and decorrelate the two landscape gradients in different areas (see Appendix 1 section 1. 3. ). In total, 263 sites were monitored: 132 sites by experts in 2018-2019 (called PRO, see Figure  1 ) and 131 by 75 volunteers in 2019 (called VOL, see Figure  1 ). All data (CS. 0, PRO and VOL) were projected on the same 500m resolution grid. One further problem, common in citizen science programmes  (Geldmann et al. , 2016) , is that only aquatic sites are surveyed while areas known to be very unsuitable for amphibians such as urban areas and intensive agriculture are generally excluded.
So, if species A with the same detectability as species B is detected at a site, then species B is likely to be truly absent (see Appendix 1 section 1. 5. for target species list); (4) stratified sampling on final prediction maps (see STRAT_CS and STRAT_ALL in Figure  1 ). We established independent datasets, stratified by model predictions for each species  (Newbold et al. , 2010; Guisan, Thuiller & Zimmermann, 2017)  with equal random sampling of validation cells with presence or absence data in 4 habitat suitability classes predicted (i. e. [0;  0. 3[, [0. 3;0. 5[, [0. 5;0. 7[ and [0. 7;1] ). We obtained an equal number of validation cells by predicted suitability class (see 1. 4. 3. for predictive map used).
Environmental dataset and variables We assembled environmental data relevant to amphibian ecology and of importance in the study region  (Guisan, Thuiller & Zimmermann, 2017). A more detailed description of variables with associate references is available in Appendix 2 Table  1. Bioclimatic variables were accessed from a compilation of climate data for the period 1950-2000 at a spatial resolution of 5km²  (Hijmans et al. , 2005 ). An altitude variable was derived from the U. S. Geological Survey's Hydro-K data set, at the same spatial resolution. We performed a principal-components analysis (PCA) on 11 bioclimatic variables relevant for amphibians and the altitudinal layer to produce 2 uncorrelated axes (see Appendix 2 Table  4  and Figure  2 ). Land cover data were downloaded from the highly detailed vector database OCS GE 1.
1 (IGN 2019), the Theia OSO Land Cover Map 2017 (available at www. theia-land. fr) and from BDTopo (IGN 2019). This was coupled with a more detailed regional inventory of hedgerows (from 2005 to 2008) and ponds (2012) and a national farming database from the EU LPIS (Land Parcel Identification System 2016) used to classify agricultural areas (see Table  1 ). We calculated land-cover variables in windows composed of a 500m grid cell with a buffer of 300m (see Table  1 ). This took into account landscape context based on species' dispersal capacities as well as the resolution of the species data set  (Guisan & Thuiller, 2005). Distance and home range differ among amphibian species but a 1km circle may be accepted as an average maximum range  (Collins & Fahrig, 2017). Collins and Fahrig (2017)  and  Boissinot et al.
Statistical models Different modelling algorithms can lead to varying results according to heterogeneous sensitivities and calculation processes  (Thuiller et al. , 2009). Therefore, consensus models based on multi-modelling approaches (ensemble-modelling) can improve final results by reducing 'noise' associated with individual model errors  (Araujo et al. , 2005; Thuiller et al. , 2009; Meller et al. , 2014). For each species, we used one regression-based approach (Generalized Additive Models, GAM) and one machine learning algorithm (Random Forest, RF) to predict and assess habitat suitability within the studied region with 50 bootstrap replicates. Presence points were randomly split 50 times into a training set (70% of the whole dataset) and the remaining 30% were used as testing set for internal evaluation (see 1.
The latter strategy aimed to select pseudo-absences where true absences were more likely. For this strategy we considered three main sources of bias in pseudo-absence selection: accessibility, linked to distance from roads or urban areas  (Kadmon, Farber & Danin, 2004; Barbet-Massin et al. , 2012) , attractiveness, relating to oversampling in protected sites or tourist areas  (Phillips et al. , 2009; Robinson, Ruiz-Gutierrez & Fink, 2018)  and observer effort, because certain administrative areas are covered by particularly active nature protection organisations (see Appendix 3). For each strategy, the number of artificial absences was fixed equal to the number of presence data (Barbet-Massin et al. , 2012; Liu, Newell & White, 2019) and we performed 10 replicates of the artificial absence generation processes.
Internal and external model validation We firstly use a cross-validation method using a 30% random split of the whole set to asses each model (for pseudo-absence selection strategies s1, s2 and s3) with 50 bootstraps repeated 10 times. calculated the area under the curve (AUC) of a receiver operating characteristic (ROC) plot of the predicted model habitat suitability scores with (1) the 30% test dataset for internal validation and (2) with the larger filtered external independent dataset (e. g. CS. 2+ABS+SUP, see Figure  1 ) using Biomod2 package  (Thuiller et al. , 2009). AUC is the most common metric used in SDM studies, as it has the advantage of being threshold and prevalence independent and has been accepted as the standard measure for assessing SDM accuracy  (Guisan, Thuiller & Zimmermann, 2017).
AUC > 0. 50 signifies that the model has better prediction than a random model. Secondly, we calculated AUC values, specificity (true negative rate) and sensitivity (true positive rate) of ensemble models calibrated with 100% of the presence-only data using different evaluation sets derived from the global external dataset used in the previous stage (see Figure  1 ). These calculations (with 100 bootstraps) were performed using PresenceAbsence package (Freeman 2012) with a standard threshold value for presence-absence discrimination fixed at 0. 5.
Comparison of external evaluation sets Values of AUC, sensitivity and specificity to four species are shown in Table  2  (two Anurans and two Urodeles; one forest species and one generalist specie each). Results for other species and CS. 1 (similar to CS. 2) are presented in Appendix 4 Table  3. Considering AUC values, evaluation with the external dataset from participative science without filter data (CS. 0) show more similar model selection results than internal crossvalidation except for R. temporaria. Sorting presence data led to decreased sensitivity and increased specificity for all species except for S. Salamandra. The models selected (s2 or s3) were similar for most species whether using stratified data from volunteers' only (STRAT_CS) or stratified data with added professional observations (STRAT_ALL), or professional data only (PRO).
See Table  2  and Appendix 4. We excluded the s1 model from the comparison because this model is never selected, either with internal or external evaluation. Table  2. Model performance according to different filters and complementary fieldwork applied to the external evaluation dataset. External datasets used were (see Figure  1 ): CS. 0 (all data from the standardized citizen science dataset); CS. 2 + ABS (CS. 2 with 10% supplementary absence cells in very unsuitable habitats); PRO (data collected by professionals only in 2018-2019); CS. 2 + ABS + SUP (citizen science data cited before adding all complementary fieldwork by professionals and volunteers); STRAT_CS (stratified data selection from CS. 2+ABS with complementary fieldwork by volunteers); STRAT_ALL (stratified data selection from CS.
Brotons et al. 2004  highlighted the difficulty of predicting distributions of the most generalist species. However, for such species, the use of filters increases specificity values considerably and the results are coherent with these species' wide ranging and ubiquitous distributions. Using external presence-absence data also makes it possible to exploit the whole presence-only data set for calibration and to use stricter filters to reduce sampling bias or data culling to retain higher quality data  (Steen et al. 2019 , Isaak et al. 2014). Our study shows that it is possible to apply strong filters (e. g. STRAT_CS) but finally retain reasonable sample sizes for most species. It should be noted that, for four species, R. temporaria, T. marmoratus, S. salamandra and P. punctatus, our results were ambiguous.
For the first three, all forest-dwelling species or very closely related to woodlands  (Boissinot, Besnard & Lourdais, 2019) , both internal and external validation methods selected models with sampling effort integrated (s3). Two main reasons could explain these results: firstly, presence data may have been insufficient for R. temporaria, and secondly these species' affinity for forest habitats. R. temporaria is a rare species and is more dependent on wet forest, flood meadows and small streams as breeding sites than other species  (Boissinot et al. , 2015). This species has a patchy distribution (i. e. locally abundant but regionally rare) and is difficult to detect. Hence, presence data are few in the studied area both in the opportunist dataset (N=477 presence-cells) and in the independent dataset (N=13 presence-cells).
As highlighted by  Vaughan & Ormerod (2005)  such factors can lead to model over fit, even with a relatively small number of variables, resulting in high AUC values. In addition, according to  Brotons et al. (2004) , low-density habitat (i. e. forest habitat in our region) may be over-weighted and it can be difficult to assess between good or bad suitability without adapted presence-absence data. Monitored forest sites are rare in our validation dataset and the assumptions we used to define sampling effort may not be well adapted for forest specialists. Finally, P. punctatus is a rare species but abundant on the Atlantic coast. Unlike the other species, it is a pioneer, adapted to open areas, especially primary unvegetated habitats such as sand dunes and mudflats with frequent physical disturbance  (Joly et al.
Using heterogeneous data from citizen science in SDM Our results show that it is possible to obtain useful external and independent datasets for model validation from filtered standardized citizen science data. Indeed, the use of filters have successfully reduce bias and noise in citizen science data sets for SDM in others studies  (Robinson, Ruiz-Gutierrez & Fink, 2018; Isaac et al. , 2019; Steen, Elphick & Tingley, 2019). In addition, filtered evaluation dataset showed coherent results according to  Phillips et al. 2009. Indeed, choosing pseudo-absence data with the same bias as occurrence data improved model performance. Since external independent data is necessary for more robust assessment of SDM  (Araujo et al. , 2005) , but prohibitive to collect, filtering low quality but large datasets from monitoring to obtain more standard and independent data may be worthwhile.
In addition, AUC appears to be more informative when presenceabsence data is used to assess and compare models than when presence-background data alone is used (Jiménez-Valverde, 2012). However, using detection-nondetection citizen data without filters may also lead to erroneous results because of overlapping sources of bias in both datasets (e. g. CS. 0 selects the same model as cross-validation). The large amount of available data allows us to strongly select data according to our research objective. Our results using PRO datasets are inconclusive for rare species perhaps because their detection was insufficiently frequent (e. g. only two observations of R. temporaria for 132 sampled sites). Finally, we found that general rules to guide data sorting were difficult to define. Our results were sensitive to the type of data used, and the species studied, reinforcing the need to define filters according to available data and species' ecology  (Steen, Elphick & Tingley, 2019).
Independence between training and testing sets is an essential criterion, but data should also be unbiased or corrected. Selection methods have been developed to try to divide the opportunistic dataset strategically to increase the independence of the testing set for cross-validation (see  Block-cross-validation in Robert et al. 2017 ). However, this method does not make it possible to escape from the general biases linked to sampling effort and/or can create extrapolation problems (see  Robert et al. 2017). Using a more standardized dataset from a participatory science program (e. g. CS0) for the evaluation makes it easier to understand the sources of bias (presence of metadata and non-detection data), to better control them and to obtain more robust information on the absence data. However, these data may also share biases with the opportunistic dataset used for calibration.
In our case, the sampling of the monitored sites (CS0) was partly biased towards volunteers' place of residence and areas with a higher density of observers. These biases were reduced through additional sampling involving volunteers. Our results show that certain filters, as well as targeted complementary fieldwork, it possible to reduce the biases identified and produce conclusive results. In addition, the use of a stratified sampling of the testing set along the suitability gradient from the SDM results (e. g. our STRAT_CS and STRAT_ALL datasets) appears to be a particularly interesting method showing stable and consistent results according to Phillips et al 2009. However, our method may be not applicable in all cases. In our study, external data came from a program with general population monitoring objectives, using standard methods designed to be accessible to a wide audience (eg.
Acknowledgements This work would not have been possible without the support of the  Pays de la Loire Herpetological Group , the  CPIE Regional Union  and the  French BirdLife International partner (LPO). We are especially grateful to  Benoit Marchadour  ( LPO ) who coordinates regional distribution atlas. We also acknowledge the many naturalists involved, for access to data and for additional fieldwork, especially  Dorian Angot ,  Anne-Lise Charpentier ,  Baptiste Gaboriau ,  Ludovic Aubry ,  Jeanne Abbou ,  Flore Gamet ,  François Varenne ,  Adrien Maitrepierre ,  Leslie Michal , and  Martin Bonhomme. We thank  Xavier Lozupone  for his help for mapping of environmental variables at regional level and  Véronique Beaujouan ,  Adeline Bulot ,  Hervé Daniel ,  Gilles Martel  and  Aurélien Besnard  for providing helpful discussion.
The investigation conforms to the principles outlined in the Declaration of Helsinki. All parameters are presented as means ± standard deviation or counts (percentages) when appropriate. All variables were tested for normal distribution of data with the Shapiro-Wilk test. Comparisons of continuous parameters between patients with and without HF were conducted with t-tests when normality was confirmed, or otherwise, with the Mann-Whitney test; the c 2 test was performed for the comparison of qualitative parameters. All results were considered statistically significant when their p-value was < 0. 05. The Statistica package, version 13. 0 (StatSoft, TIBCO Software Inc. ), was used for the statistical analysis. Out of 52 patients 50 (96%) received empagliflozin and 2 (4%) dapagliflozin. Most patients (40; 77%) were also treated with metformin, 15 (29%) received insulin and 10 (19%) sulphonylurea.
| MATERIAL S AND ME THODS Our first national survey of PLHIV took place on the week of 12-18 October 2016. All centers which had been informed beforehand by the anti-HIV regional coordination task group (COREVIH), supervisors and the pharmacists' branch of the Regional Union of Health Professionals (URPS), and which agreed to participate were recruited. All the consecutive PLHIV under ARV treatment admitted in clinical units or hospital pharmacies who agreed to participate were offered to answer the questionnaire, unless they were minors or under guardianship. Those who accepted answered anonymous selfquestionnaires collecting demographic and medical data (the latter mostly comprised their biological results available on site), current dispensing patterns for all drugs, co-prescriptions and self-medication, their views on the medication file and medication interviews, and any requests they might have.