dataesr/scientific-paragraphs-categorization huggingface dataset:
We only perform hyper-parameter tuning for the NER and QA tasks. See Appendix C for task-specific details. Bold text shows the best statistically significant score over 5 seeds. Question Answering. We evaluate our model on the FQuAD 1. 0 dataset  (d'Hoffschmidt et al. , 2020) , which is a SQuAD  (Rajpurkar et al. , 2016)  style French question-answering dataset with 20731 examples for training, and 3188 for evaluation. The results shown in Table  2  show that our model outperforms CamemBERT 30% by 6. 01 F1 points, but shows no statistically significant improvement over CamemBERT CCN et F1 score, and exact match (EM) score. Part-of-Speech and Dependency Parsing. We report our results on 4 diverse French treebanks. For the parser training, we make use of the HOPS parser  (Grobol and Crabbé, 2021)  implementation, which is a graph-based dependency parser inspired by  Dozat and Manning (2017)  tion to FastText embeddings  (Bojanowski et al.
FLUE Benchmark We use datasets from the French Language Understanding Evaluation (FLUE) benchmark  (Le et al. , 2020) , namely the French part of the paraphrase identification dataset PAWS-X  (Yang et al. , 2019) , and of XNLI  (Conneau et al. , 2018) , in addition to CLS, a binary classification dataset with Amazon reviews taken from Amazon. Our results (Table  3 ) show that our model outperforms all models on the CLS movie classification task, and matches the performance of CamemBERT CCN et on the other FLUE tasks.
Pre-training Dataset Choice We choose CCNet as our pre-training dataset instead of the more common OSCAR dataset  (Ortiz Suárez et al. , 2019) , as (i) it was shown to produce less offensive output  (Launay et al. , 2022)  and (ii) it allowed us to be fully comparable with many of the Camem- BERT models  (Martin et al. , 2020) , enabling thus meaningful comparisons. Nevertheless, we also ran experiments with CamemBERT OSCAR , and found that it performed slightly worse than CamemBERT CCN et , as shown in Table  5  Appendix A. Pre-training Compute and CO2 Impact Our model was trained for 8 days on 6 A40 GPUs, compared to CamemBERT which was trained on 256 V100 GPUs for one day, which is roughly equivalent to 28 days of training on 6 A40 GPUs, since an NVIDIA A40 GPU is about 1. 5x faster than a V100 GPU on language modeling tasks according to recent benchmarks.
8  Following the reports by Luccioni et al. (  2022 ) and  Cattan et al. (2022)  on the environmental impact of language model training, we use Lannelongue et al. 's (2021) online carbon footprint calculator to provide the following estimates: CAMEM-BERTA's pre-training used 700kWh and emitted 36kg CO 2 compared to 3. 32MWh and 170kg for CamemBERT. 9.
Taking a step back and looking at the only De-BERTa model that includes French, mDeBERTa  (He et al. , 2021a)  we can see (cf. Table  4 ) that our model only requires 6. 6% of its multilingual counterpart training samples to achieve competitive performance while additionally also outperforming the XLM-R model  (Conneau et al. , 2020)    This confirms the interest in using such training paradigms in compute limited scenarios for semantically demanding tasks such as question-answering or natural-language inference. Last but not least, other competitive language models for French are available and although not the primary focus of this paper, we conducted a comparative analysis involving FlauBERT  (Le et al. , 2020)  and FrALBERT  (Cattan et al. , 2021). The results, presented in Table  5  in Appendix A, demonstrate the better performance of our model across all evaluated tasks in comparison to these French models.
Conclusion We presented CAMEMBERTA, a data-efficient French language model trained on a large corpus of French text and the first publicly available DeBERTaV3-style pretrained model and implementation. For a fair evaluation we reused the same corpus and tokenizer as CamemBERT CCN et , but using only 30% of the total number of input training tokens. We compared the performance of both models in addition to an MLM model trained from scratch under the same setup as CAMEM-BERTA, CamemBERT 30% , on a variety of downstream tasks. Our experiments showed that our model outperforms CamemBERT 30% on all tasks except NER on FTB, and that it is able to match and even surpass CamemBERT CCN et. Furthermore, we have also made our optimized code implementation and pretrained model weights publicly available for others to use.
Ethics Statement We propose a model trained using DeBERTaV3 style pre-training along with an optimized training implementation, which reduces training computation cost when compared to previous models, and hence greatly reduces the energy cost and environmental impact of language model training. We trained our model using the CCNet dataset, for which we direct the reader to for further discussion on bias and ethical considerations. Our experiments do not include any additional data collection or human annotators. Like other language models trained on massive corpora, there may be potential biases present in the training data, which could affect the output of our models. Therefore, we advise against using these models in production without thorough testing. All our experiments were carried out on clusters with energy sources consisting of nuclear (65-75%), 20% renewable, and the remaining from gas.
Acknowledgements This work was partly funded by  Benoît Sagot 's chair in the  PRAIRIE institute  funded by the  French national reseach agency  (ANR as part of the " Investissements d'avenir" programme  under the reference  ANR-19-P3IA-0001. This work also received funding from the  European Union 's  Horizon 2020 research and innovation programme  under grant agreement No. 101021607. The authors are grateful to the OPAL infrastructure from  Université Côte d'Azur  for providing resources and support.
Habitat amount, temperature, and biotic interactions We inferred habitat amount using a combination of landcover data and remote-sensing-based indices (Table  S4 ). First, we estimated habitat amount using metrics based on land-cover data. Using the European Urban Atlas (EEA, 2010), we calculated the local patch size (total area of the study site) and patch isolation (using the proximity index, which weights the contribution of the area of neighbouring patches by the distance to the focal patch within a given search radius). We also assessed habitat amount by using the local landcover map developed by Alós-Ortí et al. (2022). This local landcover map distinguishes between vegetation and build-up types (grasslands, woody vegetation, and impervious surfaces within 32 m) and can be used an indirect proxy for habitat amount and particularly for food resource availability for bees and wasps.
At the trap-nest level, data loggers (1-Wire/Data Logger model DS1921G-F5, Analog Devices, Wilmington, MA, USA) recorded temperature hourly between February and June 2018, and we calculated the mean temperature per study site over this period (local temperature). We expect this scale of measurement to reflect the local thermal nesting conditions. Within the 800 m buffer (landscape temperature), we inferred temperature using remote sensing indices of land surface temperature (LST; Text S4). We expect this scale of measurement to reflect the thermal landscape surrounding our study sites. Additionally, we calculated LST for 50, 100, 200 and 400 m buffers, but ultimately did not use these estimates due to high inter-correlations (Pearson r > 0. 8). The distribution of values of the used predictors can be found in Figure  S1.
One site in Antwerp (An057) was not colonized and was excluded from the analyses, leading to a sample size of 79 sites. We checked collinearity among the predictors using the variance inflation factor (VIF) and the Pearson correlation coefficient. We discarded variables with VIF > 3 and a Pearson correlation coefficient > 0. 7  (Dormann et al. , 2013; Zuur et al. , 2010). Prior to the analyses, we standardized all predictors by z-transformation. We performed multimodel inference to study CNBW responses. We first formulated a global mixed model that included all fixed effects and the random structure using the R package glmmTMB. We.
ACKNOWLEDGEMENTS This research was supported by the  Swiss National Science Foundation  (project  31BD30_172467 ) within the  ERA-Net Bio-divERsA project  " BioVEINS: Connectivity of green and blue infrastructures: living veins for biodiverse and healthy cities " ( H2020 BiodivERsA32015104 ) and by the  Swiss Federal Office for the Environment (FOEN)  in the framework of the project " City4Bees " (contract no. 16. 0101. PJ/S284-0366 ). We also acknowledge  Göhner Stiftung  for supporting this project ( 2019-2917/1. 1 ). CA was supported by the  FCT  ( SFRH/BD/141822/2018 ). P. T. and Ł. M. acknowledge Polish funding through  NCN/2016/22/Z/NZ8/00004. PP  to  FCT  ( CEECIND/03415/2020 ,  DivRestore/001/2020 ). We thank the local authorities  of Zurich  (particularly  Grün Stadt Zurich [GSZ]) ,  Antwerp, Paris , Poznan and  Tartu  for supporting this study.
Data Sources/Measurements The online questionnaire was created with the software LimeSurvey. First, the age of the participants was asked, because they had to be at least 18 years old. Then, the questionnaire was divided into three categories in order to create patient group categories: COPD patients, PH patients and family members each patient group. Further questions were: demographic data, mobility, smoke status, suffering pressure. Most of the questions were designed as single choice questions. The question regarding their average level of suffering should be rated with a scale from one to six, with one being the lowest and six being the highest level of suffering. The suffering pressure is intended to represent the respondent's individual level of suffering, from low to high, in relation to their current condition.
Before evaluation, a plausibility check was performed to exclude out-of-norm records. By submitting the questionnaire, consent was given to anonymous data processing. In order to conduct semi-structured F2F interviews in which participants are encouraged to speak freely, pulmonologists and researchers developed an interview guide to increase the interviewer's flexibility. 13  Participants were asked about the average walking distance, feedback on the dummy of the portable oxygenator and their average level of suffering, which they were asked to indicate on a scale of one to five, with one being the lowest and five being the highest. In Germany, as soon as patients fulfilling the inclusion criteria were admitted as hospitalized, the treating physician informed the patient about the study and waited for consent.
Statistics The data of the online questionnaire were evaluated descriptively with the SPSS software (V26, 2019). P-values below 0. 05 were considered statistically significant. Continuous variables were reported as means ± standard deviation. Comparison between suffering pressure and acceptance of the device was performed with one-sided Fisher´s exact test. Therefore, values of suffering pressures between one and three were categorized as low and values between four and five as high. The data of the F2F interviews were evaluated following the Framework analysis by Green and Thorogood. 14  After familiarization with the data, identifying themes, data coding, organizing codes and themes the data were merged and mapped and interpreted using Microsoft Excel (Microsoft 2010).
Participants Online Questionnaire The link to the online questionnaire was sent by email to 99 self-help groups, 306 pulmologists, to all German professional associations of pulmologists and to 606 lung sports groups in Germany. During the collection period, the online questionnaire was accessed n=609 times. After a plausibility check of the given answers five records were removed, resulting in a total sample of 604 participants. Participant characteristics are provided in Table  1. Family members were not asked about their weight or height, thus reducing the total for the height and weight data by n=71 participants. Most of the questionnaire participants were female (55. 79%), living in Germany (54. 47%) and diagnosed with COPD (65,23%).
Due to the broad bandpass of the 50CORON mode, we estimate the conversion from counts per second on the detector to surface brightness by using the stsynphot package and the average spectrum of TW Hya obtained with STIS. We find that the conversion is 1 mJy arcsec -2 = 3. 54 DN s -1. We find that the flux profile of TW Hya is slightly different in 2021 compared to 2016, primarily fainter by 20% interior to 1″ as well as beyond 1 5.
A Fiducial Model for the TW Hya Disk The first step is to create a disk model that is broadly consistent with both the visible total-intensity scattered-light morphology and the SED measured for TW Hya without any shadows. To that end, we recreated the model proposed by VB17 with MCFOST. VB17 was focused on simultaneously fitting the polarized-intensity radial surface brightness profile of the disk NIR wavelengths with the TW Hya SED. This model was also derived from work done by  Menu et al. (2014) , which was a model focused on self-consistently fitting multiwavelength interferometry data with the TW Hya SED. Both models relied on a one-dimensional axisymmetric disk structure with a vertical scale height dictated by hydrostatic equilibrium. Both models assumed that the inner gas disk was truncated with a rounded inner taper and that the largest dust grains were decoupled from the gas radial density profile.
Radial Surface Density of Gas and Small Dust The previous two models of the TW Hya disk calculated an analytical radial surface density profile for the gas while assuming a gas-to-dust ratio for the small dust grains and calculated a self-consistent scale height for the disk as a function of radius. Our approach is to calculate a fully analytical gas density distribution with the following form: ⎜ ⎟ ⎛ ⎝ ⎞ ⎠ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ò r p = G S G - -¥ ¥ r z r r H r f r z H r dz , 2 exp 2 , 2 out in 2 2  where Γ in (r) is the interior density taper, Σ(r) is the surface density profile, f (r) is a multiplicative combination of density gaps to mimic the gaps observed in various scattered-light data sets, Γ out is the exterior density taper, and H(r) is the vertical scale height of the gas (VB17, and references therein).
Stellar Parameters The fundamental stellar parameters of TW Hya cover a wide range in mass and T eff , primarily because of the intrinsic difficulty of determining the parameters of young stars. For the purposes of the fiducial model, we assume similar values to those assumed in VB17, accounting for the larger distance to TW Hya determined by Gaia Data Release 3 (DR3; Gaia  Collaboration et al. 2021). In MCFOST, we chose T eff = 4000 K and g log  = 4. 0, assuming a mass of 0. 6 M e. These parameters are quite close to those determined by recent studies of TW Hya's temperature and gravity  (Sokal et al. 2018). They are also consistent with the inferred  M i sin  from CO measurements of Keplerian velocities  (Huang et al. 2018; Teague et al. 2018).
Additionally, the recent discovery of a similar shadow in submillimeter CO line emission  (Teague et al. 2022)  suggests that, instead of pure rigid rotation, the shadow(s) may oscillate in PA due to the mutual precession of the disks. Future observations will reveal whether the behavior of the shadow is truly periodic, and what that period relates to physically. If the shadow angular motions can be better disentangled, it could open up more powerful predictions for physical processes driving the shadow motions. The presence of multiple tilted rings is highly suggestive of multiple planetary companions in the innermost regions of the TW Hya disk. While planets have been searched for in NIR data at around 6 au by VB17, no candidates were found down to a few tens of Earth masses in the existing disk gaps, assuming the level of disk extinction and planet evolutionary state is well known.
Data collection and processing We monitored the sites for dissolved oxygen (DO; mg L -1 ) and stream temperature ( • C) for variable periods between July 2019 and October 2021, but not during winter (November -February; Fig. 1 ). At each site, DO and stream temperature were measured every 15 min with an in-situ sensor (HOBO U26-001, Onset Computer Corporation, Massachusetts, USA) instrumented with a copper anti-biofouling guard. At 12 sites in 2020 and 18 sites in 2021, we installed conductivity sensors (HOBO U24-001, Onset Computer Corporation, Massachusetts, USA). We cleaned DO and conductivity sensors with a toothbrush every-two weeks to remove biofouling. Prior to deployment, we lab-calibrated DO sensors with both 100 % water-saturated air and with sodium sulphite for 0 % saturation. Conductivity sensors were calibrated based on field measurements obtained with a calibrated handheld probe (Pro Plus, YSI Inc.
Storm, drying, and rewetting events We observed three environmental events conducive to hypoxia across our sites: 1) storms, 2) drying, and 3) rewetting. Each of these environmental events were identified according to the following definitions. First, we classified storm events as a doubling of baseflow  (Carter et al. , 2021) , with storms at least 24 h apart counting as separate events. We used the functions high. spells and baseflow (based on a Lyne-Hollick recurvise digital filter) from the hydrostats R package  (Bond, 2022)  to determine these events. Through sensitivity analyses, we determined these criteria to accurately and reproducibly capture each distinct event in the observed discharge time series. To determine storm effects on DO, we examined the three days before the storm peak flow and the week following the peak flow  (Carter et al.
e. , binary: hypoxic or oxic) with logistic regression and classification trees (R package rpart;  Therneau and Atkinson, 2022). Potential predictors of DO conditions measured included stream habitat of the DO sensor (pool, riffle, or run; visually assessed), hourly stream temperature, daily specific discharge of the catchment, reach slope, Strahler order, distance from the source, and altitude. Data were highly skewed towards oxic conditions, so we balanced the data with combined over-and under-sampling using the R package ROSE  (Lunardon et al. , 2014). We split the dataset into training (70 %) and testing (30 %) data for model building and validation, respectively.
, 2011). The GUTS toxicokinetic-toxicodynamic model, based on bioassay survival data, is regularly used in prospective hazard assessment of pesticides and fluctuating concentrations of toxic chemicals  (Ockleford et al. , 2018). GUTS quantifies mortality rate evolution based on the internal concentrations of hazardous compounds in organisms, which are controlled both by uptake rates and internal contaminant elimination rates  (Baudrot et al. , 2018). We adapted GUTS to include hypoxic stress instead of toxic stress by considering the DO deficit below an arbitrary value of 12 mg L -1 as the stressor input metric (instead of contaminant concentration). Using the web-interface MOSAIC (https://mosaic. un iv-lyon1. fr/guts)  (Charles et al. , 2018) , we calibrated the GUTS model with an experimental dataset of gammarid mortality in laboratory conditions under different constant levels of DO deficit (Recoura-  Massaquant et al.
, 2022). We used the reduced individual tolerance version of the model (GUTS-RED-IT), which assumes a log-logistic distribution of sensitivity threshold among individuals. The calibration experiment consisted of monitoring mortality over a five-day laboratory exposure of 300 male organisms of homogenous body size (~10 mm) to 10 constant nominal DO concentration conditions (8, 6, 5, 4, 3. 5, 3, 2. 5, 2, 1. 5 and mg L -1 , respectively), with three replicates of 10 individuals per concentration condition (Fig. S1 ). DO deficits were obtained by bubbling N gas through water columns. All detailed experimental data are available from the open access data repository Recherche Data Gouv  (Recoura-Massaquant et al. 2022). We then used the calibrated GUTS-RED-IT model to estimate potential mortality responses under observed hypoxic scenarios from three storm events and five drying events.
Acknowledgments This project was supported by The  Loire-Bretagne Water Agency  and the  Rhône-Méditerranée-Corse Water Agency. JD was also supported by the  EUR H2O'Lyon , French  ANR-17-EURE-0018  grant. JZ contributions were partially supported by  US NSF  awards ( 1846855 ,  1916567 ) and a fellowship from the  Collegium -Lyon, Institute for Advanced Study. We thank  Pierre Martinache  and  Nicolas Delorme  for technical support with lab tests and field work, and  Sandrine Charles  ( UMR 5558 Lyon  1) for her advice on modelling gammarid mortality. We also thank  Michèle Gailleton  and  Paul Courbière  for their very kind hospitality to access field sites and to develop experiments on their properties.
Numerical model DHRA follows a bottom-up approach to engineering built as an Excelbased tool. , It requires data from current building stocks and energy systems, climate data and the selection of DH generation technologies in order to calculate the techno-economic and environmental potential of DH in new markets. The energy demand is calculated based on data gathered from the building stock and individual technologies currently being utilized. This forms the baseline or business-as-usual (BAU) scenario. After identifying the total demand, desired DH generation technologies can be selected from a database. The model is then able to calculate the costs of developing the DH case and the potential savings in emissions of CO2, PM10 and PM2. 5 through comparison with the BAU case. Financial indicators such as net present value (NPV), internal rate of return (IRR) and payback time (PBT) are calculated to determine whether the project is economically attractive.
HDD and annual load profile An annual daily average temperature profile for Temuco is extracted from the national climate database and is then automatically applied to the analysis. The heating degree-days (HDD) are then calculated based on the annual daily temperature profile and indoor set temperature. The HDDs increase with the increases in the set temperature for heating. In the case of Temuco, HDDs are only calculated for days that fall under the heating season, which begins in April and ends in October. Additionally, HDDs are only calculated for days when the ambient temperature is lower than the set temperature. Total HDDs in a year, 𝐻𝐷 𝐷 is calculated by aggregating HDDs per day, 𝐻𝐷𝐷 𝑑 using equations 8 and 9 where 𝑇 is the set temperature and 𝑇 𝑑 is the average daily temperature. 𝐻𝐷𝐷 𝑑 = 𝑇 -𝑇 𝑑 [ 𝑑 𝑒𝑔𝑟𝑒𝑒𝑠 ∕𝑑 𝑎𝑦 ] ( 8 ) 𝐻 𝐷𝐷 = ∑ 𝐻 𝐷𝐷 𝑑 [ 𝑑 𝑒𝑔𝑟𝑒𝑒𝑠 ∕𝑑 𝑎𝑦 ] (9) Based on the HDDs, the demand calculated in the BAU case needs to be for the district and across the heating season.
These DH technology prices were borrowed from the District Heating Assessment Tool, DHAT, developed by the Danish Energy Agency, DEA, for the Danish market represented in  Table 6 [29]. This model was selected due to transparency in calculations and assumptions and constant update of the tool to reflect available best practices within the Danish DH experience. Hence, these values indicate the prices of such technologies for purchase in Denmark. To adjust the prices to the Chilean context, a purchasing power parity (PPP) rate is applied to convert Danish technology prices to the Chilean market. Updated PPP rates were obtained from the OECD statistical database and were found to be 52% for Chile (a 2007-2019 inflation rate and adjustment of 50% discount for new projects has been implemented while determining the PPP rate).