dataesr/scientific-paragraphs-categorization huggingface dataset:
We use the mean frequency of the 4,316 relational adjectives in Google Ngrams as a threshold. 5 We propose a train/dev/test split of the SCAL-REL dataset (65/10/25%), observing a balance between the two classes (scalar and relational) in each set. To obtain contextualised representations, we collect for each relational adjective ten random sentences from ukWaC. For scalar adjectives, we use the ukWaC-Random set of sentences (cf. Section 2. 1). 3 Multilingual Scalar Adjective Ranking.
Methodology Models We conduct experiments with state-ofthe-art contextual language models and several baselines on the MULTI-SCALE dataset. We use the pre-trained cased and uncased multilingual BERT model  (Devlin et al. , 2019)  and report results of the best variant for each language. We also report results obtained with four monolingual models: bert-base-uncased  (Devlin et al. , 2019) , flaubert_base_uncased  (Le et al. , 2020) , bert-base-spanish-wwmuncased  (Cañete et al. , 2020) , and bert-basegreek-uncased-v1  (Koutsikakis et al. , 2020). We compare to results obtained using fastText static embeddings in each language  (Grave et al. , 2018). For a scale s, we feed the corresponding set of sentences to a model and extract the contextualised representations for ∀ a ∈ s from every layer.
When an adjective is split into multiple BPE units, we average the representations of all wordpieces (we call this approach "WP") or all pieces but the last one ("WP-1"). The intuition behind excluding the last WP is that the ending of a word often corresponds to a suffix with morphological information. The DIFFVEC method We apply the adjective ranking method proposed by Garí Soler and Apidianaki (2020) to our dataset, which relies on an intensity vector (called ---→ dV ec) built from BERT representations. The method yields state-of-the art results with very little data; this makes it easily adaptable to new languages. We build a sentence specific intensity representation ( ---→ dV ec) by subtracting the vector of a mild intensity adjective, a mild (e. g. , smart), from that of a ext , an extreme adjective on the same scale (e.
g. , brilliant) in the same context. 5 Nine scalar adjectives from our datasets are also annotated as pertainyms in WordNet (e. g. , skinny, microscopic) because they are denominal. We consider these adjectives to be scalar for our purposes since they clearly belong to intensity scales. We create a dV ec representation from every sentence available for these two reference adjectives, and average them to obtain the global ---→ dV ec for that pair. Garí Soler and Apidianaki (2020) showed that a single positive adjective pair (DIFFVEC-1 (+)) is enough for obtaining highly competitive results in English. We apply this method to the other languages using the translations of a positive English (a mild , a ext ) pair from the CROWD dataset: perfect-good. 6 dditionally, we learn two dataset specific representations: one by averaging the ---→ dV ec's of all (a ext , a mild ) pairs in WILKINSON that do not appear in DEMELO (DIFFVEC-WK), and another one from pairs in DEMELO that are not in WILKINSON (DIFFVEC-DM).
This is captured by the SENSE baseline which orders the words according to their number of senses: Words with more senses have lower intensity. Frequency is taken from Google Ngrams for English, and from OSCAR for the other three languages. The number of senses is retrieved from WordNet for English, and from BabelNet (Navigli and Ponzetto, 2012) for Spanish and French. 7 or adjectives that are not present in BabelNet, we use a default value which corresponds to the average number of senses for adjectives in the dataset (DEMELO or WILKINSON) for which this information is available. We omit the SENSE baseline for  Greek due to low coverage. 8.
Tokenisers of the monolingual models instead tend to split words in a way that more closely reflects the morphology of the language (e. g. , ES: fantástico → fantás-tico; EL: γιγάντιος→γιγά-ντι-ος. Detailed results are found in Appendix A. We observe that DIFFVEC-1 (+) yields comparable and sometimes better results than DIFFVEC-DM and DIFFVEC-WK, which are built from multiple pairs. This is important especially in the multilingual setting, since it shows that just one pair of adjectives is enough for obtaining good results in a new language. The best layer varies across models and configurations. The monolingual French and Greek models generally obtain best results in earlier layers. A similar behaviour is observed for the multilingual model for English to some extent, whereas for the other models performance improves in the upper half of the Transformer network  (layers 6-12).
This shows that the semantic information relevant for adjective ranking is not situated at the same level of the Transformer in different languages. We plan to investigate this finding further in future work. The lower results in French can be due to the higher amount of ties present in the datasets compared to other languages. 9 The baselines obtain competitive results showing that the underlying linguistic intuitions hold across languages. The best models beat the baselines in all configurations except for Greek on the DEMELO dataset, where FREQ and static embeddings obtain higher results. Overall, results are lower than those ---→ dV ec and to its opposite (which represents low intensity). The red vector describes a relational adjective that is perpendicular to ---→ dV ec. reported for English, which shows that there is room for improvement in new languages.
Scalar Adjective Identification For each English adjective in the SCAL-REL dataset, we generate a representation from the available ten sentences (cf. Section 2. 2) using the bert-base-uncased model (with WP and WP-1). We experiment with a simple logistic regression classifier that uses the averaged representation for an adjective (ADJ-REP) as input and predicts whether it is scalar or relational. We also apply the DIFFVEC-1 (+) method to this task and measure how intense an adjective is by calculating its cosine with ---→ dV ec. The absolute value of the cosine indicates how clearly an adjective encodes the notion of intensity. In Figure  1 , we show two scalar adjective vectors with negative and positive cosine similarity to ---→ dV ec, and another vector that is perpendicular to ---→ dV ec, i.
e. describing a relational adjective for which the notion of intensity does not apply. 10 We train a logistic regression model to find a cosine threshold separating scalar from relational adjectives (DV-1 (+)). Finally, we also use as a feature the cosine similarity of the adjective representation to the vector of "good", which we consider as a prototypical scalar adjective (PROTO-SIM). The best BERT layer is selected based on the accuracy obtained on the development set. We report accuracy on the test set. The baseline classifiers only use frequency (FREQ) and polysemy (SENSE) as features. We use these baselines on SCAL-REL because the WordNet pertainyms included in the dataset are rarer than the scalar adjectives. The intuition behind the SENSE baseline explained in Section 3. 1 also applies here.
Conclusion We propose a new multilingual benchmark for scalar adjective ranking, and set performance baselines on it using monolingual and multilingual contextual language model representations. Our results show that adjective intensity information is present in the contextualised representations in the studied languages. We also propose a new classification task and a dataset that can serve as a benchmark to estimate the models' capability to identify scalar adjectives when relevant datasets are not available. We make our datasets and sentence contexts available to promote future research on scalar adjectives detection and analysis in different languages.
A Comparison of Wordpiece Selection Methods Table  3  of the main paper contains results of the DIFFVEC method with the best approach for selecting wordpieces (WPs) for each model. In Table  5 , we present results obtained using the alternative approach for each model and language: • for all monolingual models and the multilingual model for English, Table  5  contains results obtained with the WP approach; • for the multilingual models in the other languages, we show results with WP-1. The best approach was determined by comparing their average scores across the different methods. Some configurations improve, but they yield overall worse results per model, especially in Spanish. Differences between WP and WP-1 are generally more pronounced in the multilingual models than in the monolingual models.
Persistence, which is defined as the act of continuing to take treatment for the prescribed duration  [5] , is influenced by several factors. These include perceived treatment benefit, side effect profile  [6] , frequency and route of administration  [7]. Persistence with injectable glucose-lowering therapy is associated with better glycaemic control  [8] [9] [10]. In contrast, poor persistence with glucose-lowering medications can increase the risk of long-term complications in patients with T2D, leading to worse health status  [11, 12]. Many publications on persistence describe data from healthcare databases in the USA. The primary aim of this study was to use real-world pharmacy claims to report persistence with GLP-1 RAs in patients with T2D in France. Treatment dosing patterns and the use of concomitant therapies were also reported by documenting treatment augmentation/intensification, and off-label dose increases and decreases.
METHODS The methods used in this retrospective cohort analysis of a French longitudinal retail pharmacy database (IQVIA Real World Data [RWD] Adjudicated Pharmacy Claims) have already been published as part of an analysis of data from six countries  [13]. While the results of this larger multi-country analysis have been reported, we focus in more detail on the analysis for France in this paper. Included here are aspects of the methodology that are relevant for an understanding of the results or that relate specifically to France. During the study period, the IQVIA Real-World Pharmacy Claims-France Database covered approximately 7500 pharmacies distributed across France and captured 35% of all retail prescriptions in the country. Patient consent and ethics approval were not required for the use of these anonymized prescription data.
Analysis All data were analysed descriptively (frequency and percentage distribution for categorical variables; mean and standard deviation/median and range, 95% confidence interval [CI] or interquartile range for continuous and count variables). KM survival analysis was used for estimating index therapy persistence and time to first modification of the index treatment over the variable follow-up period. SAS v9. 2 or above software (SAS Institute, Cary, NC, USA) was used for all analyses; no formal statistical tests were employed. Further details related to the methods used in this analysis can be found in previous publications  [13] [14] [15].
In this randomized crossover trial, significantly more patients with T2D expressed a preference for the dulaglutide device than for the semaglutide device (84. 2 vs. 12. 3%; P \ 0. 0001)  [26]. A final factor that may influence persistence is the specialty of the prescribing physician-an association that has been documented in other conditions  [27]  but one that we did not analyse in our study. In order to compare our results with published data on persistence to GLP-1 RAs in France, we performed a systematic search of the PubMed database and abstracts from scientific congresses (details available in the ESM [Literature searches]). This yielded 167 publications, of which four were relevant. Of the GLP-1 RAs, only exenatide BID and liraglutide were included in these publications; persistence was consistently higher for liraglutide than for exenatide BID.
Since there is a positive relationship between the maximum allowable gap between prescriptions that defines persistence and estimated persistence rate  [30] , it is not surprising that Roussel et al. [28, 29]  reported higher persistence rates than the current study. However, it should be noted that Guerci et al. [31]  have identified differences in persistence among real-world studies, even when the same GLP-1 RA was studied in the same country using the same definition of persistence. Differences in databases, patient populations and study methodology mean that comparisons of persistence rates among studies or countries should be interpreted with caution. Despite these reservations, data from two studies  [28, 29] -both involving French patients-show similar persistence rates for GLP-1 RAs and insulin using a discontinuation definition of 6-month interruptions in index therapy.
Limitations This analysis was subject to a number of limitations. First, this was a retrospective database study that used data from participating pharmacies only. As a result, the study population may not be fully representative of the French population with T2D, even though the number of participating pharmacies accounts for onethird of total pharmacies in France. Further limitations include the absence of a definitive diagnosis of T2D and reliance on pre-index OAD treatment as a proxy for diagnosis. Also, as in other pharmacy claims databases, data relating to the reasons for treatment modifications are unknown and can only be hypothesized. Furthermore, because baseline data on which to base adjustments were not available, we did not adjust for any potential treatment selection bias or baseline confounding factors, and no formal statistical comparisons were performed.
The authors had permission to access the data from the IQVIA Real World Data [RWD] Adjudicated Pharmacy Claims database. Permission was granted by IQVIA. Data Availability. The original de-identified data used in this analysis were obtained from and are the property of IQVIA. IQVIA has restrictions prohibiting the authors from making the data set publicly available. Interested researchers may contact IQVIA to apply to gain access to the study's data in the same way the authors obtained the data (see https://www. iqvia. com/contact/sf). Open Access. This article is licensed under a Creative Commons Attribution-NonCommercial 4. 0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
ACKNOWLEDGEMENTS Funding. Sponsorship for this study and the journal's Rapid Service Fee were funded by  Eli Lilly and Company. Medical Writing Assistance. The authors would like to thank  Dr. Janet Douglas  and  Karen Goa  ( Rx Communications, Mold, UK ) for medical writing assistance with the preparation of this article, funded by  Eli Lilly and Company. Authorship. All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.
Methods Observational datasets of temperature, precipitation, and bioclimatic indices were obtained from the WorldClim database version 2  41, 42. Monthly future projections of climate variables were supplied by the Institut Pierre Simon Laplace and correspond to their low-resolution coupled ocean-atmosphere model (IPSL-CM5-LR). Five simulations were used in this paper: a control run based on the RCP8. 5 scenario 2006-2100 and; four hosing experiments, using the same RCP8. 5 scenario, but on which 0. 11, 0. 22, 0. 34, and 0. 68 Sv (1 Sv = 10 6 m 3 /s) of freshwater released in the North Atlantic from 2020 to 2070 are superimposed (see SI section S2. 1 for model and experiments descriptions). To correct for any systematic bias in the climate projections, these projections were shifted by the mean bias between the modeled and observed climatology over the period 1970-2000  29.
We calculated five bioclimatic indices to evaluate the impacts on amphibian's distributions (see below; Figs. S3-S16 and Supplementary Information S2). We used distributional data for all amphibian species from IUCN database (5547 species; http://www. iucnredlist. org). Ranges were converted in a raster format with a pixel resolution of 0. 3°(~33 km × 33 km equal-area grid cells) and 1°( ~100 km × 100 km equal-area grid cells). We selected endemic species from each biogeographical realm that had at least 50 pixels of presence (Table  S1 ). The spatial resolutions (0. 3°and 1°) were selected to reduce potential errors coming from these distributional sources. The same data and spatial resolution have been widely used in other macroecological studies either for evaluating impacts of climate change or for addressing ecological and/or evolutionary questions at these coarse-grain scales  [43] [44] [45] [46] [47] [48].
Several studies have found that distributional data either from GBIF occurrence data or IUCN expert range maps produce highly similar estimates of climatic niche conditions experienced by species  49, 50. Ficetola et al. 51  found that amphibian range maps from IUCN represent the known distributions of most amphibian species relatively well. We focus on six realms including Afrotropical, Australasian, Indomalayan, Nearctic, Neotropical, and Palearctic (Fig. S1 )  52. Species from other realms were excluded due to low sample size (i. e. , <50 presence points). We used five observed bioclimatic variables from the Worldclim database to calibrate models that have been considered important for amphibian's biology in previous studies  44, 53. These bioclimatic variables have a strong influence on species' distributions and functional traits  44, 48, 53, 54  and have been used to model species' distributional areas both in current  44  and future climate change conditions  48, 55.
The bioclimatic variables are Annual mean temperature (bio1), maximum temperature of the warmest month (bio5), minimum temperature of coldest month (bio6), annual precipitation (bio12), and precipitation seasonality (bio15). These variables represent both means (bio1, bio12), extremes (bio5 and bio6), and seasonality (bio15) conditions which jointly determine the probability of occurrence of a species in each grid cell. These variables were up-scaled from 10 km to a 33 km and 100 km resolution, respectively. Statistics and reproducibility. Species distribution modeling for 2509 amphibian species (Table  S1 ) was performed using the sdm R package  56. We randomly partitioned the present data into two sets for model calibration (70%) and model validation (30%). We generated a set of pseudoabsences in each biogeographical region using the ecospat.
rand. pseudoabsences function from ecospat R package  57. The number of pseudoabsences was three times the number of presences for each species  56. We adopt this strategy to maximize the number of pseudoabsences used for model validation using a set of standard validation metrics  58, 59. We use each biogeographical realm as the background area for all endemic species to maximize the potential available area for each species. This allows to estimate relatively well the entire set of climatic conditions that a given species likely experienced through its evolutionary and/or biogeographic history  60, 61. We selected a set of 15 species at random for each biogeographical realm and estimated species distribution models using five algorithms: MaxEnt, MARS, CART, ANN, GLM, and BRT. We explored which algorithms exhibit the best predictive performance across a set of metrics (e.
Acknowledgements J. A. V. acknowledges financial support from  DGAPA/PAPIIT  under grant number  IA201320. For the climate simulations, this work was supported by the  French Atomic Commission (CEA)  within the framework of the  Variations Abruptes du Climat: Conséquences et Impacts éNergétiques  project funded by the  Département des sciences de la matières (DSM)  with the  DSM-Energie Program. It benefited from the high performance computing (HPC) resources made available by  Grand Equipement National de Calcul Intensif, CEA , and  Center National de la Recherche Scientifique.
Data availability The distributional data for amphibian species is available from the IUCN database (http://www. iucnredlist. org). The data about observed climatology are available from Worldclim database (http://www. worldclim. org). The data that support the findings of this study  65  are available in Figshare with the identifier https://doi. org/10. 6084/m9. figshare. 13280951. v1.
Data acquisition and post processing 2. 3. 1. Data acquisition For each of the two samples, neutron and X-ray tomographies were acquired. The pixel size was of 26 μm px -1 for the neutron images, and 36 μm px -1 for the X-ray, as summarised in Table  1. The resolution allowed for the observation of the composition of the samples, and the corrosion distribution. The data from this study included, however, different sources of noise which made it more challenging to identify all the constituents. The main source of noise in the neutron data for the naturally corroded sample stems from the presence of scandium (Sc) in the cement paste, which was revealed by a neutron spectroscopy of the sample. The element, classified as a rare-earth material, is rarely present in concrete, but has a high adsorption coefficient in neutron imaging, similar to that of hydrogen.
Multimodal registration Post-processing of the reconstructed images was mainly carried out with a combination of MATLAB®  [46]  and ImageJ  [47, 48]. Postprocessing aimed at matching data acquired from neutron and X-ray tomography. Multimodal registration aligns the two types of images, obtained from the same sample, onto each other. This is done by moving the two obtained 3D attenuation fields (X-ray and neutron) in the 3D space. This requires for the scans to have similar resolutions (a negligible rounding error may affect the final pixel size). Hence, the neutron data was rescaled to a lower theoretical pixel resolution of 36 μm to match the X-ray data. This multimodal registration is essential to take full advantage of the multimodal nature of the information to identify the phases, which in turn improves subsequent quantitative analysis.
For example, pores are characterised by a low X-ray and neutron attenuation, while aggregates have high X-ray and low neutron attenuation. This approach was used to identify the key components. Initially, the histogram describing the distribution of the attenuation coefficients in the X-ray data was used to divide the concrete sample from its background. The histogram resulting from the superimposed data was used to identify the reinforcing bar, and the histogram of the attenuation coefficients in the neutron data was used to identify the aggregates. Once these three components were identified, a new set of data was generated, where the three defined phases were removed from the sample (i. e. set to be equal to zero). Thresholding was then used again to define the remaining components from the histogram of the superimposed data (voids, corrosion products and cement paste).
Acknowledgements The authors would like to acknowledge the  ILL  for granting the possibility to carry on the experimental work at the D50T instrument (research proposal  UGA-57  and  UGA-78 ). The data used in this work are made available at https://doi. org/10. 5291/ILL-DATA. UGA-57 and https://doi. org/10. 5291/ILL-DATA. UGA-78. Additionally, the computations were enabled by resources provided by the  Swedish National Infrastructure for Computing (SNIC)  at  Chalmers Centre for Computational Science and Engineering  ( C3SE ) partially funded by the  Swedish Research Council  through grant agreement no. 2018-05973.