dataesr/scientific-paragraphs-categorization huggingface dataset:
) produced in time, which is known to be exponentially increasing  [24]. As shown in Fig. 1 (a), the fraction of papers mentioning (de)centralization has been exponentially increasing in time since the 1950s, rising to around one paper every 315 in 2021. The growing interest in this topic is also reflected by the increasing number of authors involved in such academic research. Indeed, as shown in Fig. 1 (b), the fraction of authors producing such research has risen exponentially by more than one order of magnitude, with almost one academic every 154 writing a paper mentioning the topic in 2021. This growth is also seen in terms of raw number of publications and authors, as shown in Fig. S1  in the SM, where we compare these numbers for the S2AG corpus and the (de)centralization dataset and find a stronger exponential rise for the latter.
Notice that for both papers and authors there are some periods with a higher or lower increase in the fraction, showing spikes of interest at particular times. For example, in Fig. 1(a, b ) we can see that between the late 1970s and the 1980s the growth rate was faster than the overall exponential fit. In order to understand what has characterized the origins and evolution of the topic, we set to identify topics and clusters of papers in the dataset by using the hSBM algorithm  [20, 21]  described in The pipeline section. In the following analysis, we focus our attention only to years after 1950. Before this date there are only around 100 papers in our (de)centralization dataset. The very first is a political science one from 1851 on local selfgovernments versus centralized governments  [25].
Additionally, it can be utilized to explore the interplay between collective and independent innovation in the field of science. Our pipeline relies on two key methods, the multilayer hierarchical stochastic block model  [21]  and knowledge flows  [23]. On the one hand, we employ the first one to cluster documents and words in the dataset to identify different themes and topics, using information of both citations between papers and of the words used in each document. On the other hand, knowledge flows allow us to identify significant influences between clusters over time. With the present paper, we publicly release the pipeline code to allow other researchers to perform similar analyses on other concepts. Our study presents some limitations which also represent directions for future work.
Availability of data and materials All the code used for the pipeline presented in this paper can be freely accessed and used through the Github repository available at https://github. com/alberto-bracci/decentralization. The data used in this work can be obtained applying the pipeline to the open-access S2AG corpus, available at https://www. semanticscholar. org/. All computational work has been conducted on high memory nodes of the HPC cluster of Queen Mary University of London  [37].
we consider three different periods of time T: 1970-1989, 1990-2007 and 2008-2020. Similarly to what we will do in the next figures, we have excluded the year 2021 as a source of knowledge flow, because our dataset ends at the end of 2021, thus meaning that we cannot evaluate knowledge flows from papers of 2021 to future years. In the figure, all clusters are ordered as in Fig. 1(c), with the representative keywords shown in the legend below. For each period T, the color of the cell of row a and column b of the.
Dataset Granularity # images #classes CUB  [35]  Fine 11,788 200 SUN  [24]  Fine 14,340 717 FLO  [22]  Fine 8,189 102 AwA2  [37]  Coarse 37,322 50 Table  1 : Main characteristics of the dataset used. The granularity stands for fine-grained or coarse-grained classification With its generic formulation, the proposed regularization can be applied to a large number of generative models. In the following, we choose to integrate it to f-VAEGAN  [38]  as illustrated in Fig. 2. 4 Evaluation of Semantic Ambiguity Regularization.
Datasets and Metrics We evaluate our method on four datasets that are commonly used in the ZSL literature, namely Caltech UCSD Birds 200-2011 (CUB)  [35] , SUN Attribute dataset  [24] , Oxford Flowers (FLO)  [22]  and Animals with Attributes (AWA2)  [37]. Their main characteristics are reported in Table  1  Following the previous works in the field  [37] , we report the average per-class top-1 accuracy (T1) for ZSL settings, and the average per-class top-1 accuracy on seen classes denoted as s, average per-class top-1 accuracy on unseen classes denoted as u, and the harmonic mean H = 2 * (s * u)/(s + u) for the GZSL settings. We follow the exact ZSL and GZSL splits, known as the "Proposed Splits", as well as the evaluation protocol of  [37]  insuring that none of the test classes appear in ImageNet, which is used to pre-train the visual feature extractor.
Unless otherwise specified, we use 2048-dimension 101-ResNet features  [11]  as visual embeddings for all the datasets. For class semantic prototypes of CUB and FLO, we adopt the 1024-dim sentence embeddings of character-based CNN-RNN model generated from fine-grained visual descriptions  [26]. To compare the methods over several benchmarks and estimate their aggregated merit, we adopt the median normalized relative gain (mNRG)  [30]. Indeed, such a comparison can be biased if one uses a simple average over different benchmarks. mNRG exhibits several interesting features such as an independence to outlier scores, coherent aggregation or time consistency. Its main drawback is that a reference method has to be chosen, from which the performance of each method is measured, according to a unique aggregated score, possibly negative if the method performs globally worse.
In our case, we choose CLSWGAN  [36]  in inductive settings as reference. For the comparison with fine-tuned features, we use f-VAEGAN-D2 in inductive settings as reference. We compute the mNRG by aggregating the accuracy for ZSL and the harmonic mean H of seen and unseen accuracy for GZSL. By definition, the score of the reference is 0. If mN RG < 0 then the method performs globally worse than the references over all datasets. When mN RG = 1 the method obtains the best performances on a majority of datasets (at least 3 on 4 in our experiments).
State-of-the-art comparison Table  2  and Table  3  show state-of-the-art comparison on four zero-shot learning datasets. For inductive ZSL setting, our model performs globally better than all other methods with the highest mNRG score. It also achieves the best score on CUB and SUN. In the transductive ZSL setting, our approach obtains a mNRG score of 1, establishing a new transductive ZSL stateof-the-art on CUB, SUN and AWA2. Unsurprinsingly, In the GZSL setting, feature generating approaches such as our model, TF-VAEGAN  [21] , f-VAEGAN-D2  [38] , CLSWGAN  [36] , LisGAN  [19]  obtain better results than others. We also note that the accuracy on unseen classes (u) and that on seen classes (s) are better balanced. Our model outperforms the existing methods for both inductive and transductive GZSL settings.
Influence of the mixing proportion In this section, we perform an ablation study on four ZSL datasets. We evaluate our model with different values of the random mixing proportion λ (Results are shown in Table  4 ). Note that when λ is sampled from a distribution, a new value is selected for each minibatch. We found that the best performances accross all datasets were met when λ = 0. 5 or λ ∼ N (0. 5, 0. 25), i. e. setting equal weights for the two terms of the convex combination. Other settings for λ, such as λ = 0. 2 or λ ∼ U nif orm(0, 1), deteriorates the performances (∼ 1% worse for CUB, SUN and FLO and ∼ 4% worse for AWA2). Even poorer performances were found when setting λ ∼ Beta(0. 3, 0. 3) (∼ 1% worse for CUB and SUN and ∼ 8% worse for FLO and AWA2). For image classification  [40] , the random mixing proportion is sampled from the Beta distribution with a small value of α, as it assumes that the examples in the neighborhood of each data sample share the same class.
2. For fair comparison, the results reported in Table  5  were all obtained with f-VAEGAN using unlabeled images at training time, such that the 's+u' results are the same as the transductive settings in Table  3  and 2. In the ZSL setting, we obtain the same results whether we use all prototypes or unseen prototypes only, this makes sense since the test images are from the unseen classes only and there is no point in modelling Table  3 : State-of-the-art comparison Accuracy for GZSL on the "proposed split" of  [37] , both inductive (IN) and transductive (TR) results are shown. We measure Top-1 accuracy on seen (s) and unseen (u) classes as well as their harmonic mean (H). H is the average over the four datasets. Models marked with * were partially re-implemented. the ambiguities with seen classes.
Integration to TF-VAEGAN Our contribution is generic and can be used in other conditional generative-based ZSL architectures. Therefore, we evaluate the generalization capabilities of our proposed method, by integrating our contribution in the TF-VAEGAN  [21]  framework. We first learn the model end-to-end, adding the proposed regularization. Table  6  shows the comparison on CUB, between the original TF-VAEGAN model and that learned with the proposed regularization (equation 3). Our contribution improves the performance of the vanilla TF-VAEGAN for both ZSL and GZSL tasks, either in inductive and transductive settings, by 1. 5 to 3 points. [21]  and that augmented with our loss (Eq. 3), on CUB dataset for both inductive and transductive ZSL/GZSL settings, either when the model is learned from scratch or fine-tuned (ft) To further investigate the effectiveness of our contribution, we perform an additional experiment consisting in fine-tuning the generator learned by TF-VAEGAN with our method.
Conclusion We propose a novel approach to train a conditional generative-based model for zero-shot learning. The approach improves the discriminative capacity of the synthesized features by training the generator to rec-ognize virtual ambiguous classes. We construct the corresponding ambiguous class prototypes as convex combinations of the real class prototypes and then we train the generator to recognize these virtual classes. This simple procedure allows the generator to learn the transitions between categories and thus, to better distinguish them. Our approach can be integrated to any conditional generative model. Experiments on four benchmark datasets show the effectiveness of our approach across zero-shot and generalized zero-shot learning. To push further our approach, one could explore nonlinear interpolation for constructing ambiguous classes, or considering more than two real classes to construct an ambiguous one.
Sampling, DNA extraction, microsatellite genotyping, and genetic diversity analyses The final dataset included 407 individuals from 13 sampling sites for P. clavata and 450 individuals from 12 sampling sites for C. rubrum, genotyped with seven and nine microsatellites, respectively (Table  1 ; Fig. 1 ). These two sets of markers were developed and used in previous genetic studies of the two species (e. g. ,  Ledoux et al. 2010 , Mokthar-Jamaï et al. 2011). To allow for the most rigorous comparison possible, both species were sampled at eleven shared or nearby (tens of meters) sites and, at each site, sampling was performed to cover the entire site. We provide specifics on sampling methods, DNA extraction, microsatellite genotyping, quality check, tests for linkage disequilibrium and departure from panmixia in Appendix A.
001, 0. 002, 0. 003, 0. 004, 0. 005, 0. 01, 0. 025, 0. 047). Those simulated F ST are within the range of the pairwise F ST computed in each species (see Results). The power of each dataset to detect genetic structure for a given F ST value was calculated by the proportion of significant Fisher's test per 1,000 replicates (see Appendix B).
The contribution of all the matrices defines the distance between populations  (Borcard et al. 2018). For each species, allele frequencies were scaled using scaleGen function in ADE-GENET  (Jombart 2008). A third matrix of latitude and longitude was included to account for the spatial location of each site. Correlations among the three matrices were described using RV coefficients, a multivariate generalization of the squared Pearson's correlation coefficients. The MFA among the three matrices and estimation of RV coefficients were conducted in the R package FACTOMINER  (Le et al. 2008). Secondly, for each sampling site, we calculated the population-specific F ST and 95% High Probability Density Intervals in GESTE  (Foll & Gaggiotti 2006)  for each sampling site. This method measures the genetic differentiation proper to each sampling site and estimates the relative impact of genetic drift on the differentiation of the considered sampling site relative to the remaining ones.
Individual-based assignment to sampling sites Simulation study  (Carlson 2008 ) demonstrated that assignment accuracy can reach 90% with F ST as low as 0. 05, which is close to the mean F ST computed in our study for both species (see Results). Thus, for each species, we conducted a filtered assignment analysis following  Lukoschek et al. (2016)  to compare their relative patterns of demographic connectivity. This three-step analysis was conducted using GENECLASS2  (Piry et al. 2004). First, we identified firstgeneration migrants (FGMs) in each species using the Bayesian criteria of  Rannala and Mountain (1997)  with 10,000 simulated genotypes and an alpha of 0. 01. Then, for each species considered separately, FGMs were removed from the dataset and re-assigned to the reference dataset (i.
Pattern of genetic structure The simulations conducted in POWSIM supported the statistical power of our dataset to detect genetic structure corresponding to F ST equals to 0. 005 (see Appendix B). This value is one order of magnitude lower than the global F ST computed for P. clavata (0. 049) and C. rubrum (0. 041). In P. clavata, the Evanno's method (see Appendix C Figure  C1 ) identified four different genetic clusters (K = 4; Fig. 2A ). These four genetic clusters were concordant with   C1 ) identified two different genetic clusters (K = 2) that encompassed the individuals from the northern part of Cap de Creus (NPCC) and GAT (mean membership coefficient = 0. 81) vs. the remaining individuals (mean membership coefficient = 0. 69), except individuals from PAI, which were almost equally shared between the two clusters.
The DAPC analyses corroborated these results. The first two axes represented similar levels of the total variation in the two datasets (47. 8% and 49. 3% in P. clavata and C. rubrum, respectively). The scatter plot is more concordant with a geographic imprint in P. clavata compared to C. rubrum (Fig. 2B ). The first axis segregated P. 1 ) and areas (NPCC: northern part of Cap de Creus, SPCC: southern part of Cap de Creus, MI: Medes Islands, MCA: Montgrí coastal area) are shown below and above the assignment plots. Thick yellow doted lines delineate main genetic discontinuities. B Scatter plots of the discriminant analysis of principal components (DAPC). Each dot corresponds to one individual from each sampling site, which are represented by different colors. Inertia ellipses centre on the mean for each location and include 67% of the sampling points.
2019 (Garrabou et al. , 2021)). In this context, characterizing the adaptive genetic diversity and more particularly potential genomic factors and processes driving the response to thermal stress is urgently needed to pave the way for effective management plans in the context of climate change. project funded by the Foundation Prince Albert II Monaco. This work was supported by the European Union's Horizon 2020 research and innovation program under grant agreement SEP-210597628 (Future-MARES). This study was also supported by the Spanish Government through the Smart project (CGL2012-32194) the HEATMED project (RTI2018-095346-B-I00, MCIU/AEI/FEDER, UE). RL was supported by the Agence Nationale de la Recherche (projects DISLAND ANR-20-CE32-00XXX, GENOSPACE ANR-16-CE02-0008 and INTROSPEC ANR-19-CE02-0011; and project PROLAG from the CeMEB LabEx;), and by recurrent funding from INRAe and CNRS.
This research has been funded from the European Union's Horizon 2020 research and innovation programme under grant agreement No 869300 "Future-MARES". SD is supported by theFonds National de la Recherche Scientifique  (FNRS, Belgium). JG and PL acknowledge the funding of the Spanish government through the 'Severo Ochoa Centre of Excellence' accreditation (CEX2019-000928-S). CL gratefully acknowledges the financial support by ICREA under the ICREA Academia program.
Acknowledgements Field work and sampling were done in accordance with legislations. Part of this work was carried out by using the resources of the national INRAe MIGALE (http:// migale. jouy. inra. fr, MIGALE, INRAE, 2020. Migale bioinformatics Facility, https:// doi. org/ 10. 15454/1. 55723 90655 34329 3E12) and GENOTOUL (Toulouse Midi-Pyrénées) bioinformatics HPC platforms, as well as the local  Montpellier Bioinformatics Biodiversity  (MBB, supported by the  LabEx CeMEB   ANR-10-LABX-04-01 ) and  CBGP HPC  platform services. This is publication  ISEM 2023-192. JBL, DGG, PLS, CL and JG are part of the  Marine Conservation research   group MEDRECOVER  ( 2017 SGR 1521 ) from the  Generalitat de Catalunya.
(2016) , WCRP (2018) and  Meyssignac et al. (2019). Additionally, specific Argo-based products are listed on the Argo web page (http://www. argo. ucsd. edu/, last access: 7 August 2020). Although all products rely more or less on the same database, near-global OHC estimates show some discrepancies which result from the different statistical treatments of data gaps, the choice of the climatology, and the approach used to account for the MBT and XBT instrumental biases  (Boyer et al. , 2016; Wang et al. , 2018). Argo-based products show smaller differences, likely resulting from different treatments of currently undersampled regions  (e. g. , von Schuckmann et al. , 2016). Ocean reanalysis systems have been also used to deliver estimates of near-global OHC  (Meyssignac et al. , 2019; von Schuckmann et al.
, 2018) , and their international assessments show increased discrepancies with decreasing in situ data availability for the assimilation  (Palmer et al. , 2017; Storto et al. , 2018). Climate models have also been used to study global and regional ocean heat changes and the associated mechanisms, with observational datasets providing valuable benchmarks for model evaluation  (Cheng et al. , 2016; Gleckler et al. , 2016). International near-global OHC assessments have been performed previously (e. g. ,  Abraham et al. , 2013; Boyer et al. , 2016; Meyssignac et al. , 2019; WCRP, 2018). These assessments are challenging, as most of the gridded temperature fields are research products, and only few are distributed and regularly updated operationally (e. g. , https:// marine. copernicus. eu/, last access: 7 August 2020).
Equation (3) provides a decomposition of the atmospheric energy content AE into sensible heat energy (sum of the first two terms, internal heat energy and gravity potential energy), latent heat energy (third term) and kinetic energy (fourth term), where ρ is the air density, c v the specific heat for moist air at constant volume, T the air temperature, g the acceleration of gravity, L e the temperature-dependent effective latent heat of condensation (and vaporization) L v or sublimation L s (the latter relevant below 0 • C), q the specific humidity of the moist air, and V the wind speed. We neglect atmospheric liquid water droplets and ice particles as separate species, as their amounts and especially their trends are small. In the AE derivation from observational datasets based on Eq. (  3 ), we accounted for the intrinsic temperature dependence of the latent heat of water vapor by assigning L e to L v if ambient temperatures are above 0 • C and to L s (adding in the latent heat of fusion L f ) if they are below -10 • C, respectively, with a gradual (half-sine weighted) transition over the temperature range between.
(  3 ), we hence refer to the energy content AE as atmospheric heat content (AHC) hereafter. Turning to the actual datasets used, atmospheric energy accumulation can be quantified using various data types, as summarized in the following. Atmospheric reanalyses combine observational information from various sources (radiosondes, satellites, weather stations, etc. ) and a dynamical model in a statistically optimal way. This data type has reached a high level of maturity, thanks to continuous development work since the early 1990s  (Hersbach et al. , 2018). Especially reanalyzed atmospheric state quantities like temperature, winds and moisture are considered to be of high quality and suitable for climate studies, although temporal discontinuities introduced from the ever-changing observation system remain a matter of concern  (Berrisford et al.
Moreover, JRA-55C is fully independent of satellite-derived Global Positioning System (GPS) radio occultation (RO) data that are also separately used and described below together with the observational techniques. In addition to these four reanalyses, the datasets from three different observation techniques have been used for complementary observational estimates of the atmospheric heat content. We use the Wegener Center (WEGC) multisatellite RO data record, WEGC OPSv5. 6  (Angerer et al. , 2017) , as well as its radiosonde (RS) data record derived from the highquality Vaisala sondes RS80/RS92/VS41, WEGC Vaisala  (Ladstädter et al. , 2015). WEGC OPSv5. 6 and WEGC Vaisala provide thermodynamic upper air profiles of air temperature, specific humidity and density from which we locally estimate the vertical AHC based on the first three integral terms of Eq.
(3)  (Kirchengast et al. , 2019). In atmospheric domains not fully covered by the data (e. g. , in the lower part of the boundary layer for RO or over the polar latitudes for RS), the profiles are vertically completed by collocated ERA5 information. The local vertical AHC results are then averaged into regional monthly means, which are finally geographically aggregated to global AHC. Applying this estimation approach in the same way to reanalysis profiles subsampled at the observation locations accurately leads to the same AHC anomaly time series records as the direct estimation from the full gridded fields. The third observation-based AHC dataset derives from a rather approximate estimation approach using the microwave sounding unit (MSU) data records  (Mears and Wentz, 2017). Because the very coarse vertical resolution of the brightness temperature measurements from MSU does not enable integration according to Eq.
(3), this dataset is derived by replicating the method used in IPCC AR5 WGI Assessment Report 2013  (Rhein et al. , 2013; chap. 3, Box 3. 1 therein). We used the most recent MSU Remote Sensing System (RSS) V4. 0 temperature dataset  (Mears and Wentz, 2017) , however, instead of MSU RSS V3. 3  (Mears and Wentz, 2009a, b)  that was used in the IPCC AR5. In order to derive global time series of AHC anomalies, the approach simply combines weighted MSU lower tropospheric temperature and lower stratospheric temperature changes (TLT and TLS channels) converted to sensible heat content changes via global atmospheric mass, as well as an assumed fractional increase in latent heat content according to water vapor content increase driven by temperature at a near-Clausius-Clapeyron rate (7. 5 % • C -1 ).