dataesr/scientific-paragraphs-categorization huggingface dataset:
Conclusion and Future Work We propose a novel Quality Estimation model for summarization which does not require human references to estimate the quality of automatically produced summaries. SUM-QE successfully predicts qualitative aspects of summaries that recall-oriented evaluation metrics fail to approximate. Leveraging powerful BERT representations, it achieves high correlations with human scores for most linguistic qualities rated, on three different datasets. Future work involves extending the SUM-QE model to capture content-related aspects, either in combination with existing eval- uation metrics (like Pyramid and ROUGE) or, preferably, by identifying important information in the original text and modelling its preservation in the proposed summaries. This would preserve SUM-QE's independence from human references, a property of central importance in real-life usage scenarios and system development settings.
The datasets used in our experiments come from the NIST DUC shared tasks which comprise newswire articles. We believe that SUM-QE could be easily applied to other domains. A small amount of annotated data would be needed for fine-tuning -especially in domains with specialized vocabulary (e. g. , biomedical) -but the model could also be used out of the box. A concrete estimation of performance in this setting will be part of future work. Also, the model could serve to estimate linguistic qualities other than the ones in the DUC dataset with mininum effort. Finally, SUM-QE could serve to assess the quality of other types of texts, not only summaries. It could thus be applied to other text generation tasks, such as natural language generation and sentence compression.
Site-graph rewriting languages, such as Kappa or BNGL, offer parsimonious ways to describe highly combinatorial systems of mechanistic interactions among proteins. These systems may be then simulated efficiently. Yet, the modeling mechanisms that involve counting (a number of phosphorylated sites for instance) require an exponential number of rules in Kappa. In BNGL, updating the set of the potential applications of rules in the current state of the system comes down to the sub-graph isomorphism problem (which is NP-complete). In this paper, we extend Kappa to deal both parsimoniously and efficiently with counters. We propose a single push-out semantics for Kappa with counters. We show how to compile Kappa with counters into Kappa without counters (without requiring an exponential number of rules).
We will make n diverge towards the infinity so as to empirically estimate the combinatorial complexity of several encoding schemes. The rate of phosphorylation/dephosphorylation of each site, depends on the number of sites that are already phosphorylated. In Fig. 1 (a), we provide the example of a rule that phosphorylates the site a of the protein, assuming that the sites b and c are already phosphorylated and that the site d is not. Proteins are depicted as rectangles. Sites are depicted clockwise from the site a to the site d starting at the top left corner of the protein. Phosphorylation states are depicted with a black mark when the site is phosphorylated, and with a white mark otherwise. To fully encode this model in Kappa, we would require n • 2 n rules. Indeed, we need to decide whether this is a phosphorylation or a dephosphorylation (2 possibilities), then on which site to apply the transformation (n possibilities), then what the state of the other sites is (2 n-1 possibilities).
Hence, at this level of representation, the phosphorylation of the four sites become independent, which flatten the causal trace. Related Works. Many modeling languages support arbitrary data-types. In Spatial-Kappa  [41] , counters encode the discrete position of agents. More generally, in Chromar  [29]  and in colored Petri nets  [30, 35] , agents may be tagged with values in arbitrary auxiliary programming languages. In ML-Rules  [28] , agents with attributes continuously diffuse within compartments and collide to interact. We have different motivations. Our goal is to enrich the state of proteins with some redundant information, so as to reduce the number of rules that are necessary to describe their mechanistic interactions. Also we want to avoid too expressive data-types, which could not be integrated within simulation, causality analysis, and static analysis tools, without altering their performance.
Generic Abstraction of Reachable States So far, we have provided two encodings to compile Kappa with counters into Kappa without counters. These encodings are sound under some assumptions over the range of counters. Now we propose a static analysis not only to check that these conditions are satisfied, but also to infer the meaning of the counters (in our case study, that they are equal to the number of phosphorylated sites). Firstly, we provide a generic abstraction to capture the properties of the states that a Kappa model may potentially take. Our abstraction is parametric with respect to the class of properties. It will be instantiated in Sect. 5. Our analysis is not complete: not all the properties of the program are discovered; nevertheless, the result is sound: all the properties that are captured, are correct.
Yet these simplifications are done without any clue of their impact on the behavior of the systems. By providing ways of describing and handling some complex details, we offer the modelers the means to incorporate these details and to test empirically their impact. Our framework is fully integrated within the Kappa modeling platform which is open-source and usable online (https://kappalanguage. org). It is worth noting that we have taken two radically different approaches to deal with counters in simulation and in static analysis. Encodings are good for simulation, but they tend to obfuscate the properties of interest, hence damaging drastically the capability of the static analysis to infer useful properties about them. The extension of the categorical semantics provides a parsimonious definition of causality between computation steps, as well as means to reason symbolically on the behavior of the number of occurrences of patterns.
Environmental information We measured an array of environmental variables to estimate the environmental dissimilarity among plots: 1) mean annual soil temperature was estimated from field meteorological stations placed in each site; 2) growing season length (GSL) and annual number of frost days were based on daily maps of snow cover and air temperature values following the methodology defined in  Carlson et al. (2015) ; 3) top soil (0-10 cm) characteristics were determined for each plot from the average values obtained from three soil samples collected in August 2012 (C/N ratio, organic matter content and five variables describing soil nitrogen content and fluxes); 4) topographic wetness index and slope, inferred from airborne LiDAR data acquired during the year of the sampling. The environmental dissimilarity between each plot was estimated with Euclidean distances from the first five axes of a principal component analysis of all environmental variables, which were first standardized.
These axes explained 90% of the total variance of the environmental dataset. Due to our sampling design, spatial distances among plots were highly correlated with elevation differences (Mantel test, Pearson's r = 0. 99, p < 0. 001) and environmental dissimilarities (Mantel test, Pearson's r = 0. 74, p < 0. 001), and were thus ignored in the analysis of microbial β-diversity at the landscape scale. Details about the sampling of environmental variables are available in Supplementary material Appendix 1.
Data processing We used the data curation procedure detailed in  Zinger et al. (2019)  and  Ohlmann et al. (2018). This procedure sequentially performs the following steps: assembly of paired-end reads; assignment of reads to samples; removal of low-quality reads, singletons and chimeras; and building of MOTUs (more details are available in Supplementary material Appendix 2). We chose a dissimilarity threshold of three mismatches to define MOTUs. Taxonomic assignments of MOTUs were obtained by comparing sequences against full-length references from the EMBL, SILVA and UNITE databases  (Pruesse et al. 2007 , Koljalg et al. 2013 ) and a database specific to arctico-alpine plants  (Willerslev et al. 2014). Finally, we removed MOTUs with less than or equal to two reads from each sample. Basic characteristics of the sample-by-MOTUs matrices that were created for bacteria, archaea, fungi and plants (number of MOTUs, sequence length, number of reads) are available in Table  1  and Supplementary material Appendix 2 Table  A2.
2012, Zinger and Philippe 2016). Here, we applied a more rigorous approach that consists in placing the observed MOTUs on reliable reference phylogenies (i. e. based on longer DNA sequences and/or multiple loci) from their taxonomic assignment. Second, the taxonomic resolution of such short DNA metabarcoding markers can be relatively low and/or heterogeneous across lineages, which prevents a consistent identification at the species level for every detected organism of the targeted taxon. In addition, while under constant improvement, DNA reference databases for microbes are still incomplete. Consequently, taxonomic information about MOTUs does not always allow precise location on a reference phylogeny, which can increase uncertainty in diversity estimates. To overcome these issues, we propose here a new method: based on MOTU taxonomic assignations and reference phylogenies (Fig.
1  -blue frame) that seeks to characterize the uncertainty in phylogenetic placement. Briefly, for each marker, we produced a distribution of 50 MOTUs phylogenies to capture the uncertainty in the placement of each MOTU on the reference phylogenies. Reference phylogenies -we used four reference phylogenies describing the evolutionary relationships of each lineage (archaea, bacteria, fungi and plants). The bacterial and archaeal phylogenies were taken from  Lang et al. (2013) , and the fungal phylogeny from  James et al. (2006). These three phylogenies were transformed into ultrametric phylogenies using the R-function 'chronos' (R-package ape  Paradis et al. 2004 ). The plant phylogeny was an ultrametric genus-level phylogeny of alpine plants from  Roquet et al. (2013). Taxonomic information -the taxonomic classification of each tip of the reference phylogenies and each MOTU was automatically retrieved from the NCBI taxonomy database  (Sayers et al.
2009 ) using the R-function 'classification' (R-package taxize  Chamberlain et al. 2014). Placing MOTUs on reference phylogenies -we conceived an algorithm that 1) compares the taxonomic information of the reference phylogeny tips to that of MOTUS, and 2) optimally grafts each MOTU sequentially on the reference phylogenetic tree. The algorithm works as follows: given a clade to which the MOTU is assigned, the MOTU is stitched randomly to a tip or a branch in the relevant clade in the reference phylogeny. In some cases, the taxonomic information of the MOTU was too precise compared to the resolution of the reference phylogeny. In this case, the taxonomic information was downgraded to the most precise taxonomic level represented in the tips of the reference phylogeny. The updated phylogeny was then used for subsequent MOTU grafting until all MOTUs were placed on the reference phylogeny (Fig.
Funding -The research received funding from the  French Agence Nationale de la Recherche (ANR)  through the  METABAR  ( ANR-11-BSV7-0020 ),  GlobNets  ( ANR-16-CE02-0009 ) and  OrginAlps  ( ANR-16-CE93-004 ) projects, and from  'Investissement d'Avenir ' grants managed by the  ANR  ( Trajectories :  ANR-15-IDEX-02 ; Montane: OSUG@2020:  ANR-10-LAB-56 ,  CEBA :  ANR-10-LABX-25-01 ). The  Alpine Ecology Lab (LECA)  is part of  LabexOSUG@2020  ( ANR10 LABX56 ).
INTRODUCTION The XPath language  [41]  is arguably the most popular querying language for selecting elements in XML documents. It is embedded in the XML processing languages XSLT  [19]  and XQuery  [42] , and widely used in general-purpose languages like Java or C# through third-party libraries. It combines the ability to navigate the XML tree-which finds its roots in modal logic  [3] -with that of comparing data values found in several, distantly related attributes. Alongside the evaluation of the set of elements selected in a document [e. g. 4, 10, 13, 30] , the main computational problem associated with a query is the satisfiability problem: given an XPath query, and optionally a schema for the type of XML documents under consideration, does there exist at least one XML document on which this query would select some node? This abstract question Unfortunately, the satisfiability problem is in general undecidable.
Even CoreXPath 1. 0, a simplified syntax based on the navigational fragment of XPath 1. 0, is untractable, more exactly EXPcomplete already when only using the child axis  [3, 9, 40]. Thus with the exception of the work of Genevès et al. [27]  on Core-XPath 1. 0, most of the literature on the topic is of a theoretical nature [e. g. 2, 5, 12, 20-22, 25, 26, 34] , and focuses on decidability and complexity questions in variants of CoreXPath 1. 0 that allow limited forms of data joins. While these results are technically impressive, it is not immediately clear how much is gained in practice by handling data joins. Indeed, data joins are not the sole source of difficulty in XPath: many real-life XPath queries perform calls to a standard library of functions  [18] -including arithmetic and string-manipulating functions-that also lead to an undecidable satisfiability.
5. 1). In turn, answering this type of question requires the definition of a palatable semantics for a substantial subset of XPath 3. 0, which we provide in Sec. 3. As could be expected, the coverage of the naive implementations of the syntactic fragments from the literature is not very good (Sec. 4. 2), and one should extend them whenever reasonable. In Sec. 5, we propose six extensions of the original fragments and evaluate their coverage on the benchmark. These extensions are root navigation, free variables, data tests against constants, positive data joins, and restricted calls to the functions last() and id(). Just as interestingly, we exhibit several cases where these extensions cannot be handled. We analyse our experimental results in Sec. 6, notably concluding that higher coverage is obtained through basic extensions than by using complex academic fragments.
Parser We have modified the W3C parser for XQuery 3. 0, 1 which is (almost) a superset of XPath 3. 0, so that we can also use it for XPath queries extracted from XSLT documents. This parser uses a grammar automatically extracted from the language specification, so we are confident in its results. Our implementation further  (1)  extracts XPath queries from XQuery files, by selecting 'maximal XPath subtrees' in the abstract syntax tree of the XQuery document, and (2) outputs syntax trees in the standard XML format XQueryX  [17] ; this is what we process to determine to which XPath fragments each query belongs. See App. B for an example XQuery document and the corresponding parser output.
Among the features that such a front-end should support, the most impactful ones would be free variables, data tests against constants (and constant expressions), positive joins, and positional predicates. According to our benchmark results, such a front-end combined with the decidable fragments from the literature would cover about 70%-75% of the XPath queries found in XSLT files. However, due to the high reliance on user-defined or ill-supported functions, this drops to less than 30% for XPath queries from XQuery files: fullblown program analysis techniques seem necessary for XQuery. As the the support of XPath functions is a key factor, a promising approach would be to harness the power of modern SMT solvers to handle string-manipulating functions and linear arithmetic, which might cover 77.
In addition, we use RBM to design new protein sequences with putative properties by composing and turning up or down the different modes at will. Our work therefore shows that RBM are a versatile and practical tool to unveil and exploit the genotype-phenotype relationship for protein families. Sequencing of many organism genomes has led over the recent years to the collection of a huge number of protein sequences, gathered in databases such as UniProt or PFAM Finn et al. (2013). Sequences with a common ancestral origin, defining a family (Fig. 1A ), are likely to code for proteins with similar functions and structures, hence providing a unique window into the relationship between genotype (sequence content) and phenotype (biological features). In this context, various approaches have been introduced to infer protein properties from sequence data statistics, in particular amino-acid conservation and coevolution (correlation) Teppa et al.
Description The majority of natural proteins are obtained by concatenating functional building blocks, called protein domains. The Kunitz domain, with a length of about 50-60 residues (protein family PF00014 Finn et al. (2013) ) is present in several genes and its main function is to inhibit serine protease such as trypsin. Kunitz domains play a key role in the regulation of many important processes in the body such as tissue growth and remodeling, inflammation, body coagulation and fibrinolysis. They are implicated in several diseases such as tumor growth, Alzheimer, cardiovascular and inflammatory diseases and, therefore, have been largely studied and shown to have a large potential in drug design Shigetomi et al. (2010) ; Bajaj et al. (2001). Some examples of Kunitz domain-containing proteins include the Basic Pancreatic Trypsin Inhibitor (BPTI, 1 Kunitz domain), the Bikunin (2 domains) Fries and Blom  (2000) , Hepatocyte growth factor activator inhibitor (HAI, 2 domains) and tissue factor pathway inhibitor (TFPI, 3 domains) Shigetomi et al.
Description WW is a protein-protein interaction domain found in many eukaryotes and human signalling proteins, involved in essential cellular processes such as transcription, RNA processing, protein trafficking, receptor signalling. WW is a short domain of length 30-40 amino-acids (Fig. 3A , PFAM PF00397, = 7503 sequences, = 31 consensus sites), which folds into a three-stranded antiparallel -sheet. The domain name stems from the two conserved tryptophans (W) at positions 5-28 (Fig. 3A ), which serve as anchoring sites for the ligands. WW domains bind to a variety of proline (P)-rich peptide ligands, and can be divided into four groups, based on their preferential binding affinity Sudol and Hunter  (2000). Group I binds specifically to PPXY motif -where X is any amino acid; Group II to PPLP motifs; Group III to proline-arginine containing sequences (PR); Group IV to phosphorylated serine/threonine-proline sites [p(S/T)P].
1  in Zuiderweg et al. (2017). We have constructed a multiple sequence alignment for HSP70 with = 675 consensus sites and = 32, 170 sequences, starting from the seeds of Malinverni et al. (2015) , and queried SwissProt and Trembl UniprotKB databases using HMMER3 Eddy  (2011). Annotated sequences were grouped based on their phylogenetic origin and functional role. Prokaryotes mainly express two Hsp70 proteins: DnaK ( = 17, 118 sequences in the alignment), which are the prototype Hsp70, and HscA ( = 3, 897), which are specialized in chaperoning of Iron-Sulfur cluster containing proteins. Eukaryotes Hsp70 were grouped by location of expression (Mitochondria: = 851, Chloroplaste: = 416, Endoplasmic reticulum: = 433, Nucleus/Cytoplasm and others: = 1, 452). We also singled out Hsp110 sequences, which, despite the high homology with Hsp70, correspond to nonallosteric proteins ( = 294).
Cross-validation of the model and interpretability of the representations Each RBM was trained on a randomly chosen subset of 80% of the sequences in the MSA, while the remaining 20% (called test set) were used for validation of its predictive power. In practice, we compute the average log-probability of the test set to assess the performances of the RBM for various values of the number of hidden units, of the regularization strength 2 1 and for different hidden-unit potentials. Results for the WW and Kunitz domains and for Lattice Proteins are reported in Fig. 8  and in Appendix 2 (Model Selection). The dReLU potential, which include quadratic and Bernoulli (another popular choice for RBM) potentials as special cases, is consistently better than the latters. As expected, increasing allows RBM to capture more features in the data distribution and, therefore, improves performances up to some level, after which overfitting starts to occur.
Data preprocessing We use the PFAM sequence alignments of the V31. 0 release (March 2017) for both Kunitz (PF00014) and WW (PF00397) domains. All columns with insertions are discarded, after which duplicate sequences are removed. We are left with, respectively, = 53 sites and = 8062 unique sequences for Kunitz, and = 31 and = 7503 for WW; each site can carry = 21 different symbols. To correct for the heterogeneous sampling of the sequence space, a reweighting procedure is applied: each sequence with = 1,. , is assigned a weight equal to the inverse of the number of sequences with more than 90% amino-acid identity (including itself). In all that follows, the average over the sequence data of a function is defined as ⟨ ( )⟩ = ∑ =1 ( ) ∑ =1. ( 7 ) Learning procedure.
⟩ , but the probability ratio can be exponentially large. In Annealed Importance Sampling, we address this problem by constructing a continuous path of interpolating distributions ( ) = 1 ( ) 0 ( ) 1-, and estimate 1 as a product of ratios of partition functions: 1 = 1 -1 -2. 1 0 × 0 , ( 12 ) where we choose a linear set of interpolating inverse temperatures of the form = max. To evaluate the successive expectations, we use a fixed number of samples initially drawn from estimate log 1 0 ≈ ⟨ log * 1 ( ) * 0 ( ) ⟩ ∼ 0 , which is justified if 1 and 0 are close. In practice, we used = 20 chains, = 5 × 10 4 steps. For the initial distribution 0 , we take the closest (in terms of KL divergence) independent model to the data distribution. The visible layer fields are the ones of the independent model inferred from the MSA, and the weights are =0 = 0.
Hidden-unit potentials Lastly, we discuss the choice of the hidden-unit potentials. A priori, the major difference between Bernoulli, quadratic and dReLU potentials are that (i) Bernoulli hidden unit take discrete values whereas quadratic and dReLU take continuous ones and (ii) After marginalization, quadratic potentials create pairwise effective interactions whereas Bernoulli and dReLU create non-pairwise ones. It was shown in the context of image processing and text mining that non-pairwise models are more efficient in practice, and theoretical arguments also highlight the importance of high-order interactions Tubiana and Monasson  (2017). In terms of generative performance, our results on Lattice Proteins and WW domain MSAs show that, for the same number of parameters, dReLU RBM perform better than Gaussian and Bernoulli RBM.
Similar results, not shown, were obtained for the Kunitz domain MSA. Although RBM with Bernoulli hidden units are known to be universal approximators as → ∞ Le  Roux and Bengio (2008) , they require more hidden units than the other types; hence more data. This can be intuitively explained by the fact that Bernoulli units cannot naturally express modulation in the degree of presence of a feature. To overcome this issue, one needs more than one hidden unit to encode each feature, as in Nair and  Hinton (2010). This is consistent with the heavier distribution of hidden units correlations observed in all data sets, see Appendix 1, Fig. 4. For example, for RBM for Bernoulli potentials, 51 out of 100 hidden units encode gap stretches, as opposed to 23 for quadratic and 15 for dReLU potentials; on WW, the numbers are respectively 18, 15 and 9.