dataesr/scientific-paragraphs-categorization huggingface dataset:
In particular, the PDA representing {s} • G is constructed so that the translation grammar score appears in both w 1 and w 2 (i. e. , it is duplicated). In the first-pass language model, w 1 has the n-gram language model scores and the w 2 are 0. After composition, the resulting automata have the combined translation grammar score and language model score in the first dimension, and the second dimension contains the translation grammar scores alone. Pruning can be performed under the lexicographic semiring with a threshold set so that only the combined scores in the first dimension are considered. The resulting automata can easily be mapped back into the regular tropical semiring such that only the translation scores in the second dimension are retained (this is a linear operation done by the fstmap operation in the OpenFST library).
To create a suitable baseline we therefore reduce G by excluding rules that have a forward translation probability p < 0. 01, and refer to this reduced grammar as G small. This process reduces the number of strictly hierarchical rules that apply to our tune-nw set from 511K to 189K, while the number of standard phrases is unchanged. Under G small , both HiFST and HiPDT are able to exactly compose the entire space of possible candidate hypotheses with the language model and to extract the shortest path hypothesis. Because an exact decoding baseline is thus available, we can empirically evaluate the proposed two-pass strategy. Any degradation in translation quality can only be due to the modeling errors introduced by pruning under β with respect to the entropy-pruned M θ 1. Figure  15  shows translation performance under grammar G small for different values of entropy pruning threshold θ.
BLEU score is measured over the entire test set test-nw but speed is calculated only on sentences of length up to 20 words (∼500 sentences). In computing speed we measure not only the PDA operations, but the entire HiPDT decoding process described in Figure  14 , including CYK parsing and the application of M 1. We note in passing that these unusually slow decoding speeds are a consequence of the large grammars, language models, and broad pruning thresholds chosen for these experiments; in practice, translation with either HiPDT or HiFST is much faster. In these experiments we find that the language model entropy pruning threshold θ and the likelihood beamwidth β work together to balance speed against translation quality. For every entropy pruning threshold θ value considered, there is a value of β for which there is no degradation in translation quality.
Rescoring with 5-Gram Language Models and LMBR Decoding r Does the HiPDT two-pass decoding generate lattices that can be useful in rescoring? We now report on rescoring experiments using WFSAs produced by the two-pass HiPDT translation system under the large translation grammar G. We demonstrate that HiPDT can be used to generate large, compact representations of the translation space that are suitable for rescoring with large language models or by alternative decoding procedures. We investigate translation performance by applying versions of the language model M 2 estimated with stupid backoff. We also investigate minimum Bayes risk (MBR) decoding  (Kumar and Byrne 2004)  as an alternative search strategy. We are particularly interested in lattice MBR (LMBR)  (Tromble et al. 2008) , which is well suited for the large WFSAs that the system can generate; we use the implementation described by  Blackwood, de Gispert, & Byrne (2010).
Because LMBR relies on posterior distributions over n-grams, we conclude that HiPDT is able to generate compact representations of large search spaces with posteriors that are robust to pruning conditions. Finally, we find that increasing θ degrades performance quite smoothly for β ≥ 9. Again, with appropriate choices of θ and β we can easily reach a compromise between decoding speed and final performance of our HiPDT system. For instance, with θ = 7. 5 × 10 -7 and β = 12, for which we decode at a rate of 3 words/sec as seen in Figure  16 , we are losing only 0. 5 BLEU after LMBR compared to θ = 7. 5 × 10 -7 and β = 15.
We developed similar procedures for our HiFST decoder  (Iglesias et al. 2009a; de Gispert et al. 2010 ) via a different route, after noting that with the space of translations represented as WFSAs, alignment can be performed using operations over WFSTs  (Kumar and Byrne 2005). Although entropy-pruned language models have been used to produce real-time translation systems  (Prasad et al. 2007) , we believe our use of entropy-pruned language models in two-pass translation to be novel. This is an approach that is widely used in automatic speech recognition  (Ljolje, Pereira, and Riley 1999)  and we note that it relies on efficient representation of very large search spaces T for subsequent rescoring, as is possible with FSAs and PDAs.
r As predicted by the analysis of algorithmic complexity, intersection and expansion algorithms based on the PDA representation are able to perform exact decoding with large translation and weak language models. By contrast, RTN to FSA expansion fails with large translation grammars, regardless of the size of the language model. With large translation grammars, language model composition prior to expansion may be more attractive than expansion prior to language model composition. r Our experimental results suggest that for a translation grammar and a language model of a particular size, and given a value of language model entropy pruning threshold θ, there is a value of the pruned expansion parameter β for which there is no degradation in translation quality with HiPDT. This makes exact decoding under large translation grammars possible.
We present strategic portgraph rewriting as a basis for the implementation of visual modelling and analysis tools. The goal is to facilitate the specification, analysis and simulation of complex systems, using port graphs. A system is represented by an initial graph and a collection of graph rewriting rules, together with a user-defined strategy to control the application of rules. The strategy language includes constructs to deal with graph traversal and management of rewriting positions in the graph. We give a small-step operational semantics for the language, and describe its implementation in the graph transformation and visualisation tool PORGY.
Introduction In this paper we present strategic portgraph rewriting as a basis for the design of PORGY -a visual, interactive environment for the specification, debugging, simulation and analysis of complex systems. PORGY has a graphical interface  [23]  and an executable specification language (see Fig. 1 ), where a system is modelled as a portgraph together with portgraph rewriting rules defining its dynamics (Sect. 2). Reduction strategies define which (sub)expression(s) should be selected for evaluation and which rule(s) should be applied (see  [19, 8]  for general definitions). They are present in programming languages such as Clean  [24] , Curry  [17] , and Haskell  [18]  and can be explicitly defined to rewrite terms in languages such as ELAN  [7] , Stratego  [31] , Maude  [21]  or Tom  [4].
They are also present in graph transformation tools such as PROGRES  [29] , AGG  [12] , Fujaba  [22] , GROOVE  [28] , GrGen  [14]  and GP  [27]. PORGY's strategy language draws inspiration from these previous works, but a distinctive feature is that it allows users to define strategies using not only operators to combine graph rewriting rules but also operators to define the location in the target graph where rules should, or should not, apply. The main contribution of this paper is the definition of a strategic graph program (Sect. 3): it consists of an initial located graph (that is, a portgraph with two distinguished subgraphs P and Q specifying the position where rewriting should take place, and the subgraph where rewriting is banned, respectively), and a set of rewrite rules describing its dynamic behaviour, controlled by a strategy.
Unlike those papers, the notion of portgraph considered in this paper includes attributes for nodes, ports and also edges, which are taken into account in the definition of portgraph morphism. In addition, the strategy language includes a sublanguage to deal with properties of graphs, which facilitates the specification of rewrite positions and banned subgraphs (to be protected during rewriting). The operational semantics of the language, including the non-deterministic sublanguage, is formally defined using a transition system.
The syntax presented here extends the one in  [13]  by including a language to define subgraphs of a given graph by selecting nodes that satisfy some simple properties (see Table  2 ). The focusing constructs are a distinctive feature of our language. They are used to define positions for rewriting in a graph, or to define positions where rewriting is not allowed. They denote functions used in strategy expressions to change the positions P and Q in the current located graph (e. g. , to specify graph traversals). We describe them briefly below. • CrtGraph, CrtPos and CrtBan, applied to a located graph G Q P , return respectively the whole graph G, P and Q. • AllNgb, OneNgb and NextNgb denote functions that apply to pairs consisting of a located graph | F ∪ F | F ∩ F | F \ F | / 0 Table 1: Syntax of the strategy language.
It typically tests a property on nodes, ports, or edges, allowing us for instance to select the subgraph of nodes with active ports: Property((Port, Active == true), F). It is also possible to specify a function to be used to compute the subgraph: Property((Function, Root), CrtGraph) uses the built-in (or user-defined) function Root to compute a specific subgraph from the current graph. • ∪, ∩ and \ are union, intersection and complement of port graphs which may be used to combine multiple Property operators; / 0 denotes the empty graph. Other operators can be derived from the language constructs. A useful example is the not construct: • not(S) if(S)then(Fail)else(Id). It fails if S succeeds and succeeds if S fails. Strategic graph program A strategic graph program consists of a finite set of located rewrite rules R, a strategy expression S (built with R using the grammar in Table  1 ) and a located graph G Q P.
Implementation PORGY is implemented on top of the visualisation framework Tulip  [3]  as a set of Tulip plugins. The strategy language is one of these plugins. A version of Tulip bundled with PORGY can be downloaded from http://tulip. labri. fr/TulipDrupal/?q=porgy. Our first challenge was to implement port graphs, because Tulip only supports nodes and edges from a graph theory point of view. We had to develop an abstract layer on top of the Tulip graph library to be able to easily work with portgraph. When applying a rule L ⇒ R on a graph G, the first operation is to compute the morphism between the left-hand side L and G. This problem, known as the graph-subgraph isomorphism, still receives great attention from the community. We have implemented Ullman's original algorithm  [30]  because its implementation is straightforward and it is used as a reference in many papers.
Single Player Implementations Perhaps the most famous of these tools is Moped  [21, 39]  and its incarnation as a model checker for Java, JMoped  [52, 50]. In taking the algorithm from a theoretical tool to a practical one, a number of new concerns had to be taken into account. The rules of a pushdown system roughly correspond to the statements in a program. In a program with thousands of lines, a fixed point iteration that checks, during each iteration, whether each rule leads to new transitions in the automaton would be woefully inefficient. In constructing Moped, Esparza et al. [20]  showed how this naive outer loop can be reorganised such that, at each iteration, only the relevant rules of the system were considered, leading to a significant improvement in performance. A second consideration of applications to the analysis of program models is the handling of data values.
Boolean programs are essentially pushdown systems where each control state and stack character contains a valuation of a set of global and local boolean variables respectively. These boolean programs are the natural output of predicate abstraction tools such as SATABS  [18]  as well as the target compilation language of JMoped. Since there are only finitely many valuations of sets of boolean variables, they can directly be encoded as control states or characters and standard pushdown analysis techniques can be employed. However, since they are also exponential in number, such an approach is inherently inefficient. Hence, Esparza et al. introduced symbolic pushdown systems  [21]  which make boolean valuations first class objects. The saturation technique was extended by adding BDDs representing variable valuations to the edges of the P-automata, leading to an implementation capable of analysing symbolic pushdown systems derived from real-world programs.
Primarily this work has focussed on a specification language that is an extension of CTL and its translation into symbolic pushdown Büchi games  [44, 46]  resulting in the tool PuMoC  [45]. The main application of this work has been in the detection of malware. More recently still, this work has been developed for LTL-like properties to deal with situations where the CTL approach was insufficient  [47] , culminating in the PoMMaDe tool  [49]. However, the combination of BDD representations and alternating automata is not an easy one, since BDDs lack the necessary alternation for a direct embedding. Hence, Song's algorithm pays an extra exponential in its worst-case complexity (doubly exponential rather than exponential), although the practical runtime is improved. The optimal inclusion of symbolic representations into the analysis of two-player games remains an open problem.
Computational results The GRASP×ELS is benchmarked over the 60 generated instances and it is compared with solutions obtained by linear programming. All procedures are implemented under Delphi 6. 0 package and experiments were carried out on a 2. 8 GHz computer under Windows 7 with 12 GO of memory. The results for 13 instances whose optimal solutions are obtained using CPLEX are given in Table  1. As we make ten replications, the average value of the makespan, CO 2 and CPU times are considered in Table  1. For these instances our approach found ten optimal solutions concerning the makespan objective and an average deviation from the optimal solutions of 0. 17%. Whit an average deviation of 0. 08% from the CO 2 objective optimal solutions we can state that the quality achieved is very good.
Solutions are found in rather short computational time (in average of 1. 20 s), which proves that GRASPxELS is a powerful method. To estimate the quality of proposed framework we compared the results with the lower bounds (LB) calculated by the linear programming with CPLEX 24 hours' time limit. For 33 instances lower bounds are obtained and the average improvement of the lower bounds is 29. 53% for makespan and 37. 33% for the CO 2. For the rest of the instances  (14)  no CPLEX solutions are obtained during time limit. Moreover, all 60 solutions (job sequences) obtained by our GRASPxELS are compared with those obtained by linear programming as follows: for each best job sequence generated by GRASPxELS linear program found the optimal solution concerning the makespan and the CO 2. CPLEX results are compared with these obtained by our metaheuristic.
The objective of this article is to show that SEAMLESS, a designed and developed method from PCO Innovation, allows improving PLM approach efficiency by proposing to set up a business/Information System referential. PLM, as a PLM solution, is a software which basically supports product conception and product industrialization business processes (and potentially other upstream and/or downstream business processes). The functional scope may be really wide and the various amounts of subjects are strongly connected to each other. A PLM approach (which seeks to establish a PLM solution in the organization) is basically composed of a program with several projects. Among all PLM approach activities; upstream activities (and specifically specification activities) are absolutely crucial. This kind of approach is, generally, lengthy and complex.
Finally, a business/information system referential allows allocating to a particular element one or several labels in one or several languages. In this manner, we are able to capture several designations for a same thing (as it is often observed in the same enterprise) and also in a multilingual way. For example, the same business concept is named differently by R&D and marketing team. Furthermore, we are able to determine new informations (from existing informations) using inference mechanism. For example, interaction between projects is induced by interactions between processes Also, knowledge referential can be considered like a structured database which can be requested to only extract some elements (depending on the context). For example, we may consider these extractions:  Compact or detailed representation of business process (with or without interims informations)  Provide adapted representation for each kind of actors in the different projects of a PLM program  Publish representations for all actors (synthesis).
g. a formal semantic model with concepts). SEAMLESS is assisted, in the sense that this methodological semantic approach could theoretically be implemented without any particular tool, but in practice relies on a software solution which is specifically adapted to sustain it. It is a key point to note that SEAMLESS has been initially designed to be applied on a knowledge domain (even far away from PLM); and that's why, in the scope of this section, the primary interest will appear to be focused on SEAMLESS approach and then on the particular application field: "SEAMLESS applied to PLM specification activities".
In order to achieve this, SEAMLESS designs, develops and deploys solutions which allow equipping activities of increasing knowledge, management and necessaries informations restitution to execute a mission. These solutions are respectful of knowledge management and engineering best practices, adaptable to context (because they are comprised of generic software components, re-usables), consistent to web semantic technologies, norms and standards and based on open-source tools (like Jena API and Protégé software). SEAMLESS approach is an iterative approach which allow specifying knowledge domain model, building knowledge model-driven referential (eg. populate the knowledge base with respect to the specified model) and, then, exploiting them using pre-cabled requests which allow representing and visualizing all or some parts of the knowledge referential.
The large collections of ChIP-seq data rapidly accumulating in public data warehouses provide genome-wide binding site maps for hundreds of transcription factors (TFs). However, the extent of the regulatory occupancy space in the human genome has not yet been fully apprehended by integrating public ChIP-seq data sets and combining it with ENCODE TFs map. To enable genome-wide identification of regulatory elements we have collected, analysed and retained 395 available ChIP-seq data sets merged with ENCODE peaks covering a total of 237 TFs. This enhanced repertoire complements and refines current genome-wide occupancy maps by increasing the human genome regulatory search space by 14% compared to ENCODE alone, and also increases the complexity of the regulatory dictionary. As a direct application we used this unified binding repertoire to annotate variant enhancer loci (VELs) from H3K4me1 mark in two cancer cell lines (MCF-7, CRC) and observed enrichments of specific TFs involved in biological key functions to cancer development and proliferation.
INTRODUCTION Differences in gene expression programs are believed to play a major role in cell identity and phenotypic diversity in the human body. With the advances of next generation sequencing techniques it became possible to study the genome-wide occupancy maps of transcription factors (TFs) by chromatin immunoprecipitation followed by sequencing (ChIPseq). The rapid accumulation of ChIP-seq results in data warehouses provides a unique resource of hundreds of occupancy maps. With the success of the Encyclopedia of DNA elements (ENCODE) project to identify all functional elements in the human genome, the description and annotation of TF binding sites (TFBS) entered a genome-wide era by integrating a hundred TFs. The extent to which the regulatory space is organized along the genome is only starting to unfold with large consortia studies (1-5) but remains largely matter of discoveries (e.
g. super enhancers)  (6, 7). Indeed, recent studies have uncovered hundreds of genomic loci that are cooccupied by multiple TFs in various cell types suggesting the importance and abundance of combinatorial regulation in cells  (2, 7, 8). So far, those diverse regulatory features generated from various studies have not been yet integrated to form a global map of regulatory elements. Here, we report the complex landscape of TFBS in the human genome. We have constructed a global map of regulatory elements by compiling the genomic localization of 132 different TFs across 83 different cell lines and tissue types based on 395 selected human public (non-ENCODE) data sets. The integration of the genome-wide TFBS allows for the construction of a catalogue of cis-regulatory modules (CRMs) of variable complexity.
Specifically, we report a complex map of TFBS increasing by 14% (+439 Mb, +993 421 regulatory features) the human genome regulatory search space compared to the EN-CODE catalogue alone. Different studies have proposed to integrate various NGS ChIP-seq data sets but either from a workflow/platform approach  (9)  or from a quality assessment perspective  (10). We performed a detailed comparison of the TFs occupancy map generated from public data against the ENCODE TF catalogue. Both maps are examined at the levels of TFs, TFBS and CRMs scales; finally, both maps are merged allowing the creation of a large catalogue of complex organization of bound regions. In total, after including ENCODE TF data to complement public data, we examined 237 TFs across multiple cell types. This map has been compiled into public tracks in genome browsers allowing users to assess regulatory elements com-Public ChIP-seq processing Bowtie 2  (11)  with options -end-to-end -sensitive was used to align all reads on the human genome (GRCh37/hg19 assembly).
Motif discovery The RSAT (Regulatory Sequence Analysis Tools) suite was used for de novo motifs analyses in non-redundant binding sites of each TF. The RSAT program peak-motifs  (18)  with the options --markov auto -minol 6 -maxol 8 -merge lengths -2str -scan markov 3 was run to detect overrepresented words and dyads, which were then compared against known motifs from JASPAR core vertebrates database (version November 2013). ENCODE motifs were extracted from FactorBook  (19) , and JASPAR motifs from the JASPAR core vertebrates database (20) (version 5. 0 alpha).
Conservation scores For each of the 237 TFs present in our catalogue we assessed the DNA constraint for each base pairs 1 kb around the summit of each peak. Conservation scores were obtained from the Ensembl Compara database (21) release v67. Genomic Evolutionary Rate Profiling (GERP;  (22) ) score was used to calculate the conservation of each nucleotide in multi-species alignment. The multiple whole genome alignment used to derive GERP score is the 20-way amniota vertebrates Enredo-Pecan-Ortheus (EPO) alignment.