dataesr/scientific-paragraphs-categorization huggingface dataset:
• LINU. The communication graph of the Linux kernel mailing list. An edge (u, v, t) means that user u sent an email to user v at time t  [44]. • TWIT. Tweets about the migrant crisis of 2015. A directed edge (u, v, t) means that user u retweeted a tweet of user v at time t  [47, 48]. For all graphs (apart from TWIT), we have computed the exact values of the closeness centrality for all nodes, in order to evaluate the quality of our approximation algorithm and of our ranking algorithm. In the case of TWIT, we have executed only the approximation algorithm in order to deduce some properties of the graph. Our computing platform is a machine with Intel(R) Xeon(R) CPU E5-2620 v3 at 2. 40 GHz, 24 virtual cores, 128 Gb RAM, running Ubuntu Linux version 4. 4. 0-22-generic. The code was written in Java, using Java 1.
8, and it is available at https://github. com/piluc/TemporalCloseness. Hereafter, we will refer to the exact algorithm as EXACT, and to our approximation algorithm as APX-h, where h denotes the sample size in the definition of the closeness estimator. For each graph in our dataset, we ran APX-h setting h equal to 32, 64, 128, 256, 512, 1024, and repeating each experiment 50 times.
89 for h = 1024 (as shown in Table  A3  in the Appendix A, in the entire dataset, the τ value is always higher than 0. 865 for h = 1024). In the case of TWIT, we used, as the reference ranking r, a ranking obtained by running APX-1024 and we analyzed the τ values of APX-h up to h = 512. Once again, the obtained τ is greater than 0. 9 already for h = 128. This result, combined with the analysis of the relative error depicted in Figure  5 , strongly suggests that the strategy described at the end of Section 3 to find the top-k nodes might turn out to be very efficient: the verification of this hypothesis is the goal of the next section.
In particular, for each h, it reports the median of the γ  (20)  values found among 50 experiments (together with the minimum, maximum, first, and third quartile). As can be seen, despite the fact that the variability is relatively high for small sample sizes (namely, for h = 32 and h = 64), already with h = 128 it significantly reduces. In particular, by setting h = 512, we have that the top-20 nodes are always in the first 512 positions of the ranking found by APX-512. This suggests that, in order to find the exact top-20, it is enough to run APX-512 and then compute the exact closeness of the top-512 found (in O(m) time each). We have verified this hypothesis for all the graphs in our dataset (apart from TWIT) and for different values of k (see Table  2  for the graphs in our sample dataset and Table A4 in the Appendix A for the entire dataset).
As a matter of fact, in the case of large graphs (that is, with more than 20,000 nodes), we can compute the top-20 nodes by executing APX-h with h = 1024 and then compute the exact closeness of the top-h found: the time complexity of this approach would then be O(2048 × m) (which is between 10 and 100 times better than the exact approach, when applied to the large graphs in our dataset). In the case of smaller graphs, the experimental results, described in Table  A4  of the Appendix A, show that a smaller value of h (that is, h = 256) is almost always sufficient, thus giving a similar speed-up. Even more impressive is the fact that, in the case of large graphs, the same value of h (that is, h = 1024) can be actually be used for finding the top-100 nodes.
Funding: C. M. was partly funded by the  ANR (French National Agency of Research)  by the  Limass  project (under grant  ANR-19-CE23-0010 ), and by the  ANR FiT LabCom. A. M. has been partially supported by  MIUR  under the  PRIN  Project  AHeAD  ( Efficient Algorithms  for HArnessing Networked Data) and by the  University of Florence  under Project  GRANTED  ( GRaph Algorithms  for Networked TEmporal Data).
Xatkit tackles these issues by providing a set of Domain Specific Languages to define chatbots (and voicebots and bots in general) in a platform-independent way. Xatkit also comes with a runtime engine that automatically deploys the chatbot application and manages the defined conversation logic over the platforms of choice. Xatkit's modular architecture facilitates the separate evolution of any of its components. Xatkit is open source and fully available online.
II. PRELIMINARIES AND RUNNING EXAMPLE This section defines the key concepts of a chatbot application that are reused through this article. Chatbot design [12] typically relies on parsing techniques, pattern matching strategies and Natural Language Processing (NLP) to represent the chatbot knowledge. The latter is the dominant technique thanks to the popularization of libraries and cloud-based services such as DialogFlow [13] or IBM Watson Assistant [14], which rely on Machine Learning (ML) techniques to understand the user input (based on a set of training sentences provided as part of the chatbot definition) and provide user-friendly interfaces to design the conversational flow. However, Pereira and Díaz have recently reported that chatbot applications can not be reduced to raw language processing capabilities, and additional dimensions such as complex system engineering, service integration, and testing have to be taken into account when designing such applications [15].
g. a typed attribute such as a city name, a date, etc) available to the underlying application. In our approach, Actions are used to represent simple responses such as sending a message back to the user, as well as advanced features required by complex chatbots like database querying or external service calling. Finally, we define a conversation path as a particular sequence of received user intentions and associated actions (including non-messaging actions) that can be executed by the chatbot application. In Xatkit, bots can also be triggered by events. They may subscribe to external events that trigger a reaction on their side without being prompted by a user starting a conversation with them. Same as for intents, the chatbot designer can define the set of actions to be executed in response to an event.
In particular, in this example, we will assume that the project in question is a WordPress plugin and therefore the bot will take care of asking two crucial questions: what WordPress version the user is on and what PHP version the host is running. These two data points are critical for the maintainer to reliably reproduce the issue and efficiently debug the user error, saving time for everybody. On the other hand, this same chatbot can also be useful to alert the owner every time the status of the GitHub repository changes (either because somebody has used the chatbot as a user or because somebody directly interacted with the repository's issue tracker in GitHub). Instead of forcing the owner to keep an eye on GitHub or subscribe to its complex notification system, our bot will ping him on her platform of choice (in this case, we assume that she wants to be pinged on both, Slack and Discord, since it is there where she spends most of its online time).
III. XATKIT FRAMEWORK OVERVIEW Our approach applies Model Driven Engineering (MDE) principles to the chatbot building domain. As such, chatbot models become the primary artifacts that drive all software (chatbot) engineering activities [17]. Existing reports have emphasized the benefits of MDE in terms of productivity and maintainability compared to traditional development processes [18], making it a suitable candidate to address chatbot development and deployment, which, as discussed before, goes much further than simply processing natural language sentences. The following Figure  3  presents an overview of our MDEbased chatbot approach and its main components. At design time the chatbot designer specifies the chatbot under construction using two domain-specific languages (DSLs) part of Xatkit Modeling Infrastructure: • Intent Package to describe the user intentions using training sentences, contextual information extraction, and matching conditions (e.
IV. XATKIT MODELING LANGUAGE In the following we introduce the Xatkit Modeling Language, composed by a set of interrelated chatbot Domain Specific Languages (DSL) that provides primitives to design the user intentions, execution logic, and deployment platforms of the chatbot under construction (this latter one will be described in Section VI). The Xatkit language is defined through two main components [19]: (i) an abstract syntax (metamodel) defining the language concepts and their relationships (generalizing the primitives provided by the major intent recognition platforms [13], [14], [20]), and (ii) a concrete syntax in the form of a textual notation to write chatbot descriptions conforming to the abstract syntax. 6 In the following we use the abstract syntax to describe the DSL packages and primitives, and the textual to show, via examples based on our running case study, how those concepts can be used to create bots.
Let's focus first on the sub bot facing the user (Listing 2). Once the OpenBug IntentDefinition is matched, the bot replies asking the user to provide more information. Note that, this bot directly uses the Slack platform but we could redefine it in a more generic way by using the abstract Chat Platform if the chatbot designer would like to redeploy the bot as a web chat window, for instance. We would write a very similar behaviour for the other intents that collect information about the bug until we have everything we need and are ready to open the issue reporting the bug in GitHub. This is simply done by calling the OpenIssue method provided by the GithubPlatform using as parameters the data stored so far in the bug context. As we have discussed above, the bot can also react to events. As shown in Listing 3, when the event Issue_Opened (one of the events generated by the GitHub platform) arrives, we can get all the details of the new issue and immediately alert the project owner in Slack and Discord.
V. XATKIT RUNTIME The Xatkit Runtime component is an event-based execution engine that deploys and manages the execution of the chatbot. Its inputs are the chatbot model (written with the Xatkit Modeling Language) and a configuration file holding deployment information and platform credentials. In the following we detail the structure of this configuration file, then we present the architecture of the Xatkit Runtime component. Finally, we introduce a dynamic view of the framework showing how input messages are handled by its internal components.
VI. PLATFORM PACKAGE Xatkit comes with a set of platforms packaged as part of the release. 10 8 shows a taxonomy of the 13 platforms available so far including their inheritance links. Abstract classes (e. g. the text ChatPlatform) can be used in place of the concrete ones when the bot does not require any specific method only available in the concrete platform, thus facilitating the reusability of the bot. Nevertheless, it is often the case that a chatbot designer requires a new platform (e. g. to talk with the internal services in the company). If so, a platform designer (remember Figure  3 ) will take care of defining and implementing the platform. This platform designer could be the same person that is designing the chatbot or somebody else hired to perform this specific task. One way or the other, once a platform is created, it can be reused by any past, present or future chatbot.
B. PLATFORM IMPLEMENTATION Xatkit platforms are implemented as standalone Java projects implementing the Xatkit Platform Interface. This interface is used internally by the Xatkit Runtime component to dynamically load platforms, create action instances, and run them when an execution rule is matched. A Xatkit platform consist of a main class holding platformspecific data accessible to all the actions (e. g. OAuth token for our GitHub platform), and a collection of classes representing the Action that can be called on the platform (e. g. the GitHub platform contains an OpenIssue class that contains the code to actually open an issue on GitHub). In addition, optional classes can be defined to add ProviderDefinitions to the platform, and containing the code responsible of parsing received inputs/events into Xatkit IntentDefinition and EventDefinition instances.
VII. TOOL SUPPORT The Xatkit framework is open source and released under the Eclipse Public License v2. 13  The source code of the project and the Eclipse update site are available on the Xatkit GitHub organization. 14 s part of this organization, we provide the different releases of the framework, the runtime component and the repositories for the several platform components and connectors. There is also a wiki (linked from the GitHub organization) and an external website  15  that provide additional documentation, installation instructions and news around the project. To facilitate the specification of the chatbots we provide and Eclipse editor that supports auto-completion, syntactic and semantic validation, and can be installed from the Xatkit Eclipse update site. The concrete syntaxes of the Xatkit modeling languages are implemented with Xtext [21], an EBNFbased language used to specify grammars and generate the associated toolkit containing a meta-model of the language, a parser, and textual editors.
The execution of the chatbots mainly relies on the Xatkit Runtime engine. At its core, the engine is a Java library that implements all the execution logic available in the chatbot languages. In addition, Xatkit provides a full implementation of the IntentRecognitionProvider interface for Google's DialogFlow engine [13], as well as the concrete PlatformInstance implementations for the Slack, Discord, and Github platforms used in the running example (and a few others as mentioned in the previous section). The runtime component can be downloaded and deployed on a server as a standalone application, or integrated in an existing application using a dedicated Maven dependency. Additional integrations (in particular with nlp. js  16  to enable a local NL analysis) are underway. Note that all these new integrations become immediately available to all existing bots due to the clean and modular architecture of Xatkit.
Xatkit also embeds a monitoring component that stores and computes metrics to evaluate the quality of the underlying IntentRecognitionProvider. These metrics can be accessed using a dedicated REST API provided by the Xatkit server,  17  and describe: • The average number of intents matched per user session • The average number inputs that have not been matched per user session  15  https://xatkit. com/ 16 https://github. com/axa-group/nlp. js  17 Presenting this data in a more user-friendly way (e. g. via a dedicated dashboard) is left for future work. • The distribution of matched intents (to help designers understand which conversation flows are the most followed) • The list of unmatched inputs to help designers integrate them in their existing intents when relevant • The average recognition confidence level per user session, as well as per intent, in order to ease the detection of intents that should be improved • The average user session time Note that for now the computed metrics focus on the quality of the intent recognition, but we are currently working on it to cover other aspects of running Xatkit bots such as performance measurements, or user engagement.
A. EMPIRICAL SETTING The seminar lasted three days and was taught by Robert Clarisó (a member of our research group, but not a co-author of Xatkit). The first two days focused on teaching core modeling principles while the last one included a remote presentation by Gwendal Daniel introducing Xatkit as an example of the use of model-based solution to build advanced software systems. After the three teaching days, the students were asked to spend a minimum of 30 additional hours of work developing a model-based solution around Xatkit. This could either be: • A bot reusing existing Xatkit platforms • An implementation of a new platform that extended Xatkit to a new domain The first assignment was considered feasible for everybody while the second one was targeting students that could have a special interest in the chatbots domain and wanted to go deeper in their exploration of Xatkit.
B. SURVEY RESULTS The following figures show that Xatkit was evaluated very positively in a number of categories, ranging from the overall experience with Xatkit (Figure  11 ), the usability of Xatkit's modeling language (Figure  12 ), the power of the platform abstraction mechanism (Figure  13 ), the benefits of defining chatbots at a higher-abstraction level (Figure  14 ), how this helps to the portability between messaging platforms (Figure  15 ) and the quality of the Xatkit IDE (Figure  16 ). Regarding the improvement suggestions, most were around adding new platforms to Xatkit, especially input platforms such as Facebook and WhatsApp that would have allowed them to build chatbots for those platforms as part of their assignment. Two students also asked for support for other Intent Recognition Providers beyond DialogFlow.
However, their fine granularity is cumbersome for network-scale research. Omics and systems biology formats, such as BioPAX,  27  the Biological Expression Language (BEL),  28  the MODOMICS nomenclature,  7  the PRO notation,  13  ProForma,  29  and the Synthetic Biology Open Language (SBOL),  30  use abstractions that are conducive to network-scale research. However, these formats have limited abilities to represent non-canonical residues, caps, crosslinks and nicks, and they do not concretely represent the primary structures of macromolecules. Toward fine-grained global networks of intracellular biochemistry, we developed BpForms-BcForms, an open-source toolkit for abstractly representing the primary structure of polymers and complexes. BpForms includes extensible alphabets of hundreds of DNA, RNA and protein residues; an ontology of common crosslinks; and a human and machine-readable grammar for combining residues, residue modifications, intra-chain crosslinks, and nicks into polymers.
BcForms includes a human and machine-readable grammar for combining polymers, small molecules, and inter-chain crosslinks into complexes. Both tools include software for validating descriptions of macromolecules, calculating properties of macromolecules such as their formula, visualizing macromolecules, and exporting macromolecules to molecular formats such as SMILES. Both tools are available as a web application, REST API, command-line program, and Python library. Here, we describe the toolkit and demonstrate how it can facilitate omics, systems modeling, and synthetic biology. First, we describe the toolkit, including the alphabets of residues, the ontology of crosslinks, the grammars, the software tools, and the user interfaces. Second, we describe how BpForms and BcForms can be integrated with knowledge about pathways, kinetic models, and genetic designs through formats such as BioPAX, CellML, 31 SBML, and SBOL.
Toolkit for abstracting non-canonical polymers and complexes The BpForms-BcForms toolkit includes several interrelated tools for describing, validating, visualizing, and calculating properties of the primary structure of DNA, RNA, proteins, and complexes (Figure  1 ). Here, we describe the components of the toolkit including the abstractions and grammars for polymers and complexes; the alphabets of residues; the ontology of crosslinks; the software tools for quality controlling, analyzing, and visualizing macromolecules; the protocols for integrating BpForms and BcForms with formats for network research; and the user interfaces. Abstract representation of the primary structure of polymers and complexes. BpForms represents polymers as a sequence of residues, a set of crosslinks, a set of nicks, and a Boolean indicator of circularity (Figure  2B, D ).
To support a broad range of research, Bp-Forms includes the most extensive alphabets of DNA, RNA, and protein residues to date. The DNA alphabet includes 422 deoxyribose nucleotide monophosphates and 3' and 5' caps derived from data about DNA damage and repair from REPAIREtoire, 4 structural data from the Protein Data Bank Chemical Component Dictionary (PDB CCD),  32  and chemoinformatics data from DNAmod. 3  The RNA alphabet includes 378 ribose nucleotide monophosphates and 3' and 5' caps derived from biochemical data from MODOMICS  33  and the RNA Modification Database 8 and structural data from the PDB CCD. The protein alphabet has 1,435 amino acids and carboxy and amino termini derived from biochemical data from RESID 10 and structural data from the PDB CCD. The BpForms website contains pages which display the residues in each alphabet.
For compatibility with genomics research, the toolkit can also export the canonical sequences of BcForms-encoded polymers to the IUPAC/IUBMB format 34 and FASTA documents. 35 tegration with frameworks for network-scale research. BpForms and BcForms can facilitate network-scale research through integration with omics and systems and synthetic biology frameworks such as BioPAX, CellML, SBML, and SBOL. Additional File 1. 9 illustrates how Bp-Forms and BcForms can be incorporated into these frameworks. User interfaces. BpForms and BcForms each include four user-friendly interfaces: a web application, a REST API, a command-line program, and a Python library.
Additional File 1. 11. 1 and Table  S1  provide a detailed comparison of BpForms with several other formats. Comparison of BpForms alphabets with existing databases. The BpForms alphabets are the most extensive alphabets of DNA, RNA, and protein residues because they are based on structural, biochemical, and physiological data from several sources. In addition, the BpForms alphabets and the PDB CCD are the only alphabets which consistently represent DNA, RNA, and protein residues and which represent the inter-residue bonding sites of each residue, enabling residues to be combined into concrete molecular structures. In contrast, DNAmod, REPAIRtoire, MODOMICS, RESID, and the RNA Modification Database each only represent DNA, RNA, or protein residues; the residues in DNAmod, REPAIRtoire, MODOMICS, and the RNA Modification Database are hard to compose into polymers because they represent nucleobases and nucleosides rather than nucleotides; and DNAmod, REPAIRtoire, MODOMICS, RESID, and the RNA Modification Database do not capture bonding sites.
Case studies We believe that the BpForms-BcForms toolkit can support a wide range of omics and systems and synthetic biology research. Here, we illustrate how we have used the toolkit to improve the quality of the PRO database of proteoforms; analyze the metabolic cost of tRNA modification in Escherichia coli; refine, expand, a compose a model of MAPK signaling with models of other pathways; and identify constraints on designing new strains of E. coli. Proteomics: Quality control of the Protein Ontology. One of the goals of proteomics is to characterize the proteoforms in cells. Toward a comprehensive catalog of proteoforms, the PRO consortium has manually integrated several different types of data into PRO, a database of 8,095 proteoforms. Because the consortium constructs PRO, in part, by hand, automated quality controls could help the consortium identify and correct errors in PRO.
We have used BpForms quality control PRO. First, we encoded each entry in PRO into the BpForms grammar and used the BpForms software to validate each entry. This identified several types of syntactical and semantic errors. For example, we identified annotated processing sites that have invalid coordinates that are greater than the length of the translated sequence of their parent protein. We also identified modified residues whose structures are inconsistent with the translated sequences of their parent proteins, such as a phosphorylated serine which is annotated at the position of a tyrosine in the translated sequence of its parent. Second, the consortium corrected these errors. These improvements will be published with the next release later this year. To enable the consortium to continue to use BpForms to quality control PRO, we developed a script which automates this analysis.