dataesr/scientific-paragraphs-categorization huggingface dataset:
V. PERFORMANCE EVALUATION In order to evaluate DeepCog, we consider a real-world scenario, consisting of a mobile network deployed in a large metropolitan region of around 100 km 2. We leverage measurement data about the demands for individual mobile services in the target region, collected by the network operator and generated by several millions of users. In our experiments, we use demands expressed in bytes, and collected at the gateway of an operational mobile network by monitoring the GPRS Tunneling Protocol (GTP). Independent network slices are then assigned to a representative set of services, listed in Tab. I. Our selection covers popular applications with diverse requirements in terms of bandwidth and latency: this lets us provide an adequate picture of the performance of DeepCog under the heterogeneous network traffic that characterizes current mobile service demands.
e. , T = 6) arranged in a 47 × 10 matrix as an input  1. Thus, the next-step forecast occurs over an horizon of 5 minutes. The rationale for this choice roots in practicality: resource reallocation updates in the order of minutes are typical for computational and memory resources in architectures implementing NFV  [29] , and are in line with those supported by any state-of-the-art Virtual Infrastructure Manager (VIM)  [30]. We leverage the reference scenario described above to evaluate the performance of DeepCog. We employ two months of mobile traffic data for training and another two weeks of data for actual experiments. This setting is also used for all benchmark approaches. All results are derived with a high level of confidence and low standard deviation.
ACKNOWLEDGMENTS The work of  University Carlos III of Madrid  was supported by the  H2020 5G-MoNArch  project (Grant Agreement No. 761445 ) and the work of  NEC Laboratories Europe  by the  5G-Transformer project  (Grant Agreement No. 761536 ). The work of  CNR-IEIIT  was partially supported by the  ANR   CANCAN  project ( ANR-18-CE25-0011 ).
Genome analyses To conduct the bioinformatic analyses we developed a series of scripts in Python (version 2. 7. 12) (www. python. org), freely available upon request. These codes were applied to a panel of 25 bacterial genomes, including relevant pathogens, and chosen to span a wide range of GC compositions. A summary of the main features of these genomes is presented in Supplementary Table  S1. For all strains, the predicted coding sequences (CDSs) and their functional classification (COG) were retrieved from the Microscope platform from Genoscope (www. genoscope. cns. fr) (Vallenet et al. , 2006). After formatting and parsing, we estimated the average protein-disrupting effect of different mutations for all CDSs across the panel of genomes. We achieved this by computing the Grantham and BLOSUM100 scores for all of the possible substitutions per codon associated with each mutational spectrum.
The Grantham and BLOSUM100 matrices were obtained from the AAindex database (www. genome. jp/aaindex) (Kawashima and Kanehisa, 2000)  and the NCBI FTP server (ftp. ncbi. nih. gov/blast/matrices), respectively. Codons harbouring incompletely specified bases (e. g. N, R, Y) were excluded from the analyses. S1.
Metabolic engineering aims to produce chemicals of interest from living organisms, to advance towards greener chemistry. Despite efforts, the research and development process is still long and costly and efficient computational design tools are required to explore the chemical biosynthetic space. Here, we propose to explore the bio-retrosynthesis space using an Artificial Intelligence based approach relying on the Monte Carlo Tree Search reinforcement learning method, guided by chemical similarity. We implement this method in RetroPath RL, an open-source and modular command line tool. We validate it on a golden dataset of 20 manually curated experimental pathways as well as on a larger dataset of 152 successful metabolic engineering projects. Moreover, we provide a novel feature, that suggests potential media supplements to complement the enzymatic synthesis plan.
Introduction Efficient computational tools are required for metabolic engineering to achieve its true potential as a game-changer in the bioeconomy. Such tools include pathway design software, to assist the metabolic engineer in finding new pathways for production of valuable chemicals. While some tools restrict themselves to reactions already present in databases  (Rodrigo et al. 2008; Moriya et al. 2010) , others allow the generation of de novo reactions, using retrosynthesis algorithms  (Henry, Broadbelt, and Hatzimanikatis 2010; Carbonell et al. 2011; Yim et al. 2011; Carbonell et al. 2014; Liu et al. 2014; Campodonico et al. 2014; Hadadi et al. 2016 ; Kumar et al. 2018; Delépine et al. 2018). At its core, a retrosynthesis algorithm is simple: break down a target molecule into simpler molecules that can be combined chemically or enzymatically to produce it and iterate recursively until all required compounds are either commercially available or present in the microbial strain of choice.
To address those limitations, we propose to make use of the Monte Carlo Tree Search (MCTS) reinforcement learning algorithm which has already revolutionised the field of Artificial Intelligence, as illustrated by the stunning victory of Google's AI (AlphaGo) against a Go master in 2016  (Silver et al. 2016 (Silver et al. , 2017 (Silver et al. , 2018 ). An interesting application used this algorithm combined with neural networks in chemical retrosynthesis, but acknowledging that natural compounds synthesis was beyond their scope  (Segler, Preuss, and Waller 2018). To make available the algorithm to the wider audience, we implement it as RetroPath RL, an open source python package freely available on GitHub. RetroPath RL provides a programmatic access to discover biosynthesis pathways using MCTS, while allowing a number of augmentations and features to be used or developed by expert users to tune it to their needs.
Reaction rules for representing enzymatic reactions Reaction rules describe the changes in bonding patterns when a set of substrates is transformed into a set of products in order to encode enzymatic reactions for retrosynthesis. An important feature of rules for retrosynthetic applications is that they need to be generalisable, so that they can be applied to a new substrate that was not from amongst the substrates the rules were learned on. Moreover, using generalised reaction rules is the first step towards predicting promiscuous reactions, as those reactions are often missing from metabolic databases. Modelling promiscuity is a key feature in metabolic engineering, as it has for example been estimated that 37% of Escherichia coli K12 enzymes have a promiscuous activity on structurally similar substrates  (Nam et al.
2012). In our data-driven approach, we learn rules at various levels of specificity around the reaction present in the database, by keeping in the described pattern of the rule a varying number of atoms around the reaction centre. We select those atoms using a number we call diameter that represents the distance in bonds around the reaction centre: a rule at diameter 2 will include atoms at a distance of 1 around the reaction centre, while a rule a diameter 10 will include atoms at a distance of 5 around the reaction centre. Therefore, the rule at diameter 2 can apply to more diverse substrates and therefore encode more promiscuity than the rule at diameter 10. A more detailed description of reaction rules can be found in  (Delépine et al. 2018; Duigou et al. 2019 ) and in the Methods section.
Chemical similarity and sequence availability for reaction ranking Chemical similarity between query (applied on a new substrate) and the native chemical transformation has been used in various studies  (Cho et al. 2010; Hadadi et al. 2019; Coley et al. 2017; Campodonico et al. 2014). We adapted the strategy from Coley et al. that proceeds in a 2-step evaluation of the reaction. In a first step, before rule application, similarity between query and native substrates is calculated. After rule application, similarity between native and query products is also calculated. This allows accounting of similarity in a manner straightforward to use with mono-component reaction rules (Figure  1A ). Using this metric allows us to select chemical reactions similar to the ones present in metabolic databases, increasing our chances that this predicted reaction can be catalysed.
Results and Discussion Evaluating RetroPath RL with a golden dataset We first evaluate our implementation of MCTS for bio-retrosynthesis on a manually curated dataset of 20 compounds to identify the best settings for a retrosynthetic search on those compounds (chosen compounds and rationale for selection are available in the Methods section). This dataset allows us to verify bio-retrosynthesis tools suggest pathways that have been experimentally described in the literature, ensuring the biological relevance of the predicted pathways. Comparing RetroPath RL's features for expert users (see Supplementary Note 1) on this golden dataset allows us to select the best parameters possible for biological relevance. While various metrics could be available to describe what a good bio-retrosynthesis algorithm should do, there is no obvious consensus.
One step lacking (light blue) means the search algorithm found a pathway identical to the experimental one, except one step which was short-cut. Fully found (green) means the experimental pathway was found without restriction. Not found (orange) means the experimental pathway was not found. The best parameters set we found used chemical and biological thresholds of 0. 3, and a maximum of 10 allowed children per node (detailed configuration is available as Supplementary Table  2  and effects of various parameters were investigated and presented in Supplementary Note 1). To generate the results presented in Figure  3 , we ran RetroPath 2. 0, and RetroPath RL with its default configuration and a configuration with more tolerant chemical scores for rule application. Detailed analysis of results on this dataset is available in Supplementary Note 2.
Evaluating RetroPath RL on successful metabolic engineering projects After validating biological relevance of predictions for metabolic engineering, we tested RetroPath RL on a larger dataset. Our previous tool  (Delépine et al. 2018 ) was tested on the LASER database  (Winkler, Halweg-Edwards, and Gill 2015, 2016 ) that compiles successful metabolic engineering projects, completed with compounds taken from the Metabolic Engineering journal (see Methods, available as Supplementary Data 1). Given the curation level of this dataset, we checked the number of compounds for which we could find a pathway, and not the exact experimental pathway. We ran our RetroPath 2. 0 software using the same RetroRules rules (diameters 6, 10 and 16) and solved 77. 6% compounds (118 out of 152), finding a median number of pathways of 4.
Supplement finder for media supplementation The literature pathway that we identified for TPA used xylene for media supplementation, i. e. xylene is not a metabolite from the microbial strain but was necessary for producing TPA and experimentally added to the strain's growth media. Knowing this allowed us to add xylene to the list of starting compounds for retrosynthetic search. Media supplementation is commonly done in metabolic engineering but rarely integrated into retrosynthesis tools. We therefore developed a RetroPath RL feature that analyses search trees and suggests potential media supplements, available from the chemical provider Sigma. We first extracted supplement information from LASER, as well as two TPA experimental pathways known from the literature. Filtering out gene inducers, antibiotics or early precursors, we obtain a list of 8 curated pathways with supplementation (available as Supplementary Data 3).
In 5 cases out of 8, we retrieve the compound that was used for supplementation in the described experimental pathway. For 2 cases, we did not find the experimentally described supplement but suggest other possibilities. Here, we used availability in the Sigma-Aldrich catalogue as a criterion on whether a compound could be an interesting supplement. However, this feature can be used with any database of interest, for example with in-house compounds of a laboratory. One could include criteria such as capacity to cross membranes, solubility, toxicity, cost or any other feature of interest and select compounds that are biologically relevant for their application of interest.
Custom use of RetroPath RL: avoid toxic intermediates We sought to make our tool as modular and flexible as possible, therefore allowing expert users to input their knowledge. We showcase this by implementing a toxicity score to bias the search away from toxic compounds  (Planson et al. 2012; Carbonell et al. 2014). This toxicity score is negative (between 0 and -10 in our training dataset) for toxic compounds and set to 0 otherwise. The strength of using bias in MCTS is that it favours preferred routes (here, avoiding toxic intermediates) but can still find results if those preferred routes are not successful, in contrary to other algorithms that would exclude those intermediates altogether. While implementing this feature did not change our results on the golden dataset (i. e. the experimental pathway was identified for the same compounds), the order as well as the total number of returned pathways was impacted (Supplementary Table  3 ).
Conclusion and remarks We showcase here the use of Monte Carlo Tree Search for bio-retrosynthesis, and provide an implementation as RetroPath RL, an open-source Python package available on GitHub. While MCTS has already been implemented for chemical synthesis  (Segler, Preuss, and Waller 2018) , RetroPath RL is to our knowledge the first application of this algorithm to biochemistry and metabolic engineering. It is a versatile, modular command line tool, build for metabolic engineers, that takes as input a compound of interest, a microbial strain as sink and a set of reaction rules. While RetroPath RL can take as input any set of chemical rules in encoded into the SMARTS linear notation, giving great freedom to users to use reactions from their pathways or microbial strain of expertise, the tests presented in this article use rules from our RetroRules database  (Duigou et al.
2019). RetroRules was built using a data-driven approach showcased in earlier version of RetroPath  (Delépine et al. 2018) , allowing promiscuity encoding. Using both reaction rules at various diameters and chemical similarity scoring, we can tune the allowed promiscuity within our search, which is an advantage over building reaction rules from EC numbers as is commonly done in metabolic engineering. Another advantage of a data-driven approach is that it is bound to get more precise and information-rich as metabolic databases expand. We validated RetroPath RL by verifying on a manually curated dataset that a literaturedescribed pathway could be found for 20 different compounds. The fact that the experimental pathways were found for 75% of compounds using default settings and 95% of the time using more tolerant ones, with much better results when using biochemically guided search, confirms the ability of RetroPath RL to suggest experimentally viable pathways for metabolic engineers on numerous compounds.
Moreover, following the standards we set in our previous paper  (Delépine et al. 2018) , we tested RetroPath RL on a larger dataset and found a pathway for 83. 6% of compounds that were results of successful metabolic engineering projects. These results confirm that RetroPath RL generalises well to metabolic engineering compounds outside the manually curated golden dataset. Moreover, RetroPath RL suggests more pathways (median number of 11. 5 versus 4. 5), giving metabolic engineers more suggestions with which to exercise their expertise. While a restricted number of microbial strains was provided with RetroPath RL, the user can provide his own strain or supplement the media with a given compound of interest. For example, when supplementing xylene to the microbial strain an experimentally described pathway to TPA (Bramucci, M.
G. , McCutchen, C. M. , Nagarajan, V. , Thomas, S. M. 2001) is found, and when adding thebaine to the media, a pathway for morphine production is also found  (Thodey, Galanie, and Smolke 2014). A novel feature that should be welcomed by the metabolic engineering community is the ability of RetroPath RL to suggest media supplements to complement the enzymatic synthesis plan. This feature, tested on 8 pathways, found the experimentally described supplement for 5 of them, and also suggests other supplements to test. We showcase RetroPath RL's modularity by biasing the search towards less toxic intermediates, and also provide expert users 2 strategies to speed-up the retrosynthetic search, either by storing results in a database (Supplementary Note 3) or extending a search from previously run search trees (Supplementary Note 4).
Another feature of interest is the ability to use RetroPath RL for biosensor design, as we also demonstrated with our previously developed tools  (Delépine et al. 2016; Libis, Delépine, and Faulon 2016). This can be used in conjunction with a dataset of detectable compounds  (Koch et al. 2018)  to allow for design of Sensing Enabling Metabolic Pathways. Despite the advantages presented above, RetroPath RL still presents some limits, mostly related to pathway ranking. The authors of this article believe modular design, with a downstream analysis of selected pathways, to be a more appropriate course of action than ranking within the tool: so many genome or growth conditions modifications can increase a pathway yield that including all ranking schemes into one to come up with a 'best' pathway neglects both metabolic engineers' expertise and years of lessons from the industry.
However, most developers present all integrated tools that perform pathway search and ranking together. Ranking schemes in such tools can involve accounting for enzyme integration into the microbial strain, kinetics, toxicity, carried flux, thermodynamics or preferred cofactor usage  (Kumar et al. 2018; Tokic et al. 2018). Our approach provides two advantages over integrated tools. First, ranking the pathways separately allows for a much more modular integration, and the ability to integrate new ideas into ranking much more easily. Moreover, as shown in the custom use section, for advanced users attached to using these schemes, it is possible to integrate them easily given RetroPath RL's modular code and the ability of the Monte Carlo Tree Search algorithm to bias the search towards or away from properties of interest.
However, the authors learned values from Reaxys (Elsevier Life Sciences n. d. ) which contains 12. 4 million single-step reactions (compared to around 80k in MetaNetX, including reactions without chemical structures  (Moretti et al. 2016) ). Therefore, using learned values in bioretrosynthesis seems out of reach for the moment, but could become available in the coming years due to the intense curation efforts under way in the community. Moreover, those learned values in bio-retrosynthesis would have to be strain-dependent, as an intermediate compound's value for bio-retrosynthesis depends highly on the strain of interest. In conclusion, we present here a highly modular tool using one of the latest tree search algorithms for bio-retrosynthesis. This tool is modular enough for expert users to input their expert knowledge and has been thoroughly tested on datasets of interest to the community.
Reaction rule encoding Our reaction rules are generated as presented in our RetroRules database  (Duigou et al. 2019). Briefly, we extracted known biochemical reactions from the MetaNetX  (Moretti et al. 2016 ) database version 3. 1 and filtered out incomplete reactions. We identified the reaction centres based on atom-atom mappings we performed using the Reaction Decoder software (  (Rahman et al. 2016 ), version 2. 1. 0). We decomposed multi-substrates reactions into monosubstrate components by considering one substrate per component and only the subset of products that share at least one atom with the substrate. Mono-components involving a typical cofactor (such as CO2, ATP, NADH. ) as substrate were excluded. Finally, each monocomponent reaction is encoded into a collection of reaction rules using the SMARTS (Daylight Chemical Information Systems, Inc.
Branching factor calculation We generated what we called the Extended Metabolic Space (EMS) at 1 step, i. e. the metabolic space that can be reached by applying our reaction rules once on all compounds of MetaNetX  (Moretti et al. 2016 ) being used as a template for generating at least one reaction rule. We filtered out substrates having a molecular weight greater than 1 kDa. We set a timeout cut-off of 1 second on rule application. We then analysed the results presented in Table  1  and Supplementary Table  1  using the NetworkX python library  (Aric A. Hagberg 2008).
Biological score calculation A penalty is calculated as presented in our RetroPath 2. 0 paper  (Delépine et al. 2018). Briefly, we clustered reaction rules according to the EC number annotations inherited from the template reactions. Independently, we clustered enzyme sequences collected from UniProt (The UniProt Consortium 2017) (release 2019_04) according to sequence similarity using the cd-hit software (version 4. 6. 8) (Li and Godzik 2006). We establish the sequences to reaction rules associations based on Rhea (v98)  (Morgat et al. 2017), MetaCyc (v21. 5) (Caspi et al. 2018)  and Reactome (v66)  (Fabregat et al. 2018)  crosslinks. The penalty score was then computed as   () =  10 (  ) with   the number of distinct clusters that contains sequences associated to the rule. In addition, we normalised this penalty into a score comprised between 0 and 1 using the following function: 𝑠𝑐𝑜𝑟𝑒(𝑟𝑢𝑙𝑒) = 1 1 + 𝛼 • 𝑤𝑒𝑖𝑔ℎ𝑡𝑒𝑑 𝑝𝑒𝑛𝑎𝑙𝑡𝑦 (𝑟𝑢𝑙𝑒) with 𝑤𝑒𝑖𝑔ℎ𝑡𝑒𝑑 𝑝𝑒𝑛𝑎𝑙𝑡𝑦 (𝑟𝑢𝑙𝑒) = 𝑝𝑒𝑛𝑎𝑙𝑡𝑦 𝑠𝑐𝑜𝑟𝑒 (𝑟𝑢𝑙𝑒) √𝑟𝑎𝑑𝑖𝑢𝑠(𝑟𝑢𝑙𝑒) and  = 1 (ℎ  ).
Sinks construction Except stated, sink compounds have been extracted from genome-scale metabolic models by only collecting the chemicals that lie in the cytosol compartment. In addition, we filtered out "dead-end" compounds, i. e. compounds that cannot be produced by any reactions in a steadystate metabolic model, that we detected by performing a Flux Variability Analysis using the COBRApy package (v0. 15. 3)  (Ebrahim et al. 2013 ). Chemical structures have been obtained using crosslinks from the models to metabolic databases. In case no crosslink or not any valid structure was found, the PubChem  (Kim et al. 2019 ) database was examined using compound names as query. Finally, all chemical structures were standardised as described in the "Compound standardisation" section.
Available sinks The available sinks provided with our software are iML1515  (Monk et al. 2017) , iJO1366  (Orth et al. 2011)  and core E. coli metabolism  (Orth, Fleming, and Palsson 2010) , as well as Bacillus Subtilis iYO844 model  (Oh et al. 2007 ) and our set of detectable compounds for biosensor design  (Koch et al. 2018). The genome scale models were obtained from the BiGG Models database  (King et al. 2016). By default, we used sinks from the iML1515 model.
RetroPath 2. 0 configuration For all tests made with RetroPath 2. 0, we perform all executions on a recent work station. We use the sink extracted from the iML1515 model, and we set the maximum pathway length to 5, the maximum number of structures to keep for next iteration to 1000, and a 3 hours per execution time budget. We then use the rp2paths software available on GitHub at https://github. com/brsynth/rp2paths to extract the pathways from RetroPath 2. 0 output  (Delépine et al. 2018).