dataesr/scientific-paragraphs-categorization huggingface dataset:
Standalone Generation and Analysis of Bipartite Gene Family-Genome Graphs The standalone program bitwin. py performs the construction and the analysis of bipartite gene family-genome graphs (fig. 2 ). • Construction of gene families. By default, MultiTwin assumes that the sequences have been subjected to an all-against-all BLAST or Diamond run (that can optionally be performed if the raw sequences are supplied in FASTA format). Only the reciprocal best hit is kept, and the sequence similarities are filtered for !80% mutual coverage and E-value 10 À5 Then MultiTwin constructs a sequence similarity network (SSN) by filtering the remaining similarities above a user-defined similarity threshold (!30% sequence identity by default)  (Bittner et al. 2010)  The user-supplied genome-sequence file is seen as the edge file of a bipartite graph, and the previously constructed gene families are stored as a community file.
Results We have applied the MultiTwin suite to the study of gene sharing in the microbial world  (Corel et al. 2018). Here, we propose a small application for the study of pathogenicity traits in prokaryotes. We assembled a data set of 20 pairs of genomes with comparable sizes, coming from phylogenetically closely related pathogen and nonpathogen organisms (supplementary table 1, Supplementary Material online), to test the existence of genes exclusively associated with pathogenicity or nonpathogenicity. According to  (Merhej et al. 2009; Georgiades and Raoult 2011) , there are good reasons to expect finding such genes. Organisms were assigned as "pathogens" or "nonpathogens" based on metadata from the GOLD  (Mukherjee et al. 2017 ) and PATRIC  (Wattam et al. 2017)  databases. Protein sequences from these genomes were used in an all-against-all BLAST search with parameters as described in  (Bittner et al.
3. -Twin nodes in a toy example of tripartite graph. Twin classes are formed by all the nodes having exactly the same neighborhood. In this example, we highlighted in the same color the nodes forming the graph's three twin classes containing more than one node. All nodes in black have a different set of neighbors (and form thus each their own twin class). In a multipartite graph, twins can be homogeneous, like twin 1 (in yellow) or heterogeneous, like twins 2 and 3. The detect_twins. py script implements an option to detect only homogeneous twins (possibly even of a given type). In a tripartite graph where nodes of respective types 1, 2, and 3 are gene families, genomes, and environments, it may be interesting to detect patterns like twin 2, where a gene family is found in the strict subset of those genomes that thrive in the same environment.
Though some pathogens are known to encode genes involved in the production of public goods, it would be interesting to explore whether there is a broad trend toward the production of public goods by nonpathogens. MultiTwin would allow to test this hypothesis on a larger scale data set, as well as extend the analysis to additional levels of organization: investigating the subgenic level, by using tripartite domain-gene-genome graphs or, at the other end of the scale, the environmental conditions, by using tripartite genegenome-environments graphs (or even considering 4-partite domain-gene-genome-environments graphs). Another likely direction would be to use MultiTwin to define core and shell genes of pangenomes.
Acknowledgments We thank  J. O. McInerney ,  M. Habib ,  F. de Montgolfier , and  T. Hujsa  for critical discussions, and  R. Lannes  for help with the Python code. This work has been supported by the  European Research Council  (grant FP7/2007-2013 Grant Agreement # 615274  to J. S. P. , A. K. W. , E. C. , and E. B. ) and a grant from  R egion Ile-de-France  (DIM Malinf 2011-2013) to S. K.
Introduction One of the main objectives of particle physics in the post-Higgs boson discovery era is to exploit the full physics potential of both the Large Hadron Collider (LHC) and its upgrade, the high luminosity LHC (HL-LHC), in addition to present and future neutrino experiments. The HL-LHC will deliver an integrated luminosity that is 20 times larger than the present LHC dataset, bringing quantitatively and qualitatively new challenges due to event size, data volume, and complexity. The physics reach of the experiments will be limited by the physics performance of algorithms and computational resources. Machine learning (ML) applied to particle physics promises to provide improvements in both of these areas. Incorporating machine learning in particle physics workflows will require significant research and development over the next five years.
Motivation The experimental high-energy physics (HEP) program revolves around two main objectives that go hand in hand: probing the Standard Model (SM) with increasing precision and searching for new particles associated with physics beyond the SM. Both tasks require the identification of rare signals in immense backgrounds. Substantially increased levels of pile-up collisions from additional protons in the bunch at the HL-LHC will make this a significant challenge. Machine learning algorithms are already the state-of-the-art in event and particle identification, energy estimation and pile-up suppression applications in HEP. Despite their present advantage, machine-learning algorithms still have significant room for improvement in their exploitation of the full potential of the dataset.
Brief Overview of Machine Learning Algorithms in HEP This section provides a brief introduction to the most important machine learning algorithms in HEP, introducing key vocabulary (in italic). Machine learning methods are designed to exploit large datasets in order to reduce complexity and find new features in data. The current most frequently used machine learning algorithms in HEP are Boosted Decision Trees (BDTs) and Neural Networks (NN). Typically, variables relevant to the physics problem are selected and a machine learning model is trained for classification or regression using signal and background events (or instances). Training the model is the most human-and CPU-time consuming step, while the application, the so called inference stage, is relatively inexpensive. BDTs and NNs are typically used to classify particles and events.
Theory Applications The theoretical physics community has a number of challenges where machine learning can have an impact. These include areas of theoretical model optimization with hundreds of parameters, searches for new models, understanding and estimation of the parton distribution functions and possibly quantum machine learning. The following details one such application: learning of the parton distribution functions with machine learning. Making progress towards the objectives of the HL-LHC program (see Section 2) requires not only obtaining experimental measurements of the physical processes but also reliable theory inputs to compare to. This becomes increasingly challenging as the experimental data gets more precise. There are numerous examples of phenomenologically relevant processes where the experimental uncertainty is comparable to the estimate of the theoretical uncertainty of the corresponding calculation.
The PDFs then need to be determined from experimental data. The NNPDF collaboration uses machine learning techniques to obtain a PDF determination that is accurate enough to be suitable for high-precision collider data comparison. The NNPDF fitting procedure is described in full details in  [46]. The idea is to combine data from all relevant physical processes and fit a neural network representing each PDF. The difficulty of the procedure stems from the fact that multiple experimental inputs need to be combined to obtain a PDF fit. Each of these inputs adjoins only indirect constraints on the PDFs, leaving some regions of the PDF completely unconstrained by data. NNPDF fit includes around 50 datasets from different physical processes, and results that are not always consistent among themselves.
Therefore, it is crucial to propagate the uncertainty of the experimental inputs into uncertainty on the PDFs via Monte Carlo Dropout  [47]. While the dataset is small, each experimental point has an indirect relation to the PDFs, as it is the result of the convolution of one or two PDFs with the corresponding partonic cross section. Code has been developed to compute these convolutions called APFELgrid  [48]. Future research directions include the possibility of using standard ML frameworks to efficiently express the PDF fitting problem. The uncertainties of the theory calculations need to be taken into account as well in the fits. A fully systematic treatment of theory errors in PDFs is a topic of research where machine learning could play an important role. The dominant uncertainties in the data are no longer statistical and instead arise from correlated systematics.
Computing Resource Optimization and Control of Networks and Production Workflows Data operations is one of the significant challenges for the upcoming HL-LHC. In the current infrastructure, LHC experiments rely on in-house solutions for managing the data. While these approaches work reasonably well today, machine learning can help automate and improve the overall system throughput and reduce operational costs. Machine learning can be applied in many areas of computing infrastructure, workflow and data management. For example, dataset placement optimization and reduction of transfer latency can lead to a better usage of site resources and an increased throughput of analysis jobs. One of the current examples is predicting the "popularity" of a dataset from dataset usage, which helps reduce disk resource utilization and improve physics analysis time turn-over.
Data volume in data transfers is one of the challenges facing the current computing systems as thousand of users need to access thousands of datasets across the Grid. There is an enormous amount of metadata collected by application components, such as information about failures, file accesses, etc. Resource utilization optimization based on this data, including Grid components and software stack layers, can improve overall operations. Understanding the data transfer latencies and network congestion may improve operational costs of hardware resources. Networks are going to play a crucial role in data exchange and data delivery to scientific applications in HL-LHC era. The network-aware application layer and configurations may significantly affect experiment's daily operations. ML can be applied to network security in order to identify anomalies in network traffic; predict network congestion; detect bugs via analysis of self-learning networks, and optimize WAN paths based on user access patterns.
Introduction Discovery science provides a challenge that attracts brilliant minds eager to push the boundaries of scientific understanding of nature. Particle physics has a rich problem domain that offers avenues for intellectual reward. The goal is to achieve a vibrant collaboration between the data science and high-energy physics communities by finding a common language and working together to further science. Both communities can benefit from such collaboration. The HEP community can explore new research directions and applications of machine learning, novel algorithms, and direct collaboration on HEP challenges. The ML community can benefit from a diverse set particle physics problems with unique challenges in scale and complexity, and a large community of researchers that can expand the machine learning horizon by contributing to solving problems relevant to both communities.
Academic Outreach and Engagement Direct collaboration between HEP researchers and computer scientists working in the area of machine learning is an important possible driver of innovation in the area of machine learning applications described in Section 3. It is important for high-energy physics to engage and collaborate with the academic community focused on new machine learning algorithms and applications as they will naturally be interested in applying new ideas to interesting and complex data provided by HEP. Conferences and workshops are a core aspect of the academic ML community, and organizing or contributing to key conferences is a means of gaining interest. Organizing sessions or mini-workshops within major ML conferences, such as NIPS, would increase the familiarity of HEP within the ML community and jump-start future collaborations.
Machine Learning Challenges To engage the wider ML community, challenges such as the Higgs Boson Challenge (2014) or the Flavor Physics Challenge  [52, 53]  have been organized on Kaggle. These types of challenges draw considerable attention from the machine learning community and additional similar challenges should be organized in the future. Organization of a challenge requires a well documented dataset, a starting-kit and an evaluation metric to rank the solutions. This forces the organizers to simplify the problem as much as possible, while retaining its intrinsic complexity. The drawback of challenges is that once they are launched, participants priority is winning the challenge and not eventual collaboration with HEP. It is important to foresee upfront a way to integrate incoming solutions, for example via forums and post-challenge workshops (like  [53] ) where a diversity of competitive algorithms can be presented.
Collaborative Benchmark Datasets There is a strong incentive for HEP to develop public benchmark datasets, beyond just challenges. Access to a dataset makes the discussion much more concrete and productive. Within the HEP community, common datasets enable comparisons of algorithms with much better accuracy, which is very useful for research and development. The same benchmark datasets can also be used for teaching, tutorials and training. These benchmark datasets could be built based on public simulation engines or released by experiments within the bounds of their data access policy. Even a small subset of an experiment's simulated data can be the base of a very valuable benchmark dataset. For example, the CMS experiment has released a significant amount of its simulated and collected data via the CERN Open Data Portal  [56].
• Feedback results of successful applications The HEP community should organize and curate a variety of such benchmark datasets covering its current physics drivers and make them publicly available. To improve the reproducibility of results and algorithm comparisons, some of the data used for evaluation of the solutions should be kept private. Additionally, after investing heavily into producing highly-detailed and realistic simulations, the HEP community can provide the machine learning community with labeled datasets with high statistical power to test algorithms and develop novel ideas.
Industry Engagement Industry has been focused on the development and adoption of machine learning techniques. In addition to algorithm and software development, one of the promising areas is the adoption of dedicated specialized hardware and high performance co-processors. GPUs, FPGAs, and high core count co-processors all have the potential to dramatically increase performance of machine learning applications relevant to the HEP community. One of the challenges is gaining the human expertise for development and implementation. Industry interactions bring specific technology opportunities and access to specialized expertise that can be difficult to hire and support internally. There are specific areas of development where industry has expressed interest in collaborating with HEP. Automated resource provisioning, data placement, and scheduling are similar to industrial applications to improve efficiency.
Applications such as data quality monitoring, detector health monitoring and preventative maintenance can be automated using techniques developed for other industrial quality control applications. There are two more forward looking areas that coincide with HEP physics drivers, namely computer vision techniques for object identification and real-time event classification. These present a challenge to industry due to its complexity and benefit outside of HEP. CERN OpenLab is a public-private partnership that accelerates the development of cutting-edge solutions for the LHC community and wider scientific research. CERN OpenLab has established the infrastructure to maintain non-disclosure agreements, to arrange ownership of intellectual property, and to provide an interface between CERN and industry.
D. Katie Malone) • Partially Derivative • Talking Machines • Data Skeptic • Becoming a Data Scientist Podcast • Not So Standard Deviations • This Week in ML & AI Another form of engagement is through outreach-style blog posts to explain HEP challenges in a way that is easy to understand by the public. Another outreach opportunity is to make HEP related presentations at machine learning Meetups across the world to generate awareness, engage community, foster cross pollination of ideas between HEP and industry. Some popular ML meetups are: • NYC: https://www. meetup. com/NYC-Machine-Learning/ • Berlin: https://www. meetup. com/Advanced-Machine-Learning-Study-Group/ • SF: https://www. meetup. com/SF-Bayarea-Machine-Learning/ In conclusion, existing outreach efforts should be expanded to attract greater collaboration between the HEP and ML communities.
Software Methodology Presently, there are two machine learning software methodologies in high-energy physics. The first approach is to implement abstract ML algorithms in HEP-developed toolkits, such as the Toolkit for Multivariate Analysis (TMVA) in ROOT. This provides on site support to physicists and dedicated development for HEP data formats and applications. The second approach is to rely on externally developed software, of which there are many examples. This often requires tedious and repetitive work to adapt HEP data formats to external software, breaks analysis workflows and introduces difficulties in the analysis software development. Historically, a variety of approaches and competition among them has led to important breakthroughs in the field. On the other hand, having too many choices increases repetition and leads community segmentation and possible issues with reproducibility.
I/O and Programming Languages The sheer amount of data accumulated by HEP experiments requires data access optimization because. Such I/O performance is very dependent on data formats. Moreover, support for reading data in different formats is required for certain use-cases. Exploration of new file systems and methods to improve I/O limitations are important and the following R&D studies should take place: • Explore new file systems to assess I/O limitations; • Use alternative industry approaches such as Google BigQuery to explore various data access patterns; • Explore parallel data processing platforms such as Apache Spark for ML training. Although particle physics has been reliant on C++ over the past decade, the machine learning community has explored other programming languages, in particular the python-based ecosystem.
Internal and External ML tools Internally developed ML software, such as the Toolkit for Multivariate Analysis (TMVA,  [60] ), has been developed to apply a variety of machine learning algorithms to HEP challenges. Currently, many published HEP analyses with machine learning have made use of TMVA. There is also software developed in HEP, such as NeuroBayes  [61, 62]  and RuleFit, that have gained popularity outside of HEP. At the same time, the ML landscape has evolved and many different ML tools have emerged and gained popularity. There are a growing number of published results based on externally developed tools. The latter, often developed directly by industry for specific applications, are constantly undergoing development, incorporating the latest algorithms from academia. Currently, both internal and external tools are used by the HEP community.
TMVA has also undergone significant development in recent years. In addition, there are smaller tools developed in the HEP community, extending either internal or external ML tools for specific use cases and applications within HEP experiments, such as hep ml  [63]  for training ML classifiers in HEP specific tasks, or tmva-branch-adder  [64]  and lwtnn  [57]  to facilitate classifier inference tasks. This begs the question: what aspects of ML development and use should the HEP community focus on in the next 5-10 years? There are several aspects to consider including data formats, community size, and interfaces.
Machine Learning Data Formats HEP and ML communities currently make use of different data formats. HEP heavily relies on the ROOT software framework for data storage, data processing, and data analysis. The machine learning community uses a large variety of formats, as shown in Figure  1. This figure also shows the relationship of machine learning data formats with ROOT: the ROOT file format is very flexible, though it requires a significant time investment to learn the proper use. Table  1  summarizes the current machine learning toolkits and file formats they support. Table  1 : This table lists various data formats (rows) and ML tools (columns). The indicates that there is a native solution, while × means that the conversion from one data-format to another is straightforward (the conversion mechanisms from Table  2  are omitted here).
Desirable HEP-ML Software and Data Format Attributes A desirable data format should have the following attributes: high read speed for efficient training, sparse readability without loading the entire dataset into RAM, compression, and common use by the machine learning community. HEP machine learning applications require highly performant and flexible algorithms to address the variety of use cases. Some applications, such as triggering, also have to work under tight latency constraints of the order of a few microseconds and below. The data sets are extremely large, which comes with I/O challenges as described in Section 5. 2. This is expected to become even more challenging, as the LHC continues to ramp-up and deliver increasingly large amounts of data. As discussed in Section 5. 3, machine learning tools use a number of languages.
Those have the advantage of providing a homogeneous interface and require little training overhead for those already knowledgable in using TMVA. A more general approach to file format conversion is to build middleware solutions that export HEP-specific formats like ROOT to formats used by external machine learning tools. Existing middleware solutions are shown in Table  2. Approaches to bridge the different languages and data formats inside and outside of HEP include providing interfaces or building middleware solutions that translate HEP-specific data formats to external ML tools. It is a topic of current research to determine the most efficient solution.
PyROOT Python extension module that allows the user to interact with ROOT data/classes. [69]  root numpy The interface between ROOT and NumPy supported by the Scikit-HEP community. [65]  root pandas The interface between ROOT and Pandas dataframes supported by the DIANA/HEP project. [  Converts ROOT files containing TTrees into HDF5 files containing HDF5 tables. [74]  The result of this processing is filtered data and extracted physics parameters. At present, training of machine learning algorithms is done using dedicated or private resources. These vary in configuration and processing power, depending on the size of the data and complexity of the algorithm. For a given event, the evaluation of algorithms is performed on a single core producing a single discriminator or regressor output. In order to progress to evaluation of complex machine learning algorithms, more computing power is needed in both the training and evaluation stages, as larger amounts of data are needed to feed models with tens or hundreds of thousands of parameters.