dataesr/scientific-paragraphs-categorization huggingface dataset:
exploitation. Human interpretability is also an important factor in these systems. The algorithm chooses a set of experimental conditions to be generated by robot synthesizers. It is not sufficient to understand what the system generates; we must also know why certain recommendations are made. Interpretability is clearly very important for modern ML research. To aid interpretability, researchers have used hierarchical optimization approaches that operate on one or more variables. In multifactorial systems it is often necessary to understand the pareto-optimal regions of the problem space. A mathematical function called CHIMERA was recently introduced to address these problems;  192  it can be used with any optimizer, such as PHOENICS. Such systems require an operating system that is open-source and capable of controlling experimental equipment, storing data in databases, coupling with optimization approaches, and interacting with researchers.
Neural network potentials A ML approach applicable to chemical systems containing large numbers of atoms, originally proposed by Behler and Parrinello (BP method) in 2007, used high-dimensional neural network potentials (NNP, Fig. 7 ). 210 As in many conventional empirical potentials, the potential energy E is the sum of local atomic energies of all atoms in the system. Since this seminal publication, a substantial number of articles and reviews have been published on the use of NNPs for bulk chemical systems (e. g. , bulk silicon or water) or for describing single molecule potential energy surfaces and reaction coordinates. 211 ecently, Smith et al. introduced the first NNP designed for organic molecules, ANI-1. 212 It is applicable to molecular systems well outside its training set. The ANI-1 potential was trained on a dataset of small organic molecules of up to 8-heavy atoms (while sampling both conformational and configurational space).
Furthermore, ANI-1 demonstrated its applicability to much larger systems, up to 70 atoms, including known drugs and molecules randomly selected from the GDB-11  213  database and containing up to 10 heavy atoms. It predicted DFT energies of the test set molecules with up to 10 heavy atoms very well, with the resulting RMSE values below 0. 57 kcal mol À1. Many techniques for improving the accuracy and transferability of general-purpose ML potentials have been employed. Among these, active learning methods, already proven successful in conventional QSAR modeling, have been especially popular. 214 ctive learning methods provide a consistent and automated improvement in accuracy and transferability and have contributed greatly to the success of general-purpose models. An active learning algorithm decides what new QM calculations should be performed then adds the new data to the training set.
Datasets As previously stated, one of the most important aspects of building a model in chemistry is the choice of the training dataset. Various datasets of organic and materials systems for training ML models have been developed over the last decades. Two of the most popular organic molecule benchmark sets are the QM7  199  and QM9 216 collections. The QM7 benchmark was developed by subsampling the GDB-13 217 database of small molecules. QM7 contains 7165 energy-minimized molecules consisting of up to 7 heavy atoms and several properties computed with density functional theory (DFT). This benchmark is difficult to model by ML because of its relatively small size. Initial mean absolute errors (MAE), using the coulomb matrix representation,  199  were around 10 kcal mol À1. The ANI-1 dataset includes organic molecules with a large number of non-equilibrium DFT total energy calculations.
It includes B24 M conformations for 57 462 molecules from the GDB database, with the total energy values computed for each conformation. This dataset samples both chemical and conformational degrees of freedom at the same time and thus provides 100Â more data. Therefore, we expect that this dataset will become a new standard for comparing the ability of current and future ML methods to improve on the best model accuracy (1 kcal mol À1 ) achieved for the QM9 benchmark. More importantly, this data source is a foundation for development of future general-purpose machine-learned approaches. The COMP6 benchmark dataset 214 was developed to validate the transferability of ML potentials. COMP6 is a benchmark suite containing five rigorous benchmarks that cover broad regions of organic and bio-chemical space of isolated molecules and a sixth built from the existing S66x8 218 noncovalent and intermolecular interactions data.
Advanced approaches In addition to active learning, there are other ML techniques that aim to reduce training data requirements. Some ML-based methods (such as NN) can take advantage of information from multiple sources. The key concept is to train a model using a large dataset of medium accuracy, then retrain the model with a smaller, more accurate and difficult to obtain data set. This process called transfer learning (TL) relies on the assumption that less accurate data sets contains some information that makes it easier to learn models for the smaller datasets of higher accuracy data. For example, TL could be performed by taking a DL model that was pretrained to medium-fidelity DFT, holding some number of parameters in the model constant, then retraining the remaining parameters using a much smaller, higher accuracy CCSD(T)/CBS dataset.
,  240  leading to the discovery of two magnets Co 2 MnTi and Mn 2 PtPd, the first ever discovered by computational means. Ko ¨rner et al. performed a ML high-throughput-screening of intermetallic ThMn 12-type phases and rare-earth-lean systems with YNi 9 In 2type. 241 Mo ¨ller et al. 242  built kernel-based ML models to optimize chemical compositions for permanent magnets. Light conversion and emission. To overcome input constraints of common ML pipelines, Duvenaud et al. 243  developed a convolutional neural network operating directly on graphs (representing molecules of arbitrary size/shape), demonstrating enhanced predictive performance over traditional fingerprinting for solubility, drug efficacy, and organic photovoltaic efficiency datasets. Gomez-Bombarelli et al. integrated neural networks as part of a larger computational discovery pipeline to prioritize molecules for quantum simulations.
This section has given a short-unavoidably incomplete-snapshot of the current state of the art. What do our colleagues say about future frontiers? Jain et al. 249  identified challenges as follows: (i) streamlining the use of large data resources (even with rational APIs, large databases remain difficult to interrogate, especially when mixing data from different repositories); (ii) developing descriptors for crystalline, periodic solids; and (iii) balancing interpretability (physical meaning) of descriptors versus accuracy of models. The latter represents a well-known challenge resolved in cheminformatics about a decade ago. 250 Butler et al. 198  added the following extra challenges to this list: (iv) dealing with smaller datasets (of critical importance especially for the experimental world); (v) quantum learning (to enhance calculation speed); and (vi) establishing new principles (not only data, but also laws, somewhat similar to Jain's point about balancing interpretability and accuracy).
What do we say about future frontiers? There is no need to add further elements to the philosophical discussion of ML/AI's future. We should not underestimate the critical issues of the following additional challenges: (vii) dealing with the disordered/amorphous systems (e. g. , it is not a coincidence that the field of high-entropy alloys is still lacking a compelling ML work); (viii) sustainability and organization of big-data in terms of computational infrastructure, standardization of dataentries and prototypes, development of materials database languages, e. g. , AFLUX; 251 (ix) further exploration of web-, cloud-, and frameworks-directions, and the last but the most important point (x) unless ML can generate new useful materials faster than experiments alone, materials scientists' interest in ML will dissipate quickly.
To conclude this section, we highlight the clear similarity between materials informatics with the traditional workflow of QSAR modeling (see Fig. 1  and 8 ). As with cheminformatics, the starting point of materials informatics is the accumulation of large datasets of materials with experimental or computational properties. The need for developing novel materials descriptors and their use in building property prediction models using ML techniques follows. Finally, current challenges outlined in the concluding part of this section parallel many of those facing traditional QSAR modeling of bioactive compounds. Thus, materials informatics (and a closely related field of nanomaterials informatics described in the next section) represents a prime example of a new discipline, whose development was enabled and immensely catalyzed by the experience and approaches developed in QSAR.
1 ). However, the high structural diversity and complexity of nanomaterials typically lead to specific challenges, 253 especially when it comes to the choice of molecular descriptors. Two types of representations are clearly emerging from the literature -studies in which the whole nanoparticle is characterized computationally, experimentally, or both or when such characterization is applied to the surface chemistry of the nanoparticle (especially, organic decorators) only. Naturally, the choice of descriptors and the associated software is different for these two types of QNAR modeling. For the second type of study the QNAR model is similar to a traditional QSAR model, trained using descriptors for surface chemistry, to predict biological activity of the nanomaterials. Another challenge of QNAR modeling, similar to materials informatics is the relatively small size of the datasets currently available in the public domain.
The importance of data on ENMs structure and properties Like other area of materials science, nanotechnology has generated various datasets of physicochemical properties, environmental fate and transport parameters, and bioactivity of nanomaterials. 255 They contain both literature curated and raw data from various experimental investigations, useful for QNAR modeling. For example, the OCHEM database 255 contains experimental data on ENMs and provision for generating descriptors for model building, NanoMiner 256 contains data (including omics data) on 634 types of ENMs. The NM-biological interactions knowledge base contains over 200 toxicological evaluations for embryonic zebrafish exposed to metal Fig. 9  General scheme representing the development of quantitative nanostructure-activity relationships for the calculation of properties of nanomaterials using both computed and experimentally determined molecular descriptors.
This journal is © The Royal Society of Chemistry 2020 and metal-oxide ENMs. NanoDatabank 257 has raw data for over 1000 different nanomaterials and associated characterization and toxicity data. Early nanoinformatics efforts were focused on organizing data into structured datasets (i. e. , with fixed fields or records). 39 owever, there is growing recognition that significant data are available as unstructured datasets (i. e. , with no predefined fixed fields or records), often are scattered across multiple literature and online sources. Thus, significant recent efforts have been devoted to the development of public databases, meta data, and data management systems for nanomaterials. These efforts included incorporation and integration of information from multiple sources, addressing data security, effective data sharing, intelligent data queries, and data integration.
258 The joint EU-US Nanoinformatics Roadmap 2030 255 has stressed the need for guidelines concerning the development of nanoinformatics datasets that are structured, have controlled ontology for ENMs properties and bioactivity, and interoperability with other databases and modeling tools. Raw data (free from preprocessing by data curators) that can be curated and analyzed in a context-dependent way are most useful for QNAR development. Substantial amounts of experimental data on the toxicity of ENMs have been generated, primarily in various cell lines such as, macrophages, pancreatic and other human cells and bacteria. There are still limited studies with simple organisms like zebrafish and even fewer on higher animals. Toxicity data include experimental results across multiple assays and cell lines/types with ENMs having different surface modifications and core compositions.
There are different levels of confidence and consistency across the toxicological studies. Currently, efforts to derive generalized toxicity models based on ENMs characteristics have been based on datasets from single studies rather than integrated from the collective body of published data. 259 Clearly, to develop predictive nano-SAR models of ENMs toxicity, it is useful to identify critical biological pathways that can lead to adverse outcomes. 260 Understanding relationships between the structural and physicochemical properties of ENMs and the biological responses and correlation between such responses can be very useful for deriving causal relationships. Although QNAR models provide valuable insight on ENMs toxicity, they generally cannot provide direct mechanistic interpretation that can be validated and tracked back directly to experimental data.
271 N models construct a network where the modes are ENMs characteristics and the edges (links) represent conditional dependences of target outcomes on various attributes. This provides a visual representation of causal relationships. 272 The model allows interpretation of ''if/then'' causal relationships where the parent (antecedent) and child (descendent) nodes are at the outgoing and incoming links in the BN structure, respectively. The set of model attributes and their conditional dependencies represents knowledge from the dataset(s) of attributes and toxicity outcomes in the form of probability distributions. BN models can identify, for example, the conditional dependence that would lead to a toxicity outcome within a specific range. Previous studies have demonstrated the value of BNs for developing qualitative ''toxicity/hazard'' classification of ENMs based on using physicochemical and specialized descriptors.
Subsequently, research teams at the University of Nottingham, CSIRO, Monash University, and MIT generated polymer microarrays  285  and conducted high throughput screening to elucidate structure-property relationships in their interactions with cells. The use of biomaterials as cell factories  286  shows great promise, and the large generated stem cell attachment, proliferation, and differentiation datasets were modelled by ML methods. These could make robust and accurate predictions of stem cell behavior of materials not used to train the models. In one study, the attachment of embryoid bodies (a surrogate and stable cell system to mimic embryonic stem cells) to a polymer library was modelled using sparse feature selection and optimally regularized neural networks. 287 These models relied on DRAGON descriptors and Bayesian regularized neural networks to quantify the attachment of embryoid bodies to the polyacrylate libraries.
The algorithm then identified the patient with the most ''neighbors'' within a specified similarity cut-off, forming the first cluster. These patients were excluded from further iterations. The process was repeated until only patients without neighbors (i. e. , singletons) were left. For this dataset, the optimal balance between the number of clusters and clustering overlap was found at similarity threshold 0. 62. Using the sphere exclusion algorithm for clustering reduces the risk of bias since the method does not make a priori assumptions regarding numbers of clusters or similarity thresholds. After leaving 189 patients (6%) as outliers, the following nine COPD clusters (phenotypes) were identified, with the number of patients given in brackets: 1: depression-COPD (1748); 2: malignancy-COPD (312); 3: coronary artery disease-COPD (291); 4: young age-low comorbidity-high readmission-COPD (152); 5: advanced malignancy-COPD (144); 6: cerebrovascular disease-COPD (120); 7: atopy-COPD (81); 8: diabetes mellituschronic kidney disease-COPD (64) and 9: advanced disease-COPD (43).
Outlook The field of QSAR modeling based on simple approaches used to predict chemical reactivity was initially popularized by Corwin Hansch and his colleagues more than 55 years ago. 1  For many years, even decades, this field was focused on the prediction of physicochemical properties and biological activities using descriptors representing intrinsic properties of chemical structures. However, as the size and diversity of chemical datasets expanded, the QSAR modeling field has evolved to include larger and more diverse types of chemical descriptors and increasingly more complex statistical and machine learning techniques. We reflected on these trends earlier, 2 and foreshadowed the impact that these developments in the QSAR modeling community would have on many other areas of research. We projected that, with the continuing strong growth of publicly accessible data, this field will become essential for extracting knowledge from, and making predictions with, these massive data sets.
Conclusions This contribution was conceived by a group of scientists who have dedicated significant portions of their professional careers to the development and use of quantitative methods in computational chemistry and molecular modeling. Following the previous highly cited comprehensive survey of QSAR modeling that was coauthored by many contributors to this paper and published in 2014,  2  we felt it was time to reflect on the new and exciting developments in QSAR modeling that have emerged in the last five years due to proliferation of large and diverse (Big Data) molecular bioactivity datasets and of burgeoning use of associated Big Data analytical methods such as DL. We also intended to share our observations and excitement concerning the prolific use of similar ML approaches in areas beyond chemical domain; the latter excitement and observations were in part influenced by the transition to other fields that some original cheminformaticians, including several co-authors of this paper, have made in their own research evolution and career development.
, 2018). The EMODnet Chemistry portal is now built on a network of 45 marine research and monitoring institutes and oceanographic data management experts from 27 countries. Many of these are National Oceanographic Data Centres (NODC), actively involved in managing, indexing and providing access to ocean and marine data sets, acquired from research cruises and monitoring activities in European marine waters and global oceans. The objective of EMODnet Chemistry is to provide easy and open access to marine chemistry data sets and data products related to three main categories: eutrophication (e. g. , nutrients, oxygen and chlorophyll), contaminants (e. g. , hydrocarbons, pesticides, heavy metals, antifoulants) and marine litter (e. g. , beach litter, seafloor litter and floating micro-litter).
Data derive from inputs gathered and collated from national monitoring efforts and research activities from all European coastal states. The large heterogeneity in data managed by EMODnet Chemistry derives from the different kinds of collated variables: eutrophication data are mainly available for the water column, conversely, contaminants and marine litter are collected in the water, sediments and in biota. Furthermore, samples have been collected by a wide range of instruments used for in situ data acquisition, analyzed by heterogeneous laboratory protocols, with different method accuracy and precision that need to be described and archived, together with the data, to allow data reusability and solid scientific analysis. A lot of information included in metadata is therefore needed to correctly archive data and allow their long-term use.
DATA COLLECTION, AGGREGATION, AND VALIDATION Data collections are the result of harmonization, standardization and validation process, applied when merging observations from different sources, sensors and purposes  (Giorgetti et al. , 2018). All data entering EMODnet Chemistry are managed according to standard protocols developed in the framework of the European consolidated SeaDataNet marine data infrastructure 1  , which implements consolidated communication standards and tools, common data and metadata models and common file formats. In particular, the following SeaDataNet standards are adopted to ensure interoperability with other data platforms: • Metadata services for standard description of cruise, organizations, projects, observing systems, datasets. (CSR, EDMO, EDMED,. ), • Ocean Data View (ODV) format for data exchange, • NVS Controlled Vocabularies providing common terms to describe data and metadata, • Common Data Index (CDI) service to access and download data according to the related data policy, • Security services for user registrations, • Products viewing services for discovery, visualization, and downloading of products, • Dedicated tools (NEMO, MIKADO,.
) to prepare data and metadata, • Quality control procedures for data validation. The use of controlled vocabularies (i. e. , standardized terms that cover a broad spectrum of disciplines) is an important prerequisite to allow consistency and interoperability. The SeaDataNet NVS controlled vocabularies are technically managed and hosted by the British Oceanographic Data Centre by means of the NERC Vocabulary Server (NVS2. 0). Heterogeneity in marine chemical data is extremely high, with regard to sampled matrix characteristics, different sampling and analytical protocols. To keep all relevant information linked to the data, a very specific vocabulary (BODC Parameter Usage Vocabulary terms, P01 vocabulary 2  ) was implemented by SeaDataNet, which allows to classify the different substances, but also record matrix characteristics and the analytical techniques.
The P01 vocabulary is based on a semantic model which uses a defined set of controlled vocabularies (the semantic building blocks), and which allows to keep the relevant information and to label parameters with a standard description. EMODnet data and metadata format is compliant with the INSPIRE themes and SeaDataCloud data models  (Partescano et al. , 2018). The data files can be imported into the Ocean Data View (ODV,  Schlitzer, 2002)  visualization and analysis software package, which is freely available for non-commercial, nonmilitary research and for teaching purposes. In order to obtain data aggregation per sea region from the heterogeneous datasets originating from multiple institutions, an automatic Robot Harvester, properly configured with predefined criteria of geographical and temporal coverage and parameters, was adopted to retrieve specific data sets from distributed data centers.
The resulting collection is aggregated, and quality controlled using ODV software and following a dedicated methodology. Regional leaders were in charge of both steps, to be performed according to a common protocol, shared between all sea regions. Parameter aggregation included unit conversions, harmonization of parameter coding and meaning (taking into consideration the possible difference in the collection of new and historical data). Regional quality control follows procedures compiled in discussion with the wider international community (e. g. , IOC/IODE, ICES and JCOMM, SeaDataNet Data Quality Control Procedures) and involves: metadata format correctness and completeness check, data format checks, identification of negative and zero values, identification of wrong measurement units and "broad range" check which consists in the comparison with minimum and maximum regional values derived from previous statistics (SeaDataNet, 2010;  Barth et al.