dataesr/scientific-paragraphs-categorization huggingface dataset:
The GNN Model A graph is a non-linear data structure composed of a collection of nodes and edges. Formally, it is defined as a tuple G = (N, E), where N is the set of nodes and E ⊆ N × N is the set of edges. Nodes represent objects or entities, and relationships between them are represented by edges. Nodes and edges can be associated with values or vectors of values; describing their attributes are defined as a node feature vector x n for each node n ∈ N and an edge feature vector e n,m for each edge (n, m) ∈ E, respectively. A GNN assigns a state s n to each node n ∈ N and updates it iteratively by sending messages through the edges connecting n to its neighbors Ne(n). The GNN theoretical model is fully described by two parametric functions, f w and g w , which, respectively, regulate the state updating process and the output calculation.
• A molecular embedding module: A GNN-based architecture fed with the graph representations of the molecular structures-extracted from the SMILES strings describing the drug molecules-to produce a task-based NF, which is concatenated to the drug node features and used by the following sub-model to predict DSEs on drug nodes; this approach should encourage the molecular embedding sub-model to extract information, which is not captured by the MF used in the original work, as it remains the same overall training procedure. In this case, since the NF is produced by a ML model, it is dynamically adapted during the learning procedure for the DSE prediction task to maximize the performance of the general model. • DruGNN: The model is similar the one introduced in  [14]. This network is a recurrent GNN model  [16]  which exploits an inductive-transductive learning scheme to predict DSEs on drug nodes.
More detail about its implementation and its hyperparameters can be found in the original paper  [14]. The main difference with the original model is the fingerprint used as part of the drug node features, since in this work it is not a pre-calculated or standard MF, but a NF produced by the molecular embedding sub-model. To ensure that the information extracted from the molecule complement the information contained in the DruGNN graph, the two aforementioned modules are jointly trained as a single model. The model was implemented using the GNNkeras library  [38]  which is publicly available on GitHub https://github. com/NickDrake117/GNNkeras, accessed on 1 November 2022. Please refer to the GNNkeras paper  [38]  and the Data Availability section for more details about GNN and DruGNN models and implementations.
Dataset Description The dataset for the DSE prediction task was the one constructed in  [14]. Please refer to the Data Availability Statement section for data and code availability. It consists of a single heterogeneous graph where genes and drugs are represented as nodes, connected by three different sets of edges, with relationships based on gene-gene interactions, drug-gene interactions, and drug-drug similarity. In particular, gene node features are retrieved from the BioMart database  [39]  and consist of chromosome and strand location, percentage of GC content, and a one-hot vector representing the molecular function ontology term, clustered from molecular function ontology  [40] ; instead, drug nodes are described by a set of chemico-physical molecular descriptors provided by the PubChem database  [41] , such as molecular weight, polar surface area, xlogp coefficient, heavy atom and rotatable bonds count, number of hydrogen bond donors, and acceptors.
MFs were extracted from the SMILES strings describing each molecular structure of the drugs, using the rdkit python library  [11]. MFs were used to define the drug-drug relationships, based on the drug similarity measured in terms of Tanimoto similarity  [42] , the most common measure of similarity between molecules. Drug-gene relationships are based on drug-to-protein interactions (DPI) extracted from the STITCH database  [43] , one of the most extensive and up-to-date DPI databases available online, and Biomart, allowing us to retrieve a mapping between drugs and genes whose products are the proteins the drugs interact with. Finally, gene-gene relationships are retrieved from the Human Reference Interactome (HuRI)  [44]  and from a gene-to-protein mapping provided by Biomart. The labels for the drug nodes were extracted from the SIDER database  [45]  and consist of 360 single DSEs, retrieved from medical documents, with no information about the concentration at which the side effects appear: the final task is then a multi-label, multi-class, node-focused classification.
In particular, belonging to the positive class for a generic drug means that it produces the particular side effect, corresponding to the value 1 in the related position of the target vector. Note that each drug can cause multiple side effects. The original dataset construction, with all the source databases and preprocessing operations, is sketched in Figure  3. The molecules' SMILES strings were also used to obtain graph-based representations of the drug molecules, with which the molecular embedding module was fed during training time. Even if the graph representation of a drug as a molecule may seem natural, it can still be extremely complex: a simple way to represent them is to set each atom as a node (except for hydrogen atoms) and each bond between them as an edge in the graph, both associated with feature vectors, as the GNN model  [16]  supports features on both the edges and the nodes.
Since some atoms are quite rare in the dataset-for instance, arsenic is present in only one molecule-they are grouped in accordance with their chemical families which are encoded by a 1-hot vector: in total, 18 classes of atoms have been identified, as shown in Figure  4. Moreover, atom feature vectors are enriched by additional chemical information, such as the degree of the atom (number of electrons involved in chemical bonds), the number of hydrogen atoms linked to them, the number of radical electrons around the atom (which constitute a rare feature, as they introduce instability), and their formal charges. Edge features describe the type (simple, double, triple, or aromatic) of bond the edge represents, encoded by a 1-hot vector: this is the only feature used on edges, as other important quantities should be deductible from the atoms on both sides of the edge, and from their neighborhood.
The DruGNN dataset consists of a single graph composed of 9222 nodes (of which 1341 are drug nodes and 7881 are gene nodes) and 331,623 edges (of which 12,002 are gene-gene interaction links, 314,369 are DPIs, and 5252 are drug-drug similarities). At a high-resolution level, drug nodes are represented as a single homogeneous graph each, where nodes represent atoms and edges represent chemical bonds. In the original dataset, only side effects present in more than 100 drugs were considered for the task. However, since the dataset contains many DSEs occurring in a very limited number of cases-some of them even in less than 10 drugs-only the 280 most frequent DSEs, out of the 360 originally described in  [14] , were considered in the present study. The final dataset, represented by a single graph of graphs, was then fed to the model, for a multi-label, multi-class classification task.
• Using binary focal crossentropy instead of simple binary crossentropy; • Not considering the DSEs with less than 100 positive occurrences, as described in Section 2. 3; • Using a weighting scheme to encourage better prediction of rarer side effects. Focal cross-entropy was first introduced in  [47]  for object detection applications. The only difference from the standard cross-entropy loss is the introduction of a factor (1p) γ for positive example and p γ for negative examples, as defined in Equation (3): Loss(p, y) = -(1 -p) γ log(p), if y = 1 -(p) γ log(1 -p), otherwise (3) From a practical point of view, it results in a loss which penalizes big errors and gives little to no importance to small ones in predictions. Despite its early development purpose, it can be effectively applied on any classification task where the model has to learn from unbalanced datasets, as the additional factor encourages the model to take some risks by predicting the minority class more often.
Effects of γ on accuracy and F1-score metrics in the grid search procedure are shown in Figure  6. In the dataset, some DSEs occur in only 100 cases, and others in more than 1000, suggesting that the difficulty of prediction for a given DSE is somehow related to the number of positive examples. To tackle this disparity, the model should give more relevance to rarer DSEs when predicting class probabilities: the ideal behavior should be for the network to have close to uniform predicting power over the whole drug set, as in perfectly balanced datasets. To achieve this, a weighting scheme inversely proportional to the number of positive examples has been used, as defined in Equation (  4 ): w = 100 n positive µ (4) where w is the weight given to a DSE with the n positive positive example, and µ is a geometric coefficient which determines the weight of this weighting process (no weighting for µ = 0 and inversely proportional weighting for µ = 1).
This metric shows a notable increase in 0. 6. The threshold of 90% is still a somewhat low value for such a use case; however, it constitutes a robust metric for predictors. Some metrics, such as the accuracy, do not improve, resulting in a drop in performance of about 10 percentage points: that is in line with the expectations, as the dataset is characterized by an imbalanced class distribution-very common in most real-life classification problems or datasets-thus making the F1-score a better evaluation metric, since it gives a better measure of the incorrectly classified cases than the accuracy metric. An ablation study was carried out to assess the importance of the contributions of the different changes in the learning procedure and in the dataset. The results are shown in Table  3  as the means of 10 learning procedures with different initialization parameters.
4, is very small, but it is mostly beneficial, with the exception of the recall which showed a very slight improvement using µ = 0 (no weighting). Additionally, a comparison with previously developed methods for drug side effect prediction is provided in Table  4. It is worth noticing that each method was developed on different datasets and with different and often heterogeneous data types. Moreover, these differences led to different dataset sizes, as not all the data available online can be exploited in all the settings. More precisely, MolecularGNN was compared to: DruGNN  [14] ; Pauwels  [48] , which is a good baseline that calculates the sparse canonical correlations between the drug structure fingerprints and the side effects; DeepSide  [49] , which is the only other approach based on deep learning and exploits data concerning drug targets, structural fingerprints, and gene expression; DrugClust, which formulates the prediction as a clustering problem based on gene expression data  [50].
Please keep in mind that the different nature of the predictors, the data used by each, and the number of examples mean that the comparison is purely qualitative. The comparison was made in terms of AUC metric, since this is the only metric that was measured for each of the predictors involved. The presented method, with an unbalanced dataset, did not reach an AUC comparable to simpler predictors operating on less unbalanced datasets. The F1-score was the dominant metric in this case, and it was significantly better than the F1-score of DruGNN-the only other predictor trained on a dataset with a comparable level of unbalancedness between classes.
Conclusions In this paper, an enhanced and innovative version of DruGNN was proposed: the new model, called MolecularGNN, is able to work directly on the molecular structure and to work on unbalanced datasets and a graph-of-graphs domain. The novelty of this model is represented by the generation of drug structural fingerprints, which can be adapted to the task the general model is trained for. In particular, a drug's molecular structure is processed by a GNN sub-model, producing a NF which is then used to enrich drug node features in the DruGNN graph dataset, which are eventually processed by the following GNN sub-model to predict the DSEs of drug nodes. The new ML-based technique, trained with an inductive-transductive approach, together with the NF generation, results in an increase of 0.
2 in the F1-score and 0. 06 in precision at recall of 90% with respect to the original DruGNN model, though they can be further improved. The overall accuracy is not improved, but MolecularGNN succeeded in identifying more experimentally observed DSEs than DruGNN: this is an acceptable compromise, since this task is considered much more important than identifying DSEs which do not arise for a given drug, as a non predicted DSE could cause little harm or be potentially fatal. Although MolecularGNN is still limited, with an F1-score of less than 0. 5, its precision at recall of 90% performance indicates that it could possibly be used as an aide during drug development processes. Although a limitation of this work is represented by the fact that the model has been tested only on one dataset, please notice that the dataset construction itself is an important part of the methodology.
Moreover, the dataset contains all the side effects for which all the data involved in the prediction are currently available. As a consequence, changing the dataset would also mean changing the data sources, and consequently, the problem's and model's definitions. For the same reason, it is common practice to test the models for DSE prediction only on the dataset for which they are conceived, as each model usually uses a unique combination of the many different pieces of information needed to model these complex biological phenomena. Possible future developments in this context include a dynamic learning approach, by considering a dynamic topology of the graph, in which the connections between the drug nodes do not remain the same throughout the learning procedure, since they can be re-calculated on the basis of the similarity between the neural fingerprints generated at each epoch by the appropriate sub-model, possibly adding some constraints to respect a certain level of similarity given by the one obtained with the standard, preprocessed fingerprints.
Data Availability Statement: The data presented in this study are openly available on GitHub https: //github. com/YohannPerron/MolecularGNN, accessed on 1 November 2022 along with DruGNN python code https://github. com/PietroMSB/DrugSideEffects, accessed on 1 November 2022. Please refer to  [14]  for further information on the dataset and to  [38]  for the GNNkeras python code.
Complete genome of T. humireducens strain PIV-1 In order to identify the key enzymes and their encoding genes, we generated a high-quality genome sequence of T. humireducens through a hybrid sequencing approach combining both long and short reads. A single contig corresponding to the main bacterial chromosome was obtained, encompassing 4 498 018 bases (66% GC content) and harbouring 4365 coding sequences, as well as the sequence of a 39 874 base plasmid (60% GC content) containing 34 coding sequences (Fig. S4 ). Both sequences are available from the ENA/NCBI/DDBJ databases under accession numbers OW011623 and OW011624 (study identifier PRJEB51144). Overall, about 88% of the coding sequences of T. humireducens are classified in at least one EGGNOG group (79% into at least one COG group), and its closest sequenced organisms in terms of genome-level gene syntenies were several Thauera strains isolated from various wastewater treatment systems  (Liu et al.
, 2013). Thauera species are well known for their capability of degrading numerous aromatic compounds and terpenoids under denitrifying conditions, thereby contributing to their elimination from the environment. Comparative genomic analyses (see Experimental procedures) indicated that this new genome and the genome of the type strain of T. humireducens shared an average nucleotide identity (ANI) of 98. 16% and diverged by less than 0. 5% in GC content. The overall difference in genome length was 326 kb with the genome presented here encoding 363 additional proteins. Taxonomic assignments performed using the Genome Taxonomy Database Toolkit [GTDB-Tk,  (Chaumeil et al. , 2020) ] and DSMZ's Type Strain Genome Server platform  [TYGS, (Meier-Kolthoff and Göker, 2019) ] both indicated that the new genome is a new member of the T.
Within the Microscope platform, the taxonomic assignment of the new genome was performed using the Genome Taxonomy DataBase Toolkit  (Chaumeil et al. , 2020)  on the basis of the topology of the genome tree and of ANI values  (Jain et al. , 2018). In this pipeline, the Bacteria reference tree is inferred with FastTree  (Price et al. , 2010)  under the WAG model from the concatenated alignment of 120 ubiquitous bacterial genes  (Parks et al. , 2018). An independent whole genome-based taxonomic assignment was also performed using the Type Strain Genome Database (TYGS, Meier-Kolthoff and Göker, 2019). Briefly, the taxonomic assignment was based on so-called 'digital DNA-DNA hybridization' (dDDH) values, computed according to the GBDP method  (Meier-Kolthoff and Göker, 2019)  between our genome and the 10 most closely related genomes (Table  S4 , provided as supplementary data file) initially identified by MASH sketching  (Ondov et al.
Full MS spectra (350-1550 m/ z) were acquired in the Orbitrap at a resolution of 120 000 with automatic gain control (AGC) target value of 3 Â 10 6 ions. Data resulting from LC-MS/MS measurements were analysed with the Proteome Discoverer (v. 2. 4, Thermo Fischer Scientific) using SEQUEST HT. Protein identification was performed using a self-built reference database downloaded from UniProt (Thauera humireducens PIV1). Searches were conducted with the following parameters: Trypsin as enzyme specificity and two missed cleavages allowed. A peptide ion tolerance of 10 ppm and an MS/MS tolerance of 0. 02 Da were used. As modifications, oxidation (methionine) and carbamidomethylation (cysteine) were selected. Peptides that scored a q-value >1% based on a decoy database and with a peptide rank of 1 were considered identified.
Kinetic modeling of HCD experiments was performed using MassKinetics  19  Scientific Demo software (Version 1. 17. 2. 648 Beta, http://proteomics. ttk. mta. hu/masskinetics/#myPage, generously supplied by Prof. Lászl o Drahos). 4] [15] [16]  Calculation of the frequencies of vibrations for precursor ions was performed using the GAMESS  20, 21  computational package at the Hartree-Fock level of theory employing a Slater-type basis set. Despite the fact that this level of theory is not high, it has been demonstrated  22, 23  that RRKM calculations are not sensitive to the vibrational frequencies when considering only the initial state. A decomposition time of 5 ms 2 was used for simulation of SY curves. For this purpose, deposited internal energy and its evolution with time were modeled utilizing a truncated Maxwell-Boltzmann distribution and characteristic temperature (T char ).
For this simple model, the rate of deactivation is presumed to be negligible, and activation and then fragmentation are considered to be distinct, consecutive steps. [26]  From Arrhenius plots obtained previously using low-energy CID experiments  11  (fig. 6  in reference  11  ), activation energies (E a ) where experimentally measured. For the purpose of assigning critical energies (E 0 ) that are required for RRKM modeling of survival yield curves, MassKinetics 19 Scientific software (Version 1. 17. 2. 648 Beta) was employed. For computation purposes, ions were considered to remain in thermal equilibrium upon fragmentation. Employing this model, critical energies were fine-tuned until the optimal fit was attained between the experimentally recorded Arrhenius plots and those that were derived computationally.
For this reason, the experimentally measured pre-exponential factor (log A = 15. 0 (±1. 1))  11  is used in the MassKinetics software. Then, considering a thermal system, the critical energy was adjusted in the software until the best fit was attained between the experimentally acquired Arrhenius plot and the computationally obtained one (Figure  2 ). By doing so, an E 0 of 1. 54 eV was found for the reference system. Afterwards, for HCD measurements, the deposited mean internal energy, <E int >, and corresponding T char for each point of the SY curve of the reference complex ([Host +1-H] + ) were obtained using the truncated Maxwell-Boltzmann model. A linear relationship between the energy in the laboratory frame of reference and the mean internal energy was established as follows 2,24-26 (Figure  3 ): < E int > ¼ 0:1605 Â E lab þ 2:464 ð1Þ In Equation  1 , the y-intercept of 2.
Study design Full details of the study design have been reported previously  (28). Briefly, the study was a fiveweek diet switch-over from a habitual Western diet to a MedDiet, and then back to a habitual Western diet with a nested, two-week, parallel-group, randomized, total diet replacement, dietary intervention trial comparing organic versus conventional food consumption during the MedDiet phase. The study was carried out in accordance with the Declaration of Helsinki. Ethical approval was obtained from the Faculty of Science Agriculture and Engineering Research Ethics Committee, Newcastle University, UK (Reference number 17-SEA-017) and the trial was registered in ClinicalTrials. gov, NCT03254537.
Randomization and masking Participants self-selected into small single-sex clusters based on shared accommodation during the intervention in Crete. Individuals within these clusters were then randomly allocated to the conventional (n=14) or organic (n=13) group by simple lottery. Clusters with three participants therefore had an uneven allocation which resulted in a skewed sex balance between the study groups (Supplementary Table  1 ; 28). Participants were aware of study group assignment since potential sensory differences between foods and color-coding of dishes of foods to ensure compliance made blinding impossible. All collected samples were coded until after biochemical and statistical analyses were completed. All data were analyzed according to group allocations without deviation from the protocol.
Study outcomes The primary outcome was urinary excretion (UE) of mineral micronutrients, toxic metals and phenolics and plasma concentrations of selected mineral micronutrients (Cu, Fe, Se, Zn). The study was registered in Clinical Trials prior to enrolment of the study-participants. At the time of data analyses (i. e. when the trial was completed) we were able to include additional elements compared O R I G I N A L U N E D I T E D M A N U S C R I P T with those registered, e. g. zinc. We chose this primary outcome because both MedDiet and organic food consumption were reported to lead to higher intakes of phenolics (the most abundant secondary metabolites/antioxidants in plants) and mineral micronutrients, but lower exposure to the toxic metal Cd, which in turn has been linked to positive effects on human health  (1) (2) (3) 13, 14, (17) (18) (19) (20) (21) (22) (23) (24) (25) 34, 35).