dataesr/scientific-paragraphs-categorization huggingface dataset:
Suppression of CRISPR immunity by Acr was measured through a transformation assay. Overnight cultures of CRISPR-KO, BIM2 or BIM5 bacteria (approximately 5*10 9 cfus) grown in LB broth were either not infected or infected with 1. 6*10 9 pfus DMS3mvir, DMS3mvir-AcrIF1 or DMS3mvir-AcrIF4 in 50 mL Falcon tubes containing 10 mL of Luria-Bertani (LB) broth, and incubated at 37 C while shaking at 180 rpm for 2 hours. Bacteria were harvested by spinning at 3500 rpm for 30 minutes. A sample from the supernatant was used to quantify phage titers following chloroform extraction as described above (data not shown). The bacterial pellet was further processed for the transformation assay. First, cells were washed twice in 1 mL of a 300 mM sucrose solution to make them competent  (Choi et al. , 2006). Next, the pellet was resuspended in 300 mL of 300 mM sucrose, and 100 mL from this was used for plating on LB agar to enumerate total bacterial cfus after the infection and sucrose-washing steps (data not shown).
ACKNOWLEDGMENTS M. L. was supported by funding from the  Wellcome Trust  (https://wellcome. ac. uk) ( 109776/Z/15/Z ), which was awarded to E. R. W. E. R. W. further acknowledges the  Natural Environment Research Council  (https://nerc. ukri. org)( NE/ M018350/1 ), the  BBSRC  ( BB/N017412/1 ), and the  European Research Council  (https://erc. europa. eu)( ERC-STG-2016-714478 -EVOIMMECH ) for funding. S. v. H. acknowledges funding from the  People Programme  ( Marie Curie Actions ; https://ec. europa. eu/research/mariecurieactions/) of the European Union's  Horizon 2020  ( REA grant  agreement no. 660039 ) and from the  BBSRC  ( BB/R010781/1 ). S. G. acknowledges funding ( Visiting Professorship ) from the  Leverhulme Trust. A. B. acknowledges funding from the  Royal Society. The authors thank  Olivier Fradet  for experimental contributions and  Adair Borges  and  Joe Bondy-Denomy  ( UCSF ) for providing DMS3mvir-AcrIF4 and phage  JBD26.
DATA AND SOFTWARE AVAILABILITY Sequence analysis scripts and draft genome assemblies of DMSmvir, DMS3mvir-AcrIF1, and DMS3mvir-AcrIF4 can be accessed from https://github. com/s-meaden/landsberger under https://github. com/s-meaden/landsberger/blob/master/DMS3mvir. gff. gz, https://github. com/s-meaden/landsberger/blob/master/DMS3mvirAcrF1. gff. gz, and https://github. com/s-meaden/landsberger/ blob/master/DMS3mvirAcrF4. gff. gz. Sequence data have been deposited in the European Nucleotide Archive under accession number ENA: PRJEB25016. Raw data have been deposited at Mendeley Data under https://doi. org/10. 17632/vt434wb4b4. 1. e6 Cell 174, 908-916. e1-e6, August 9, 2018.
Adverse drug reactions, also called side effects, range from mild to fatal clinical events and significantly affect the quality of care. Among other causes, side effects occur when drugs bind to proteins other than their intended target. As experimentally testing drug specificity against the entire proteome is out of reach, we investigate the application of chemogenomics approaches. We formulate the study of drug specificity as a problem of predicting interactions between drugs and proteins at the proteome scale. We build several benchmark datasets, and propose NN-MT, a multi-task Support Vector Machine (SVM) algorithm that is trained on a limited number of data points, in order to solve the computational issues or proteome-wide SVM for chemogenomics. We compare NN-MT to different state-of-the-art methods, and show that its prediction performances are similar or better, at an efficient calculation cost.
This would indeed require, for each of the considered proteins, that the binding affinities of multiple ligands were available. The second category is docking (refer to  [7]  for a recent review on docking), also called target-based approaches. Docking is a molecular modeling method that predicts the affinity of a ligand for a protein based on the estimated interaction energy between the two partners. However, it relies on the 3D structure of the proteins, which strongly limits its application on a large scale. Finally, chemogenomic approaches  [8]  can be viewed as an attempt to fill a large binary interaction matrix where rows are molecules and columns are proteins, partially filled with the known protein-ligand interaction data available in public databases such as the PubChem database at NCBI  [9].
Given a molecule kernel K molecule and a protein kernel K pro- tein , the Kronecker kernel K pair is defined by: K pair ððm; pÞ; ðm 0 ; p 0 ÞÞ ¼ K molecule ðm; m 0 Þ Â K protein ðp; p 0 Þ ð2Þ Thus, the Kronecker kernel K pair captures interactions between features of the molecule and features of the protein that govern their interactions (see Supplementary Materials S2 for an explicit definition of the Kronecker product of two matrices). If K molecule is a n × n matrix and K protein is a p × p matrix, their Kronecker product K pair has size np #x00D7; np. In the context of chemogenomics, this can correspond to a very large size, leading to intractable computations. However, one interesting property of the Kronecker kernel is that calculating its values on a data set of (m, p) pairs does not require storing this entire matrix since it is sufficient to store K molecule (m, m 0 ) and K protein (p, p 0 ).
While the applicability of multi-task approaches can be limited in practice by computational times, our approach only requires training on a dataset of size similar to those used by single-task methods. We evaluate the performance on various assembled datasets in which the protein and/or the ligand are orphan. Finally, compare the NN-MT algorithm to state-of-the-art approaches in drug-target interaction prediction  [18] [19] [20] [21] [22] [23] [24]. We used and updated the PyDTI package  [24] , adding an implementation of NN-MT together with key cross-validation schemes and a DrugBank-based dataset built in the present study. In addition to all other experiments performed in the present study, this benchmark study concludes that NN-MT is a good default method providing stateof-the-art or better performances, in a wide range of prediction scenario that can be encountered in real-life studies: proteome-wide prediction, protein family prediction, test (protein, ligand) pairs dissimilar to pairs in the train set, and orphan cases.
Evaluation of prediction performance Prediction performance is commonly evaluated with a cross-validation (CV) scheme  [52] : 1) the dataset is randomly split into K folds 2) the model is run K times, each run using the union of (K-1) folds as the training set, and measuring the performance on the remaining fold. Prediction performance are averaged over all folds. When hyperparameters had to be selected, we used a nested cross validation (Nested-CV) scheme  [53]. It consists in a (K-1) folds cross validation (inner-CV) nested in a K folds cross validation (outer-CV). At each step of the outer-CV, the inner-CV is repeated for all considered values of the hyperparameters. The values leading to the best prediction performance are retained as optimal. We used K = 5, a classical value in CV. We also considered leave-one-out cross-validation (LOO-CV), for which the number of folds is the number of available points in the dataset.
The LOO-CV scheme is particularly useful when the number of samples is small. It was used in the present paper when the size of the considered dataset was too small to perform 5-fold-CV. We estimated prediction performance using two scores that are classically employed to judge the quality of a classifier in case of drug-target interaction prediction. The first one is the area under the Receiver Operating Characteristic curve  [54]  (ROC-AUC). The ROC curve plots true positive rate as a function of false positive rate, for all possible thresholds on the prediction score. Intuitively, the ROC-AUC score of a classifier represents the probability that if a positive and a negative interaction are each picked at random from the dataset, the positive one will have a higher positive score than the negative one.
The second one is the area under the Precision-Recall curve  [55]  (AUPR). It indicates how far the prediction scores of true positive interactions are from false positive interactions, on average. Although we used both the ROC-AUC and AUPR scores, since negative interactions are actually unknown interactions in protein-ligand interaction datasets, the AUPR is considered a more significant quality measure of the prediction method than the ROC-AUC. Indeed, it emphasizes the recovery of the positive samples and penalizes the presence of false positive examples among the best ranked points. We used the Python library scikit-learn  [56]  to implement our SVM-based machine learning algorithms and we used the PyDTI package  [24]  for the other recent algorithms.
Datasets Many publicly available databases such as KEGG Drug  [57] , DrugBank  [58] , or ChEMBL  [59]  can be used to build a learning dataset of protein-ligand interactions. We chose to build all the datasets used in the present study from the DrugBank database v4. 3, because it contains FDAapproved drugs, or drug candidate molecules. This allowed optimize and test our models on drug-like molecules, on which they intend be applied. In addition, we assumed that the list of human proteins appearing as targets for molecules of DrugBank can represent a relevant "druggable" human proteome on which we could train models that predicting the specificity of drug-like molecules. We built a first learning dataset called S, based on Version 4. 3 of the DrugBank  [58]. We selected all molecules targeting at least one human protein, and having a molecular weight between 100 and 600 g.
mol -1 , a range in which most small molecule marketed drugs are found  [60]. This leads to a dataset composed of 3980 molecules targeting 1821 proteins, and including 9536 protein-ligand interactions that correspond to the positive training pairs. All other protein-ligand pairs are unlabeled because no interactions were recorded for them in the database. Most of these pairs are expected not to interact, but a small number of them are in fact missing interactions. However, we considered that all unlabeled pairs as negative examples, allowing the predictor to re-classify some of these pairs as positive examples. We built several other datasets using exactly the same training pairs as those in S, but 5-folded in various ways. Datasets S 1 , S 2 , S 3 , and S 4 are folded so as to correspond to random, orphan protein, orphan ligand, and double orphan prediction situations.
The construction of these four datasets is detailed in Section 3. 2, where they are used. Datasets S 0 1 , S 0 2 , S 0 3 , and S 0 4 are also folded to mimic the same situations, but with the additional constraint that proteins and ligands were clustered based on their similarities, and each fold contains only one cluster of proteins and of ligands. The goal is to test the performance of the method in situations similar to those of S 1 , S 2 , S 3 , and S 4 , but with the added difficulty that the test set (one fold) and the train set (the 4 other folds) contain pairs that have low similarities. This setting is relevant when considering proteome-wide predictions: many of the proteins to consider may not have close neighbors among the proteins for which the most information (i. e. ligands) are known.
The construction of these four datasets is detailed in Section 3. 3, where they are used. We also built a dataset called S 0 by keeping only molecules and proteins in S which are involved at least in two interactions, in order to compare the prediction performance of the proposed methods with those of ligand-based and target-based approaches. Indeed, these two single-task approaches require at least two data points, one used as train, and one as test. Consequently, when a LOO-CV scheme is used, no ligand and no protein are orphans in S 0. S 0 contains 5908 positive interactions and was used in Sections 3. 4 and 3. 5. In addition, we randomly generated four sets of 5908 negative interactions involving proteins and ligands found in the positive interactions, while ensuring that each protein and each ligand are present in the same number of positive and negative interactions.
Then, we assessed performance by computing the mean and standard deviation of the AUPR scores over test sets including the positive interactions set and one of the negative interactions sets. Finally, we built three protein family datasets by extracting from S 0 all protein-ligand interactions involving respectively only G-Protein Coupled Receptors (GPCR set), ion channels (IC set), and kinases (Kinases set). These datasets were used to evaluate performance of our method within a family of proteins, and compare it to those of single-task approaches. They were extracted from S 0 (and not from the larger dataset S) since again, these comparisons used the LOO-CV scheme, which requires at least two data points per protein and per molecule. Table  1  gives some statistics about the datasets, including the distribution of targets per drug and the distribution of ligands per protein.
Kernel selection and parametrization The goal of this section is to choose a protein kernel and a molecule kernel that we will use throughout the remainder of this study. We assumed that kernels optimized on a large dataset of interactions between drug-like molecules and druggable human proteins such as dataset S would be good default kernels for the prediction of drug candidates specificity. Therefore, we optimized kernels on dataset S (the largest dataset built in the present study), and used the best-performing couple of kernels in the remainder of the paper. The set of (protein, ligand) pairs in S were randomly 5-folded, and we performed a nested 5-fold-CV experiment in order to evaluate the six possible kernel combinations and their best hyperparameters. Table  2  gives the best prediction performance for the six combinations of protein and molecule kernels, together with the corresponding hyperparameters.
Performance of multi-task approaches in orphan situations The goal of this section is to evaluate the performance of MT in cases where the queried (protein, molecule) pairs contain proteins and/or molecules that are not in the training set, as proposed by  [26]. For that purpose, all the pairs of dataset S were used and 5-folded as follows in order to create four cross-validation data sets: • S 1 : randomly and balanced in positive and negative pairs; • S 2 (corresponding to the "orphan ligand" case): (protein, molecule) pairs in one fold only contain molecules that are absent from all other folds; prediction on each test set (each fold) is performed using train sets (the four other folds) in which no the ligands of the test set are absent. • S 3 (corresponding to the "orphan protein" case): (protein, molecule) pairs in one fold only contain proteins that are absent from all other folds; prediction on each test set is performed using train sets in which no the proteins of the test set are absent.
• S 4 (corresponding to the "double orphan" case): (protein, molecule) pairs in one fold only contain proteins and molecules that are both absent from all other folds. Prediction on each test set is performed using train sets in which no the proteins and the ligands of the test set are absent. The folds of S 4 were built by intersecting those of S 2 and S 3 and S 4. Thus, S 4 contains 25 folds and not 5. Fig  1  shows the nested-CV AUC and AUPR scores obtained by the MT method on the S 1 -S 4 datasets. As expected, the best scores are obtained for S 1 , and the worst for S 4 , since in S 4 , no pairs of the train set contain the protein or the ligand of the tested pair to guide the predictions. The loss of performance between the random and the double orphan settings is about 0. 12 both in AUC and AUPR.
However, the performance on the S 4 dataset remains well above those of a random predictor. These results confirm that MT chemogenomics can make predictions for (protein, ligands) pairs made of unseen proteins and unseen ligands, even in datasets containing very diverse types of proteins. This confirms previous observations made on less diverse datasets  [26]. It is important to point that single-task approaches would not be able to provide any prediction on the S 4 dataset. The scores obtained in the S 2 and S 3 datasets are intermediate between those observed on S 1 and S 4. This was to be expected, as in these datasets, the algorithm can rely on training pairs  containing either the same proteins (S 2 ) or the same ligands (S 3 ) as the test set. The AUC and AUPR scores are both slightly better for S 3 than for S 2 , which suggests that predicting ligand for new protein targets is easier than predicting targets for new compounds, as already noticed in  [26].
Impact of the similarity of the training examples to the test set To evaluate the impact on performance of the dissimilarity between training and test pairs, we re-folded the pairs of S following the "clustered cross-validation" approach  [62]. More precisely, we clustered proteins (resp. ligands) into 5 clusters by hierarchical clustering  [63]. We then built four cross-validation datasets, S 0 1 À S 0 4 , generated based on folds similarly as S 1 -S 4 , but with the added constraint that all pairs in a given fold are made of proteins from a single protein cluster and ligands from a single ligand cluster. Therefore, test pairs are more dissimilar from train pairs than in the S 1 -S 4 datasets, which makes the problem more difficult. Overall, all the pairs of dataset S were 5-folded as follows in order to create four crossvalidation data sets: • S 0 1 : randomly and balanced in positive and negative pairs, each fold containing proteins and ligands belonging to the same cluster; • S 0 2 (corresponding to the "orphan ligand" case): (protein, molecule) pairs in one fold only contain molecules that are absent from all other folds; prediction on each test set (each fold) is performed using train sets (the four other folds) in which no the ligands of the test set are absent, with the additional constraint that each fold contains proteins and ligands belonging to the same cluster.
Thus, S 4 contains 25 folds and not 5. We used the same kernels as for the MT method. Fig  2  shows the prediction performance of MT on these new cross-validation folds. For all the datasets, we observed a strong decrease in prediction scores with respect to those obtained on the corresponding S 1 -S 4 datasets. This suggests that good performance on a query pair (p Ã , m Ã ) is driven by the presence in the training set of pairs made both of proteins similar to p Ã and of molecules similar to m Ã , even if the query pair (p Ã , m Ã ) is a double orphan, as in S 4. Our results also suggest that it is more important to train on pairs similar to the double orphan query pair (p Ã , m Ã ), as in S 4 , than on data containing, for example, p Ã itself, but paired only with molecules quite dissimilar to m Ã , as in S 0 2.
Multi-task approaches on reduced training sets In all the following experiments, we used the LOO-CV scheme because intra-task and extratask pairs can only be defined for each pair separately, which prevents from using K-fold-CV schemes. In addition, in single-task approaches, the size of the training set was relatively small in most cases (see datasets statistics in Section 2), which does not allow to fold the data. We checked that the LOO-CV scheme did not trigger a bias, as sometimes observed  [26]  (see Supporting Information S2 Fig and  S3  Table ). Because prediction of a given (protein, ligand) interaction can only be made by single-task if the pair partners are present in at least another pair of the train set, in the following experiments, we used the S 0 dataset in which all ligands and all proteins are involved in at least two known interactions, as explained in Section 2.
Adding similar extra-task positive examples. The results from Section 3. 3 suggest to explore the performance of the MT-intra method when trained on various datasets including extra-task pairs close to the tested pair, in addition to the intra-task pairs. Therefore, we built train sets made of: • the train set of MT-intra • n þ e closest extra-task positive pairs with respect to the tested pair (n þ e 2 f1; 5; 10; 50g).
The results we have presented so far address the issue of using kernel methods with SVM in the context of proteome-wide specificity prediction, at a tractable computational cost thanks to the choice of a reduced learning dataset, without loss in prediction performance. Reducing datasets to the most informative data points is also the underlying idea of active-learning methods  [64]. In the latter, the goal is to guide step by step which data points are needed (i. e. have to be observed and labeled) to best improve the prediction performance. These methods are used when acquiring data is the limiting factor. By essence, their goal is also to reduce the size of the data sets that are used. However, another key issue corresponds to study the specificity of a molecule within a family of related proteins.
Specificity prediction within families of proteins We considered three families of proteins because they gather a wide range of therapeutic targets, and have also been considered in other chemogenomics studies, thus providing reference prediction scores: G-Protein Coupled Receptors (GPCRs), ion channels (IC), and kinases. All the (protein, molecule) pairs involving GPCRs, ICs, or kinases that were present in the dataset S described in Section 2. 4 were used to build the three corresponding family datasets. We compared the performance of the MT-intra method (trained using only positive pairs involving the protein or the ligand of the tested pair) to those of the NN-MT and RN-MT methods, in order to evaluate the interest of the multi-task approach in family studies. We considered two versions: one in which the Profile protein kernel is used, as in the above sections, and another in which a family-based hierarchy kernel is used (Section 2.
1), because a sequencebased kernel may not be optimal to study the specificity of the molecule within a family of proteins  [11, 27]. The corresponding methods are called MT-intra-family, NN-MT-family, and RN-MT-family. As in the above section, each (protein, ligand) test pair is considered in turn in a LOO-CV scheme. We used a learning dataset containing: all positive intra-task positive pairs, ten times more random negative intra-task pairs (this value was found adequate in previous sections), a varying number of positive extra-task pairs (nearest neighbors for NN-MT or NN-MT-family, random for RN-MT or RN-MT-family), and a number of negative extra-task pairs so that the ratio of n À e =n þ e varies from 1 to 20.
In this context, the sequence kernel that was optimized in proteome-wide studies might better capture the degree of similarity between two kinases than the hierarchy kernel does. Overall, the above results on the IC, GPCR and kinase families indicate that the proposed NN-MT method leads to the best results when the train set includes all positive intra-task pairs, 10 times more random negative intra-task pairs, a small number of nearest neighbors positive extra-task pairs (in the range of 10) and around 5 times more random negative extratask pairs. These conditions are very similar to those leading to the best prediction scores when ligand specificity is studied on large scale in the protein space. Even if the performance of NN-MT on family datasets is better than those reached by other methods on similar datasets  [11, 20] , they remain in the same order of magnitude.