dataesr/scientific-paragraphs-categorization huggingface dataset:
ACKNOWLEDGMENTS We are grateful for the help with transcriptome library compilation and analyses by  Glenn Hicks ,  John Weger ,  Holly Eckelhoefer , and  Clay Clark  at the  UCR IIGB Core Facility , the advice on the gene tree generation by  John Heraty , feedback by  Elizabeth McCarthy ,  Jacob Landis , and  Patricia Springer  and  Jaimie Van Norman  lab members, and greenhouse and lab support by  Arman Baghaei ,  Alan Le , and  Victor Herrera.
FUNDING This research was funded by the  National Science Foundation  ( IOS 1456109 ). DM was partly supported by  UCR Graduate Assistance in Areas of National Need  (GAANN) and  Graduate Research Mentoring Program  (GRMP) fellowships. NP-M acknowledges grants by  COLCIENCIAS  ( 111565842812 ), the  Convocatoria  Programaticas  2017-2018 , and the  Estrategia de Sostenibilidad   2018-2019 , in the  Universidad de Antioquia given to the group Evo-Devo en Plantas.
Results and Discussion Data quality assurance. The analytical strategy adopted herein was based on two different chromatographic separation methods (HILIC and RPLC, see experimental section) coupled to detection in mass spectrometry involving both positive and negative-ion modes. This strategy was applied by distinguishing the endo-and exocellular fractions as well to deepen the coverage of the metabolome. As a result, this led to analyze each biological sample 8 times, in distinct sessions of analyses. Prior to any data processing or analysis, several quality checks were adopted (i. e column pressure checking, variations of internal standards in QC/study samples and instrumental stability in terms of retention time and accurate masses) along the whole batch. Accordingly, internal standards were spiked into each sample prior to LC-HRMS analyses to assess retention time stability, the consistency of signal intensities and mass accuracy along the whole batch.
Besides the very limited number of features appearing as dysregulated, one should note that their corresponding Fold-Change indices display modest values remaining below 5 for all features identified in the course of negative-ion mode and below 10 for those pinpointed throughout positive-ion mode analyses. The only exception was M453T283 in this latter group, for which no putative identification could be retrieved. Filtered RPLC datasets related to extracellular fractions consisted of 289 and 126 features in positive and negative-ion modes, respectively. In both ionization modes, PCA plots accounted for roughly 60% of the total variance between the two groups. However, whatever the considered ionization mode, these plots did not lead to a clear-cut discrimination of the samples according to their exposure status (Fig.
Metabolomic HILIC analyses. Endocellular data subset with features satisfying both FC > 2 and ANOVA p-value < 0. 05 were respectively represented by 210 and 58 features in positive and negative-ion modes. Generated PCA plots roughly represented 75% of the total variance and could discriminate between non-exposed and exposed samples along PC2 component in both ion modes (Fig. 3 ). Notwithstanding this discrimination, the statistical processing workflow did not lead to emphasize any relevant biomarker in the negative-ion mode data set. Positive-ion mode endometabolomics only resulted in highlighting one biomarker with a moderate FC value of 2. 8 for which two putative identifications could be retrieved from HMDB database (i. e. either 2-aminoheptanedioic acid or N-carboxyethyl-γ-aminobutyric acid) (Fig.
7 and 8. 4%. Control and exposed samples also exhibited an obvious separation along PC1 dimension (Fig. 4C ). The careful validation of both extracted ion chromatograms and box-and-whisker plot as described in the Data Processing section led applied to retainment of 111 and 99 features as dysregulated in positive and negative polarities, respectively (Figs  S5  and S6 ). The current study was performed using a single HRMS analyser, so that  the identification of the dysregulated features could only rely on exact mass data, including any limitation coming along with it in terms of structural resolution. Resultantly, the structural assignments remain tentative and most often do not lead to reach unambiguous metabolites. From a practical viewpoint, handling an array of references covering the structural diversity of candidate biomarkers is not a realistic purpose in the frame of a metabolomic study pinpointing such an elevated number of features of interest.
Statistical processing. The statistical processing of the metabolomic data considered all exposed and unexposed samples independently of one another. LC/MS data were further processed by R package XCMS (version 3. 2). The preprocessing results generated a data matrix as a feature list table comprising their integrated intensities (reconstructed ion chromatogram peak areas), along with the observed fold-changes and associated p-values 42. Applied peak picking parameters were prefilter = c(5,25), snthresh = 6, mzdiff = 0. 01 and ppm = 15. Initial alignment (bw = 20, minfrac = 0. 66, minsamp = 4, mzvid = 0. 008) and retention time correction (standard loess, plottype = c(deviation) were then applied. Further alignment steps were performed using the same processing parameters with decreasing bw values (lowered to bw = 9 for the second round and to 5 for the final stage).
Subsequently, R-package CAMERA was used for peak annotation after XCMS data processing  43. Consistently with reported guidelines, features found in less than 20% of the analyzed samples were removed according to the so-called 80% rule  44. As to univariate analyses, the coefficient of variation (CV) within QC samples was calculated by dividing the standard deviation by the mean intensity of each feature, leading to a histogram of the resulting CV distribution. Subsequently, computation of the Fold-Change (FC, ratio of abundance between non-exposed and exposed samples) along with the corresponding p-value (statistical significance from a Student t-test) streamlined the selection of metabolites of interest. Features selection was based on the following criteria: CV QC < 30%, FC > 2 and ANOVA p-value < 0.
05. Extracted Ion Chromatograms (EICs) were individually monitored to exclude potential artifacts from the ion list. Later on, Box and Whiskers Plot related to all metabolites of potential interest were also individually checked to retain the features for which no overlap existed between values obtained from non-exposed and exposed samples. Such features were tentatively identified against databases as described later on. The top and bottom of each box represent the 25 th and 75 th percentiles, the center line indicates the median and the extent of the whiskers depicts the 5th and 95th percentiles. Regarding multivariate analyses, Principal Component Analyses (PCA) were performed to get a general overview of the interrelationship between study samples as well as QC. Statistical graphs were prepared using SigmaPlot 13.
+ , [M-H 2 O + H] + , [M + Na] + and [M-H 2 O + Na] + were selected as possible adducts for positive polarity. Regarding negative-ion mode, [M-H] -, [M-H 2 O-H] -and [M-H 2 O + HCOOH-H] -were considered. Putative identifications were carried out against the freely available database Human Metabolome Data Base (HMDB). Whenever possible, efforts were made to narrow down identification possibilities among isobaric compounds according to elution order, matrix of occurrence.
The effectiveness of most cancer targeted therapies is short-lived. Tumors often develop resistance that might be overcome with drug combinations. However, the number of possible combinations is vast, necessitating data-driven approaches to find optimal patient-specific treatments. Here we report AstraZeneca's large drug combination dataset, consisting of 11,576 experiments from 910 combinations across 85 molecularly characterized cancer cell lines, and results of a DREAM Challenge to evaluate computational strategies for predicting synergistic drug pairs and biomarkers. 160 teams participated to provide a comprehensive methodological development and benchmarking. Winning methods incorporate prior knowledge of drug-target interactions. Synergy is predicted with an accuracy matching biological replicates for >60% of combinations.
A key bottleneck in the development of such models has been a lack of public data of sufficient size and variety to train computational approaches  4, 8  , particularly considering the diversity of biological mechanisms that may influence drug response. A further limit to the translatability of many computational approaches is their reliance on data features that may not be present during the course of patient care, such as on-treatment tumor molecular profiles. To accelerate the understanding of drug combination synergy, Dialog for Reverse Engineering Assessments and Methods (DREAM) Challenges partnered with AstraZeneca and the Sanger Institute to launch the AstraZeneca-Sanger Drug Combination Prediction DREAM Challenge. DREAM Challenges (dreamchallenges. org[www. dreamchallenges. org]) are collaborative competitions that pose important biomedical questions to the scientific community, and evaluate participants' predictions in a statistically rigorous and unbiased way, emphasizing model reproducibility, and methodological transparency  9.
Participants used the described data to train and test models, and were encouraged to extend computational techniques to leverage a priori knowledge of cellular signaling networks. In this manuscript, we report on the results of this Challenge where we have identified novel and performant methods using a rigorous evaluation framework on previously unpublished data. We describe the details of these approaches, as well as general trends arising from the meta-analysis of all submissions. The full dataset, along with methods and scoring functions, are freely provided to the research community, and available to benchmark future algorithms in the field. Finally, we describe putative mechanistic models derived from the observed predictive features underlying synergistic responses, particularly between receptor tyrosine kinase and PI3K/AKT pathway inhibitors.
Results A large high-throughput drug combination screen. We collated a combinatorial drug sensitivity screen comprising 11,576 experiments each measured in a 6-by-6 dose matrix (Methods) across 85 cancer cell lines. A synergy and an antagonism distribution (Loewe reference model  10, 11  ) were calculated and summarized via a single score for each experimental matrix (Methods; Supplementary Fig. 1  and Supplementary Data 1). The resulting dataset included highly reproducible cell viability response measurements and synergy scores for 910 pairwise combinations of 118 drugs (Supplementary Fig. 2  and Methods), plus information on the drugs including putative drug targets and their chemical properties. We also integrated deep molecular characterization of these same cell lines, including somatic mutations, copy-number alterations, DNA methylation, and gene expression profiles (Fig.
3 vs. 7. 3, t-test P = 2e-18) than the remaining 12% (N = 113) whose drug targets were defined to be in distinct pathways. As part of the Challenge design, we ensured that drug targeted pathways and cancer types were proportionally distributed across sub-challenges and training/test datasets (Fig. 1g ). Sparsity in the cell line drug combination data matrix (Fig. 1g ) resulted as several drug combinations were selectively profiled in clinically relevant cancer cell lines, e. g. , ESR1 inhibitors were predominantly combined with other drugs in estrogen receptor-positive breast cancer cell lines since these agents are standard of care within this cancer subtype.
Comparison of AZ-DREAM to published combination screens. We compared the AstraZeneca-DREAM (AZ-DREAM) challenge dataset with the independently published in vitro combination screening studies by O'Neil et al. 4  and ALMANAC  13. All three studies used different experimental designs and protocols (Fig. 2a  and Supplementary Table  1 ), with AZ-DREAM exploring screens with a 5-by-5 concentration titration format, O'Neill et al. 4  a 4by-4 format, and ALMANAC a 3-by-3 format. The overlap is limited in cell lines, targets and drugs explored: 10 cells and 7 targets, 4 drugs and 0 combination-cell pairs shared between AZ-DREAM and O'Neil et al. 4  ; and 15 cells, 28 targets, 19 drugs, and 10 combination-cell pairs shared between AZ-DREAM and ALMANAC. AZ-DREAM provides greater coverage of targeted agents and greater numbers of cell lines per tumor type (Supplementary Table  1 ).
For the AZ-DREAM and ALMANAC comparison, only nine experiments passing quality control (Supplementary Methods) had the same combination tested in the same cell line, comprising gefitinib combined with either tamoxifen, fulvestrant, vorinostat, crizotinib, or everolimus, and tested in two breast cancer cell lines (MCF7 or T47D). Encouragingly, all but one experiment were concordantly identified synergistic with the same effect signs seen in AZ-DREAM and ALMANAC (Fig. 2b ). Within the nine overlapping experiments, 38 titrations were identically used by both ALMANAC and AZ-DREAM (Supplementary Fig. 3  and Supplementary Fig. 4a ). For these 38 titrations we observed a correlation >0. 7 (Fig. 2c  and P < 10 -7 , test for association) between cell viability measures from AZ-DREAM and ALMA-NAC, maintained for individual combinations and cell lines    4  and ALMANAC datasets.
2c, d ) and fewer instances of extreme synergy scores. The technical reproducibility within these screens, however, was comparable, with a similar correlation observed between replicated experiments (Spearman = 0. 63 for O'Neil et al. ,  4  Spearman = 0. 56 for AZ-DREAM). We derived a RECIST-like response measure from the AZ-DREAM in vitro data (see Methods) to enable comparison to in vivo response metrics for the eight overlapping combinations in patient-derived tumor xenograft (PDX) models published by Gao et al. 8  (Supplementary Data 1 and Supplementary Fig. 5a ). We compared the % PDX models with synergy in Gao et al. 4, 8  to the % cell lines with synergy in AZ-DREAM. A concordant trend was observed (Pearson r = 0. 34, P = 0. 42; Methods; Supplementary Fig. 5b ), although too few combinations were shared between the datasets to conclude with statistical confidence.
Random Aggregation is an ensemble combining a random combination of models. Standard error of mean represented by error bars are estimated from ten random splits of the data teams (Fig. 3b ). All performance statistics and team rankings are available at the Challenge website (synapse. org/DrugCombina-tionChallenge[https://www. synapse. org/#!Synapse:syn4231880/ wiki/235649]). To benchmark the performance of teams in the final rounds of SC1A/B and SC2, we established lower and upper bounds of performance. We defined the lower bound as the null model, i. e. , random permutation of the synergy data across each cell line (see 'Code availability'' section). We would not expect algorithms to predict better than the reproducibility observed between replicate experiments. We therefore identified cases for which replicate measurements (same drug/combination tested independently in the same cell line) were available and assessed the primary metric achieved when using one measure to predict the other.
Mutational and copy-number variation (CNV) data can similarly offer a barcode of cell identity to encode cell line label. However, where mutation data improved performance when replacing cell line labels, replacement with CNV decreased performance significantly (Fig. 4a , t-test P = 8. 8e-6). Importantly, in all cases additional feature data increased performance when added to the NAD baseline model (Fig. 4a , t-test P = 0. 009, 0. 009, 0. 002, 0. 008, 0. 021 adding drug target, KEGG pathway, Gene Ontology, signaling network, and mutation features, respectively). Ensemble of different feature sets improved prediction most when collectively increasing coverage of biological (pathway) complexity (Fig. 4a , t-test P = 1. 2e-6). Inspecting consistently poorly predicted drug combinations. While a global performance metric applied to all cell-lines and drug combinations provides a broad assessment of model prediction accuracy, we hypothesized that some models may be optimized for certain subclasses of combinations and/or tumor types.
The drugs targeting these pathways were prevalent in our dataset, but these specific combinations of those drugs were not (Supplementary Fig. 9a ). Assessment of the interactions between drug targets and neighboring proteins from OmniPath, a comprehensive compendium of literature-based pathway resources  19  , revealed no differences in the somatic alteration frequency for targets or their first neighbors between Size of node is proportional to number of drugs targeting a specific pathway in the entire dataset and width of edges is proportional to the number of drug combinations within the poorly or well predicted combination subset. Red edges highlight target pathway combinations unique to either the well predicted or poorly predicted combinations. c Types of interactions between the nearest neighboring gene and the two drug targets of poorly and well predicted combinations.
Among the prioritized feature-to-combination associations were several genetic variants associated with synergistic responses to the combination of receptor tyrosine kinase (RTK) inhibitors with inhibitors of the downstream PI3K/AKT pathway. Amplifications or activating mutations in EGFR or ERBB2 consistently predicted synergy from inhibition of the RTK + PI3K/AKT pathways across multiple independent drugs and datasets (Fig. 7a ). Less direct relationships were also observed, including combined AKT inhibition with either IGFR inhibition in the ERBB2 mutant setting or FGFR inhibition in the EGFR mutant setting (Supplementary Data 3). Despite kinase domain homology it is unlikely these observations are explained by off-target effects since EGFR, ERBB2, and FGFR mutations were only predictive of respective monotherapy responses (Supplementary Fig.
We assessed the performance of top-performing AstraZeneca-DREAM models on the independent screening datasets by O'Neil et al. 4  and ALMANAC  13. Since no combination-cell experiments directly overlapped AZ-DREAM and O'Neil et al. 4  , we collapsed drugs by shared targets to expand the overlap. We observed that SC1A models from NAD and DMIS outperformed random models (Fig. 8a , mean primary metric = 0. 07, top 1% of random models) for cell lines and drug target combinations nonoverlapping between O'Neill et al. and AZ-DREAM data (Supplementary Data 1). Interestingly, no substantial performance increase was observed when independent model predictions were made for the ten cell lines in common between the two datasets, nor the 30 combinations with similar chemical/target properties (Supplementary Table  2  and Fig.
8b-d ). As in the main Challenge, combining these two or more models in an ensemble led to an improved prediction performance (Fig. 8a-d ). Considering the limited overlap and correlation of synergies between AZ-DREAM and ALMANAC datasets, high performance was not expected when predicting ALMANAC synergy Fig. 7  Post-hoc analysis of putative synergy biomarkers. a Synergy markers suggested by DMIS and NAD, when focusing on top weighted features from predictive models filtered for biological relatedness to drug targets, ANOVA with FDR of triple asterisks refers to 5%, double asterisks refers to 20% and single asterisks to 35%. b Venn-diagram showing independent and overlapping cell lines between AZ-DREAM and O'Neil et al. 4. Independent dataset reproducibility of biomarker predictions in c the overlapping and d O'Neil et al.
4  exclusive cell lines. The effect size is the mean difference in synergy scores of mutant vs. wild-type cell lines. e Comparison of ADAM17 combined with PIK3CB/D against ADAM17 in combination with pan-PIK3C inhibitor. f Network cartoon of PI3K signaling and role of ADAM17 scores with models trained on AZ-DREAM data points alone, as was observed for the NAD model. We trained two NAD models on 50% of the ALMANAC data: first a baseline model to show maximum performance achievable when using only cell and drug label features as input; and a second allowing use of the full feature set as input. The full model consistently predicted significantly better than baseline in the remaining 50% over ten randomized iterations, giving confidence in the transferability of the method designed for AZ-DREAM.
Best performance was observed for targeted combinations (r = 0. 369 vs. 0. 287 for full NAD and baseline NAD model, ANOVA P = 1. 322e-31 for model type and P = 5. 5e-05 for model type and targeted therapy association, Fig. 8e ). Exploring AZ-DREAM biomarker associations prioritized as described in earlier sections, we assessed statistical association for drug combinations with consistent targets in the O'Neil et al. 4  dataset. In the ten cell lines overlapping the AZ-DREAM and O'Neil et al. 4  datasets, seven of the prioritized biomarkerdrug combinations were present, of which six (86%) showed reproducible directionality (FDR < 35%, Fig. 7c ). In the 21 O'Neil et al. 4  cell lines not used within AZ-DREAM training, 11 of the prioritized biomarker-drug combinations were present, of which 8 (72%) showed reproducible directionality (Fig.
Discussion The objective of this AstraZeneca-DREAM Challenge was to drive the development of innovative computational approaches to predict novel drug combinations and to comprehensively benchmark these approaches. To enable this we made one of the largest in vitro drug combinatorial screens to date available to the scientific community. We cover largely non-overlapping experiments to existing datasets and, in particular, offer extensive data for targeted therapies complimenting the non-targeted chemotherapeutic agents covered in the NCI-ALMANAC. Despite little overlap we demonstrated an encouraging reproducibility of data and predictions between screens, particularly when considering the confounding differences in experimental designs and assay formats  21. Furthermore, we showed that some trends represented in these data can be reproduced in vivo, and that clinically efficacious combination pairs can be identified.
Our double-blinded benchmark of 78 methods provides an unbiased comprehensive evaluation of the state-of-art of drug synergy prediction. Collectively this Challenge manuscript equips the scientific community with data and a methodological baseline for algorithm development, alongside a suite of computational methods to direct new experiments towards likely synergistic drug combinations. The results of the AstraZeneca-Sanger Drug Combination Prediction DREAM Challenge have shed important light on the best practices and limitations in predicting drug synergy. By evaluating predictions from a large number of teams, we were able to discern important strategies for predicting drug synergy from molecular and chemical traits. As with most DREAM Challenges, we observed that the machine learning method itself has little impact on overall performance.
Aggressive pre-filtering that incorporates clean sparse network data to consider feature relevance to drug targets and cancer was successfully used by top Fig. 8  Translatability AstraZeneca-DREAM models to the independent screens. Performance of SC1A models for predicting synergy scores in the O'Neil et al. 4  dataset by the best performing teams are plotted along with distributions of predictions from the random model and replicate experiments. Performance of predictions are shown for a all experiments in the O'Neil et al. 4  dataset, and three subsets of the dataset; b experiments that tested same cell lines as AZ-DREAM, c tested similar drugs as in AZ-DREAM (one drug in the combination with the same target), and d tested similar combinations as in AZ-DREAM (same targets for both drugs in the combination).