dataesr/scientific-paragraphs-categorization huggingface dataset:
Actives Selection A two-step process was applied to rationally select final active compounds for the MUV data sets. (1) Potential active compounds were filtered to eliminate artifacts caused by organic chemicals aggregation in aqueous buffers ("Hill slope filter"), as well as off-targets, cytotoxic effects or interference with optical detection methods ["frequency of hits filter" and "autofluorescence  (Simeonov et al. , 2008)  and luciferase inhibition  (Auld et al. , 2008)  filters"]. (2) A "chemical space embedding filter" was applied to ensure that actives located in regions of the chemical space devoid of decoys were eliminated from the dataset (Figure  1 ). Subsets of 30 actives with the maximum spread per target were generated using a Kennard-Jones algorithm. Selected active compounds were exchanged with remaining potential active compounds until all datasets were adjusted to a common level of spread.
The level of separation between the decoy compounds and the active compounds was adjusted to the same level of spread so that the data clumping is null. In total, 500 decoys were selected per selected active, resulting in 15,000 decoys per dataset. The minimization of analog bias and artificial enrichment makes the MUV datasets fitted for LBVS. The availability of structures in the PDB (2008) for seven of the MUV targets makes it suitable for SBVS as well  (Löwer et al. , 2011). Thus, the MUV constituted the first dataset that enabled comparative evaluations of SB and LBVS methods and protocols.
Initial Compounds Database Decoy compounds from the DEKOIS 2. 0 benchmarking datasets are selected from a subset of the ZINC database of 15 million molecules. Eight physicochemical properties are evaluated: molecular weight, octanol-water partition coefficient, hydrogen bonds acceptor and/or donor, number of rotatable bonds, positive and negative charges, and the number of aromatic rings. For each physicochemical property, bins are defined, and all possible combinations of bins are used to split the database compounds into cells. The initial bins are defined so that each bin is equally populated, and each final cell is characterized by a set of 8 physicochemical properties. Each user-provided active compound is associated with the closest cell (in terms of physicochemical properties), and 1,500 decoys are randomly preselected from this parent cell, or from the direct neighbor cells if the parent cell is not populated enough to provide 1,500 decoy compounds (Figure  1 ).
Initial Compounds Database Active compounds assigned to each target of the DUD-E were collected from the ChEMBL09 database if their activity/affinity (Ki, Kd, IC50, EC50, or associated logP) was ≤1µM  (Gaulton et al. , 2012). Additionally, 9,219 experimental decoys displaying no measurable affinity up to 30 µM were included in the decoy sets.
Decoys Selection The decoy compounds were extracted from the ZINC database  (Irwin and Shoichet, 2005)  and selected by narrowing or widening windows around 6 physicochemical properties: molecular weight, octanol-water partition coefficient, rotatable bonds, hydrogen bonds acceptors, hydrogen bonds donors, and the net charge. To avoid active compounds in the decoy sets, a topological dissimilarity filter was applied. Molecules were sorted according to their Tanimoto distance to any ligands using CACTVS fingerprints, and the 25% most dissimilar decoy molecules were retained. Finally, up to 50 decoys were randomly selected for each ligand and pooled with the 9,219 experimental decoys. An automated tool was made available online to generate decoys from user-supplied ligands using the same protocol (http://decoys.
docking. org). The possibility to generate decoy sets for any target has been revealed successful and is now widely used by the scientific community  (Lacroix et al. , 2016; Nunes et al. , 2016; Allen et al. , 2017; Meirson et al. , 2017). Despite the success of the DUD-E, some weaknesses should be corrected in the DUD-E benchmarking database. The 102 targets are defined as a UniProt gene prefix (such as DRD3) and not a full gene_species (such as DRD3_HUMAN or P35462), which can bias the actives selection when the binding site composition differs between species. Additionally, only one single structure was considered for each protein while many docking studies pointed out that the structure selection is crucial for screening and docking, particularly for proteins that accommodate ligands with different binding modes  (May and Zacharias, 2005; Ben Nasr et al.
Ligands Preparation The NRLiSt BDB is composed of 9,905 active molecules targeting 27 nuclear receptors (NRs). Active compounds are divided into 2 datasets per target according to their agonist or antagonist profile. All active compounds were extracted from the ChEMBL database and included in the NRLiSt after a manual inspection of the corresponding ligands bioactivity data in the original papers. All inverse-agonists, modulators, agonists/antagonists, weak to partial agonists, weak to partial antagonists and ligands with unknown pharmacological profile were discarded. In addition 339 human holo structures extracted from the PDB are provided, among which 266 are agonists-bound, 17 are antagonists-bound and 56 are others-bound. Valid active compounds extracted from literature were clustered using chemical fingerprints, and a Tanimoto cut-off of 0.
Decoys Selection In total 458,981 decoys generated with the DUD-E online tool were provided, with a mean ratio of 1/51 for each dataset. In further studies, Lagarde et al. integrated the antipharmacological profile ligands in the decoy set to orient the screening toward the desired pharmacological profile  (Lagarde et al. , 2014b). For instance, antagonists were considered as the decoy compounds set for agonists screening research, while agonists were considered as the decoy compounds set for antagonists screening research. In agreement, the corresponding agonist-and antagonist-bound structures were used for SBVS, when available. Results showed that the enrichment is better when the pharmacological profile is considered prior to screening and should therefore be systematically considered to avoid artificially bad ligands enrichment.
Maximal Unbiased Benchmarking Data Sets for HDACs (MUBD-HDACs) So far, most of the decoy datasets [such as DUD-E  (Mysinger et al. , 2012)  and DEKOIS  (Vogel et al. , 2011; Bauer et al. , 2013) ] or decoys generator [such as DecoyFinder  (Cereto-Massagué et al. , 2012)  or the DUD-E generator server] are designed for SBVS purpose. Few databases [i. e. , MUV  (Rohrer and Baumann, 2009) , NRLiSt BDB] are intended to propose benchmarking datasets for LBVS. Xia et al. thus proposed a workflow to fulfill this need, and built up decoy datasets for LBVS targeting the histone deacetylases protein family (HDACs).
Ligands Preparation Active compounds were retrieved from the ChEMBL18 database  (Gaulton et al. , 2012) , among molecules annotated with quantitative data (i. e. , IC50), manually checked, and filtered (exclusion of salts, molecules with more than 20 rotatable bonds or with a MW of 600 or more). Finally, ligands displaying a Tanimoto coefficient greater than 0. 75 based on MACCS fingerprints were removed to exclude analog molecules, and 6 physicochemical properties (MW, logP, HBAs, HBDs, RBs and net Formal Charge-nFC) were computed for all HDACs inhibitors (HDACIs).
Decoys Selection The "All-Purchasable Molecules" subset of the ZINC database was used as the initial set of molecules before a two-step filtering: (1) Compounds outside of the bounds of the HDACIs physicochemical properties were removed, as well as molecules with a Tanimoto coefficient ("similarity in structure" or sims) greater than 0. 75 to any active compounds to circumvent the introduction of potential active structures (false negatives) into the decoy set. (2) To retain only 39 decoys per HDACI, compounds were further filtered to ensure similar physicochemical properties and a random spatial distribution of the decoys around the ligands. A specific metric was assigned to each step, specifically the simp ("similarity in properties") and the simsdiff ("sims difference"). The simp is the Euclidian distance of the physicochemical properties between a target compound and a reference compound.
The simsdiff between a potential decoy and a query ligand is the average difference between (a) the topological similarity sims between the potential decoy and the remaining ligands and (b) the topological similarity sims between the query ligands and the remaining ligands. First, a cut-off is applied on the simp to ensure properties similarity between ligands and decoy compounds and second, the 39 lowest simsdiff decoys for each ligand are selected. Last, for each ligand, the PDB  (Berman et al. , 2000)  structures of the targeted HDAC isoform were prepared and provided for SBVS data sets. Unlike DUD-E  (Mysinger et al. , 2012) , only Homo sapiens 3D-data were considered. The MUBD-HDAC datasets for HDAC2 and HDAC8 isoforms were compared to DUD-E  (Mysinger et al. , 2012)  and DEKOIS 2. 0  (Ibrahim et al.
, 2015a)  datasets, in terms of structural diversity [Bemis-Murcko atomic frameworks  (Bemis and Murcko, 1996) ], property matching and ligand enrichment in SB-and LB-VS approaches. The MUBD-HDAC displayed similar to better results in terms of structural diversity and property matching and was more challenging as measured by ligand enrichment using GOLD  (Jones et al. , 1997)  or fingerprints similarity search, in agreement with a higher structural similarity. Finally, the MUBD-HDACs sets displayed small to great improvement in terms of nearer ligands bias (i. e. , ligands that are more similar structurally to a ligand than to any decoy), compared to DUD-E and DEKOIS 2. 0, respectively. This bias is known to produce artificially positive LBVS evaluation outcomes  (Cleves and Jain, 2008)  and thus, should be minimized.
Ideal Benchmarking Database The ideal VS benchmarking datasets composition should mimic real-life cases, where a small number of diverse active ligands is embedded into a much larger fraction of inactive compounds. Moreover, both sets of molecules are usually indistinguishable using simple descriptors like their physicochemical properties and share common fragments or functional chemical groups; such features should therefore be transposed to benchmarking datasets design, so that the putative inactive compounds constitute good "decoy" compounds in line with the active compounds and ensure a robust evaluation of the VS methods  (Good and Oprea, 2008; Lagarde et al. , 2015; Xia et al. , 2015).
Comparison of Decoys Selection Methods for SBVS Among the recent tools to help create benchmarking sets (MUV, DEKOIS, DUD-E, and MUBD), the main difference resides in the strategy used to achieve their respective objectives: the DUD-E and DEKOIS data sets are designed for evaluating SBVS methods while MUV and MUBD are conceived for benchmarking LBVS approaches. Following this basic distinction, the respective algorithms to generate decoy datasets differ significantly. In the former case, the topological dissimilarity between ligand compounds and decoy compounds is maximized to avoid inclusion of active compounds into decoy datasets. In the latter case, the proper embedding of decoy compounds into the ligands chemical space is of primary importance. For the DUD-E, the final decoys were randomly selected from the 25% most topologically dissimilar molecules compared to the ligands to ensure unbiased selection of decoy compounds.
However, several studies pointed out that bias are still present into DUD-E data sets. For instance, Chaput et al. recently evidenced that the performance of four VS programs (Glide, Gold, FlexX and Surflex) is biased (over-estimated) using the DUD-E. Good performance (as measure by BEDROC curves) could be achieved for all programs when original DUD-E datasets were used, while only Glide was considered successful when chemical library biases (i. e. , datasets whose decoys and active compounds differ for nine physicochemical properties) were removed. While the DUD-E was successfully used for numerous studies, this observation clearly showed that there is still place for improvements. Boeckler's group proposed a similar workflow in DEKOIS and DEKOIS 2. 0. A physicochemical similarity over eight properties (and represented by the physicochemical similarity score PSS) is used and the topological dissimilarity between the active compounds and the future decoy compounds is computed as in the DUD-E.
However, two main differences have to be noted: (1) the topological dissimilarity was computed using the more elaborated weighted LADS score rather than a 2D fingerprint based Tanimoto coefficient filter and (2) the LADS score was combined with the PSS prior to final selection of the decoys. Therefore, the final decoys selection was balanced by both parameters (physicochemical similarity and topological dissimilarity) rather than using successive arbitrary (even if widely used) thresholds, and was successfully used by  Hamza et al. (2014)  for drug repurposing. This balance may come at a cost, as evidenced by Xia et al. : DEKOIS datasets for HDAC2 and HDAC8 were shown to be less efficient in terms of property matching between the active compounds and the decoy compounds  (Xia et al. , 2015).
Comparison of Decoys Selection Methods for LBVS Both DUD-E and DEKOIS databases share the same overall decoy selection procedure by combining topological dissimilarity and physicochemical properties similarity. While adapted to SBVS, this approach may hinder the objective evaluation of LBVS that is very sensitive to topological difference between active and decoy compounds. The MUV datasets  (Rohrer and Baumann, 2009)  was designed to overcome this specific weakness of the benchmarking datasets. The authors introduced the notion that decoy compounds and active compounds should be homogeneously spread in the chemical space rather than decoy compounds should be topologically dissimilar to the active compounds (as in the DUD-E for instance). The authors tested 18 datasets and claimed that MUV benchmarking datasets displayed neither analogous bias nor artificial enrichment.
Furthermore, they noticed that their data sets were SBVS compliant and compared advantageously to the biased DUD sets, leading to a potential broader use of their sets. MUV sets were applied to the evaluation of VS tools  (Tiikkainen et al. , 2009; Abdo et al. , 2010) , the training of new QSAR models  (Marchese Robinson et al. , 2017)  or molecular graph convolutions  (Kearnes et al. , 2016). As highlighted by Xia et al. "MUV is restricted by the sufficient experimental decoys (chemical space of decoys)"  (Xia et al. , 2015). Indeed, MUV relies on the availability of experimental data and is restricted to well-studied targets. The authors subsequently proposed the Maximum Unbiased Benchmarking Data sets (MUBD, see section Benchmarking Databases) that was applied to GPCRs  (Xia et al. , 2014) , HDACs  (Xia et al.
, 2015; Hu et al. , 2017)  and Toll-like receptor 8  (Pei et al. , 2015). The MUBD-DecoyMaker algorithm relies on both a minimal and required topological dissimilarity (sims) between decoy and active compounds, but makes use of an additional criterion that minimizes the simsdiff parameter, i. e. , ensures that decoy and active compounds are as similar as possible. One should note that this additional step (the decoyactives similarity check) yield datasets also suitable for SBVS; they seemed even more challenging in SBVS (for HDAC2 and HDAC8) as they provided datasets with higher structural similarity  (Xia et al. , 2015). Thus, these approaches are particularly appealing as they provide benchmarking datasets that (1) are adapted to LB and SB-VS approaches, (2) subsequently allow comparative evaluations of the performance of LB and SB-VS approaches, and (3) may be more challenging for SBVS.
Fine-Tuned Benchmarking Datasets The quality of an evaluation lies in the consistency between the retrospectively screened benchmarking datasets and the prospectively screened compound collections as well as the target binding site properties  (Ben Nasr et al. , 2013). The recent trend to publish protein family-specific datasets or user-provided active compounds dependent decoys generation tools paves the way for a valuable and systematic use of benchmarking datasets prior to prospective VS of large compound collections. In SBVS, tuned datasets should be used to identify the protocol, conformational sampling, and/or scoring methods that induces the best enrichment in active compounds  (Allen et al. , 2015 (Allen et al. , , 2017;; Lacroix et al. , 2016; Li et al. , 2016; Nunes et al. , 2016; Meirson et al.
, 2017). For instance,  Allen et al. (2015 Allen et al. ( , 2017) )  evaluated different scoring schemes using DUD-E generated decoys and successfully identified dual EFGR/BRD4 inhibitors. In LBVS, the choice of the dataset is crucial to build a reliable model that can be used to distinguish active compounds from decoy compounds. For example,  Ruggeri et al. (2015)  used DUD-E generated decoys to define and optimize pharmacophore models that led to the identification of 2 dual competitive inhibitors of P. Falciparum M1 (Pf A-M1) and M17 (Pf A-M17) aminopeptidases. Of note, when using automatic decoy datasets generation tools, the provided active compounds should be carefully selected to avoid the previously detailed biases.
Integration of True Inactive Compounds Despite the open-data initiatives that should ease the access to data in the near future, the low documentation about negative data (inactive and/or non-binding) is still an open issue. The inclusion of experimental data in a dataset requires great attention since (1) publicly available databases may present annotation errors that should be manually corrected  (Lagarde et al. , 2014a) , and (2) diversity in the type of value and experimental conditions make some data barely comparable. The selection and the use of negative compounds (inactive and/or non-binding) in the evaluation/development of methods is a delicate step that strongly influences the quality of the resulting model. In agreement with  Lagarde et al. (2014a)  and  Kaserer et al. (2015) , we recommend that: (1) Interaction data should be extracted from receptor binding or enzymatic activity assays on isolated or recombinant protein; cell-based assays should be avoided because of the many factors that can influence the outcome of the assay (non-specific binding.
One should note that the integration of inactive/non-binding compounds comes with new basics for datasets design. This case is particularly challenging since the inactive/non-binding compounds are usually extracted from the same chemical series as the active compounds. In this case, small fragments modification can induce important bioactivity loss or gain, thus, clustering active compounds to guarantee diversity and minimize analogous bias would have no meaning. Since the final objective of using such data is to harshly evaluate ability of VS methods to discriminate active from inactive compounds based on small signals, the proximity between active and inactive compounds within a chemotype should be conserved, as well as the similarity within the active compounds of a chemotype. However the over representation of a given chemotype could hinder the evaluation of VS method by masking the enrichment of low populated chemotypes.
CONCLUSION Benchmarking databases are widely used to evaluate virtual screening methods. They are particularly important to compare performance of virtual screening methods and therefore to select appropriate protocol prior to large compounds collections screening, and to estimate the reliability of the results of a screening. The characterization of the weaknesses of the first published databases helped designing improved benchmarking datasets with minimized bias. The rational selection of decoy compounds is particularly important to avoid artificial enrichment in the evaluation of the different methods. The diversification of public datasets gathering both active and decoy compounds for a given protein family, and the publication of online decoys generation tools contributed to the democratization of the use of benchmarking studies to help identifying protocols adapted for the query/target system under study.
Largest decoy data set so far (40 Decoy compounds selection Remarks DUD datasets Extracted from previous successful studies Designed for LBVS only Same as DUD, but does not consider synthetic feasibility Purely virtual decoys, availability is not considered Class decoys and ligands into "cells" based on 6 physicochemical properties and select the closest decoys based on (1) a weighted physicochemical similarity and (2) a LADS score based on functional fingerprints similarity elaborated from the active set Original treatment of the physicochemical similarity, and introduce the concept of Latent Active in Decoy set, i. e. , false false positives Physico-chemical properties fit and topological dissimilarity filter. Final selection based on MW First extensive database targeting a specific protein family partial agonists.
The selection is based on both the proteins) and first attempt to select topologically dissimilarity to ligands decoys topologically dissimilar decoys Same as DUD and the fit of physicochemical Graphical tool to generate decoy data sets with adaptable thresholds for properties physicochemical properties -DUD clusters more relevant for Physico-chemical properties fit along scaffold hopping with a topological dissimilarity filter. Largest database so far (1,420,433 decoys and 66,695 actives) -Design to decrease the analog bias Random selection of decoys is then applied on 13 of the 40 DUD targets, enrich DUD active data sets with Same as DEKOIS with 3 additional compounds from WOMBAT database physicochemical properties (nFC, Two functions measure the Ligands and decoys are from nPC, Ar), a PAINS filter and an improved, weighted LADS score active-active and decoy-active biologically actives and inactive distances using 2D chemical compounds, i.
e. , are true actives and Use the DUD-E decoy generation tool Ligands can be either agonists or descriptors. Actives with the inactives, respectively antagonists (other actives are maximum spread within the active set removed), depending on the purpose were chosen and decoys with similar of the study spatial distribution were selected Subset of the DUD database, with Initially designed for Select decoys based a weighted physicochemical similarity (6 more stringent criteria on MW (≤450) "scaffold-hopping" studies physicochemical properties are and AlogP (≤4,5), and a minimal considered), and ensure a random number of chemotypes spatial distribution of the decoys (i. e. , Applicable both to SBVS and LBVS bioactivity strategies, uses ligands with proved (Continued) decoys should be as distant to the other actives as a reference ligand) No.
of targets / No. of classes No. of targets / 40/6 No. of classes 2/2 - 40/6 9/4 40/6 6/1 34/7 147/1 40/6 - 40/6 102/8 13/4 81/11 18/7 27/1 13/4 14/1 Origin of the decoys Origin of the ZINC decoys ACD Literature ZINC MDDR ZINC MDDR ZINC "drug-like" and Rognan's decoy set ZINC ZINC "drug-like" User -ZINC - ZINC PubChem ZINC and DUD-E decoys generator DUD ZINC DB name BENCHMARKING DATABASES Year Download address Origin of the ligands DB name Charge Matched DUD Year Download address Origin of the 2010 http://dud. docking. org/ DUD ligands charge-matched/ Rognan's decoy set REPROVIS-DB 2000 http://bioinfo-pharma. u-strasbg. fr/ Literature 2011 -Literature (Bissantz et al. , 2000) labwebsite/download. html Virtual Decoy sets 2011 http://compbio. cs. toronto. edu/VDS DUD Shoichet's decoy set (VDS) (Wallach and 2003 -MDDR (McGovern and Lilien, 2011) Shoichet, 2003) DEKOIS (Vogel et al.