dataesr/scientific-paragraphs-categorization huggingface dataset:
highlighted that machine-learning SFs improve the rank-ordering of series of related molecules and that, as structural interatomic databases grow, machine-learning SFs are expected to further improve. 24 In a review dedicated to drug repurposing by structure-based virtual screening, Ma et al. explained that machine-learning SFs are attracting increasing attention for the estimation of protein-ligand binding affinity. 25 Last, Yuriev et al. noted that machine-learning approaches in docking remain an area of active and accelerating research. 26 Taken together, these reviews show that machine-learning SFs are indeed a recent and fastgrowing trend. However, no review has been devoted yet to this emerging research area, namely machine-learning SFs where machine learning is used to replace a predetermined functional form.
Benchmarks for binding affinity prediction provide a dataset suitable for constructing and validating SFs. With this purpose, one or more partitions of these data are proposed, each consisting of a training set and a test set. Examples of these benchmarks are those proposed by PDBbind  36, 37  and CSAR. 38 Here, the predictive performance of the regression-based SF is assessed using the Pearson correlation coefficient (R p ), Spearman rank-correlation (R s ), and Root Mean Square Error (RMSE) between predicted and measured binding affinity. More specifically, for a given model f, G bind w 0 + w 1 G vdW + w 2 G HBonds + w 3 G rotor + w 4 G hydrophobic x 0 x 1 x 2 x 3 x 4 x m x 1 M w m x m m=1 k=1 K j = p f RF (x m ) = x ij (d cutoff -d kl ) H kl (d kl ) l=1 L i k=1 j i K j l=1 L i p n ð Þ = f x ! n ð Þ is the predicted binding affinity given the features x ! n ð Þ characterizing the nth complex and the performance metrics are defined as: RMSE = ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 N X N n = 1 y n ð Þ -p n ð Þ 2 v u u u t R p = N X N n = 1 p n ð Þ y n ð Þ - X N n = 1 p n ð Þ X N n = 1 y n ð Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N X N n = 1 p n ð Þ 2 - X N n = 1 p n ð Þ 2 0 @ 1 A ðN X N n = 1 y n ð Þ 2 -ð X N n = 1 y n ð Þ 2 0 @ 1 A v u u u t Rs = N X N n = 1 pr n ð Þ yr n ð Þ - X N n = 1 pr n ð Þ X N n = 1 yr n ð Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N X N n = 1 pr n ð Þ 2 - X N n = 1 pr n ð Þ 2 0 @ 1 A ðN X N n = 1 yr n ð Þ 2 -ð X N n = 1 yr n ð Þ 2 0 @ 1 A v u u u t where N is the number of complexes in the set, a and b are the intercept and coefficient of the linear correlation between p n ð Þ È É N n = 1 and y n ð Þ È É N n = 1 on the test set, whereas p r n ð Þ È É N n = 1 and y r n ð Þ È É N n = 1 are the rankings of p n ð Þ È É N n = 1 and y n ð Þ È É N n = 1 , respectively.
On the other hand, VS aims at retrieving a subset of molecules containing the highest possible proportion of the actives contained in the screened library. SFs approach this task by ranking molecules in decreasing order of activity against the target, whether this is done using predicted binding affinity (regression) or the probability of belonging to the class of actives (classifier), and retaining the top-ranked molecules by applying a tight cutoff to the resulting ranking. This is an early recognition problem, as only a few top-ranking molecules can be tested in prospective applications and thus performance metrics should measure the proportion of actives that are found at the very top of the ranked list (e. g. , BEDROC  39  ). The enrichment factor (EF) is the most popular early recognition metric because of its simplicity and it is given by the ratio between the proportion of actives at the top x% of ranked molecules and the proportion of actives in the entire dataset (EF x% ).
GENERIC MACHINE-LEARNING SFs TO PREDICT BINDING AFFINITY This section focuses on the development of machinelearning SFs to predict the binding affinity of any protein-ligand complex. A prototypical application of generic SFs would be to identify those chemical derivatives of a known ligand that enhance their potency and selectivity against the considered target (e. g. , drug lead optimization  7, 46  ). The more accurate the ranking provided by the SF, the better guided such optimization task will be. Consequently, the SF performance at this task is measured by correlation and error metrics between predicted and binding affinities on a test dataset, as explained in the previous section. The earliest machine-learning SF we are aware of was introduced by Deng et al. in 2004  47  using Kernel partial least squares (K-PLS)  48  as the regression model.
The radial basis function (RBF) kernel is used here to carry out a transformation from the feature space to an internal representation upon which the PLS is applied. The occurrences of each protein-ligand atom pair within a distance of 1-6 Å were used as features. Two small datasets were considered, with 61 and 105 diverse protein-ligand complexes each. Results showed that some of the models were able to accurately predict the binding affinity of the selected test set complexes. This study constituted a proof-of-concept that nonlinear machine learning could be used to substitute the additive functional form of a SF. Two years later, Zhang et al. 49  adopted k-nearest neighbors as the regression technique and utilized a total of 517 protein-ligand complexes from the literature. Instead of geometrical descriptors, the electronegativities of ligand and protein atom types were used as features by mapping each four neighboring atoms to one quadruplet where the corresponding atomic electronegativities were summed up.
The best performance on the PDBbind benchmark was achieved by RF::XR (R p = 0. 806, R s = 0. 796), which is incidentally similar to RF::VinaElem 20 both in their sets of features and performances (R p = 0. 803, R s = 0. 798). Until now, we have reviewed how the combined choice of regression model and features affects the performance of the resulting SF. Recently, the future performance of these combinations was analyzed using a series of blind tests with increasingly large training sets. 20 In the case of a classical SF, its linear regression model could not assimilate data effectively beyond a few hundred training complexes and thus its test set performance remained flat regardless of training set size. In contrast, performance of RF models on the same test set continued to increase even after having exploited 3000 training complexes (see Figure  4 ).
70 Therefore, it is still not clear whether applying a family-specific SF is better than using a generic SF with a training set that includes all available complexes for that target. A few more family-specific machine-learning SFs compiling their own datasets have been presented to date. The first of these SFs, support vector inductive logic programming (SVILP), was presented by Amini et al. ,  72  where the regression model was SVR and the features were inductive logic programming (ILP) rules. The latter are logic rules inferred from distances between predetermined fragments of the ligand and protein atoms. Over the five considered targets, it was observed that the accuracies of the classical SFs were much lower than SVILP. This study also demonstrated for the first time that docking poses can be used to train a machine-learning SF in the absence of crystal structures to produce satisfactory results, at least in the studied target classes (HIV protease, carbonic anhydrase II, thrombin, trypsin, and factor Xa).
The training set was formed by ligand-bound crystal structures, which ranged in number between 9 and 197 depending on the target class, as actives. The test set was formed by 100 actives per target from the StARlite database (currently known as ChEMBL database  79  ). For each set, 2000 decoy molecules per target were selected at random from the Pub-Chem database  80  and assumed inactives. For those actives that were not cocrystallized with the target and all inactives, docking poses were generated using Glide. The machine-learning classifiers based on Pharm-IF features showed a better performance for targets with large number of complexes for training (PKA, carbonic anhydrase II, and HIV protease) and worse screening for those with few (SRC and cathepsin K). The screening efficiencies for SRC and cathepsin K were improved by adding the docking poses of their actives to the training set.
16). Kinnings et al. 83  argued that data imbalance, in this context having many more actives than inactives, reduces the performance of machine-learning classifiers. Thus, they proposed a multiplanar SVM classifier, which is made of n different SVM classifiers trained on a common set of actives and one of n disjoint subsets of inactives. In this way, the data imbalance problem was alleviated by having a more similar number of actives and inactives in each individual SVM, while the diversity of inactives was incorporated by making the prediction through the consensus of all balanced SVMs. The resulting SVM classifier and the classical SF eHiTS-Energy were tested on the DUD dataset for the target of interest (INHA) and the 12 DUD datasets with the largest number of chemotypes, with former outperforming the latter in all 13 targets.
SVR-KBD was in this way strongly improved from SVR-KB (AUC = 0. 71 vs 0. 52), now also outperforming all classical SFs, but not the SVM-SP classifier. Prospective VS was carried out with the first version of RF-Score in order to search for new inhibitors of antibacterial targets (Mycobacterium tuberculosis and Streptomyces coelicolor DHQase2). 85 A hierarchical VS strategy was followed combining ligand-based  86  and structure-based techniques. 18, 87 While the VS protocol involving RF-Score performed poorly retrospectively, it obtained the best performance prospectively. Overall, this study identified 100 new DHQase inhibitors against both M. tuberculosis and S. coelicolor, which contained 48 new core scaffolds. Recently, RF-Score has been incorporated into a user-friendly large-scale docking webserver  88  to carry out prospective VS of up to 23 million purchasable molecules from the ZINC database.
89 ing et al. 90  applied molecular interaction energy components (MIECs) features in combination with SVM to VS. MIECs features include van der Waals, electrostatic, hydrogen bond, desolvation energy, and the nearest distance between the target protein atoms and the ligand atoms. Two datasets with positive/active to negative/inactive ratio of 1:100 and 1:200 were compiled and the performance of MIEC-SVM was assessed by 500 cross-validation runs. With a very small fractions of binder in the library and <1% expectation of true positives, the average performance was very high (average MCC 0. 76). Furthermore, MIEC-SVM outperformed classical SFs (AUC 0. 56 for Glide, 0. 88 for X-score and 0. 99 for MIEC-SVM in the first data sets; 0. 57 for Glide, 0. 89 for X-score and 0. 99 for MIEC-SVM in the second dataset) in both above mentioned scenarios and in predicting true positives out of top 20 ligands in the 500 cross validations.
BOX 1 SOFTWARE IMPLEMENTING MACHINE-LEARNING SFs To permit the application of these SFs and facilitate the development of further machine-learning SFs, the following software has been released to date: PESD-SVM 51 : http://breneman. chem. rpi. edu/ PESDSVM/ RF-Score-v1  18  : http://bioinformatics. oxfordjournals. org/content/suppl/2010/03/18/btq112. DC1/bioinf-2010-0060-File007. zip NNScore series  53, 65, 82  : http://nbcr. ucsd. edu/ data/sw/hosted/nnscore/ RF-Score-v2 64 : https://bitbucket. org/aschreyer/ rfscore RF-Score-v3 (stand-alone  91  ): http://crcm. marseille. inserm. fr/fileadmin/rf-score-3. tgz RF-Score-v3 (embedded in a webserver for prospective VS  88  ): http://istar. cse. cuhk. edu. hk/idock Open Drug Discovery Toolkit 92 : https://github. com/oddt/oddt Machine-learning SFs have been used to improve our understanding of particular molecular recognition processes.
Machine-learning SFs have also been found to obtain better average performance than classical SFs at VS. The largest differences have been achieved by three classifiers: SVM-SP,  58  NNScore,  82  and MIEC-SVM  90  (NNScore has the additional benefit of being released as open-source software). In various studies,  58, 67, 82  it has been observed that regressionbased SFs performed substantially worse than classifier-based SFs at VS. The latter has been attributed to the low content of inactive data instances that the training sets of regression-based SFs have, which is much higher in the training set of classifiers and screening libraries. Meroueh and coworkers demonstrated  58  that the performance of regression-based SFs can be boosted by increasing the content of training set inactives in each DUD target.
It is also important that the developers of these SFs make the resulting software available to others. This will increase awareness of their accuracy and potential and permit their application to many more prospective studies. Currently, there are two freely available machine-learning SFs that are designed to be used for lead optimization (RF-Score-v3  20  ) and VS (NNScore 82 ), respectively. Very recently, the Open Drug Discovery Toolkit (ODDT)  92  has been released, which aims at facilitating the development and application of machine-learning SFs as a part of computeraided drug discovery pipelines. Even without any further modeling advance, future training data will increase the accuracy of machine-learning SFs for lead optimization, VS, and emerging applications on a wider range of targets and ligand chemistries.
ACKNOWLEDGMENTS This work has been carried out with the support of the A* MIDEX grant  (n  ANR-11-IDEX-0001-02 ) funded by the  French Government «Investissements d'Avenir» program  and the  UK Medical Research Council   Methodology Research Fellowship  (grant no. G0902106 ), both awarded to P. J. B. We would like to thank  Dr Christophe Dessimoz  ( Department of Computer Science, UCL, London, UK ) and  Dr James Smith  ( MRC Human Nutrition Research, Cambridge, UK ) for preliminary discussions. P. J. B. designed and wrote the review using the drafts provided by Q. U. A, A. A, and F. R, who contributed equally. All authors commented on the article.
We generate also network and cluster configuration details. Table  2  synthetizes values of the expected data input of machines. The first column indicates the category of the machine. Despite the evolution in Hadoop, we adopt the principle of separation between slots; the last two columns contain the number of reduce and map cores (slots) per machine. The costs of resources consumption are expressed in columns seven, eight and nine and they depend on the type of machine. We generate the completion time needed to treat tasks; these values depend on the size of the block. We define: ( [ ( ⁄ ] (16) We take into account the needed time to start up virtual machines TimeStartUpVM, the size of block and the amount of data computed per GHz per unit of time SpeedProcessorRate. We benefit from the last variable to inject the random aspect depending on the categories of machines: for the category "compute optimized", [ for the other types [.
This elimination scheme can then be used to simplify proofs that directly follow the case-analysis and recursion behavior of the function without repeating it. EQUATIONS supports definitions using arbitrarily complex well-founded recursion schemes, including the nested kind of recursion found in hereditary substitution functions, and the generation of unfolding lemmas and elimination schemes for those as well. Additionally, EQUA-TIONS plays well with the Program extension of COQ to manipulate subset types (also known as refinement types). The purpose of this paper is to present a case study of using EQUATIONS to show the normalization of Predicative System F, in an algorithmic way such that the normalization function can actually be run inside the proof assistant. Predicative System F was introduced by Leivant  [6]  to study the logical strength of different extensions of arithmetic.
g. ML, but combining Program and EQUATIONS, it can be shown to inhabit a richer type, providing a proof that the function indeed computes normal forms given inputs in normal form. The pre and postconditions of the hereditary subsitution function, which will be explicited later, are actually necessary to justify the termination of this function. From this it is easy to derive normalization and show that the system is consistent (relatively to Coq's theory, with the K axiom currently, although we hope to have an axiom-free version working by the time of the workshop). The paper is organized as follows: in §2 we present a gentle introduction to the EQUATIONS package and its features and quickly explain the main differences with the original presentation from  [10]. Then we summarize the standard definitions and metatheoretical results on Predicative System F that we proved and highlight the main differences with the presentation of  [5].
We provide the COQ development of our proof supplemented with some commentary to help follow along. First in §3 we present the language definition, with its typing and reduction rules. Then we show in §4 some metatheoretical properties on this language, such as substitution lemmas and regularity. Section 5 is dedicated to showing strong normalization of the calculus, which includes defining a well-founded ordering to justify the termination of the hereditary substitution function, and its definition itself using the EQUATIONS package. We also provide as an appendix the code which is produced from this definition by COQ's extraction mechanism. Finally, we compare with related work and conclude in §6.
Equations EQUATIONS allows one to write recursive functions by specifying a list of clauses with a pattern on the left and a term on the right, à la Agda  [9]  and Epigram  [8]. Here is an example recursive definition on lists, where wildcards corresponds to arbitrary fresh variables in patterns: Equations length {A} (l : list A) : nat := length [] ⇒ 0; length (cons t) ⇒ S (length t). The package starts by building a splitting tree for the definition and then compiles it to a pure COQ term. From the splitting tree, it also derives the equations as propositional equalities, which can be more robust to use than reduction when writing proofs about the constant, although in this particular case the compiled definition is the same as the one from the standard library. Here we have two leaves in the computation tree hence two equations: Check length equation 1 : ∀ A : Type, length [] = 0.
Check length equation 2 : ∀ (A : Type) (a : A) (l : list A), length (cons a l) = S (length l). These two equations are automatically added to a rewrite hint database named length and can be used during proofs using the simp length tactic. In addition, an elimination principle for length is derived. Note that length comp is just a definition of the return type of length in terms of its arguments, i. e. it is λ A l, nat here: Check length elim : ∀ P : ∀ (A : Type) (l : list A), length comp l → Prop, (∀ A : Type, P A [] 0) → (∀ (A : Type) (a : A) (l : list A), P A l (length l) → P A (cons a l) (S (length l))) → ∀ (A : Type) (l : list A), P A l (length l). This elimination principle can be used in proofs to eliminate calls to length and refine at the same time the arguments and results of the call in the goal.
Indeed, the term keep (sublist p xs) would not be well-typed, as it is expected to have type sublist comp p (x :: xs) which is incl (filter p (x :: xs)) (x :: xs), but has type incl (x :: filter p xs) (x :: xs). This is just a technical limitation we hope to remove in the future. The resulting elimination principle, while maybe not so useful in that case as this program constructs a proof, shows the explicit rewriting needed in the definition of the subpredicate P0. This concludes our exposition of EQUATIONS and we now turn to the formalization of Predicative System F.
Previous research has performed comparisons of various real-life traces and identified substantial differences in connection duration, isolation duration, and number of neighbours. San Francisco Cab Trace  [2]  has been shown to be the most challenging trace with very short connectivity durations  [1] , very long isolation time  [1]  and low number of neighbours  [1]  compared to other social traces (such as Sassy  [7]  and Infocom  [14] ). For example, SF Cab  [2]  traces exhibit predominantly short contact durations (a mean of 31sec) while Infocom 2006 displays substantially longer contact durations (a mean of 3min). In this work we choose to focus on reporting on SF traces as most challenging dataset currently available.
2006b). This non-parametric method is based on the classification and regression tree (CART) approach introduced by L. Breiman  (Breiman, Friedman, Olshen, and Stone 1993). Beyond the basic function, we have developed a set of functions for an extended analysis of spatial data, providing: inference, graphical representations, spatio-temporal analysis, adjustments on quantitative covariates, spatial weighted partition, and the gathering of similar adjacent final classes. In this paper, we propose a new package SPODT for R (R Core Team 2014) which provides an extensible set of functions for partitioning spatial and spatio-temporal data. The implementation and extensions of the algorithm are described and function usage are proposed based on a field observation datafile (malaria episodes in Mali)  (Coulibaly et al.
2013)  and samples showing three different cluster shapes. The results are compared to the CART approach using the tree package  (Ripley 2014). All results were obtained using R 3. 1. 0 (Windows 7, Intel Core i7, CPU Q820 @1. 73GHz, 64-bit). The SPODT package is freely available from the Comprehensive R Archive Network at http://CRAN. R-project. org/package=SPODT.
Program developments We have developed different R functions for a complete analysis of spatial data, according to our method. On the basis of the basic algorithm, several extensions have been developed: Spatio-temporal analysis: Integration of splits of a time covariate. The statistical unit is then defined by planar coordinates and a date. On an unique location different values can be observed at different dates. As CART algorithm, SpODT algorithm can thus provide a spatial splitting or a temporal splitting. Adjustments: Following the same procedure, the SpODT algorithm can provide a classification of different quantitative covariates. For these covariates, the standard CART algorithm is applied (i. e. , no oblique split is performed). Gathering similar adjacent final classes: This option makes possible to gather similar adjacent classes at the end of the recursive splitting algorithm.
spodtSpatialLines(object, data) This function provides the SpatialLines object (see the package sp, Bivand, Pebesma, and Gómez-Rubio 2013) that contains the boundaries of the spatial classification issued from the spodt function. object is an object of class spodt, usually a result of a call to spodt. data is the initial SpatialPointsDataFrame containing the planar coordinates and the variables. The SpatialLines object obtained can be used, for example to obtain maps.