nils-herrmann/clean_scirepeval huggingface dataset:
Prediction of the effect of mobile-phase salt type on protein retention and selectivity in anion exchange systems.

This study examines the effect of different salt types on protein retention and selectivity in anion exchange systems. Particularly, linear retention data for various proteins were obtained on two structurally different anion exchange stationary-phase materials in the presence of three salts with different counterions. The data indicated that the effects are, for the most part, nonspecific, although various specific effects could also be observed. Quantitative structure retention relationship (QSRR) models based on support vector machine feature selection and regression models were developed using the experimental chromatographic data in conjunction with various molecular descriptors computed from protein crystal structure geometries. Star plots for each descriptor used in the final model were generated to aid in interpretation. The resulting QSRR models were predictive, with cross-validated r2 values of 0.9445, 0.9676, and 0.8897 for Source 15Q and 0.9561, 0.9876, and 0.9760 for Q Sepharose resins in the presence of three different salts. The predictive power of these models was validated using a set of test proteins that were not used in the generation of these models. Interpretation of the models revealed that particular trends for proteins and salts could be captured using QSRR techniques.
Support vector regression with applications in automated drug discovery

The purpose of this research is to develop effective machine learning or data mining techniques based on support vector machines (SVMs), and apply them to practical problems arising from computer-aided drug discovery. SVMs are a robust inference methodology originally derived for pattern recognition and later extended to other inference problems such as regression problems. In this study, we investigate novel approaches for support vector regression (SVR), variable selection and model evaluation. First, an intuitive geometric framework for SVR is developed based on duality analysis from the geometric and mathematical programming perspective. We show that SVR can be regarded as a classification problem in the dual space. The regression function is constructed by separating the convex or reduced convex hulls respectively of the training data with the response variable shifted up and down by an error tolerance e. A geometric SVR formulation is proposed based on choosing the maximal margin plane between the two shifted datasets. Second, we develop a variable selection method to address challenging drug design problems where problems usually have a relatively small number of molecules (observations) but a large number of descriptors (variables). The basic idea is to construct a series of linear models that exhibit good generalization by solving sparse linear SVMs. The subset of variables having nonzero weights in the linear models is used in nonlinear SVM to produce the final regression model. The method exploits the fact that a linear SVM (no kernels) with e 1-norm regularization inherently performs variable selection as a side-effect of minimizing capacity in the SVM method. Third, we devise a new technique for evaluation and comparison of regression models. The proposed approach, regression error characteristics (REC) curves, depicts the trade-off of the error tolerance versus the accuracy of the functions. REC curves generalize the ROC curve evaluation method for classification to regression. The REC curve estimates the error cumulative distribution function. Commonly-used error measures can be visually represented using the geometry of the REC plot.
Regression Error Characteristic Curves

Receiver Operating Characteristic (ROC) curves provide a powerful tool for visualizing and comparing classification results. Regression Error Characteristic (REC) curves generalize ROC curves to regression. REC curves plot the error tolerance on the x-axis versus the percentage of points predicted within the tolerance on the y-axis. The resulting curve estimates the cumulative distribution function of the error. The REC curve visually presents commonly-used statistics. The area-over-the-curve (AOC) is a biased estimate of the expected error. The R2 value can be estimated using the ratio of the AOC for a given model to the AOC for the null model. Users can quickly assess the relative merits of many regression functions by examining the relative position of their REC curves. The shape of the curve reveals additional information that can be used to guide modeling.
Sparsity and dimensionality reduction for kernel regression analysis

Kernel methods such as support vector machines (SVM) and kernel partial least squares (KPLS) have been proven to be very effective for nonlinear inference. A key to success in nonlinear modeling is capacity control. For example, SVM explicitly imposes a convex regularization term in the objective function. KPLS iteratively optimizes the objective function rather than solving the full problem at once. 
Dimensionality reduction and sparsity are examples of capacity control. By extracting features, dimensionality reduction methods construct a low-rank approximation of the original data matrix. The model capacity is controlled by a criterion of the feature extraction and reduced rank of the approximation. In kernel methods, a dense model must compute all the elements of the kernel matrix. Imposing sparsity reduces the number of support vectors, making the model less complex and computationally more efficient. 
The first goal of this thesis is to provide a framework for sparsifying KPLS solutions. The proposed approach, ν-KPLS, modifies KPLS by introducing the einsensitive loss, which yields sparsity in dual solutions. The ν-KPLS algorithm must solve a convex optimization problem many times. In order to compute the predictive model efficiently, two simple heuristics are proposed. 
Challenges in kernel methods involve the selection of a kernel type and kernel parameters, and the improvement of interpretability. This thesis also addresses those issues. We focus on a heterogeneous kernels framework that allows combinations of different types of kernels. Ideas in ensemble methods are utilized to select interpretable kernels and optimize the kernel parameters simultaneously. 
It can be shown that PLS solves a least squares problem. However, the squared loss is not necessarily desirable. Rather, it should depend on the data sets. A generic boosting framework, AnyBoost, is utilized to extend the PLS framework to arbitrary loss functions. The resulting generalization of PLS extrapolates the key advantages of PLS to general machine learning tasks. The framework also provides a way for a dimensionality reduction technique based on a generic loss function.
An Optimization Perspective on Kernel Partial Least Squares Regression

This work provides a novel derivation based on optimization for the partial least squares (PLS) algorithm for linear regression and the kernel partial least squares (K-PLS) algorithm for nonlinear regression. This derivation makes the PLS algorithm, popularly and successfully used for chemometrics applications, more accessible to machine learning researchers. The work introduces Direct K-PLS, a novel way to kernelize PLS based on direct factorization of the kernel matrix. Computational results and discussion illustrate the relative merits of K-PLS and Direct K-PLS versus closely related kernel methods such as support vector machines and kernel ridge regression. ∗This work was supported by NSF grant number IIS-9979860. Many thanks to Roman Rosipal, Nello Cristianini, and Johan Suykens for many helpful discussions on PLS and kernel methods, Sean Ekans from Concurrent Pharmaceutical for providing molecule descriptions for the Albumin data set, Curt Breneman and N. Sukumar for generating descriptors for the Albumin data, and Tony Van Gestel for an efficient Gaussian kernel implementation algorithm. This work appears in J.A.K. Suykens, G. Horvath, S. Basu, C. Micchelli, J. Vandewalle (Eds.) Advances  Models and Applications, NATO Science Series III: Computer & Systems Sciences, Volume 190, IOS Press Amsterdam, 2003,p. 227-250. 2 K.P. Bennett, M.J. Embrechts
Sparse Kernel Partial Least Squares Regression

Partial Least Squares Regression (PLS) and its kernel version (KPLS) have become competitive regression approaches. KPLS performs as well as or better than support vector regression (SVR) for moderately-sized problems with the advantages of simple implementation, less training cost, and easier tuning of parameters. Unlike SVR, KPLS requires manipulation of the full kernel matrix and the resulting regression function requires the full training data. In this paper we rigorously derive a sparse KPLS algorithm. The underlying KPLS algorithm is modified to maintain sparsity in all steps of the algorithm. The resulting ν-KPLS algorithm explicitly models centering and bias rather than using kernel centering. An e-insensitive loss function is used to produce sparse solutions in the dual space. The final regression function for the ν-KPLS algorithm only requires a relatively small set of support vectors.
Dimensionality Reduction via Sparse Support Vector Machines

We describe a methodology for performing variable ranking and selection using support vector machines (SVMs). The method constructs a series of sparse linear SVMs to generate linear models that can generalize well, and uses a subset of nonzero weighted variables found by the linear models to produce a final nonlinear model. The method exploits the fact that a linear SVM (no kernels) with l1-norm regularization inherently performs variable selection as a side-effect of minimizing capacity of the SVM model. The distribution of the linear model weights provides a mechanism for ranking and interpreting the effects of variables. Starplots are used to visualize the magnitude and variance of the weights for each variable. We illustrate the effectiveness of the methodology on synthetic data, benchmark problems, and challenging regression problems in drug design. This method can dramatically reduce the number of variables and outperforms SVMs trained using all attributes and using the attributes selected according to correlation coefficients. The visualization of the resulting models is useful for understanding the role of underlying variables.
MARK: a boosting algorithm for heterogeneous kernel models

Support Vector Machines and other kernel methods have proven to be very effective for nonlinear inference. Practical issues are how to select the type of kernel including any parameters and how to deal with the computational issues caused by the fact that the kernel matrix grows quadratically with the data. Inspired by ensemble and boosting methods like MART, we propose the Multiple Additive Regression Kernels (MARK) algorithm to address these issues. MARK considers a large (potentially infinite) library of kernel matrices formed by different kernel functions and parameters. Using gradient boosting/column generation, MARK constructs columns of the heterogeneous kernel matrix (the base hypotheses) on the fly and then adds them into the kernel ensemble. Regularization methods such as used in SVM, kernel ridge regression, and MART, are used to prevent overfitting. We investigate how MARK is applied to heterogeneous kernel ridge regression. The resulting algorithm is simple to implement and efficient. Kernel parameter selection is handled within MARK. Sampling and "weak" kernels are used to further enhance the computational efficiency of the resulting additive algorithm. The user can incorporate and potentially extract domain knowledge by restricting the kernel library to interpretable kernels. MARK compares very favorably with SVM and kernel ridge regression on several benchmark datasets.
Prediction of Protein Retention Times in Anion-Exchange Chromatography Systems Using Support Vector Regression

Quantitative Structure-Retention Relationship (QSRR) models are developed for the prediction of protein retention times in anion-exchange chromatography systems. Topological, subdivided surface area, and TAE (Transferable Atom Equivalent) electron-density-based descriptors are computed directly for a set of proteins using molecular connectivity patterns and crystal structure geometries. A novel algorithm based on Support Vector Machine (SVM) regression has been employed to obtain predictive QSRR models using a two-step computational strategy. In the first step, a sparse linear SVM was utilized as a feature selection procedure to remove irrelevant or redundant information. Subsequently, the selected features were used to produce an ensemble of nonlinear SVM regression models that were combined using bootstrap aggregation (bagging) techniques, where various combinations of training and validation data sets were selected from the pool of available data. A visualization scheme (star plots) was used to display the relative importance of each selected descriptor in the final set of "bagged" models. Once these predictive models have been validated, they can be used as an automated prediction tool for virtual high-throughput screening (VHTS).
Exploiting unlabeled data in ensemble methods

An adaptive semi-supervised ensemble method, ASSEMBLE, is proposed that constructs classification ensembles based on both labeled and unlabeled data. ASSEMBLE alternates between assigning "pseudo-classes" to the unlabeled data using the existing ensemble and constructing the next base classifier using both the labeled and pseudolabeled data. Mathematically, this intuitive algorithm corresponds to maximizing the classification margin in hypothesis space as measured on both the labeled and unlabeled of data. Unlike alternative approaches, ASSEMBLE does not require a semi-supervised learning method for the base classifier. ASSEMBLE can be used in conjunction with any cost-sensitive classification algorithm for both two-class and multi-class problems. ASSEMBLE using decision trees won the NIPS 2001 Unlabeled Data Competition. In addition, strong results on several benchmark datasets using both decision trees and neural networks support the proposed method.
A Generic Nearest Point Algorithm for Solving Support Vector Machines ( Tentative )

We propose a nearest point algorithm (NPA) for optimization of a family of Support Vector Machines (SVM) methods. Geometrically, optimizing SVM corresponds to finding the nearest points between two polyhedrons. In classification, the hard margin case corresponds to finding the closest points of the convex hull of each class. The soft margin case corresponds to finding the closest points in the reduced convex hulls. A prior nearest point algorithm (NPA) was very effective but limited to the convex hull case for classification. We propose a generic NPA applicable to both the regular and reduced convex hull cases. Our approach is applicable to both classification and regression SVM based on 1-norm and 2-norm error functions. The resulting algorithm is efficient, easy to implement and requires no optimization solvers. We review how SVM regression with 2-insensitive loss can be regarded as a nearest point problem for two reduced convex hulls. Experimental results for NPA for SVM classification problems indicate the method is extremely promising. We discuss extensions of the approach applicable to very large databases.
A Pattern Search Method for Model Selection of Support Vector Regression

We develop a fully-automated pattern search methodology for model selection of support vector machines (SVMs) for regression and classification. Pattern search (PS) is a derivative-free optimization method suitable for low-dimensional optimization problems for which it is difficult or impossible to calculate derivatives. This methodology was motivated by an application in drug design in which regression models are constructed based on a few high-dimensional exemplars. Automatic model selection in such underdetermined problems is essential to avoid overfitting and overestimates of generalization capability caused by selecting parameters based on testing results. We focus on SVM model selection for regression based on leave-one-out (LOO) and cross-validated estimates of mean squared error, but the search strategy is applicable to any model criterion. Because the resulting error surface produces an extremely noisy map of the model quality with many local minima, the resulting generalization capacity of any single local optimal model illustrates high variance. Thus several locally optimal SVMmodels are generated and then bagged or averaged to produce the final SVM. This strategy of pattern search combined with model averaging has proven to be very effective on benchmark tests and in high-variance drug design domains with high potential of overfitting. ∗This work is supported by the NSF Grants IIS-9979860 and IRI-97092306. †Dept. of Decision Sciences and Engineering Systems, Rensselaer Polytechnic Institute, Troy, NY 12180, mommam@rpi.edu ‡Dept. of Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY 12180, bennek@rpi.edu
A Genetic Algorithm Approach for Semi-Supervised Clustering

A novel semi-supervised clustering algorithm is proposed that synergizes the benefits of supervised and unsupervised learning methods. Data are clustered using an unsupervised learning technique biased toward producing clusters as pure as possible in terms of class distribution. These clusters can then be used to predict the class of future points. For example, in database marketing this technique can be used to identify and characterize segments of the customer population likely to respond to a specific promotion. One key additional benefit of this approach is that it allows unlabeled data with unknown class to be used to improve classification accuracy. The objective function of a traditional clustering technique, cluster dispersion in the K-means algorithm, is modified to minimize both the within-cluster variance of the input attributes and a measure of cluster impurity based on the class labels. Minimizing the within-cluster variance of the examples is a form of capacity control to prevent overfitting. For the output labels, impurity measures such as the Gini index can readily be applied to this problem. In this work, a genetic algorithm is proposed to optimize such an objective function to produce clusters. Non-empty clusters are labeled with the majority class. Experimental results show that using class information often improves the generalization ability compared to unsupervised methods based only on the input attributes. Benchmark studies also indicate that the method performs very well even when few training examples are available. Training using information from unlabeled data can improve classification accuracy on that data as well.
Strip mining for molecules

Quantitative structure-activity relationship (QSAR) problems deal with "in-silico" chemical design for the virtual invention of novel pharmaceuticals. The goal of QSAR is to predict the bioactivities of molecules based on a set of descriptive features. QSAR problems are notoriously challenging for machine learning because a typical QSAR predictive data mining problem set is characterized by a large number of descriptive features (300-1000), often for a relatively small number of molecules (50-300). This paper introduces data strip mining for QSAR modeling. Strip mining is a general approach for feature selection and predictive modeling based on successive stages of feature elimination done by performing a sensitivity analysis to a predictive model.
Stripmining For Molecules

QSAR (Quantitative Structure-Activity Relationship) problems deal with "in-silico" chemical design for the virtual invention of novel pharmaceuticals. The goal of QSAR is to predict the bioactivities of molecules based on a set of descriptive features. QSAR problems are notoriously challenging for machine learning because a typical QSAR predictive data mining problem set is characterized by a large number of descriptive features (300-1000), often for a relatively small number of molecules (50-300). This paper introduces data strip mining for QSAR modeling. Strip mining is a general approach for feature selection and predictive modeling based on successive stages of feature elimination done by performing a sensitivity analysis to a predictive model.
CINF 14 [ 499453 ] : Prospective Evaluation of Structure-based ADME Predictions : Knowing When the Experiment is the Prudent Course of Action

3D-pharmacophores have become popular in the last decade as descriptors for chemical similarity searching and combinatorial library design. These pharmacophores are typically encoded in large bitstrings, up to hundreds of millions of bits long. Research on other classes of chemical descriptors suggests that "chemical space" requires on the order of 10-20 dimensions. Many of the millions of dimensions are correlated or irrelevant and their large size makes them a less efficient representation for computation. We have developed an approach based on classical multidimensional scaling to determine the inherent dimensionality of these large chemical bitstrings (pharmacophore or otherwise), resulting in real-valued cartesian coordinates. We have also developed a novel approach to calculate much smaller bitstrings (hundreds of bits) that preserve the original pairwise similarity of the original large bitstrings. The new smaller space is general enough that it can be used for many common descriptor-based library design approaches: coverage, diversity, D-optimal, or informative design.  [522224]: A hierarchy of structure representation Johann Gasteiger, Thomas Kleinöder, Jens Sadowski, Markus Wagener, and Markus C. Hemmer, ComputerChemie-Centrum and Institute of Organic Chemistry, University of Erlangen-Nuremberg, Naegelsbachstr. 25, Erlangen 91052, Germany, Fax: +49-9131-85 26566, Gasteiger@chemie.uni-erlangen.de ACCEPTED Topic Selection: Chemical Descriptors: I Invited: Y Preferred Presentation  OralOnly Consider for Sci-Mix: N Conforms to Bylaw 6: Y Last Modified: 2001-12-13  Modern drug design generates massive amounts of data that have to be related to chemical structures. Therefore, methods are needed that encode the physicochemical effects of chemical structures responsible for Abstract Listing Page 1 of 47 CINFAbstract ListingModern drug design generates massive amounts of data that have to be related to chemical structures. Therefore, methods are needed that encode the physicochemical effects of chemical structures responsible for Abstract Listing Page 1 of 47 CINFAbstract Listing
Feature selection for in-silico drug design using genetic algorithms and neural networks

QSAR (quantitative structure activity relationship) is a discipline within computational chemistry that deals with predictive modeling, often for relatively small datasets where the number of features might exceed the number of data points, leading to extreme dimensionality problems. The paper addresses a novel feature selection procedure for QSAR based on genetic algorithms to reduce the curse of dimensionality problem. In this case the genetic algorithm minimizes a cost function derived from the correlation matrix between the features and the activity of interest that is being modeled. From a QSAR dataset with 160 features, the genetic algorithm selected a feature subset (40 features), which built a better predictive model than with full feature set. The results for feature reduction with genetic algorithm were also compared with neural network sensitivity analysis.