# Algorithms for Drug Sensitivity Prediction
## 4 Predictive Models
### Kernel Based Methods  
The most commonly-used kernel-based method for regression modeling consists of support vector regression. Support vector regression is based on the statistical learning theory or Vapnik-Chervonenkis (VC) theory developed by [PERSON] and [PERSON] during the 1960s [72; 73; 74]. VC theory attempts to provide the statistical underpinnings of learning algorithms and their generalization abilities. Support Vector (SV) classifiers and subsequently SV regression showed high accuracy in comparison to existing approaches and became a commonly-used tool in classification and regression modeling [75; 76; 77; 78].  
To explain SV regression, consider \(x(i,j)\) and \(y(i)\) (\(i=1,\ldots,n;j=1,\ldots,M\)) to denote the training predictor features and output response samples, respectively. In terms of SV regression, the goal is to find a function \(f(x(i,1:M))\) that has at most deviation \(\varepsilon\) from actual values \(y(i)\)[77], and a characteristic of the function is minimized. Thus, the deviation of the function approximator \(f\) from the actual values is being limited while minimizing some characteristic of the approximator. For explanation purposes, consider the following linear function:  
\[f(x(i,1:M))=\sum_{j=1}^{M}w(j)x(i,j)+b \tag{10}\]  
Here, the function characteristic will be the norm of \(w\). The problem can be posed as an optimization problem as follows:  
minimize \[\frac{1}{2}||w||^{2}\] subject to \[|y_{i}-\sum_{j=1}^{M}w(j)x(i,j)-b|\leq\epsilon\] (11)To extend the SV algorithm to the nonlinear scenario, kernels are usually employed. Some commonly-used kernels are [77]:  
* homogenous polynomial kernels \(k\) with \(p\in\mathbb{N}\) and: \[k(x,x^{\prime})=<x,x^{\prime}>^{p}\] (12)
* inhomogenous polynomial kernels \(k\) with \(p\in\mathbb{N}\), \(c>0\) and: \[k(x,x^{\prime})=(<x,x^{\prime}>+c)^{p}\] (13)
* radial basis kernels: \[k(x,x^{\prime})=e^{-\frac{\|x-x^{\prime}\|^{2}}{2\sigma^{2}}}\] (14)  
Support vector machines can handle non-linearity based on the use of kernels and tend to have better predictive performance compared to regularized regression approaches [79]. However, the direct biological interpretability of the generated model is limited. Support vector machines strike a balance on generalization error performance and can handle various noise models. For further details on SV regression models, readers are referred to a tutorial on SVMs by [PERSON] and [PERSON] [77].  
Support vector regression has been used for drug sensitivity prediction on the Cancer Cell Line Encyclopedia database in [39] using baseline gene expression as features. Other studies involving support vector regression for drug sensitivity prediction include [25; 52; 66]. The work in [52] considers a comparative study of multiple machine learning algorithms, including support vector regression for drug sensitivity prediction on CCLE and Sanger datasets. In their analysis, ridge regression and elastic net outperformed support sector regression. However, the likely reason for this behavior is the inbuilt embedded feature selection inherent in ridge regression and elastic net as compared to support vector regression.  
Other forms of kernel-based approaches include Bayesian kernel-based multitask learning [62], which was the best performer in the NCI-DREAM drug sensitivity prediction challenge [25].