# Algorithms for Drug Sensitivity Prediction
## 4 Predictive Models
### Linear Regression Models
When the additional penalty term consists of the square of the \(l_{2}\) norm of \(\beta\), it is known as ridge regression [54]:  
\[\hat{\beta}^{ridge} =\operatorname*{arg\,min}_{\beta}(\mathbf{Y}-\mathbf{X}\beta)^{ T}(\mathbf{Y}-\mathbf{X}\beta)+\lambda\sum_{i=1}^{M}\beta(i)^{2} \tag{3}\] \[=\operatorname*{arg\,min}_{\beta}||\mathbf{Y}-\mathbf{X}\beta|| _{2}^{2}+\lambda||\beta||_{2}^{2} \tag{4}\]  
\(\lambda\geq 0\) is a tuning parameter that provides a control on fitting a linear model and shrinking the parameter coefficients. When the penalty term consists of the \(l_{1}\) norm of the parameter vector, it is known as Least Absolute Selection and Shrinkage Operator (LASSO):\[\beta^{lasso} =\operatorname*{arg\,min}_{\beta}(\mathbf{Y}-\mathbf{X}\beta)^{T}( \mathbf{Y}-\mathbf{X}\beta)+\lambda\sum_{i=1}^{M}|\beta(i)| \tag{5}\] \[=\operatorname*{arg\,min}_{\beta}||\mathbf{Y}-\mathbf{X}\beta||_{ 2}^{2}+\lambda||\beta||_{1} \tag{6}\]  
LASSO [44] has shown to provide good performance and feature selection properties, but also suffers from the following limitations: (a) when \(n\) is smaller than \(M\), LASSO selects at most \(n\) features before it saturates; and (b) LASSO usually selects one variable from a group of highly correlated variables, ignoring the others. To avoid the limitations of LASSO, elastic net combines the penalties of ridge regression and LASSO to arrive at:  
\[\beta^{EN}=\operatorname*{arg\,min}_{\beta}||\mathbf{Y}-\mathbf{X}\beta||_{ 2}^{2}+\lambda_{1}||\beta||_{1}+\lambda_{2}||\beta||_{2}^{2} \tag{7}\]  
It has been shown that the elastic net regression can be formulated as a support vector machine problem, and resources applied to support vector machine solutions can be applied to elastic net, as well [55]. The initial publication on the Cancer Cell Line Encyclopedia [23] considered elastic net as the model for predicting drug sensitivity based on over 100,000 genomic features. For molecular features, the authors considered an integrated set of gene expression, gene copy number, gene mutation values, lineage, pathway activity scores derived from gene expression data and regions of recurrent copy number gain or loss derived from the Genomic Identification of Significant Targets in Cancer (GISTIC) [56]. Ridge regression has been used in [57] to predict chemotherapeutic response in patients using only before-treatment baseline tumor gene expression data. The authors compared the performance with other approaches like random forests [58], nearest shrunken centroids [59], principal component regression [60; 61], LASSO [42] and elastic net [44] regression and observed that ridge regression was the best performer. However, other studies comparing multiple algorithms on a drug sensitivity database [25] have observed that random forests or kernelized Bayesian multitask learning [62] perform better than ridge regression or other regularized linear regression approaches. As mentioned earlier, both the drug type that is being predicted or the different molecular features being used to design the predictive model can play a significant role. For instance, in the study [57], they used chemotherapeutic, whereas [25] used primarily targeted drugs. Furthermore, [57] used gene expression data alone, whereas [25] used multiple types of genomic data, including gene expression, RNA-Seq, RPPA, methylation, CNV and SNP. We next attempt to provide an overview of the other modeling algorithms that are commonly used for comparing the performance of drug sensitivity predictive models.