# Algorithms for Drug Sensitivity Prediction
## 4 Predictive Models
### Prediction Performance  
This section considers a predictive performance comparison between different genetic characterization based models for the CCLE dataset. We have selected the methodologies of Elastic Net (EN), Kernelized Bayesian Multitask Learning (KBMTL), Random Forest (RF) and Deep Neural Network (DNN) as the representative methodologies for the four broad categories of regularized linear regression, kernel-based modeling, ensemble methods and deep learning, respectively.  
For EN tuning of parameters \(\alpha\) and \(\lambda\), we have followed the approach in the original CCLE paper that considered elastic net for predictive modeling of drug sensitivity [23]. The optimal choice of the two parameters is selected to minimize root mean squared error for 10-fold cross-validation of training data, where 10 random values of \(\alpha\in[0.2,1]\) and 250 random values of \(\lambda=\exp^{\gamma};\gamma\in[-6,5]\) are used for optimization.  
For RF generation, we have used 200 trees for the forest, 10 random features for each split decision and 5 minimum entries in leaf nodes similar to the parameters considered in [34]. We have implemented KBMTL using the algorithmic code provided in [62]. Based on the parameters used in [62], we have considered 200 iterations and gamma prior values (both \(\alpha\) and \(\beta\)) of one. Subspace dimensionality has been considered to be 20, and the standard deviation of hidden representations and weight parameters are selected to be the defaults 0.1 and one, respectively.  
For DNN, we have used a fully-connected structure that has one hidden layer:  
\[y(x)=W^{o}(\sigma(W^{h}x+b^{h})+b^{o} \tag{16}\]  
where \(y\) is the output layer, \(W^{o}\) and \(b^{o}\) are the weight and bias matrix for output layer and \(W^{h}\) and \(b^{h}\) are weight and bias matrix for hidden layer and \(\sigma\) is the activation function. In our model, we use a Rectified Linear Unit (ReLU) function \(\sigma(x)=ReLU(x)=max(0,x)\) as our activation function. The input layer's dimension is 200; the output layer's dimension is one; while the hidden layer has 100 neurons. During the training, a mini-batch based SGD is used [116]. The mini-batch size is five, and the learning rate is 0.0001. In order to prevent curvature issues [117], a 0.9 momentum is added to the SGD. The whole training stops at 1000 epochs.  
Table 1 shows the predictive performance in the form of correlation coefficients between actual and predicted drug sensitivity (AUC) for EN, KBMTL, RF and DNN for four drugs in the CCLE dataset [23]. We consider hold-out error estimation where 350 samples have been used as training, and the remaining samples (in the range of 140) are used for testing. Feature selection using ReliefF [33] has been used to reduce the number of gene expression features from 18,988 to 200. We observe that RF produces the best performance followed by KBMTL, EN and DNN. The worst performance of DNN is likely due to the limited number of samples used for training the model. DNN is preferable for scenarios where the number of samples is significantly higher than the number of features.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Drug Name**} & \multicolumn{4}{c}{**Correlation Coefficients**} \\ \cline{2-5}  & \(EN\) & \(KBMTL\) & \(RF\) & \(DNN\) \\ \hline AZD0530 & 0.2149 & 0.2245 & 0.2577 & 0.1986 \\ Erlotinib & 0.3905 & 0.4100 & 0.4438 & 0.1341 \\ Lapatinib & 0.4758 & 0.4641 & 0.5127 & 0.3667 \\ Crizotinib & 0.3348 & 0.2647 & 0.3622 & 0.2609 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Prediction performance in the form of correlation coefficients between actual and predicted drug sensitivity for Elastic Net (EN), Kernelized Bayesian Multitask Learning (KBMTL), Random Forest (RF) and Deep Neural Network (DNN) for four drugs in the Cancer Cell Line Encyclopedia (CCLE) dataset.