# Machine Learning Predicts Outcomes of Phase III Clinical Trials for Prostate Cancer
## 4 Discussion  
We developed a series of ML models to predict two-year outcomes of a standard treatment for prostate cancer, based on baseline data from three clinical trials. This classification problem was challenging because it involved predicting responses a considerable \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Sensitivity** & **F1** & **AUC** \\ \hline \multirow{4}{*}{\begin{tabular}{c} **Study 1** \\ (chance level 71\%) \\ \end{tabular} } & **XGBoost** & 77\% & 86\% & 92\% & 87\% & 78\% \\  & **Voting classifier** & 75\% & 84\% & 97\% & 95\% & 77\% \\  & **Catboost** & 73\% & 82\% & 99\% & 98\% & 77\% \\  & **KNN** & 71\% & 83\% & 83\% & 71\% & 71\% \\  & **Logistic Regression** & 68\% & 81\% & 83\% & 72\% & 71\% \\ \hline \multirow{4}{*}{\
\begin{tabular}{c} **Study 2** \\ (chance level 70\%) \\ \end{tabular} } & **XGBoost** & 73\% & 43\% & 86\% & 92\% & 60\% \\  & **Voting classifier** & 71\% & 42\% & 99\% & 100\% & 52\% \\  & **Catboost** & 70\% & 43\% & 100\% & 100\% & 50\% \\  & **KNN** & 71\% & 42\% & 52\% & 76\% & 52\% \\  & **Logistic Regression** & 65\% & 37\% & 56\% & 76\% & 40\% \\ \hline \multirow{4}{*}{
\begin{tabular}{c} **Study 3** \\ (chance level 84\%) \\ \end{tabular} } & **XGBoost** & 84\% & 2\% & 76\% & 94\% & 10\% \\  & **Voting classifier** & 83\% & 7\% & 100\% & 100\% & 18\% \\  & **Catboost** & 82\% & 2\% & 15\% & 84\% & 10\% \\  & **KNN** & 81\% & 15\% & 47\% & 88\% & 25\% \\  & **Logistic Regression** & 82\% & 2\% & 15\% & 84\% & 10\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance Metrics (mean %s) for the Individual Study Models (averages are given for sensitivity, precision and F1 scores, weighted by the frequency of class labels). Figure 2: ROC curves for each algorithm, for the Three-Study Merged Dataset Models (using the full dataset for the fitted models). time in the future (two years). Despite this challenge, by creating a large dataset, merged from three clinical trials, we achieved an overall accuracy of 76% (the three-study merged dataset models). This level of accuracy represents a considerable improvement on the chance accuracy level of 54%. This level of accuracy also compares well with the only other study to use ML to predict clinical trial treatment outcomes (65% accuracy for responses to anti-depressant medication [15]). Most importantly, we suggest that this is the first time that accuracy levels have been achieved which are strong enough to form the basis of real-world applications. Our results are therefore a significant addition to the scientific literature and have important implications for both precision medicine and drug development strategy. Finally, this is the first study to use ML to predict individual treatment outcomes of clinical trials for cancer. One crucial question is how far the predictive performance of such models can generalize to a real-world setting. To ensure the robustness of our results, we computed performance metrics by taking the mean of the results of a repeated (ten-fold) cross-validation procedure. More importantly, we assessed our models' generalizability by testing'separate study validation models,' in which ML models were trained and tested on data from separate studies. This procedure provided a more stringent test of generalizability, for three reasons. First, differences between studies can compromise model accuracy. In our study there were small differences in entry criteria and there may have been unknown differences in the measurement of some variables. Second, such designs constrain the number of samples that can be used for model training and testing, which may introduce inefficiencies into the modeling. In our case, using data from Studies 1 and 3 for model training meant that this training set represented 65% of the total data. ML models are usually trained on higher proportions of the data than this, with 80% being typical.