# Self-Service Data Science in Healthcare with Automated Machine Learning
## 5 Discussion
### Lessons Learned
#### 5.1.1 Suitability of AutoML Methods for Researcher-Physicians  
After the elicitation of requirements, we found that researcher-physicians prefer to create logistic regression models in their research. There are multiple reasons why we find that AutoML in the way we assessed it, is not the best way to create models for researcher-physicians. First, AutoML originates from the CASH problem. If the algorithm type is pre-selected, AutoML can only contribute to HPO. As logistic regression only has two hyperparameters [8], we question the need for using AutoML to tune these hyperparameters.  
Second, even if we drop the constraint of logistic regression for model construction, previous research found that other ML models do not significantly outperform logistic regression models in medical studies [45]. Even when we drop the logistic regression model constraint, we still doubt the usability of AutoML due to the results of [PERSON] et al. although other researchers do not find the same results on general datasets [46; 47].  
Third, there is no explanation of variable importance by the tested AutoML method. As model explainability and variable importance are essential requirements for researcher-physicians, this makes AutoML unsuitable in its current form. If the given explanations are satisfactory, AutoML might be useful in research. More on variable importance is available in the future research section below.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & **Preference** & **Score** \\ \hline
**User interaction** & & \\ Upload dataset & GUI & 4/5 \\ Create a subset & Code & 3/5 \\ Workflow & Code & 4/5 \\ Workflow explanation & Code & 4/5 \\
**Model construction** & & \\ Progress reporting & Code & 4/5 \\ Model construction & Code & 5/5 \\
**Model explanation** & & \\ Compare results & GUI & 4/5 \\ Explanation missing data & Code & 4/5 \\ Readability & GUI & 4/5 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Artefact preferences of the five domain experts.  
If AutoPrognosis can deliver on its promises it can be a promising technology considering the user stories on model explanation.  
Fourth, there is no structure in the pipelines created by TPOT; this can lead to very complicated pipelines in with three or four logistic regression models, all using each other's results as input. These constructions are hard to understand for domain experts, if we would use grammar to represent the pipeline like in RECIPE [26]. The grammar can help to create more understandable pipelines for researcher-physicians. Another option could be using fixed-pipeline methods based on BO. Although they performed worse on the benchmark test, they can compensate if their results make more sense to the researcher-physicians. Finally, we have learned that there is a gap in the knowledge level of ML between literature and practice for researcher-physicians. The literature on AutoML states that AutoML aims to aid non-expert users of ML techniques [8]. However, we find that most non-expert users have no knowledge or education in programming. Hence, the current offering of AutoML techniques are still too technical for non-expert users in healthcare.