## 5 Decision Model Generation
### Automated Approach  
The analysis using the automatic machine learning environment was carried out in the RapidMiner Auto Model [51]. This environment performs semi-automatic feature selection, suggesting which attributes are valuable. It is recommended to remove attributes that contain too many missing values, have too little (Stability) or too much (ID-ness) variety, and are characterised by too strong or too weak correlation with a decision attribute. Auto Model recommended the following attributes as the basis of the analysis: age, sore throat, headache, muscle aches, dyspnoea, blood group, cough, contact with infected person, hypertension, days of symptoms, sex, saturation, temp. > 38 \({}^{\circ}\)C, temp. max, temperature in hospital, loss of smell or taste, dizziness.  
Auto Model enables a user to define the costs of the classification mistakes. In this way, it is possible to define the false negative error, which is the prediction that someone does not have COVID-19 while they have the disease, as being more undesirable than a false negative error. In the analysis performed, several values of the cost were verified--the cost value set to \(-1\) means that there were no additional costs, and the lower values mean larger costs.  
Finally, Auto Model allows a user to verify the quality of several optimised classifiers. The classifiers are not evaluated by means of a well-known and typically recommended cross-validation procedure, but by a proprietary solution. The reported performance is the average of the scores over 5 out of 7 subsets on a hold-out data set consisting of 40% of data that were not used for model optimisation. The performance (in terms of sensitivity and specificity) of the models automatically generated on training data is presented in Table 4. Moreover, the models generated for the larger cost values \(\{-5,-7\}\) (distinguished by better quality) were applied to the test data set to additionally verify their quality--the results of this analysis are presented in Table 5.  
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Sensitivity**} & \multicolumn{4}{c}{**Specificity**} \\ \hline
**Cost** & \(-\mathbf{1}\) & \(-\mathbf{3}\) & \(-\mathbf{5}\) & \(-\mathbf{7}\) & \(-\mathbf{1}\) & \(-\mathbf{3}\) & \(-\mathbf{5}\) & \(-\mathbf{7}\) \\ \hline Naive Bayes & 0 & 72.2 & 100 & 100 & 100 & 8.53 & 0 & 0 \\ Generalised Linear Model & 0 & 0 & 64.58 & 100 & 100 & 85.39 & 16.18 & 0 \\ Logistic Regression & 0 & 16.17 & 62.79 & 84.29 & 100 & 56.8 & 18.68 & 8 \\ Deep Learning & 0 & 38.57 & 100 & 100 & 100 & 32.57 & 0 & 0 \\ Decision Tree & 0 & 0 & 0 & 100 & 100 & 100 & 100 & 1 \\ Random Forest & 0 & 32.68 & 100 & 100 & 100 & 83.29 & 5.05 & 0 \\ Gradient Boosted Trees & 0 & 14.05 & 64.4 & 98.33 & 100 & 96 & 69.93 & 9.05 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance of the classifiers generated by RapidMiner Auto Model on training data.  
The obtained results presented in Tables 4 and 5 show that the undertaken task is non-trivial, and it is challenging for the automatically generated classifiers. This motivated further research conducted by data science experts. Moreover, the adopted value of the cost for false negative errors has a significant impact on the obtained quality of the results--only higher cost values enabled the classifiers to obtain acceptable sensitivity (see Table 4); however, the specificity then drastically decreased. Finally, treating the obtained results as a starting point for more advanced analyses carried out by data science experts, it can be noticed that the highest sensitivity with non-zero specificity is achieved by two models: logistic regression and gradient boosted trees.