A respiratory syndrome COVID-19 pandemic has become a serious public health issue nowadays. The COVID-19 virus has been a ecting more than tens of millions people worldwide. Some of them have recovered and have been released. Others have been isolated and few others have been unfortunately deceased. In this paper, we apply and compare di erent machine learning approaches such as decision tree models, random forest, and multinomial logistic regression to predict isolation, release, and decease states for COVID-19 patients in South Korea. The prediction can help health providers and decision makers to distinguish the states of infected patients based on their features in early intervention to take an action either by releasing or isolating the patient after the infection. The proposed approaches are evaluated using Data Science for COVID-19 (DS4C) dataset. An analysis of DS4C dataset is also provided. Experimental results and evaluation show that multinomial logistic regression outperforms other approaches with 95% in a state prediction accuracy and a weighted average F1-score of 95%. Keywords: COVID-19, prediction, isolation, release, decease, classiﬁcation, decision tree, random forest, multinomial logistic regression. Corresponding author. Email addresses: tkafif@uqu.edu.sa (Tarik Alaﬁf), ralotibi@kau.edu.sa (Reem Alotaibi), asbassam@uqu.edu.sa (Ayman Albassam), s44181770@st.uqu.edu.sa (Abdulelah Almudhayyani)1. Introduction The COVID-19 pandemic has ﬁrst appeared in Wuhan, Hubei province, China in December 2019. It has become a serious public health issue worldwide as it spreads quickly. The COVID-19 virus is an infectious disease which directly a ects people lungs. It is a branch of Coronaviruses family. It is believed that it has been transmitted from animals to human. Preprint submitted to Elsevier January 11, 2021 The virus also transmits from human respiratory to another which is already noticed worldwide. The virus may cause mild or severe symptoms to a ected people during the virus replications in their bodies. A ected people may develop symptoms such as dyspnea, fever, sore throat, cough, fatigue, and pneumonia. The virus has become a serious disease since it may cause death after developing the symptoms. Fortu- nately, many people have recovered and have been released while others are still isolated. While many people are recovered, the number of deaths is very few compared to the number of recovered ones. Machine learning (ML) algorithms play an important role in today’s research because of the ability to automate complex tasks. These al- gorithms can learn from previous experience to predict future outcomes. Many research works have been developed and used in machine learn- ing. Some of the well-known ML classiﬁers are Decision Trees (DTs), Random Forest (RF), and Multinomial Logistic Regression (MLR). DT, so-called CART, is a popular supervised machine learning and a top-down predictive modelling approach [1]. It uses a simple tree structure representation to classify negative and positive examples recursively based on their in- put features. Various decision tree approaches have been proposed over the past years. ID3 [2], C4.5 [3] and CART [4] are the most common ones [5]. RF is an ensemble learning approach that is based on generating a large number of decision trees using a di erent subset of the training data [6, 7]. These subsets are chosen by random sam- pling of the original training data. The ﬁnal pre- dictions are made by taking the majority votefrom all individual classiﬁcation trees. RF is a powerful algorithm in machine learning. One of the advantages of RF is the ability to han- dle large datasets with high-dimensional feature space. It can handle thousands of input variables and identify the most signiﬁcant variables. In addition to DT and RF, regression ap- proaches are also commonly used to predict fu- ture events in many research ﬁelds. In the ma- chine learning context, logistic regression is a well-known technique that predicts binary out- comes. On the other hand, MLR is an ex- tended version of a binary logistic regression ap- proach that allows performing multi-class pre- diction where prediction outcome can be either dichotomous or continuous [8]. Our work is motivated by the success of CARTs, RF, and MLR approaches in many pre- dictive applications such as in vegetation dis- tributions [9], microRNA precursors [10], and response variables [11]. In this paper we pro- pose to use them to predict isolation, release, and decease states for COVID-19 patients in South Korea. The proposed approaches are evaluated using Data Science for COVID-19 (DS4C) dataset. An analysis for DS4C dataset is provided. Experimental results and evaluation show that MLR outperforms other approaches with 95% in a state prediction accuracy and a weighted average F1-score of 95%. The remainder of this paper is organized as follows. In Section 2 , related work is re- viewed. Section 3 introduces basic notations and mathematical models used throughout the paper. In Section 4 , we brieﬂy describe and an- alyze DS4C dataset. In Section 5 , we present our work. In Section 6 , experimental results and evaluation are provided using public DS4C 2 dataset. We provide discussion details in Sec- tion 7 . Finally, our conclusion and future work are provided in Section 8 . 2. Related Work Many research works have been published recently to analyze and tackle the problem of COVID-19 virus in many ﬁelds. Several methods were experimented in [12– 24] for COVID-19 analysis and predictions. Fanelli and Piazza [12] analyzed and fore- casted COVID-19 spreading in China, Italy, and France. Nadia and Hazem [13] studied the ef- fects of sex, region, infection reason, birth year, release date, and the diseased date on the re- covered and deceased cases for COVID-19 pa- tients. Hu et al. [14] applied a stacked auto- encoder and K-means to model the transmission dynamics of the epidemics based on provinces and cities. The model was built to forecast the conﬁrmed cases of COVID-19. Remuzzi and Remuzzi [15] predicted the number of infected patients in Italy using an exponential curve. Tar- tai and Varallyay [16] attempted to predict the results of COVID-19 epidemic in a region us- ing a logistic model. Pedersen and Meneghin [17] analyzed and predicted the spread of the COVID-19 virus using restrictions to reduce the epidemic in Italy. Peng et al. [18] pro- posed a SEIR model to analyze the epidemic in China. Petropoulos and Makridakis [19] fore- casted the conﬁrmed cases, death cases, and re- covery cases of COVID-19 using time series forecasting approaches. Chatterjee et al. [20] and Jia et al. [21] developed mathematical models to study the impact of COVID-19 epi- demic. Weissman et al. [22] proposed a non-learning predictive mathematical model, called SIR, to predict hospital capacity needs during the COVID-19 pandemic. However, the SIR model predicts only the epidemic spread over time for the whole population within a speciﬁc area. Yang et al. [23] used a long term short memory neural network to predict the COVID- 19 epidemic in China. Similar to [23], Vattay [24] attempted to predict the epidemic in Italy. Di erent from the aforementioned research works to tackle the COVID-19 problem from di erent perspectives, we apply and compare di erent machine learning approaches such as CARTS, RF, and MLR to predict isolation, re- lease, and decease states for COVID-19 patients in South Korea. The proposed approaches are evaluated using DS4C dataset to measure the performance of the predictions. 3. Preliminaries LetDa set of patients records such that D=fX1;X2; : : : ; Xng, where nis the num- ber of patients. Each patient record Xis be- longing to one of the classes jthat is used for the classiﬁcation task such that D= f(X1;j1);(X2;j2); : : : ; (Xn;Yn)g. Each patient record Xis composed of several input features such that X=fx1;x2; : : : ; xmg, where mis the total number of patient features. Then, each fea- ture becomes a predictor variable of the class j such that X=f(x1;x2; : : : ; xm;j)g,Yis the class variable taking values either in a set of classes C=f1;2; : : : ; kg, where kis the total number of classes. Next, we show the basic mathematical equa- tions used in the following approaches: 3 3.1. CARTs In CARTs, Gini impurity is used to measure the purity in the decision tree models by know- ing how good the split of input features in each node [25, 26]. Then, the Gini impurity is com- puted using the training examples with the class jas shown in Equation 1: Gini( Djx)=1kX j=1Pj2(1) where x,kandPjare the examined feature, the number of classes and the probability of the jth class respectively. The Gini Index is computed for each input feature partition and the average Gini Index for the examined feature xis deﬁned as: Gini( Djx)=vals(x)X i=1jDij jDjGini( Dijx) (2) where x,Diandvals(x) are the examined fea- ture, a data partition and number of discrete val- ues in feature xrespectively. The feature with the lowest Gini Index is chosen as the splitting feature. 3.2. RF An RF consists of a collection of decision trees T=f1; : : : ; ntreeg. Each decision tree is trained on a di erent subset of the data using bootstrap sampling (bagging) out of D. Final predictions are made using the majority votes. The RF employs random feature selection for each node of every tree in the forest, meaning that the splitting feature xis chosen at random.3.3. MLR Logistic regression is deﬁned as a generalized linear model that is usually used to classify bi- nary outcomes [8]. Ridge estimator is proposed to improve model prediction [27]. Logistic re- gression approach is slightly modiﬁed to deal with a multi-class classiﬁcation that is called the MLR, in case of predicting koutcome classes forninstances with mobservations, the parame- ter matrix of regression coe cients Bto be cal- culated will be an m(k1) matrix. eis the exponential function. Class jprobability with the exception of the last class is shown in Equa- tion 3: Pj(Xi)=eXiBj Pk1 j=1eXiBj+1(3) The last class probability is shown in Equa- tion 4: 1k1X j=1PjXi=1 Pk1 j=1eXiBj+1(4) The (negative) multinomial log-likelihood is shown in Equation 5:: L=nX j=1fk1X j=1Yi jln(Pj(Xi)+ (1(k1X j=1Yi j))ln(1k1X j=1Pj(Xi))g+ridge(B2) (5) 3.4. Evaluation Metrics The accuracy for multi-class classiﬁcation is deﬁned as the number of instances correctly 4 identiﬁed as either truly positive or truly nega- tive out of the total as shown in Equation 6. The error rate is the complement of accuracy. accuracy =T P+T N T P+FP+T N+FN(6) where T P,T N,FPandFNrepresent true posi- tive, true negative, false positive and false nega- tive respectively. F1-score is averaging the harmonic mean of precision and recall as shown in Equations 7: precision =T P T P+FP recall =T P T P+FN F1-score =2precisionrecall precision +recall(7) where precision and recall represent precision and recall respectively. The weighted aver- age F1-score is computed by averaging F1- score class-by-class while weights are computed based on each target class frequency as shown in Equation 8. Using such evaluation metric helps to keep an account for imbalance distribution of classes when computing the F1-score. Weighted average F1-score = Pk j=1wjF1-score j Pk j=1wj(8) Where kis the total number of target classes, wjthe weighting parameter of the jthclass and F1-score jis the computed F1-score of the jth class. Figure 1: The number of samples distribution for each state in DS4C dataset. Figure 2: The samples distribution for sex feature in DS4C dataset. 4. DS4C Dataset DS4C [28] has been published on Kaggle re- cently into public in February 24th, 2020 in South Korea1. The dataset consists of di er- ent data in four sheets (Case data, patient data, time-series data, and additional data). The case data describes the data for COVID-19 infectious cases. The patient data describes the epidemi- 1https: //www.kaggle.com /kimjihoo /coronavirusdataset. 5 Figure 3: The samples distribution for age feature in DS4C dataset. ological and route data of COVID-19 patients. The time-series data describes the status of time- series data of COVID-19 patients. Also, addi- tional data is reported for regions, weather, and population. Each sheet is found in the form of a CSV ﬁle. In our work, we only use the patient data ﬁle to reach the aim of our research since it contains epidemiological data for COVID-19 patients in South Korea. Patients data consists of 5,165 patients labeled records. Each patient record consists of several features such as sex, age, country, province, city, infection case, infected by, contact number, symptom onset date, con- ﬁrmed date, released date, deceased date, and state. The state feature represents the label for each patient record. The dataset has three states. Each record in the dataset is either labeled isolated, released, or deceased. Figure 1 shows the states versus the num- ber of infection cases in the dataset. The ﬁg- ure also shows clearly the unbalanced number of the sample’s distribution in this dataset which consists of 2,158 isolated states, 2,929 releasedstates, and 78 deceased states. We also show the samples distribution for sex and age features in the dataset in Figure 2 and Figure 3 respec- tively. We notice that the number of infected male is closed to the number of infected female. We also notice that the most infected age among patients is between 20 to 29 years old. We have investigated the causes of COVID- 19 infection in South Korea patients in this dataset. We have found the most patients re- ceive this infection are from the most frequent causes ”Contact patient”. Then, ”Overseas in- ﬂow” is followed. Figure 4 depicts clearly the most common causes of COVID-19 infection in South Korea patients. Therefore, a social dis- tancing is needed to halt or at least decrease the spread of the COVID-19 pandemic. Figure 4: The causes for COVID-19 infection in South Korea according to the infection cases in DS4C dataset. 5. Our Work We propose to use DT models, RF, and MLR to classify the states (isolated, released, 6 Figure 5: The generated DT architecture of our approach using the maximum DT depth of 3. and deceased) for COVID-19 patients in South Korea. The DT models learn discrimina- tive features from patient records to perform the prediction. The models predict the pa- tient’s state based on several feature vari- ables. We train our DT models using the features such as sex, age, country, province, city, infection case, contact number, symp- tom onset date, conﬁrmed date, released date, deceased date, and state. Patients case id are excluded. Before training the models, we pre-process the patients’ records to manipulate missing data and non-integer values. The miss- ing data are replaced by -1. We apply di erent maximum DT depths to build our predictive intelligent models. The models are trained using the patients’ records. For DT models, we use maximum DT depths of 3, 5, and 10 in our work. Figure 5 shows the generated DT architecture after training us- ing the depth of 3. The DT architectures with the maximum depth of 5 and 10 generate a very large tree which consists of many branches andmany leaf nodes and non-leaf nodes. A general- ization capability is lost if the depth is more than 10. For the RF model, the model is trained with the default parameters according to randomFor- est package in R [6]. The default parameters in the randomForest algorithm: ntree , the number of classiﬁcation tree (the default value is 500) andmtry, the number of features tested at each node (the default values is sqrt(m), where mis the total number of features). For the MLR model, the model-building pro- cess is carried out using weka tool with a ridge estimator which is considered as a stable imple- mentation. The predictive model is fed with a full set of features. The Quasi-Newton method is used to perform the optimization task. Ridge values of 1108are used in the log-likelihood calculation. 6. Experimental Results and Evaluation Tree-based approaches are implemented in R using rpart andrandomForest packages [6, 29] 7 Figure 6: Maximum tree depth tuning based DT. while MLR approach was built using the Weka version 3.8.4 of machine learning tool [30]. A single laptop 1.6 GHz Dual-Core Intel Core i5 CPU with 8 GB of RAM is used to train and test the models. 10-fold cross-validation is applied for training and testing the models using DS4C dataset. A seed state is set to 1234. Table 1 shows the accuracy, error rates, and the weighted average F1-score for the tested models. For DT models, we choose three di er- ent tree depths: 3, 5, and 10. In order to obtain the e ect of the maximum tree depth parameters on the experimental results, we show the perfor- mance curve to verify the e ectiveness of depth parameters on the models. Figure 6 shows the models accuracy versus di erent tree depths. It clearly shows that the DT models accuracy stops improving at the maximum depth of 12. Figure 7 shows the e ect of the number of generated trees in the RF on the classiﬁcation er- ror rate. We can see that the error rate becomes stable when the number of trees is increased. The curve in black color represents out-of-bag Figure 7: RF error rates with an increase in the number of classiﬁcation trees. The number of trees and the estimated error rates are shown on x-axis and y-axis, respectively. (OOB) error rate and the other colors represent misclassiﬁcation error rate curves. The out-of- bag error is estimated internally during building the DTs. The confusion matrices using 10-fold cross- validation are provided in Table 2. The table shows true positives, true negatives, false posi- tives, and false negatives test examples after ap- plying each model. We notice that the DT mod- els using di erent depths are unable to predict the true positives in the deceased state. This maybe due to the small number of examples found in the deceased state. Following the DT models, the RF approach is able to classify 22 true positive examples from the deceased state. Lastly, the MLR model classiﬁes 44 true posi- tive examples from the deceased state correctly. Besides, the small number of false negatives re- veals an encouraging result. From Table 1, the MLR approach outperforms other approaches with 95% in the state prediction accuracy and 8 Table 1: Prediction accuracy, error rates and the weighted average F1-scores for the applied algorithms using DS4C dataset. Algorithms implemented Accuracy Error rate Weighted average F1-score DT (Depth =3) 82.92% 17.08% 81.74% DT (Depth =5) 85.63% 14.37% 84.67% DT (Depth =10) 88.21% 11.79 % 87.47% RF 92.55% 07.45% 92.28% MLR 95.00% 05.00% 95.00% with a margin of 2.45% to RF approach. To the best of our knowledge, there is no existing simi- lar work to compare with ours since the COVID- 19 research area and the DS4C dataset are new. 7. Discussion The experimental results presented above have illustrated the ability of the proposed approaches to produce reliable predictions of COVID-19 patients who are isolated, released, and deceased. The proposed predictive instru- ments were built using a small bag of fea- tures which causes type I and type II errors to arise, as utilized features cover only limited as- pects of patients’ personal and infection factors. Though, utilizing subject features allows to pro- duce quality predictions of patient’s status even with the absence of medical-related aspects. The proposed MLR model is able to predict 96.3% and 94.6% of the released and isolated cases respectively, while it predicts only 56.4% of the decease instances as such patients belong to minority state in the dataset. The COVID-19 DS4C dataset holds a signiﬁcantly unbalanced class distribution where the minority instancesbelong to the decease state occupy only 1.5% of total instances in the dataset. The reason is that logistic regression model tends to produce more reliable classiﬁcation outcomes when it deals with few numbers of features, and big amounts of training examples which is the case in the uti- lized dataset. Therefore, MLR provides most signiﬁcant performance in prediction accuracy and the weighted average F1-score. One can notice that the DT models using dif- ferent depths have failed to predict the instances from the decease state as shown in Table 2. This is due to the small number of deceased state ex- amples exist in the dataset. Therefore, the per- formances of the di erent depth of DT models have decreased. However, the DT model using the depth of 3 has achieved the best true positive rate only in the release state compared to other models. On the other hand, the RF model performance relatively competes with the MLR model per- formance with a margin of 2.45% in state pre- diction accuracy. However, the RF model has lower true positive rates in all states compared to the MLR model as shown in Table 2. In gen- eral, based on the characteristics of features, and 9 Table 2: Confusion matrices for actual versus predicted patients’ states. DT (Depth =3) Deceased Isolated Released Deceased 0 0 0 Isolated 21 1444 90 Released 57 714 2839 DT (Depth =5) Deceased Isolated Released Deceased 0 0 0 Isolated 23 1586 92 Released 55 572 2837 DT (Depth =10) Deceased Isolated Released Deceased 0 0 0 Isolated 25 1808 181 Released 53 350 2748 RF Deceased Isolated Released Deceased 22 18 38 Isolated 0 1992 166 Released 0 163 2766 MLR Deceased Isolated Released Deceased 44 14 20 Isolated 15 2042 101 Released 19 89 2821 corresponding experimental results using DS4C dataset, we can conclude that the MLR model is more e ective than other classiﬁcation models for predicting COVID-19 patients states. Unfortunately, none of the existing COIVID- 19 public datasets provide such adequate fea- tures and labeling for predicting the states of isolation, recovery, and decease. Most of thecurrent public COVID-19 datasets focus on pro- viding a statistical data about the death num- ber, infection number, infected areas, patients tweets, infected X-ray scans, and infected CT scans from chests and brains. Also, the cur- rent DS4C public dataset lacks some of impor- tant features associated with the impact of the COVID-19 virus such as patients’ vital signs, medications usage, and chronic diseases. The lack of COVID-19 data and the mechanism of its collection and collaboration is considered a limitation, and a challenge in COVID-19 re- search area. 8. Conclusion This research work attempts to apply and compare di erent machine learning approaches such as DT models, RF, and MLR to predict iso- lation, release, and decease states for COVID- 19 patients in South Korea. The proposed approaches are evaluated using DS4C dataset. An analysis for DS4C dataset is also provided. This study ﬁnds that contact patient is the most cause for COVID-19 infection. Experimental results and evaluation show that MLR outper- forms other approaches with 95% in the state prediction accuracy and the weighted average F1-score of 95%. The proposed machine learn- ing models can help health providers and deci- sion makers in early intervention to take an ac- tion either by releasing or isolating the patient after the infection. Based on our observations from South Korean public COVID-19 statistical dashboard of daily conﬁrmed cases, it reveals that South Korea has the highest number of infection in February, 2020 with 909 conﬁrmed cases. Luckily, the 10 current infection number per day has lowered to less than 50 since the beginning of March, 2020. Still, South Korea has a less number of infection rate compared to other countries which may be due to social distancing. In the future, we plan to collect clinical data from local hospitals to analyze more important features and explore their patterns. These pat- terns may give a better understanding to the rela- tions of the states. They are maybe beneﬁcial to reduce the impact of the decease states. We also plan to use deep learning based methods such as deep neural networks to predict the state more accurately. Declarations of interest The authors declare that they have no known competing ﬁnancial interests or personal rela- tionships that could have appeared to inﬂuence the work reported in this paper. Acknowledgment This work is funded by Research and De- velopment Grants Program for National Re- search Institutions and Centers (GRANTS), Tar- get Research Program, Infectious Disease Re- search Grant Program, King Abdulaziz City for Science and Technology (KACST), Kingdom of Saudi Arabia, grant number (5-20-01-007- 0001).