# Decision Support Software for Forecasting Patient's Length of Stay
## 4 Experimental Results  
In this section, we report a series of experiments to evaluate the performance of the proposed two-level classification algorithm against some of the most popular and commonly used classification algorithms. In this regard, performance evaluation was conducted using different classification algorithms at each level and explore their classification accuracy. Our aim is to find which classifiers are best suited as A-level, B\({}_{1}\)-level and B\({}_{2}\)-level classifiers for producing the highest performance. To this end, we have selected  
* Naive Bayes (NB) algorithm was the representative of the Bayesian networks [25]. * Multilayer Perceptron (MLP) was representative of the artificial neural networks [26]. * Sequential Minimal Optimization (SMO) [27] from the support vector machines. * \(k\)NN [28] from instance-based learners with Euclidean distance as distance metric. * Random Forest (RF) [29] from decision trees. This selection was based on studies which have shown that the above classifiers constitute some of the most effective and widely used data mining algorithms [30] for classification problems. We evaluated the performance of our proposed two-level classification scheme in terms of accuracy, as one of the most frequently used measures for assessing the overall effectiveness of a classification algorithm [31] and is defined as the percentage of correctly classified instances. All classifiers have been implemented in WEKA Machine Learning Toolkit [32] and the classification accuracy was evaluated using the stratified 10-fold cross-validation [33] i.e., this approach involves randomly dividing the set of instances into ten groups (folds), of approximately equal size, so that each fold had the same distribution of classes as the entire data set. Each fold is treated as a testing set, and the classification algorithm is fit on the remaining nine folds. The results of the cross-validation  
Figure 2: Two-level classification scheme. process are summarized with the mean of the prediction model skill scores. Table 2 reports the configuration parameters of all classification algorithms. Tables 3-7 present the performance evaluation of B\({}_{1}\)-level and B\({}_{2}\)-level classifiers using NB, MLP, \(k\)NN, RF and SMO as A-level classifiers, respectively. In each case, the accuracy measure of the best performance is highlighted in bold. Firstly, it is worth mentioning the sensitivity of the two-level scheme on the selection of A-level and B-level classifiers. More specifically, the classification accuracy of the proposed classifier varies between 42.3-53.89%, 42.53-54.67%, 57.44-7  
Finally, to illustrate the efficacy of the two-level classification algorithm, we compared it against the performance of single learning algorithms. Notice that two-level stands for the proposed two-level classification scheme using RF as A-level and \(k\)NN and RF as B\({}_{1}\)-level and B\({}_{2}\)-level classifiers, respectively. Table 8 summarizes the accuracy of each individual classifier which reveals the efficacy of our two-level classifier. Clearly, the proposed scheme significantly outperforms all single classifiers, exhibiting higher classification performance. Moreover, we demonstrate the classification accuracy of the proposed algorithm with a more traditional approach, comparing the confusion matrix of two-level classification scheme with that of best single classifier (RF). Notice that confusion matrix gives an additional information about classes which are commonly mislabeled one as another. Tables 9 and 10 present the confusion matrices of the two-level classification scheme and RF, respectively. The interpretation of these tables illustrates that the proposed algorithm exhibits significantly better classification accuracy for patients which stayed in the hospital for one or two days. More specifically, the proposed algorithm correctly classified 84.7% and 79.6% of the patients which stayed in the hospital for one and two days, respectively; while RF classified only 74% and 72%, respectively.