S INCE Decemeber 2019, the outbreak of coronavirus disease-19 (COVID-19) [1] , [2] has infected more than three million people worldwide, and causing more than 230, 000 deaths. World Health Organization (WHO) has declared the COVID-19 as a global health emergency on January 30, 2020 [3] . The chest computed tomography (CT) has shown to be useful to assist clinical diagnosis of COVID-19 [4] - [9] . However, the rapid growth of COVID-19 patients results in the shortage of the clinicians and radiologists. It is highly desired to develop automatic methods for computer-aided COVID-19 classification tools with chest CT images.
A few machine learning methods have been proposed for COVID-19 classification using chest CT images. For example, [10] employs a logistic regression method for COVID-19 classification by using clinical and laboratory features. [11] , [12] use random forest model with the handcrafted features for COVID-19 classification. Moreover, some deep learning based methods are proposed for the diagnosis of COVID-19. For instance, [13] leverages a deep learning method to learn the feature representation of the chest CT images, and then uses the learned features for COVID-19 classification by combining decision tree and Adaboost algorithm. In addition, [14] employs an end-to-end network to map the CT images to label space for COVID-19 disease identification.
In summary, the existing machine learning methods for COVID-19 diagnosis are mainly based on the handcrafted features or the learned image representation by deep neural networks. However, simply adopting handcrafted features cannot fully utilize the high-level information for COVID-19 classification, while the features learned from neural networks require great effort for parameter tuning with a small amount of medical image data.
To this end, in this paper, we propose a novel adaptive feature selection guided deep forest method that takes advantage of the high-level deep features with small number of medical image data for the classification between COVID-19 and CAP. Specifically, as shown in Fig. 1 , we first extract the locationspecific features from the chest CT image. Then, a deep forest model [15] is introduced to learn the latent high-level representations of these features, which can effectively describe the high-level information within the extracted location-specific features by using a small-scale training data. Intuitively, the use of the feature selection could promote the performance of the classification task. Hence, in our study, we also introduce a task-driven feature selection method to adaptively reduce the redundancy of features. In particular, the feature selection operation will discard a portion of unimportant features based on the feature importance which is calculated from trained forests in each deep forest layer. Hence, the feature selection and classifier training are adaptively incorporated into a unified framework. Finally, the trained adaptive feature selection guided deep forest is used for COVID-19 prediction.
The major contributions of this paper are three-fold. First, a deep forest is leveraged to learn the high-level feature representation of the location-specific features from chest CT for the diagnosis of COVID-19. Second, we proposed an adaptive feature selection method to adaptively select the discriminative features for the diagnosis of COVID-19. Third, the proposed method is evaluated on the collected COVID-19 dataset, which consists of 1495 COVID-19 patients and 1027 CAP patients. Experimental results demonstrate that our method achieves superior classification performance than the comparison methods.
The rest of the paper is organized as follows. We first introduce the materials used in this study and the proposed adaptive feature selection guided deep forest in Section II. Then, in Section III, we present experimental settings and experimental results. In Section IV, we study the influence of parameters in the proposed methods and present the limitations of the current study as well as possible future directions. We finally conclude this paper in Section V.
In this section, we first introduce the dataset used in this study. Then, we present the feature extraction procedure for the location-specific features. Next, we describe the proposed adaptive feature selection guided deep forest (AFS-DF). Finally, we provide the implementation details for our proposed AFS-DF. SCENARIA from Hitachi. CT protocol includes: 120 kV , reconstructed slice thickness ranging from 0.625 to 2 mm, with breath hold at full inspiration. All images are de-identified before sending for analysis. The study are approved by the Institutional Review Board of participating institutes. Written informed consent is waived due to the retrospective nature of the study.
Similar to [12] , we extract the location-specific features (i.e., infection locations and spreading patterns) to represent the chest CT images for diagnosis of COVID-19. Specifically, the chest CT images are first automatically segmented into infected lung regions and lung fields bilaterally by using VB-Net [16] , [17] . The infected lung regions are mainly related to mosaic sign, ground glass opacity (GGO), lesion-related signs (air bronchogram) and interlobular septal thickening. The lung fields include left lung, right lung, five lung lobes, and eighteen pulmonary segments. Then, we extract four kinds of locationspecific handcrafted features, including volume, infected lesion number, histogram distribution and surface area from chest CT images. Meanwhile, we also extract the radiomics features for describing the CT images. More details are as follows.
1) Volume features: Based on the segmented infected lung regions, we extract the total volume of infected region, and then calculate the percentage of the infected region of the whole lung. Meanwhile, according to the lung field segmentation results, we further extract the volume and percentage in each lobe and each pulmonary segment, respectively. Since there are evidence that pneumonia caused by COVID-19 more likely occurs in both right and left lungs, we also calculate the infected lesion difference as well as the percentage difference between left and right lungs. 2) Infected lesion number: In comparison to CAP, most of the COVID-19 infections encompass bilateral lungs with multifocal involvement [6] , [7] , and COVID-19 generally has concentrated infection lesions while CAP shows small in volume and patchy in distribution [18] . Therefore, we calculate the features of the total number of infected regions in the bilateral lungs, lung lobes, and pulmonary segments, respectively. 3) Histogram distribution: The predominant chest CT findings show that bilateral and peripheral GGO and consolidation are a radiologic hallmark of COVID-19 [19] , [20] . GGO is a pattern of hazy increased lung opacity with preservation of bronchial and vascular margins, whereas consolidation is characterized by a homogeneous increase in lung parenchymal attenuation
or CAP ? Fig. 1 . Pipeline of the proposed adaptive feature selection guided deep forest for COVID-19 vs. CAP classification. We first extract the location-specific features from chest CT. Then, the proposed AFS-DF is leveraged to train the classifier based on the location-specific features. Finally, based on the location-specific features, the trained FAS-DF is adopted for COVID-19 identification task. that obscures the margins of vessels and airway walls on CT images [18] . To extract the intensity distribution of the infected regions in chest CT images, we calculated the histogram features of the infected regions. 4) Surface area: In previous study [21] , it has been found that COVID-19 had a predominate distribution in the posterior and peripheral lung, and the abnormalities of lung parenchyma eventually spread to the central area and bilateral upper lobes [6] . Therefore, we constructed the infection surface as well as the lung boundary surface. We further calculated the distance of each infection surface vertex to the nearest lung boundary surface, and categorized them into 5 ranges, i.e., 3, 6, 9, 12 and 15 voxels. For each feature, the number of infection surface vertices within each range of distances to the lung wall is calculated. Furthermore, the percentage of infection vertex number against the number of whole infection surface vertices in each range is also considered. 5) Radiomics features: Radiomics features extracted from infected lesions, including intensity features (e.g., average gray level intensity, range of gray values) and texture features (e.g., gray level co-occurrence matrix, graylevel run-length matrix, gray-level size-zone matrix, and neighborhood gray-tone difference matrix) are used in our study. Beside, we also adopt the age and gender into the locationspecific features for the diagnosis of COVID-19. In summary, a total of 239 dimensions features are used in our study.
As mentioned in Section II-B, we extract location-specific features from the chest CT images. But, simply using these features cannot adequately describe the high-level information for COVID-19 classification. In this work, we propose an adaptive feature selection guided deep forest to learn the latent high-level representation of the extracted location-specific features with adaptive task-driven feature selection process for diagnosis of COVID-19. The architecture of proposed adaptive feature selection guided deep forest is shown in Fig. 2 .
As shown in Fig. 2 , each layer of proposed adaptive feature selection guided deep forest consists of N independent random forests and a feature selection unit. Here, each random forest produces a probability distribution of the COVID-19 and CAP (the yellow rectangle in Fig. 2) . Then, the N probability distribution vector of the COVID-19 and CAP are concatenated with the input feature vector. To reduce the redundancy of the features, we further perform an adaptive feature selection operation. In particular, for each trained random forest, we can calculate the feature importance for each feature within the input feature vector. Thus, we calculate the overall feature importance c i for i-th feature as follows,
where c i,n is the feature importance for i-th feature in n-th random forest. Herein, we discard the features with low feature importance by a specific ratio based on the calculated feature importance. Hence, the feature selection and classifier training are adaptively incorporated into a unified framework. Thus, the selected feature vector as the input of the next layer. Finally, we cascade multiply layers to learn the deep discriminative feature representation for COVID-19 classification task.
As shown in Fig. 2 , in the proposed adaptive feature selection guided deep forest, we employ a Xgboost [22] with 20 trees, a random forest [23] with 20 trees, and two extremely randomized trees [24] with 20 and 50 trees, respectively. We empirically set the feature discard ratio as 0.2 in our study. In the training stage, we feed the extracted location-specific features to the adaptive feature selection guided deep forest. The training set further is divided to 5 subsect for 5-fold crossvalidation. In particular, each subset is sequentially selected as the validation set, while the remaining four subsets are treated as training set for model construction. Thus, the numbers of cascade layers and selected features are automatically determined by using the cross-validation strategy. Hence, the adaptive feature selection guided deep forest can adaptively train the feature selection and classification model in a taskdriven manner.
In the testing stage, we also feed the location-specific features of test subject to the trained adaptive feature selection guided deep forest model. In the last layer, each forest will produce a probability distribution p for the identification of COVID-19. For each subject, we use the following equation to ensemble the predicted value for diagnose of COVID-19,
where p(l = c|n) is the probability of subject belongs to category c (i.e., COVID-19 or CAP) that is provided by the n-th forest in last layer. Finally, we use the MAP criterion to obtain the label for each subject, i.e., argmax c p(l = c)
In this section, we first illustrate the competing methods and experimental settings in our study. Then, we present experimental results achieved by different methods on the chest CT images with 1495 patients of COVID-19 and 1027 patients of CAP.
In our experiments, we compare our proposed AFS-DF with the following four widely adopted machine learning methods. 1) Logistic Regression (LR): A Logistic regression method is employed for COVID-19 classification by using the extracted location-specific features. 2) Support Vector Machine (SVM): The extracted location-specific features are fed into the SVM classifier by using radial basis function kernel with default parameters.
The random forest classifier is applied on the location-specific features for COVID-19 classification, and the number of trees in random forest is set as 500 via cross-validation. 4) Neural Networks (NN): In this method, a fully connection neural network is employed for COVID-19 classification. Specifically, we empirically set the minibatch size as 64, the number of epochs as 100, and the learning rate as 0.001.
For all extracted location-specific features from chest CT images, we first perform the normalization with center 0 and deviation 1 for each feature. Then, we employ a 5-fold crossvalidation strategy to evaluate the performance of the proposed AFS-DF as well as its competitors. Specifically, all subjects are randomly partitioned into 5 subsets. Each subset is sequentially selected as the testing set, while the remaining four subsets are treated as training set for model construction, and the final classification results are the average over the five-fold crossvalidations.
In order to measure the classification performance of different methods, four evaluation metrics are adopted, including classification accuracy (ACC), sensitivity (SEN), specificity (SPE) and the Area Under the receiver operating characteristic Curve (AUC) [25] . Here, the ACC, SEN and SPE are defined as,
where TP, TN, FP and FN in Eqs.3-5 represent True Positive, True Negative, False Positive, and False Negative, respectively.
We evaluate the COVID-19 vs. CAP classificationin 5-fold cross-validation on the collected chest CT images dataset. Table II shows the quantitative results (i.e., ACC, SEN, SPE and AUC) achieved by different methods.
From Table II , we can observe that our AFS-DF archives the best results in the terms of ACC, SEN, SPE and AUC. In particular, our AFS-DF method achieves the highest classification accuracy (i.e., 91.79%), which is better than the LR (i.e., 89.81%), SVM (i.e., 89.97%), RF (i.e., 89.41%) and NN (i.e., 89.53%). In general, the proposed AFS-DF achieves 1.98%, 1.82%, 2.38% and 1.83% improvements in terms of ACC over the LR, SVM, RF and NN, respectively. The possible reason for improvements is that our AFS-DF not only leverages the high-level feature representation by using the deep forest to improve the performance of COVID-19 vs. CAP, but also employs the discriminative features by using the adaptive feature selection process. The deep methods (i.e., NN and AFS-DF) show better classification accuracy in task of diagnosis of COVID-19 when compared with the conventional classifiers (i.e., LR, SVM and RF). The possible reason is that, the deep models can learn the high-level feature representative, which can boost the classification performance. It is worth noting that COVID-19 is the highly contagious disease, the higher SEN should have practically meaningful advantage for timely COVID-19 diagnosis to prevent the spread of the COVID-19. The SEN achieved by our AFS-DF for COVID-19 vs. CAP is 93.05%, which is better than other baseline methods. These results imply that using the adaptive feature selection guided deep forest model can improve the identification ability of COVID-19. As shown in Fig 3, our proposed AFS-DF method produces the best classification performance when compared with the baseline methods. These results further validate that using the high-level representation and adaptive feature selection strategy could improve the performance for COVID-19 vs. CAP classification. The statistics of feature importance of extracting location-specific features is shown in Fig. 4 . We calculate the sum of feature importance of location-specific features over the last layer in 5 trained AFS-DF model. Fig. 4 shows that the surface area features have importance influence for COVID-19 classification.
In this section, we first compare our proposed adaptive feature selection guided deep forest with several state-of-theart methods for COVID-19 classification. Then, we study the influence of the adaptive feature selection strategy and the selected deep features that are learned by adaptive feature selection guided deep forest. Finally, we present the limitations of this work as well as possible future research directions. 
Since several attempts have been made for COVID-19 classification, we now compare our proposed AFS-DF with state-of-the-art methods. [13] employs a convolutional neural networks (CNN) to extract the features of chest CT images, and then combines the decision tree and Adaboost to produce the classification result of COVID-19 vs. typical viral pneumonia (a total of 670 CT scans). [14] leverages a ResNet [26] to predict the COVID-19, Influenza-A viral pneumonia, and healthy cases (a total of 618 CT scans). [10] extracts clinical and laboratory features and uses a logistic regression model for non-severe and severe patient classification (a total of 196 CT scans). [11] extracts quantitative features, and introduced a random forest method to assess the severity of the COVID-19 patient (a total of 176 CT scans). [12] extracts location-specific handcrafted features, and proposed an infection size-adaptive random forest for COVID-19 classification (a total of 2685 CT scans).
The results are reported in Table III . One can observe from Table III that the proposed AFS-DF shows competitive classification performance for COVID-19 patient identification. The underlying reason could be that our AFS-DF can utilize the high-level discriminative representation of the extracted features.
To study the effectiveness of the proposed adaptive feature selection, we compare it to two feature selection methods (i.e., Lasso and ElasticNet [27] ) and its variant (i.e., Deep Forest (DF)). Lasso is used to select a discriminative subset of features from the feature vector by using l 1 -norm sparsity constraint. The parameter for the sparsity constraint in this method is set as 0.001 by using cross-validation. In ElasticNet, a l 1 -norm is leveraged to reduce dimension of extracted features. Also, a l 2 -norm is further introduced into the classification model to ensure the smoothness of the linear model. The parameters for the sparsity constraint and smoothness constraint in ElasticNet are set as 0.001 and 0.1 by using cross-validation, respectively. DF is a variant of the proposed AFS-DF, which employs the same architecture as our AFS-DF model without feature selection block. Note that this variants method has same parameters setting with AFS-DF for training and test in our study. The experimental results are reported in Table IV . As shown in Table IV , the proposed AFS-DF achieves the best classification performance, when compared with Lasso, ElasticNet and DF. In particular, compared with the feature selection methods (i.e., Lasso and ElasticNet), the proposed AFS-DF achieves better performance by using the high-level feature representation. Meanwhile, by using the adaptive feature selection operation, the AFS-DF achieve a better performance when compared with DF. These results imply the effectiveness of the proposed AFS-DF.
To evaluate the effectiveness of the selected deep features (e.g., the features used in the last layer of AFS-DF) for COVID-19 vs. CAP classification, we further develop three methods based on LR, SVM and RF by using the selected deep features (i.e., AFSDF-LR, AFSDF-SVM and AFSDF-RF). We evaluate these six methods for COVID-19 vs. CAP classification, with the results reported in Table V. As can be seen from Table V , the proposed AFSDF-LR, AFSDF-SVM and AFSDF-RF outperform their counterparts (i.e., LR, SVM and RF) in most of evaluation metrics. Of note, the proposed methods consistently achieve better results in terms of ACC and SEN. For example, AFSDF-LR, AFSDF-SVM and AFSDF-RF achieve 1.38%, 1.15% and 1.11% improvement over LC, SVM and RF in terms of ACC for COVID-19 vs. CAP classification, respectively. Compared with LC, SVM and RF, the AFSDF-LR, AFSDF-SVM and AFSDF-RF also show improvement in terms of SEN for COVID-19 vs. CAP classification, respectively. The possible reason is that, with the selected deep features by using AFS-DF, the features include the high-level and discriminative information. Hence, the conventional machine learning methods (i.e., LR, SVM and RF) can use these selected deep features to improve the performance of COVID-19 classification task.
Beside, In Fig 5, we plot the original location-specific features, the features of the last layer in DF, and the features of the last layer in AFS-DF, which perform dimensionality reduction by using t-SNE [28] . As shown in Fig 5, our AFS-DF produces more discriminative features for COVID-19 vs. CAP classification. 
There are still several limitations in the current study. First, the adaptive feature selection guided deep forest is only validated on COVID-19 vs. CAP classification task. In the future, we plan to perform our proposed method on other COVID-19 classification tasks (e.g., COVID-19 vs. normal, severe patients vs. non-severe patients, etc.). Second, we extract the handcraft features by using prior knowledge in current work; in future, the features learned by deep learning method are expected to leverage our proposed method for further performance improvement.
In this paper, we propose an adaptive feature selection guided deep forest for COVID-19 vs. CAP classification by using the chest CT images. Specifically, the AFS-DF uses the deep forest to learn the high-level representation based on the location-specific features. Meanwhile, an adaptive feature selection operation is employed to reduce the redundancy of features based on the trained forest. Experimental results on the collected COVID-19 dataset with 1495 COVID-19 cases and 1027 CAP cases show that our proposed AFS-DF approach can achieve superior performance on COVID-19 classification with chest CT images in comparison with several existing methods.