The sudden spike in the number of patients with COVID-19 , a new respiratory virus, has put unprecedented load over healthcare systems across the world. In many countries, the healthcare systems have already been overwhelmed. There arXiv:2004.09803v1 [eess.IV] 21 Apr 2020 are limited kits for diagnosis, limited hospital beds for admission of such patients, limited personal protective equipment (PPE) for healthcare personnel and limited ventilators. It is thus important to differentiate which patients with severe acute respiratory illness (SARI) could have COVID-19 infection in order to efficiently utilize the limited resources. In this work we propose the use of chest X-Ray to detect COVID-19 infection in the patients exhibiting symptoms of SARI. Using our tool one can classify a given X-Ray in one of the four classes: normal, bacterial pneumonia, viral pneumonia, and covid pneumonia. The use of X-Ray has several advantages over conventional diagnostic tests:
1. X-Ray imaging is much more widespread and cost effective than the conventional diagnostic tests. 2. Transfer of digital X-Ray images does not require any transportation from point of acquisition to the point of analysis, thus making the diagnostic process extremely quick. 3. Unlike CT Scans, portable X-Ray machines also enable testing within an isolation ward itself, hence reducing the requirement of additional Personal Protective Equipment (PPE), an extremely scarce and valuable resource in this scenario. It also reduces the risk of hospital acquired infection for the patients.
The main contribution of this work is in proposing a novel deep neural network based model for highly accurate detection of COVID-19 infection from the chest X-Ray images of the patients. Radiographs in the current setting are in most cases interpreted by non-radiologists. Further, given the novelty of the virus, many of the radiologists themselves may not be familiar with all the nuances of the infection, and may be lacking in the adequate expertise to make highly accurate diagnosis. Therefore this automated tool can serve as a guide for those in the forefront of this analysis.
We would like to re-emphasize that we are not proposing the use of the proposed model as alternative to the conventional diagnostic tests for COVID-19 infection, but as a triage tool to determine the suitability of a patient with SARI to undergo the test for COVID-19 infection.
To help accelerate the research in this area, we are releasing our training code and trained models publicly for open access at https://github.com/ arpanmangal/CovidAID. However, we note that both the model and this report merely captures our current understanding of this rapidly evolving problem, that too on very limited data currently available. We will keep updating the model and this report as we get newer understanding and better results.
Various deep learning based approaches have been developed to identify different thoracic diseases, including pneumonia [7, 8, 11, 13] . We choose CheXNet [7] to build upon, which could detect pneumonia from chest X-Rays at a level exceeding practicing radiologists. CheXNet is trained on ChestX-ray14 [11] (the largest publicly available chest X-ray dataset), gives better performance than previous approaches [11, 13] , and has a simpler architecture than later approaches [8] .
CheXNet [7] is a 121-layer DenseNet [3] based model trained on the ChestX-ray14 [11] dataset comprising of 112,120 frontal-view chest X-Ray images. The model is trained to classify X-Ray images into 14 different thoracic disease classes, including pneumonia. Given the visual similarity of the input samples, we found this to be the closest pre-trained backbone to develop a model for identifying COVID-19 pneumonia.
Since the recent sudden surge of COVID-19 infections across the world, many alternative screening approaches have been developed to identify suspected cases of COVID-19. However there are only limited such open-source applications available for use [1, 5, 10] which use chest X-Ray images. Publicly available data on chest X-Rays for COVID-19 are also limited.
COVID-Net [10] is the only approach having an open source and activelymaintained tool which has ability to identify COVID-19 as well as other pneumonia while showing considerable sensitivity for COVID-19 detection. COVID-Net uses a machine-driven design exploration to learn the architecture design starting from initial design prototype and requirements. It takes as input a chest X-Ray image and outputs a prediction among three classes: Normal, Pneumonia and COVID-19. We treat this model as our baseline, comparing our results with it.
Given the limited amount of available X-Ray samples (including bacterial and viral pneumonia), it is hard to train a deep neural network from scratch. Hence we use a pre-trained backbone trained on a large dataset. Our approaches use the pre-trained model of CheXNet [7] , released by Weng et al. [12] .
We aim to classify a given frontal-view chest X-Ray image into the following classes: Normal, Bacterial Pneumonia, Viral Pneumonia and COVID-19. We have trained our model in two configurations, the one which classifies into the above four classes, and the other configuration with three classes (clubbing viral and bacterial pneumonia into one). The motivation behind the four class configuration is to better understand if any confusion between regular pneumonia and COVID-19 is due to the similarity of pathology between COVID-19 and viral pneumonia.
Similar to CheXNet [7] , we treat each class as a binary classification problem, with input as a frontal-view chest X-Ray image X and output being a binary 
Here, C is the number of classes and y is the ground truth for X. p c (ŷ = 1 | X; θ) and p c (ŷ = 0 | X; θ) are the probability scores for X being and not being in class c, respectively, as assigned by the network based on network weights θ. The two terms are weighted by w + c = Nc Nc+Pc and w − c = Pc Nc+Pc , where P c and N c are the number of positive and negative samples of class c, respectively in the training set.
Our model contains pre-trained CheXNet [7] , with a 121-layer Dense Convolutional Network (DenseNet) [3] backbone, followed by a fully connected layer. We replace CheXNet's [7] final classifier of 14 classes with our classification layer of 4 classes (3 classes for the clubbed pneumonia configuration), each with a sigmoid activation to produce the final output.
For training we initialize our model with pre-trained weights from CheXNet implementation by Weng et al. [12] , and then following the two stage training process described below: 1. In the first step, DenseNet's backbone weights are frozen and only the final fully connected layer is trained. Training is performed using Adam optimizer with following parameters: β 1 = 0.9, β 2 = 0.999, and learning rate 10 −4 . We use mini-batches of size 16, and train for about 30 epochs. The model with the lowest validation loss is selected for next stage. 2. In the second stage, the network weights are initialized from above, but the whole network is trained end-to-end (all layers), using the same hyperparameters. We use mini-batch size of 8 in this stage due to memory constraints, and train for 10 epochs. Again, the model with lowest validation loss is selected for testing.
We use the covid-chestxray-dataset [2] for COVID-19 frontal-view chest X-Ray images and chest-xray-pneumonia dataset [4] for frontal-view chest X-Ray images with bacterial/viral pneumonia as well as of normal lungs. We use the pre-trained CheXNet model, thus implicitly using robust features obtained after training on ChestX-ray14 [11] dataset. The covid-chestxray-dataset [2] does not contain proper data split for training purposes, so we perform our own split, as shown in Tables 1 and 2. Since multiple images for the same patient could be found in the dataset, we split the data by patient-IDs to prevent any information leakage. We choose 20% of the images as test set, and 10 images are kept as validation set. 
The combined dataset (covid-chestxray-dataset and chest-xray-pneumonia) has a high data imbalance due to scarce COVID-19 data. Note that this imbalance is different from the positive-negative class imbalance (for which w + and w − were introduced in the loss function). To ensure that training loss due to COVID-19 does not get masked by training loss due to other classes, we consider only a random subset of pneumonia data in each batch. The size of this subset should neither be too small, which will lead to overfitting on the COVID-19 data, nor too large to mask the COVID-19 loss, and is fixed empirically. In each batch we take data from classes Normal, Bacterial Pneumonia, Viral Pneumonia and COVID-19 in the ratio 5 : 5 : 5 : 1. In case of the three class classification network, this ratio is 7 : 7 : 1.
Our results indicate that this approach can lead to COVID-19 detection from X-Ray images with an AUROC (Area under ROC curve) of 0.9994 for the COVID-19 positive class, with a mean AUROC of 0.9738 (for 4-class classification configuration). Since we have modeled the problem as a binary classification problem for each class, given an input image X, we treat the class with maximum confidence score as the final prediction for calculating Accuracy, Sensitivity (Recall), PPV and confusion matrix. The confusion matrices on the test data for the two configurations of our model are shown in Fig. 4 . It can be seen that viral pneumonia, having a sensitivity (recall) value of 0.87 is often confused with bacterial pneumonia. This is likely due to the overlapping imaging characteristics. Our sensitivity (recall) for COVID-19 positive class is 1.0, which is at par with the sensitivity (recall) for bacterial pneumonia.
To compare our approach with COVID-Net [10] we evaluate their published pre-trained models on the same test data split as ours. Fig. 5 compares their ROC curves with our model along with the confusion matrix. The COVID-Net inference shown here is made using their 'Small' variant which seemed to perform better among their two variants. It can be seen that our model outperforms COVID-Net by >0.14 AUROC in detecting regular Pneumonia as well as COVID-19. It should be noted that we used a different Pneumonia dataset from that used by COVID-Net, however, the COVID-19 data used is the same. Table 5 .
The high margin of difference of the F1 scores of the two models clearly establish the superior performance of our model over COVID-Net.
To demonstrate the results qualitatively, we generate saliency maps for our model's predictions using RISE [6] . In this approach 1000 randomly masked versions of a given X-ray image are queried and their classification scores are used to create a weighted mask corresponding to each output class. The core idea behind the RISE [6] approach is that masks which preserve semantically important parts of the image will lead to a higher classification score and hence a higher weight in the final mask for the respective class. Red denotes region of greater importance.
The purpose of these visualizations was to have an additional check to rule out model over-fitting as well as to validate whether the regions of attention correspond to the right features from a radiologist's perspective. Fig. 6 shows some of the saliency maps on COVID-19 positive X-rays.
We have presented some initial results on detecting COVID-19 positive cases from chest X-Rays using a deep-learning model. We have demonstrated sig-nificant improvement in performance over COVID-Net [10] , the only publicly maintained tool for classification of COVID-19 positive X-rays, on the same chest-xray-pneumonia dataset [4] . The results look promising, though the size of the publicly available dataset is small. We plan to further validate our approach using larger COVID-19 X-ray image datasets and clinical trials.