# A Lightweight Deep Learning-Based Pneumonia Detection Approach for Energy-Efficient Medical Systems

Authors: Bandar Almaslukh
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

Early detection of pneumonia disease can increase the survival rate of lung patients. Chest X-ray (CXR) images are the primarily means of detecting and diagnosing pneumonia. Detecting pneumonia from CXR images by a trained radiologist is a challenging task. It needs an automatic computer-aided diagnostic system to improve the accuracy of diagnosis. Developing a lightweight automatic pneumonia detection approach for energy-efficient medical systems plays an important role in improving the quality of healthcare with reduced costs and speedier response. Recent works have proposed to develop automated detection models using deep learning (DL) methods. However, the efficiency and effectiveness of these models need to be improved because they depend on the values of the models’ hyperparameters. Choosing suitable hyperparameter values is a critical task for constructing a lightweight and accurate model. In this paper, a lightweight DL approach is proposed using a pretrained DenseNet-121-based feature extraction method and a deep neural network- (DNN-) based method with a random search fine-tuning technique. The DenseNet-121 model is selected due to its ability to provide the best representation of lung features. The use of random search makes the tuning process faster and improves the efficiency and accuracy of the DNN model. An extensive set of experiments are conducted on a public dataset of CXR images using a set of evaluation metrics. The experiments show that the approach achieved 98.90% accuracy with an increase of 0.47% compared to the latest approach on the same dataset. Moreover, the experimental results demonstrate the approach that the average execution time for detection is very low, confirming its suitability for energy-efficient medical systems.


---

## Body

## 1. Introduction

One of the most important diseases prevalent all across the globe is pneumonia, which affects the lungs. Pneumonia is a deadly disease that causes the air sacs in the lungs to become filled with pus and liquid [1]. Pneumonia is divided into two important types, which are bacterial type and viral type. Bacterial pneumonia causes more severe symptoms [2]. Each type of pneumonia has a different treatment plan from the other. Bacterial pneumonia is treated using antibiotic therapy, while the other type of pneumonia usually does not need any antibiotic therapy and will get better on its own [3]. The main cause of pneumonia illness is the high level of pollution around the globe. It is ranked the eighth most common cause of death in the United States [4]. According to the results presented in [5], there are approximately one million older people and 800,000 children under five years who died from pneumonia in 2017. Pneumonia affects children and adults all around the world, but it is most common in South Asia and sub-Saharan Africa. Many steps can be followed to protect children from pneumonia, and these steps could be implemented with low-cost treatment, simple interventions, and low-tech care and medication [6].In most countries, elderly patients who have pneumonia will be overlooked and untreated until their health reaches a fatal point. According to studies, pneumonia is the single largest cause of death of children worldwide, especially affecting children who are under the age of five [7]. There is an urgent and important need for people to do more research to diagnose this type of disease using computer-aided diagnostic system to protect people from death, especially children.There are many available tests to diagnose pneumonia, such as chest X-rays (CXR), chest MRI, and needle biopsy of the lung [8]. The best and most commonly used of these methods is CXR, making an important contribution to clinical care and epidemiological studies [9–11]. Using the X-ray (XR) method is preferable to the CT method because CT imaging takes longer to perform than XR imaging. Moreover, CT imaging requires high-quality scanners and these types of scanners may not be available in many underdeveloped countries. In many regions around the world, there is still a problem of a shortage of healthcare workers and radiologists, yet they have a significant role in detecting and predicting pneumonia disease [12, 13]. Nowadays, the most popular method used to diagnose pneumonia disease is using artificial intelligence (AI) based on computers, mobile devices, cloud, and edges [14, 15]. The most advantageous point of this method is that it can be available to a large number of people at a minimal cost. However, developing a lightweight AI detection approach for energy-efficient medical systems is required for improving the quality of healthcare and reducing the cost of response. Radiologists find diagnosing pneumonia a challenge because some features of this disease are similar to other features of different diseases. Recent studies show that deep learning techniques are solving all these problems sometimes with a higher prediction accuracy rate than an average radiologist [16]. There are several deep learning techniques used to diagnose pneumonia, like biomedical image diagnosis and convolutional neural networks (CNNs). CNN techniques are the most adopted technique by the research community, and they have proved to be effective for both image classification and image segmentation. Another method of deep learning is the biomedical image diagnosis, which has been proven to come up with fast and accurate results of diagnosing the disease, matching the accuracy of radiologists [17]. Currently, deep learning techniques cannot fully replace medical diagnoses made by a trained clinician, but they are useful to supplement clinical decision-making.In this paper, a lightweight deep learning approach is presented based upon a transfer learning concept and a random search optimization technique. The approach consists of a pretrained DenseNet-121-based feature extraction method and a DNN-based method with a random search fine-tuning technique. The main contributions of this work are summarized as follows:(i)
To increase the size of existing training images and reduce the overfitting problem, a data augmentation step is applied using a set of effective and reasonable augmentation operations that represent actual CXR images(ii)
To extract the most important features of CXR images, a DenseNet-121 model is applied due to its ability to provide a best representation of lung features(iii)
To classify the extracted features, a custom DNN model is built using a set of fully connected layers and dropout layers. Dropout layers are used for reducing the overfitting problem(iv)
To tune the hyperparameters of the DNN model and improve its efficiency and accuracy, a random search technique is used by defining a specific range of values which depends on the acquired knowledge in the DL domain(v)
To evaluate the accuracy and efficiency of the approach, an extensive set of experiments on a public dataset is conducted with a comparison with current and recent related workThe remaining sections of the paper are arranged as follows: Section2 presents more details of other related work and focuses on the main contribution of the suggested work. Section 3 focuses on the materials and methods of the study, including a description of the dataset used, a background of pretrained convolutional neural networks, and a detailed explanation of the proposed approach. The experimental results of the dataset are presented in Section 4. Section 5 presents the conclusion and considers future research.


## 2. Related Work

Machine and deep learning methods have been used in most of various research fields [18–23]. Deep learning methods are considered the state-of-the-art techniques for classification and regression tasks. However, they need massive amounts of data, high processing power, and fewer resource requirements. Besides, energy efficiency and security of developed systems are also required for smart different applications to enable and facilitate their services [24–26].Medical healthcare systems are one type of these applications that need to be improved in terms of accuracy and efficiency. Therefore, several biomedical image detection techniques have been proposed by different authors starting with Razzak et al. [27], who discussed the issues and the prospect of medical preprocessing. Shen et al. [28] mentioned that much work has been done already in the field deep learning techniques to detect numerous diseases. One model of deep learning was proposed by Esteva et al. [29] for the dermatologist-level classification of skin cancer. Also, Milletari et al. [30] have presented a method using CNN to depict the prostate using MRI volumes. Grewal et al. [31] applied a deep learning technique to detect brain hemorrhage from CT scans, while Varun et al. [32] used a deep learning technique to detect diabetic retinopathy in retinal fundus photographs. Moreover, Bar et al. [33] proposed using deep learning to detect chest pathologies. There are many studies related to detecting disease using CXR that had been proposed using several examination techniques [34–36]. CXR images should pass through several evaluation processes to eliminate all other unwanted parts of the body to give the best results and eliminate the error rate in the diagnosis stage. Researchers such as Hermann [37] and Nasrullah et al. [38] described the algorithms used in their research. They used customized two deep three-dimensional (3D) combined with linked network (CMixNet) architectures to detect and classify lung nodules. Moreover, Yao et al. [39] used long short-term memory (LSTM) networks with DenseNet to utilize the overreliance within abnormalities.Several research papers described work on the classification of pneumonia. First of all, Khatri et al. [40] used the EMD (earth mover’s distance) method for detecting lungs that had been infected by pneumonia and detecting normal lungs, while Abiyev and Ma’aitah [41] and Stephen et al. [42] presented models for the classification of pneumonia using CNN. Several promising results have been discovered by Cohen et al. [43] and Rajaraman et al. [44]. One of the studies used customized CNNs to recognize pneumonia and to classify the types of pneumonia, such as virus or bacterial, in CXRs of pediatric pneumonia [44]. Moreover, Sirazitdinov et al. [45] focused on pneumonia identification to segment the photographs of pulmonary together with amplified photographs using region-based CNNs. Lakhani and Sundaram [46] focused on data augmentation to get an area under the curve of 0.94-0.95 without any need to pretrain using two types of neural networks, which are AlexNet and GoogLeNet. Deep CNN using CheXNeXt with 121 layers has been used to identify fourteen different pathologies, which include frontal-view CXRs of pneumonia [47]. Another approach discussed was to use DenseNet-121 that had been trained before using the extracted features to identify different fourteen thoracic diseases [48]. Some researchers, such as Saraiva et al. [49], Ayan and Ünver [50], and Rahman et al. [51], have applied deep learning techniques to classify types of pneumonia. Furthermore, researchers introduced a three-dimensional (3D) convolutional neural network (MSH-CNN) to develop a diverse multiscale novel that depends on CT images of the chest [52]. On the other hand, another study was focused on detecting pneumonia using the hierarchical structure of CNN and loss function novel, sin-loss [53]. Considering pneumonia identification, Jaiswal et al. [54] applied Mask-RCNN, with dropout and L2 regularization to utilize general and specific features to segment the pulmonary images, while Jung et al. [55] applied a shortcut connection of 3D deep CNN (3D DCNN). They used a voting rule to combine the output of different networks to find the final prediction result [56]. Toğaçar et al. [57] proposed to combine the features extracted from a different number of deep learning models to classify pneumonia. The accuracy produced by the approach was 96.84%.Hashmi et al. [58] proposed a weighted classifier-based approach that combines the prediction weights obtained from ResNet18, InceptionV3, Xception, MobileNetV3, and DenseNet-121 models. The authors stated that the final weighted classifier model achieved an accuracy of 98.43% on the test set. However, developing a lightweight and effective pneumonia detection approach for energy-efficient medical systems is still a challenging task that needs to be improved. The current deep learning-based pneumonia detection approaches have limitation in choosing suitable hyperparameter values for constructing a lightweight and accurate model.The research question of this work can be written as is it possible to develop a lightweight and effective automatic pneumonia detection approach using a pretrained DenseNet-121 model and a random search fine-tuning algorithm? To answer this research question, many experiments are conducted on a public dataset using different splitting procedures.


## 3. Materials and Methods

The materials and methods of the study consist of three parts: a description of the dataset used, background to the pretrained convolutional neural networks, and a detailed explanation of the proposed approach, which covers three main steps: data preprocessing and augmentation, model development, and model evaluation.


### 3.1. Dataset

The data sample used in this work is a public dataset of CXR images [59], which contained 5836 images. These were divided into two sets: a test set and a training set as presented in Table 1. Each image in the dataset has one of two labels: normal or pneumonia, and the pneumonia class have either a bacterial class label or a viral class label. CXR images were taken throughout the patients’ clinical care routine; not all cases had a bacterial and viral coinfection. Two expert physicians graded the dataset images’ diagnoses before they were used for training machine learning models. Moreover, a third expert evaluated the dataset images to check for any error in the grading task. The distribution of classes in the training and testing sets was highly imbalanced. Some samples were in the test set and not in the training set. Therefore, the training and test sets were merged, shuffled, and arranged randomly into 5136 images in the new training set and 700 images in the test set (see Table 2). Moreover, the pneumonia images (4273 samples) were also resampled and divided randomly into 3873 images as a training set and 400 images as a test set to classify bacterial and viral cases as shown in Table 3. Examples of dataset images for normal, bacterial pneumonia, and viral pneumonia cases are presented in Figure 1.Table 1
Number of normal and pneumonia images in the training and test sets of the original normal and pneumonia data split.
Class nameTraining setTesting setTotalNormal13492341583Pneumonia38833904273Total52326245856Table 2
Number of normal and pneumonia images in the training and test sets of the resampled normal and pneumonia data split.
Class nameTraining setTesting setTotalNormal12833001583Pneumonia38734004273Total51567005856Table 3
Number of bacterial and viral images in the training and test sets of the resampled bacterial and viral data split.
Classes nameTraining setTesting setTotalBacterial25632182781Viral13101821492Total38734004273Figure 1
Example of dataset images for normal, bacterial pneumonia, and viral pneumonia cases: (a) column of normal images, (b) column of bacterial pneumonia images, and (c) column of viral pneumonia images [59].
(a)(b)(c)


### 3.2. Convolutional Neural Networks (CNNs)

CNNs are useful tools in the field of computer vision to classify and recognize objects input from 2D and 3D images [60]. The typical CNN model architecture usually comprises some pairs of convolution and pooling layers. Some techniques, including batch normalization [61], dropout [62], and ResNet block [63], have been applied to improve the accuracy and efficiency of deep CNN models. Dropout is usually used to reduce the overfitting problem during the training phase. Batch normalization can help to accelerate the training process. For example, the model proposed in [61] can outperform the original model by achieving the same accuracy with fourteen times fewer training speeds. In [63], the authors proposed the ResNet block and found that smoothing the information propagation can be achieved by two critical steps: identity shortcut connections and identity after addition activations. They designed the model to propagate information directly within the entire network rather than one residual unit. Moreover, the model was trained more easily than the original ResNet model design [64].


### 3.3. Transfer Learning and Pretrained Models

Deep CNN models may have days or weeks of training on a vast dataset. Another way to shorten this process is to use model weights from pretrained models on a benchmark computer vision dataset, such as ImageNet. Developing a new model by integrating the pretrained model can solve a problem in different computer vision applications. In this subsection, the use of transfer learning when developing CNNs models will be explored. Transfer learning includes using the models trained on one problem as a starting point and applied to related issues.Transfer learning is a flexible technique that allows pretrained models to work directly as feature extraction or preprocessing and integrating entirely into new models. To understand transfer learning, Pan and Yang [65] used task domain-specific and marginal probabilities to propose a transfer learning framework. We suppose that the domain, D, can be defined as a two-element tuple that consists of feature space, S, with a marginal probability, PE, where E is a data example. Hereafter, mathematically, D can be computed as follows:
(1)D=S,PE.At this point,S is the space of all vectors, ei is the ith term vector equivalent to some documents, and E is a particular learning exampleE=e1,e2,⋯,en∈S.Given a domain (D) and the specific task (T) on that domain can be written as follows:
(2)T=L,PY∣E=L,fei,Y=y1,⋯,yn,yi∈L,where L is a space of labels and f is a classification function that can learn from the feature vectors with their labelei,yi. Therefore, to classify a label of unseen feature vector is as follows:
(3)fei=yi,where an unseen feature vector is en∈S and classified label is yi∈L.In pneumonia diagnosis, training a deep learning model on an insufficient dataset makes the results unreliable. Therefore, a useful pretrained CNN model is used to extract the features from CXR images.


### 3.4. Research Approach

The research approach is presented as a flow chart in Figure2. It consists of three main generic stages: data preprocessing and augmentation, model development, and model evaluation.Figure 2
Flow chart of the research approach stages.In the data preprocessing and augmentation stage, the training images are resized, normalized, and augmented. Resizing and normalization steps are used to make the input images suitable for the pretrained DenseNet-121 model. The augmentation step is also applied to increase the size of existing training images and reduce the overfitting problem. In the model development stage, an optimized custom DNN model will be built based on a pretrained DenseNet-121-based feature extraction method and a random search technique. The optimized custom DNN model uses a pretrained DenseNet-121-based feature extraction to extract significant features from CXR images, and uses a random search technique to fine-tune its hyperparameters and improve the accuracy of the detection task. To assess the experimental results and model performance, a set of metrics are defined and described in the model evaluation stage.


#### 3.4.1. Data Preprocessing and Augmentation

In this stage, each image in the training or testing step is resized to224×224 to be suitable for the defined input size of the DenseNet-121 architecture. Moreover, each image is also normalized to a standard scale between 0 and 1 for the respective neural network architecture. The CXR images are single-channel, grayscale images, and the pretrained DenseNet-121 architecture expects RGB images of a three-channel as inputs. Therefore, this issue is resolved by repeating the first channel two more times to make them suitable as inputs for the DenseNet-121-based feature extraction method.Data augmentation is also used to mitigate the problems of data imbalance and data unavailability, which makes the generalizations of learned DL models inadequate and ineffective. The objective of data augmentation is to increase the size of the training data samples to reduce the overfitting problem. In the data augmentation of the proposed approach, the CXR training images are generated using a set of operations that makes sense of real-world CXR images. The image operations used to expand the training set images are left and right rotation with different small degree values of less than 10, shifting left and right, zooming, and shear. The shifting, zooming, and shear have small values, in which the images generated are not too different from the original images. The other image operations, such as flipping, warping, or changing the brightness and contrast of images, are not applied because they do not represent actual CXR images anymore and maybe cause a problem in pneumonia detection. The strategy of augmenting data in this stage is that the images of the class with a small sample size are augmented more than the other class to help to balance the classes. In the dataset used, the normal (healthy) class against the pneumonia class and viral class against bacterial class are small sample sizes. Therefore, these two classes are augmented more than the other classes.


#### 3.4.2. Model Development

In this research work, an automated pneumonia model is developed using a pretrained DenseNet-121-based feature extraction method and a DNN-based method with a random search fine-tuning technique. The pretrained DenseNet-121 is used to extract the significant features from CXR images. It is selected due to its ability to provide the best representation of lung features and its capability of achieving a high accuracy detection rate for the diagnosis of pneumonia from CXR images [58]. The DNN-based method is implemented using a set of fully connected dense layers. The first layer (input layer of DNN) takes its inputs from the feature extraction method. The last layer is the output layer with a softmax activation function. The number of hidden layers and their neurons need to be set up with appropriate values. Besides, selecting the learning rate for training the DNN model should be tuned to achieve a high accuracy rate. The process of selecting DNN hyperparameters is called hypertuning or hyperparameter tuning. These hyperparameters are the variables that govern the model topology and its training procedure. They have constant values over the training procedure and influence the performance of the DNN model. To find the best values for the DNN hyperparameters to classify pneumonia and normal cases from the CXR images, a random search technique is adopted in the proposed approach. Figure 3 shows the flow chart of development steps of the model. The description of each step is given as follows:Figure 3
Flow chart of the model development steps.The inputs of the development stage are the preprocessed images, and the outputs are labels for either normal lungs or pneumonia, and the pneumonia case can be labeled bacterial pneumonia or viral pneumonia. The description of each step is given as follows:(i)
The pretrained DenseNet-121-based feature extraction methodIn this method, the pretrained DenseNet-121 is used as a feature extraction method. It feeds the fine-tuned DNN-based classification model of the significant features. The DenseNet-121 architecture has been proposed by Huang et al. [66], in which each layer is connected directly to every other layer in a feed-forward manner in each dense block. The feature maps of all preceding layers for any layer are treated as separate inputs of that layer, whereas the feature maps of any layer are passed as inputs to all subsequent layers of that layer. Figure 4 shows the DenseNet-121 architecture used in this method.Figure 4
The architecture of DenseNet-121 model.The number 121 represents the number of layers with the trainable weights, excluding the batch normalization layer. The additional four layers are an initial7×7 convolutional layer and three transitional layers. The weights of DenseNet-121 are loaded without including the classifier part, which is on top of the DenseNet-121 model. The output of the DenseNet-121-based feature extraction method is a volume shape of 7×7×1024, which is fed into a 2D global average pooling layer of the fine-tuned DNN-based feature classification. The idea behind the DenseNet-121 is to connect the layers that are close to the input and close to the output to improve the performance of CNNs [66]. The DenseNet model has a number of advantages, such as fewer parameters compared to other CNNs, which means the number of hyperparameters is considerably reduced. It does not need to learn unnecessary feature maps due to dense connectivity pattern. The feature maps of the network remain constant within each dense block, while the number of filters is changed between them. Therefore, the features are reused, and the vanishing gradient problem is mitigated.
(ii)
The fine-tuned DNN-based feature classification methodThis method is responsible for building a fine-tuned DNN model for classifying the CXR image of a patient as either normal or pneumonia, and the pneumonia case can be classified as either bacterial or viral. The input layer of the model is a 2D global average pooling that takes the bottleneck features extracted from the DenseNet-121 feature extractor. The output layer is a fully connected dense layer with a softmax activation function and two neurons for binary classification. A random search technique is used for model hyperparameter tuning. In this case, a search space of other hyperparameters is defined. Thus, the search ranges are initialized as1,5,0.01,0.00015, and 64,128,256,512,1024 for the number of fully connected dense hidden layers, learning rate, and number of neurons in each hidden layer, respectively. Empirically, the activation function of each hidden layer is set up with a RELU activation function, and the dropout layer after the first hidden layer is followed by a dropout layer with a dropout ratio of 0.5. Then, the second hidden layer is followed by a dropout layer with a dropout ratio of 0.2 to avoid the overfitting problem. An Adam optimizing algorithm is used to optimize the training process. The number of epochs is initially 150, and batch size is set to 100. The validation split is prepared with a 10% ratio from the training set during the training process of the model.


#### 3.4.3. Model Evaluation

For model evaluation, a number of different evaluation metrics are used to score the classification outcomes. The main evaluation metrics used for pneumonia classification from lung X-rays include the accuracy, recall (sensitivity), andF-score, which are computed using the following equations.The experimental results are measured based on four metrics. These metrics are theF1-score, accuracy, precision, and sensitivity, calculated as in the following:
(4)Accuracy=TP+TNTP+TN+FP+FN,RecallSensitivity=TPTP+FN,Precision=TPTP+FP,F1−score=2∗Precision∗RecallPrecision+Recall,where FN and FP represent the number of false negative samples and false positive samples. TP and TN represent the number of true negative samples and true positive samples.As well as the evaluation metrics given above, the execution time of feature extraction, training, and testing are also computed to measure the efficiency of the proposed approach.


## 3.1. Dataset

The data sample used in this work is a public dataset of CXR images [59], which contained 5836 images. These were divided into two sets: a test set and a training set as presented in Table 1. Each image in the dataset has one of two labels: normal or pneumonia, and the pneumonia class have either a bacterial class label or a viral class label. CXR images were taken throughout the patients’ clinical care routine; not all cases had a bacterial and viral coinfection. Two expert physicians graded the dataset images’ diagnoses before they were used for training machine learning models. Moreover, a third expert evaluated the dataset images to check for any error in the grading task. The distribution of classes in the training and testing sets was highly imbalanced. Some samples were in the test set and not in the training set. Therefore, the training and test sets were merged, shuffled, and arranged randomly into 5136 images in the new training set and 700 images in the test set (see Table 2). Moreover, the pneumonia images (4273 samples) were also resampled and divided randomly into 3873 images as a training set and 400 images as a test set to classify bacterial and viral cases as shown in Table 3. Examples of dataset images for normal, bacterial pneumonia, and viral pneumonia cases are presented in Figure 1.Table 1
Number of normal and pneumonia images in the training and test sets of the original normal and pneumonia data split.
Class nameTraining setTesting setTotalNormal13492341583Pneumonia38833904273Total52326245856Table 2
Number of normal and pneumonia images in the training and test sets of the resampled normal and pneumonia data split.
Class nameTraining setTesting setTotalNormal12833001583Pneumonia38734004273Total51567005856Table 3
Number of bacterial and viral images in the training and test sets of the resampled bacterial and viral data split.
Classes nameTraining setTesting setTotalBacterial25632182781Viral13101821492Total38734004273Figure 1
Example of dataset images for normal, bacterial pneumonia, and viral pneumonia cases: (a) column of normal images, (b) column of bacterial pneumonia images, and (c) column of viral pneumonia images [59].
(a)(b)(c)


## 3.2. Convolutional Neural Networks (CNNs)

CNNs are useful tools in the field of computer vision to classify and recognize objects input from 2D and 3D images [60]. The typical CNN model architecture usually comprises some pairs of convolution and pooling layers. Some techniques, including batch normalization [61], dropout [62], and ResNet block [63], have been applied to improve the accuracy and efficiency of deep CNN models. Dropout is usually used to reduce the overfitting problem during the training phase. Batch normalization can help to accelerate the training process. For example, the model proposed in [61] can outperform the original model by achieving the same accuracy with fourteen times fewer training speeds. In [63], the authors proposed the ResNet block and found that smoothing the information propagation can be achieved by two critical steps: identity shortcut connections and identity after addition activations. They designed the model to propagate information directly within the entire network rather than one residual unit. Moreover, the model was trained more easily than the original ResNet model design [64].


## 3.3. Transfer Learning and Pretrained Models

Deep CNN models may have days or weeks of training on a vast dataset. Another way to shorten this process is to use model weights from pretrained models on a benchmark computer vision dataset, such as ImageNet. Developing a new model by integrating the pretrained model can solve a problem in different computer vision applications. In this subsection, the use of transfer learning when developing CNNs models will be explored. Transfer learning includes using the models trained on one problem as a starting point and applied to related issues.Transfer learning is a flexible technique that allows pretrained models to work directly as feature extraction or preprocessing and integrating entirely into new models. To understand transfer learning, Pan and Yang [65] used task domain-specific and marginal probabilities to propose a transfer learning framework. We suppose that the domain, D, can be defined as a two-element tuple that consists of feature space, S, with a marginal probability, PE, where E is a data example. Hereafter, mathematically, D can be computed as follows:
(1)D=S,PE.At this point,S is the space of all vectors, ei is the ith term vector equivalent to some documents, and E is a particular learning exampleE=e1,e2,⋯,en∈S.Given a domain (D) and the specific task (T) on that domain can be written as follows:
(2)T=L,PY∣E=L,fei,Y=y1,⋯,yn,yi∈L,where L is a space of labels and f is a classification function that can learn from the feature vectors with their labelei,yi. Therefore, to classify a label of unseen feature vector is as follows:
(3)fei=yi,where an unseen feature vector is en∈S and classified label is yi∈L.In pneumonia diagnosis, training a deep learning model on an insufficient dataset makes the results unreliable. Therefore, a useful pretrained CNN model is used to extract the features from CXR images.


## 3.4. Research Approach

The research approach is presented as a flow chart in Figure2. It consists of three main generic stages: data preprocessing and augmentation, model development, and model evaluation.Figure 2
Flow chart of the research approach stages.In the data preprocessing and augmentation stage, the training images are resized, normalized, and augmented. Resizing and normalization steps are used to make the input images suitable for the pretrained DenseNet-121 model. The augmentation step is also applied to increase the size of existing training images and reduce the overfitting problem. In the model development stage, an optimized custom DNN model will be built based on a pretrained DenseNet-121-based feature extraction method and a random search technique. The optimized custom DNN model uses a pretrained DenseNet-121-based feature extraction to extract significant features from CXR images, and uses a random search technique to fine-tune its hyperparameters and improve the accuracy of the detection task. To assess the experimental results and model performance, a set of metrics are defined and described in the model evaluation stage.


### 3.4.1. Data Preprocessing and Augmentation

In this stage, each image in the training or testing step is resized to224×224 to be suitable for the defined input size of the DenseNet-121 architecture. Moreover, each image is also normalized to a standard scale between 0 and 1 for the respective neural network architecture. The CXR images are single-channel, grayscale images, and the pretrained DenseNet-121 architecture expects RGB images of a three-channel as inputs. Therefore, this issue is resolved by repeating the first channel two more times to make them suitable as inputs for the DenseNet-121-based feature extraction method.Data augmentation is also used to mitigate the problems of data imbalance and data unavailability, which makes the generalizations of learned DL models inadequate and ineffective. The objective of data augmentation is to increase the size of the training data samples to reduce the overfitting problem. In the data augmentation of the proposed approach, the CXR training images are generated using a set of operations that makes sense of real-world CXR images. The image operations used to expand the training set images are left and right rotation with different small degree values of less than 10, shifting left and right, zooming, and shear. The shifting, zooming, and shear have small values, in which the images generated are not too different from the original images. The other image operations, such as flipping, warping, or changing the brightness and contrast of images, are not applied because they do not represent actual CXR images anymore and maybe cause a problem in pneumonia detection. The strategy of augmenting data in this stage is that the images of the class with a small sample size are augmented more than the other class to help to balance the classes. In the dataset used, the normal (healthy) class against the pneumonia class and viral class against bacterial class are small sample sizes. Therefore, these two classes are augmented more than the other classes.


### 3.4.2. Model Development

In this research work, an automated pneumonia model is developed using a pretrained DenseNet-121-based feature extraction method and a DNN-based method with a random search fine-tuning technique. The pretrained DenseNet-121 is used to extract the significant features from CXR images. It is selected due to its ability to provide the best representation of lung features and its capability of achieving a high accuracy detection rate for the diagnosis of pneumonia from CXR images [58]. The DNN-based method is implemented using a set of fully connected dense layers. The first layer (input layer of DNN) takes its inputs from the feature extraction method. The last layer is the output layer with a softmax activation function. The number of hidden layers and their neurons need to be set up with appropriate values. Besides, selecting the learning rate for training the DNN model should be tuned to achieve a high accuracy rate. The process of selecting DNN hyperparameters is called hypertuning or hyperparameter tuning. These hyperparameters are the variables that govern the model topology and its training procedure. They have constant values over the training procedure and influence the performance of the DNN model. To find the best values for the DNN hyperparameters to classify pneumonia and normal cases from the CXR images, a random search technique is adopted in the proposed approach. Figure 3 shows the flow chart of development steps of the model. The description of each step is given as follows:Figure 3
Flow chart of the model development steps.The inputs of the development stage are the preprocessed images, and the outputs are labels for either normal lungs or pneumonia, and the pneumonia case can be labeled bacterial pneumonia or viral pneumonia. The description of each step is given as follows:(i)
The pretrained DenseNet-121-based feature extraction methodIn this method, the pretrained DenseNet-121 is used as a feature extraction method. It feeds the fine-tuned DNN-based classification model of the significant features. The DenseNet-121 architecture has been proposed by Huang et al. [66], in which each layer is connected directly to every other layer in a feed-forward manner in each dense block. The feature maps of all preceding layers for any layer are treated as separate inputs of that layer, whereas the feature maps of any layer are passed as inputs to all subsequent layers of that layer. Figure 4 shows the DenseNet-121 architecture used in this method.Figure 4
The architecture of DenseNet-121 model.The number 121 represents the number of layers with the trainable weights, excluding the batch normalization layer. The additional four layers are an initial7×7 convolutional layer and three transitional layers. The weights of DenseNet-121 are loaded without including the classifier part, which is on top of the DenseNet-121 model. The output of the DenseNet-121-based feature extraction method is a volume shape of 7×7×1024, which is fed into a 2D global average pooling layer of the fine-tuned DNN-based feature classification. The idea behind the DenseNet-121 is to connect the layers that are close to the input and close to the output to improve the performance of CNNs [66]. The DenseNet model has a number of advantages, such as fewer parameters compared to other CNNs, which means the number of hyperparameters is considerably reduced. It does not need to learn unnecessary feature maps due to dense connectivity pattern. The feature maps of the network remain constant within each dense block, while the number of filters is changed between them. Therefore, the features are reused, and the vanishing gradient problem is mitigated.
(ii)
The fine-tuned DNN-based feature classification methodThis method is responsible for building a fine-tuned DNN model for classifying the CXR image of a patient as either normal or pneumonia, and the pneumonia case can be classified as either bacterial or viral. The input layer of the model is a 2D global average pooling that takes the bottleneck features extracted from the DenseNet-121 feature extractor. The output layer is a fully connected dense layer with a softmax activation function and two neurons for binary classification. A random search technique is used for model hyperparameter tuning. In this case, a search space of other hyperparameters is defined. Thus, the search ranges are initialized as1,5,0.01,0.00015, and 64,128,256,512,1024 for the number of fully connected dense hidden layers, learning rate, and number of neurons in each hidden layer, respectively. Empirically, the activation function of each hidden layer is set up with a RELU activation function, and the dropout layer after the first hidden layer is followed by a dropout layer with a dropout ratio of 0.5. Then, the second hidden layer is followed by a dropout layer with a dropout ratio of 0.2 to avoid the overfitting problem. An Adam optimizing algorithm is used to optimize the training process. The number of epochs is initially 150, and batch size is set to 100. The validation split is prepared with a 10% ratio from the training set during the training process of the model.


### 3.4.3. Model Evaluation

For model evaluation, a number of different evaluation metrics are used to score the classification outcomes. The main evaluation metrics used for pneumonia classification from lung X-rays include the accuracy, recall (sensitivity), andF-score, which are computed using the following equations.The experimental results are measured based on four metrics. These metrics are theF1-score, accuracy, precision, and sensitivity, calculated as in the following:
(4)Accuracy=TP+TNTP+TN+FP+FN,RecallSensitivity=TPTP+FN,Precision=TPTP+FP,F1−score=2∗Precision∗RecallPrecision+Recall,where FN and FP represent the number of false negative samples and false positive samples. TP and TN represent the number of true negative samples and true positive samples.As well as the evaluation metrics given above, the execution time of feature extraction, training, and testing are also computed to measure the efficiency of the proposed approach.


## 3.4.1. Data Preprocessing and Augmentation

In this stage, each image in the training or testing step is resized to224×224 to be suitable for the defined input size of the DenseNet-121 architecture. Moreover, each image is also normalized to a standard scale between 0 and 1 for the respective neural network architecture. The CXR images are single-channel, grayscale images, and the pretrained DenseNet-121 architecture expects RGB images of a three-channel as inputs. Therefore, this issue is resolved by repeating the first channel two more times to make them suitable as inputs for the DenseNet-121-based feature extraction method.Data augmentation is also used to mitigate the problems of data imbalance and data unavailability, which makes the generalizations of learned DL models inadequate and ineffective. The objective of data augmentation is to increase the size of the training data samples to reduce the overfitting problem. In the data augmentation of the proposed approach, the CXR training images are generated using a set of operations that makes sense of real-world CXR images. The image operations used to expand the training set images are left and right rotation with different small degree values of less than 10, shifting left and right, zooming, and shear. The shifting, zooming, and shear have small values, in which the images generated are not too different from the original images. The other image operations, such as flipping, warping, or changing the brightness and contrast of images, are not applied because they do not represent actual CXR images anymore and maybe cause a problem in pneumonia detection. The strategy of augmenting data in this stage is that the images of the class with a small sample size are augmented more than the other class to help to balance the classes. In the dataset used, the normal (healthy) class against the pneumonia class and viral class against bacterial class are small sample sizes. Therefore, these two classes are augmented more than the other classes.


## 3.4.2. Model Development

In this research work, an automated pneumonia model is developed using a pretrained DenseNet-121-based feature extraction method and a DNN-based method with a random search fine-tuning technique. The pretrained DenseNet-121 is used to extract the significant features from CXR images. It is selected due to its ability to provide the best representation of lung features and its capability of achieving a high accuracy detection rate for the diagnosis of pneumonia from CXR images [58]. The DNN-based method is implemented using a set of fully connected dense layers. The first layer (input layer of DNN) takes its inputs from the feature extraction method. The last layer is the output layer with a softmax activation function. The number of hidden layers and their neurons need to be set up with appropriate values. Besides, selecting the learning rate for training the DNN model should be tuned to achieve a high accuracy rate. The process of selecting DNN hyperparameters is called hypertuning or hyperparameter tuning. These hyperparameters are the variables that govern the model topology and its training procedure. They have constant values over the training procedure and influence the performance of the DNN model. To find the best values for the DNN hyperparameters to classify pneumonia and normal cases from the CXR images, a random search technique is adopted in the proposed approach. Figure 3 shows the flow chart of development steps of the model. The description of each step is given as follows:Figure 3
Flow chart of the model development steps.The inputs of the development stage are the preprocessed images, and the outputs are labels for either normal lungs or pneumonia, and the pneumonia case can be labeled bacterial pneumonia or viral pneumonia. The description of each step is given as follows:(i)
The pretrained DenseNet-121-based feature extraction methodIn this method, the pretrained DenseNet-121 is used as a feature extraction method. It feeds the fine-tuned DNN-based classification model of the significant features. The DenseNet-121 architecture has been proposed by Huang et al. [66], in which each layer is connected directly to every other layer in a feed-forward manner in each dense block. The feature maps of all preceding layers for any layer are treated as separate inputs of that layer, whereas the feature maps of any layer are passed as inputs to all subsequent layers of that layer. Figure 4 shows the DenseNet-121 architecture used in this method.Figure 4
The architecture of DenseNet-121 model.The number 121 represents the number of layers with the trainable weights, excluding the batch normalization layer. The additional four layers are an initial7×7 convolutional layer and three transitional layers. The weights of DenseNet-121 are loaded without including the classifier part, which is on top of the DenseNet-121 model. The output of the DenseNet-121-based feature extraction method is a volume shape of 7×7×1024, which is fed into a 2D global average pooling layer of the fine-tuned DNN-based feature classification. The idea behind the DenseNet-121 is to connect the layers that are close to the input and close to the output to improve the performance of CNNs [66]. The DenseNet model has a number of advantages, such as fewer parameters compared to other CNNs, which means the number of hyperparameters is considerably reduced. It does not need to learn unnecessary feature maps due to dense connectivity pattern. The feature maps of the network remain constant within each dense block, while the number of filters is changed between them. Therefore, the features are reused, and the vanishing gradient problem is mitigated.
(ii)
The fine-tuned DNN-based feature classification methodThis method is responsible for building a fine-tuned DNN model for classifying the CXR image of a patient as either normal or pneumonia, and the pneumonia case can be classified as either bacterial or viral. The input layer of the model is a 2D global average pooling that takes the bottleneck features extracted from the DenseNet-121 feature extractor. The output layer is a fully connected dense layer with a softmax activation function and two neurons for binary classification. A random search technique is used for model hyperparameter tuning. In this case, a search space of other hyperparameters is defined. Thus, the search ranges are initialized as1,5,0.01,0.00015, and 64,128,256,512,1024 for the number of fully connected dense hidden layers, learning rate, and number of neurons in each hidden layer, respectively. Empirically, the activation function of each hidden layer is set up with a RELU activation function, and the dropout layer after the first hidden layer is followed by a dropout layer with a dropout ratio of 0.5. Then, the second hidden layer is followed by a dropout layer with a dropout ratio of 0.2 to avoid the overfitting problem. An Adam optimizing algorithm is used to optimize the training process. The number of epochs is initially 150, and batch size is set to 100. The validation split is prepared with a 10% ratio from the training set during the training process of the model.


## 3.4.3. Model Evaluation

For model evaluation, a number of different evaluation metrics are used to score the classification outcomes. The main evaluation metrics used for pneumonia classification from lung X-rays include the accuracy, recall (sensitivity), andF-score, which are computed using the following equations.The experimental results are measured based on four metrics. These metrics are theF1-score, accuracy, precision, and sensitivity, calculated as in the following:
(4)Accuracy=TP+TNTP+TN+FP+FN,RecallSensitivity=TPTP+FN,Precision=TPTP+FP,F1−score=2∗Precision∗RecallPrecision+Recall,where FN and FP represent the number of false negative samples and false positive samples. TP and TN represent the number of true negative samples and true positive samples.As well as the evaluation metrics given above, the execution time of feature extraction, training, and testing are also computed to measure the efficiency of the proposed approach.


## 4. Experiments and Discussion

The experiments of this study are performed on a laptop with an Intel Core i7-8750H CPU processor, 2.20 GHz, 32 GB RAM, and a 64-bit operating system, X64-based processor. A Python programming language was used to implement the experiments. The public dataset described in Subsection3.1 was employed to evaluate and compare the proposed approaches. Three experiments were conducted on this dataset using three different splitting procedures. The first experiment was performed on the default split by the owner of the dataset. The second experiment was executed using a splitting procedure done on the dataset after merging the training set and test set of the default split. The third experiment was conducted using a splitting procedure on pneumonia images to classify bacterial and viral cases. The three splitting procedures are described in Subsection 3.1. In all of the experiments, the training images in the three training sets are preprocessed and augmented using the first stage of the research approach described in Subsection 3.4.1. Figure 5 shows an example of output images after applying the different augmentation operations. Tables 4–6 demonstrate the number of images in the training sets of the three splitting procedures.Figure 5
Output images after applying data augmentation operations: (a), (c), (e), and (g) original images, (b) output image of shifting left operation, (d) output image of the rotation operation, (f) output image of the zooming operation, and (h) output image of the shear operation.
(a)(b)(c)(d)(e)(f)(g)(h)Table 4
Number of normal and pneumonia images in the augmented training set and original test sets of the original split of the dataset.
Class nameTraining setTesting setTotalNormal45832344817Pneumonia44883904878Total90716249695Table 5
Number of normal and pneumonia images in the augmented training set and original test sets of the second splitting procedure.
Class nameTraining setTesting setTotalNormal45833004883Pneumonia44884004888Total90717009771Table 6
Number of bacterial and viral images in the augmented training set and original test sets of the third splitting procedure.
Class nameTraining setTesting setTotalBacterial32102183428Viral29901823172Total62004006600To select the best values of the model hyperparameters, an initial experiment was performed using the splitting procedure of this work. Table7 presents the best values of the DNN hyperparameters based on the random search technique.Table 7
The best values of the DNN hyperparameters based on the random search technique.
HyperparameterSelected valueNumber of fully connected hidden layers3Number of neurons in the first fully connected hidden layer1024Number of neurons in the third fully connected hidden layer512Learning rate0.000125


### 4.1. Experimental Results

In this subsection, the experimental results are presented. The curves of training accuracy and validation accuracy for the three experiments are presented in Figures6–8. The confusion matrices that give the number of detected images in the test sets are shown in Figures 9–11.Figure 6
The progress of training accuracy and validation accuracy for the first experiment.Figure 7
The progress of training accuracy and validation accuracy for the second experiment.Figure 8
The progress of training accuracy and validation accuracy for the third experiment.Figure 9
Confusion matrix of detected images in the test set of the first experiment.Figure 10
Confusion matrix of detected images in the test set of the second experiment.Figure 11
Confusion matrix of detected images in the test set of the third experiment.From Figures6–8, we can see that the progress of training accuracy and validation accuracy of the second experiment is better than the other experiments. This means that the distribution of classes in the training and validation is more appropriate to train the model. Moreover, it confirms the efficacy of data augmentation and optimization methods to improve the training process and trained model. Figures 9–11 show the efficacy of the trained model to classify test data examples from the second experiment.From the confusion matrices, the results of evaluation metrics are computed. Tables8–10 present the experimental results of these evaluation metrics for three experiments.Table 8
Evaluation results of the first experiment.
Class namePrecisionRecallF1-scoreNormal0.9630.8850.922Pneumonia0.9340.9790.956Weighted avg.0.9450.9440.943Accuracy94.4%Table 9
Evaluation results of the second experiment.
Class namePrecisionRecallF1-scoreNormal0.9770.9970.987Pneumonia0.9970.9830.990Weighted avg.0.9890.9890.989Accuracy98.9%Table 10
Evaluation results of the third experiment.
Class namePrecisionRecallF1-scoreBacterial0.9630.9680.966Viral0.9610.9560.959Weighted avg.0.9620.9630.962Accuracy96.3%From Tables8–10, we can see that the DNN model achieves accuracies of 94.4% for the first experiment, 98.9% for the second experiment, and 96.3% for the third experiment. Also, these results confirm the effectiveness of the proposed approach.


### 4.2. Comparison with the State-of-the-Art Approaches

A comparison was made of the proposed approach with the state-of-the-art approaches and models. Table11 lists the accuracies obtained, along with the number of images used in the evaluation experiments. From Table 11, we can see that even though the research work undertaken by Hashmi et al. [58] achieves a high accuracy result compared to the other works, the proposed approach attains a competitive accuracy result and outperforms this work.Table 11
Comparison of results of current related work.
Related work (year)Approach/modelNo. of imagesAccuracyAyan and Ünver [50] (2019)VGG16585684.50%Abiyev and Ma’aitah [41] (2018)CNN100092.40%Cohen et al. [43] (2019)DenseNet-121523292.80%Stephen et al. [42] (2019)CNN585693.73%Saraiva et al. [49] (2019)CNN584094.40%Rajaraman et al. [44] (2018)Customized CNN585696.20%Chouhan et al. [56] (2020)Ensemble of pretrained CNNs523296.39%Toğaçar et al. [57] (2019)Combined features of pretrained AlexNet, VGG-16, and VGG-19 with mRMR feature selection algorithm584996.84%Rahman et al. [51] (2020)Transfer learning with deep CNN524798.00%Hashmi et al. [58] (2020)Combined the weighted predictions from ResNet18, Xception, InceptionV3, DenseNet121, and MobileNetV3585698.43%This work (2021)Pretrained DenseNet-121+ optimized DNN using random search algorithm585698.90%In summary, from the results of the three experiments, the approach used in this study is effective for classifying and detecting pneumonia cases from CXR images.


### 4.3. Efficiency of Proposed Approach

In this subsection, the efficiency of the proposed approach is investigated based on the running time of its different stages. For the preprocessing stage, the average execution time taken to resize and normalize 6600 images is 60.31 seconds. The model development stage takes an average execution time 34.15 minutes to extract the bottleneck features from 6200 training images and 600 seconds of average execution time to train the DNN model on the bottleneck features of training images. For the detection task, each input image takes an average execution time of 0.33 for feature extraction and 0.001564 seconds for feature classification. Figure12 shows the time efficiency of the detection task as the number of input images of fixed size grows.Figure 12
Average execution time of detection task in seconds at different numbers of input images.In real scenarios of lightweight approaches, it is necessary to fulfill constraints, such as low execution time, low training time, and low energy usage. Because the proposed approach uses a transfer learning-based pretrained DenseNet-121 feature extraction method to train the DNN model, the training time is low compared to training a CNN model from scratch.Moreover, as shown in Figure12, the average execution time of the detection task seems to be near to linear time. Linear running time is related to the processing time of the method to process every element in the input. However, it is clear that the execution time required to detect a pneumonia disease from a CXR image is very low. The fact that the energy consumption of systems normally depends on their execution time, the proposed detection method is suitable and lightweight for energy-efficient medical systems.


## 4.1. Experimental Results

In this subsection, the experimental results are presented. The curves of training accuracy and validation accuracy for the three experiments are presented in Figures6–8. The confusion matrices that give the number of detected images in the test sets are shown in Figures 9–11.Figure 6
The progress of training accuracy and validation accuracy for the first experiment.Figure 7
The progress of training accuracy and validation accuracy for the second experiment.Figure 8
The progress of training accuracy and validation accuracy for the third experiment.Figure 9
Confusion matrix of detected images in the test set of the first experiment.Figure 10
Confusion matrix of detected images in the test set of the second experiment.Figure 11
Confusion matrix of detected images in the test set of the third experiment.From Figures6–8, we can see that the progress of training accuracy and validation accuracy of the second experiment is better than the other experiments. This means that the distribution of classes in the training and validation is more appropriate to train the model. Moreover, it confirms the efficacy of data augmentation and optimization methods to improve the training process and trained model. Figures 9–11 show the efficacy of the trained model to classify test data examples from the second experiment.From the confusion matrices, the results of evaluation metrics are computed. Tables8–10 present the experimental results of these evaluation metrics for three experiments.Table 8
Evaluation results of the first experiment.
Class namePrecisionRecallF1-scoreNormal0.9630.8850.922Pneumonia0.9340.9790.956Weighted avg.0.9450.9440.943Accuracy94.4%Table 9
Evaluation results of the second experiment.
Class namePrecisionRecallF1-scoreNormal0.9770.9970.987Pneumonia0.9970.9830.990Weighted avg.0.9890.9890.989Accuracy98.9%Table 10
Evaluation results of the third experiment.
Class namePrecisionRecallF1-scoreBacterial0.9630.9680.966Viral0.9610.9560.959Weighted avg.0.9620.9630.962Accuracy96.3%From Tables8–10, we can see that the DNN model achieves accuracies of 94.4% for the first experiment, 98.9% for the second experiment, and 96.3% for the third experiment. Also, these results confirm the effectiveness of the proposed approach.


## 4.2. Comparison with the State-of-the-Art Approaches

A comparison was made of the proposed approach with the state-of-the-art approaches and models. Table11 lists the accuracies obtained, along with the number of images used in the evaluation experiments. From Table 11, we can see that even though the research work undertaken by Hashmi et al. [58] achieves a high accuracy result compared to the other works, the proposed approach attains a competitive accuracy result and outperforms this work.Table 11
Comparison of results of current related work.
Related work (year)Approach/modelNo. of imagesAccuracyAyan and Ünver [50] (2019)VGG16585684.50%Abiyev and Ma’aitah [41] (2018)CNN100092.40%Cohen et al. [43] (2019)DenseNet-121523292.80%Stephen et al. [42] (2019)CNN585693.73%Saraiva et al. [49] (2019)CNN584094.40%Rajaraman et al. [44] (2018)Customized CNN585696.20%Chouhan et al. [56] (2020)Ensemble of pretrained CNNs523296.39%Toğaçar et al. [57] (2019)Combined features of pretrained AlexNet, VGG-16, and VGG-19 with mRMR feature selection algorithm584996.84%Rahman et al. [51] (2020)Transfer learning with deep CNN524798.00%Hashmi et al. [58] (2020)Combined the weighted predictions from ResNet18, Xception, InceptionV3, DenseNet121, and MobileNetV3585698.43%This work (2021)Pretrained DenseNet-121+ optimized DNN using random search algorithm585698.90%In summary, from the results of the three experiments, the approach used in this study is effective for classifying and detecting pneumonia cases from CXR images.


## 4.3. Efficiency of Proposed Approach

In this subsection, the efficiency of the proposed approach is investigated based on the running time of its different stages. For the preprocessing stage, the average execution time taken to resize and normalize 6600 images is 60.31 seconds. The model development stage takes an average execution time 34.15 minutes to extract the bottleneck features from 6200 training images and 600 seconds of average execution time to train the DNN model on the bottleneck features of training images. For the detection task, each input image takes an average execution time of 0.33 for feature extraction and 0.001564 seconds for feature classification. Figure12 shows the time efficiency of the detection task as the number of input images of fixed size grows.Figure 12
Average execution time of detection task in seconds at different numbers of input images.In real scenarios of lightweight approaches, it is necessary to fulfill constraints, such as low execution time, low training time, and low energy usage. Because the proposed approach uses a transfer learning-based pretrained DenseNet-121 feature extraction method to train the DNN model, the training time is low compared to training a CNN model from scratch.Moreover, as shown in Figure12, the average execution time of the detection task seems to be near to linear time. Linear running time is related to the processing time of the method to process every element in the input. However, it is clear that the execution time required to detect a pneumonia disease from a CXR image is very low. The fact that the energy consumption of systems normally depends on their execution time, the proposed detection method is suitable and lightweight for energy-efficient medical systems.


## 5. Conclusion and Future Work

Early detection of pneumonia increases the survival rate of patients. Automatic pneumonia detection from CXR images can be achieved using a CAD application. It is one of the medical systems that help to train radiologists to improve their accuracy of diagnosing pneumonia.Developing a lightweight approach for the automatic detection of pneumonia in energy-efficient medical systems is important to improve the quality of healthcare with reduced cost and time response. In this paper, a lightweight DL approach is presented using a pretrained DenseNet-121-based feature extraction method and DNN-based feature classification method with a random search fine-tuning technique. The DenseNet-121 model is used as a feature extraction method due to its ability to provide the best representation of lung features. The use of a random search facility optimizes the efficiency and accuracy of the DNN model.A number of experiments were performed on a dataset of CXR images and evaluated by a set of metrics. The results of the evaluation metrics demonstrate that the approach achieved 98.90% accuracy, which is higher than the state-of-the-art techniques and methods. Moreover, the time execution results show that the detection time of the approach is very low, which makes it a lightweight pneumonia detection approach, suitable for applying in energy-efficient medical systems. All results confirmed the effectiveness and efficiency of the proposed approach.In future work, a large-sized dataset of different lung diseases will be collected to train the model to improve the healthcare sectors. Moreover, an energy evaluation measure will be used to compute a real result of the energy consumption of the proposed approach.



---

*Source: 5556635-2021-04-22.xml*