## 3 Methods and Materials
### Combination of Deep Learning and SVM
#### 3.3.1 Deep Features Extraction  
The most important feature of deep learning models is that they have many layers for deep feature extraction. Deep learning techniques extract deep features during the training phase and check their performance when diagnosing new images (deep features) during the testing phase. As deep learning models have many layers for feature extraction, each layer has a specific task for deep feature extraction; for example, the first convolutional  
Figure 3: The set of X-ray images of tuberculosis after the enhancement process. (**a**) Shenzhen dataset. (**b**) TB Chest Radiography Database.  
layer extracts geometric features, the second layer is for edge detection, the third layer is for color features extraction, the fourth layer is for texture features, and so on [29].  
Deep learning models differ in the number of layers they contain. The most important layers in deep learning models that extract feature maps are convolutional layers, followed by auxiliary layers for the further processing of deep features and pooling layers to reduce dimensionality. In the following section, we will discuss each layer in detail.  
**Convolutional layers**: Convolutional layers are the backbone of deep learning models; the name \"convolutional layers\" derives from convolutional neural networks. Deep learning models are distinguished from one model to another according to the number of convolutional layers. Three main parameters control the efficiency of the convolutional layers' work: filter size, zero pad, and P-Step. The filter size \(f(t)\) controls the number of pixels that will be wrapped around the original image \(x(t)\) as in Equation (1) [30]. Zero padding is provided around the edges of the image to preserve the original image in its natural size. P-Step specifies how much the filter will jump over the image at a time. For example, if P-Step = 2, the filter will jump over the image by two steps during the warping process [31].  
\[y(t)=(x*f)(t)=\int x(a)f(t-a)\ da \tag{1}\]  
where \(y(t)\) represents the output,\(f(t)\) represents the filter and \(x(t)\) refers to the input.  
**Pooling Layer**: One of the most important layers in deep learning models that facilitate computational operations. Convolutional layers produce millions of parameters, which is challenging due to high computational costs. Therefore, pooling layers solve this challenge by reducing the dimensionality of high-dimensionality deep features. Deep learning models provide two techniques for dimensionality reduction, the max pooling technique and the average pooling technique [32]. The max pooling technique selects the groups of pixels according to the size of the max pooling filter [33]. It checks the value of each pixel in the selected group and replaces the selected pixels with the \(max\) value as in Equation (2). The average pooling technique selects the groups of pixels according to the size of the filter. It works by calculating the average of the selected pixels and replacing all the selected pixels with their average, as shown in Equation (3).  
\[z(i;\ j)=max_{m,n=1...k}f[(i-1)p+m;\ (j-1)p+n] \tag{2}\]  
\[z(i;\ j)=\frac{1}{k^{2}}\sum_{m,n=1...k}f[(i-1)p+m;\ (j-1)p+n] \tag{3}\]  
where \(m\) and \(n\) represent the sizes of the feature matrix, \(f\) refers to the filter pixels, \(p\) represents the step, and \(k\) refers to the feature matrix size.  
**Auxiliary layers:** Deep learning models have auxiliary layers such as the rectified linear units (ReLU) layer, which follows the convolutional layers. The main task of the ReLU layer is to pass the values of the positive features and convert the values of the negative features to zero. The dropout layer prevents overfitting; in this work, this layer was set to 50%, which means that the layer passes 50% of the features each time.  
The deep features were extracted from two chest X-ray datasets by the ResNet-50 [34] and GoogLeNet [35] models and sent to the PCA method for dimensionality reduction and stored in feature vectors.