# Pneumonia Detection on Chest X-ray Images Using Ensemble of Deep Convolutional Neural Networks
## 3 Methodology
### Deep Convolutional Neural Networks (DCNN) Models
#### 3.1.3 Vision Transformer (VIT)  
The VIT has successfully obtained perfect performance on different computer vision tasks, as discussed in [25]. The Vision Transformer (VIT) [23] divides an image into patches and uses a transformer to pattern the similarity among these patches as sequences, resulting in sufficient image classification performance. VIT's structure can be summarized as follows: (1) Divide the given image into patches. (2) Flatten patches and use these patches to produce lower-dimensional linear embeddings (Patch Embedding). (3) Add a class token and positional embedding. (4) Give the patch sequence to the transformer layer and use a class token to obtain the label. (5) To obtain the output prediction, transfer the class token values to the Multilayer Perception (MLP). Regarding inserting a 112 \(\times\) 112 image to generate the patch, we start with 16 \(\times\) 16 nonoverlapping and overlapping patches. Therefore, generating 49 patches becomes easy as well as inserting them into the linear projection layer, taking into account that each patch has three color channels. Further, the patches are loaded into the linear projection layer to achieve a long vector representation of each patch. The overall number of overlapping and nonoverlapping patches in the patch embedding is 49, and the patch size with the number of channels is 16 \(\times\) 16 \(\times\) 3. As a result, each patch's long vector is 768, and the patch embedding matrix is 49 \(\times\) 196. In addition, class tokens and position embedding have been added to the sequence of embedded patches. If positional encoding is not used, the transformer will not be able to retain the information. Due to the additional class token, patch embeddings are still sized 50. Lastly, the acquired representations of the class token are obtained by feeding patch embeddings with a positional encoding and a class token into the transformer layer. As a result, the transformer-encoding layer produces \(1\times 768\), which is then transferred to the MLP block to give an accurate prediction. The transformer encoder, which contains the Multihead Self-Attention (MHSA) block and the MLP block, is the most important element in the VIT structure. The encoded layer has 50 \(\times\) 768 as input, which this layer merged into patch embeddings, positional embeddings, and class tokens. In the VIT architecture, the previous layer's inputs and outputs for the 12 layers are 50 \(\times\) 768. Furthermore, the normalization layer normalizes the inputs before they are fed into the Multiheaded Attention (MHA) block. To obtain the query, key, and value matrix in MHA, the input data are adapted into a 50 \(\times\) 2304 (768 \(\times\) 3) shape using a linear layer. Then, these matrices are reshaped into 50 \(\times\) 3 \(\times\) 768, where each one is represented as 50 \(\times\) 768. These matrices are then reshaped once more to 12 \(\times\) 50 \(\times\) 64. Once these matrices are obtained, the attention process for the MHA block is performed using the following equation:  
\[\text{Attention(Query,Key,Value)}=\text{softmax}(\frac{\text{Query}\cdot \text{Key}^{\text{T}}}{\sqrt{\text{d}_{\text{Key}}}})\cdot\text{Value} \tag{1}\]  
The outputs from the MHSA block are delivered as an input to the skip connection. Then, the outcome of the skip connection is sent to the normalization layer before being delivered to the MLP block for processing. Due to significant advancements in VIT, MLP includes a local mechanism to understand local features [50]. Furthermore, depthwise convolution is integrated into the MLP block during the first fully connected layer to reduce parameters and achieve better results. The output of the MLP block must eventually feed the skip connection to achieve the encoder layer's output. In this paper, the vision transformer is used because it focuses on each independent patch of the image, as well as their relationships with other patches.