## 3 Methodology
### Feature Extraction Module
#### 3.3.2 CBAM Attention-Based CNN Feature Extraction  
In our work, we introduced a CBAM attention-based module within CNN so that the model can concentrate on particular features that carry essential information about gestures. Figure 4: Processing of sign image and corresponding HOG output. The architecture of the CBAM module used is shown in Figure 5. This module proposed by [PERSON] et al. [25] aims to efficiently enhance feature extraction and representation. We can represent the intermediate feature map as \(F\in\mathbb{R}^{C\times H\times B}\) where, \(C\times H\times B\) is the input feature map resolution. In the channel attention block, the channel carrying meaningful information is decided based on inter-channel relationships. Two pooling functions, global average pooling (GAP) and max pooling, individually squeeze out the spatially extracted features from the intermediate feature vector. Then, these features are fed to a multilayer perceptron (MLP) and merged. Then, the sigmoid function activates the sigmoid feature map. The channel attention module gives a 1D attention map written as \(M_{c}\in\mathbb{R}^{C\times 1\times 1}\). Then, we will obtain a refined input feature \(F^{\prime}\) by multiplying this attention map with the original output. Mathematically, the process of the channel attention module is shown in the following equations. \[M_{c}(F)=f_{sigmoid}(MLP(GlobalAvgPool(F))+MLP(MaxPool(F))) \tag{4}\]  
\[F^{\prime}=M_{c}(F)\otimes F \tag{5}\]  
The spatial attention block concentrates on the inter-spatial relation of the input feature map and it is fed with this refined feature vector and two intermediate feature maps are generated using GAP and max pooling that are concatenated. The fused feature map is passed through a 7 \(\times\) 7 kernel and sigmoid function to generate a spatial feature map with attention weight \(M_{S}\in\mathbb{R}^{1\times H\times w}\) and results in a more efficient feature representation by multiplying the spatial feature map with the refined input as given in Equations (6) and (7). \[M_{s}(F)=f_{sigmoid}\bigg{(}Conv\bigg{(} \begin{bmatrix}GlobalAvgPool(F);\\ MaxPool(F)\end{bmatrix} \bigg{)}\bigg{)} \tag{6}\]  
Figure 5: Architecture of CBAM mechanism. \[F^{\prime\prime}=M_{\rm s}\big{(}F^{\prime}\big{)}\otimes F^{\prime} \tag{7}\]  
where the symbol \"\(\otimes\)\" represents elementwise multiplication, and \(F^{\prime\prime}\) is the final refined feature map [26, 27]. In this study, we employed a CBAM attention-based module that consists of three CNN convolution blocks to generate attention-based feature maps from the input dataset. Figure 6 demonstrates the flowchart of the CBAM-CNN model. The convolutional layer is the initial layer in each block, and each convolution layer generates features and gives a feature map. Then, we call the CBAM attention module to extract channel and spatial data. Each convolution block has batch normalization and max pooling layers. Also, we used equal padding and the rectified linear unit (ReLU) activation function within each block. We used 32, 64, and 128 filters in the CNN model's first, second, and third stages. Finally, the model results in a 1D feature vector after flattening the 3D feature map obtained after the last layer of max pooling. The flattened 1D vector that contains information on image-based features extracted from convolutional layers is expressed by Equation (8). \[F_{\text{CNN}}\ =\{f_{\text{CNN1}},\ f_{\text{CNN2}},\ f_{\text{CNN3}},\dots \dots\dots f_{\text{CNNnn}}\} \tag{8}\]  
where m is the number of features. Then, the CNN features are fused with the HOG-based feature vector to obtain the training feature vector by Equation (9). \[F_{concat}=F_{\text{CNN}}\ \oplus\ F_{\text{HOG}} \tag{9}\]  
There are two fully connected dense layers in the network. The first layer, with 300 neurons, takes the combined feature vector as input and performs a linear transformation, and the second layer, with 150 neurons, further processes the previous layer's output.