## 2 Our Proposal
### Dynamic MSL Signs Classifier  
After selecting the best classifier for the 26 static letters of the alphabet, we proceed with the design of classifiers for the dynamic signs \"k\", \"rr\", and \"ll\". Developing this component requires a combination of expertise in MSL and computational skills, as the double letters \"rr\" and \"ll\" share the same hand configuration as \"r\" and \"ll\", respectively, but include distinctive horizontal movements. Similarly, the letter \"k\" shares the same hand configuration as \"p\", with the addition of a vertical movement. Thus, accurate classification of the static sign is a prerequisite for correctly identifying these dynamic signs. In scenarios where a deaf person communicates by spelling with a hearing person who has learned a limited set of signs, including these dynamic letters extends the range of words that can be expressed. This, in turn, facilitates comprehension for hearing individuals and potentially encourages more interactions with the deaf community. The process for classifying the test dataset follows the flow shown in Figure 8. We begin by classifying the dataset 11 using the model selected in step 3, generating a list of classifications for each frame. Each classification is encapsulated in a \"hand object\" that includes the following attributes: \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Epochs & 20 \\ Learning rate & \(1\times 10^{-3}\) \\ Optimizer & ADAM \\ Loss function & Categorical Cross-Entropy \\ \hline \hline \end{tabular}
\end{table} 
Table 6: CNN hyperparameters. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer** & **Neurons** & **Activation Function** & **Kernel Size** \\ \hline Convolutional & 64 & ReLU & \(3\times 3\) \\ Max Pooling & N/A & N/A & \(2\times 2\) \\ Convolutional & 32 & ReLU & \(3\times 3\) \\ Max Pooling & N/A & N/A & \(2\times 2\) \\ Convolutional & 16 & ReLU & \(3\times 3\) \\ Max Pooling & N/A & N/A & \(2\times 2\) \\ Flatten & N/A & N/A & N/A \\ Fully Connected & 64 & ReLU & N/A \\ Fully Connected & 32 & ReLU & N/A \\ Fully Connected & 32 & ReLU & N/A \\ Output & 26 & Softmax & N/A \\ \hline \hline \end{tabular}
\end{table} 
Table 5: CNN architecture. * Sign classification: The corresponding alphabet letter for the detected sign. * Confidence value: The probability from the softmax layer, indicating the reliability of the model's detection. * Coordinates \((x,y)\): The position obtained via the MediaPipe API. * Bounding box: The rectangular area around the hand's position in the image, specified by the \((x,y)\) coordinates of the upper left corner and the bounding box's width and height. We perform classification on a frame-by-frame basis. There is one hand object per image for static letters, while for video sequences, there is one hand object per frame. To classify the \"k\" sign (detailed in Algorithm 2) and the double letters \"rr\" and \"ll\" (described in Algorithm 3), we use sliding windows of five consecutive hand objects in steps 4 and 4, respectively. Each sliding window produces a single output, which is the classification with the highest confidence value among the five frames. The total number of windows varies depending on the number of frames in each video of the dynamic letter dataset. Once processing of a video is completed (end of step 4), a list containing the classification results for each window is obtained. However, the size of this list may vary; step 4, then selects the final classification, which corresponds to the result with the highest confidence value. For static signs, the sliding window contains only one frame, and the classification result proceeds directly to step 4. As illustrated in Figure 5, the sign for the letter \"k\" shares the initial hand posture of the letter \"p\" but includes a distinct vertical movement to differentiate it.