# Sign Language Interpreting System Using Recursive Neural Networks
## 3 The Proposed System
### Proposed Neural Network Structures  
The next building block of the proposed system is the RNN. Since there are several RNN types that have proven to be effective in this kind of application, we designed some variations of the proposed system for different configurations utilizing the following types of RNN: the LSTM, the Bidirectional LSTM, and the Gated Recurrent Unit (GRU).  
We implemented four configurations, two based on LSTM, one in bidirectional LSTM, and one more based on the GRU. Table 1 details the layered architecture for each case.  
We summarize the main hyperparameters of the structures we planned in two sets. The first one is in Table 2, which contains a set of hyperparameters we used in common in all cases we explored, such as the learning rate, number of batches, and other relevant hyperparameters. The second is in Table 3, which summarizes the set of particular hyperparameters; for example, the loss function. Given the nature of the model training, we use categorical cross-entropy and the mean square error (MSE) as loss functions, defined in Equations (1) and (2), respectively. Finally, we utilize the adaptive moment estimation (Adam) optimizer defined in Equation (3).  
\[CCE=-\sum_{i=1}^{N_{\text{C}}}y_{i}\cdot log(\hat{y}_{i}), \tag{1}\]  
where:  
* \(CCE\) : Categorical Crossentropy.
* \(N_{\text{C}}\): Number of classes.
* \(y_{i}\): Ground truth value (0 or 1) for the \(i\)-th class.
* \(\hat{y}_{i}\): Predicted value for the \(i\)-th class.  
\[MSE=\frac{1}{N}\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^{2}, \tag{2}\]  
where:  
* \(MSE\) : Mean Squared Error.
* \(N\): Total number of observations or data.
* \(y_{i}\): True value of the \(i\)-th observation.
* \(\hat{y}_{i}\): Estimated value for the \(i\)-th observation.  
\[\theta_{n+1}=\theta_{n}-\frac{\alpha}{\sqrt{\hat{v}_{n}+\epsilon}}\widehat{m} _{n}, \tag{3}\]  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Layer Number** & **Model 1** & **Model 2** & **Model 3** & **Model 4** \\ \hline
1 & LSTM & LSTM &
\begin{tabular}{c} Bidirectional \\ LSTM \\ \end{tabular} & GRU \\ \hline
2 & LSTM & LSTM &
\begin{tabular}{c} Bidirectional \\ LSTM \\ \end{tabular} & GRU \\ \hline
3 & LSTM & LSTM & Dense & GRU \\ \hline
4 & ReLU & LSTM & Dense & ReLU \\ \hline
5 & Dense & ReLU & ReLU & Dense \\ \hline
6 & Dense & Dense & Dense  
\[m_{n}=\beta_{1}*m_{n-1}+(1-\beta_{1})*g_{n},\]  
\[v_{n}=\beta_{2}*v_{n-1}+(1-\beta_{2})*g_{n^{2}},\]  
where:  
* \(\theta_{n}:\) Optimizer parameter at time step \(n\).
* \(\alpha\): Learning rate.
* \(\widehat{v}_{n}\): Magnitude of the gradient.
* \(\epsilon\): A small value to avoid division by zero.
* \(\hat{m}_{n}:\) First-order momentum at time step \(n\).
* \(\beta_{1}\): Hyperparameter between 0 and 1 that controls the exponential decay rate.
* \(g_{n}\): Gradient calculated at time step \(n\).
* \(v_{n}:\) Second-order momentum at time step \(n\).
* \(\beta_{2}\): Exponential decay rate of the second-order momentum.
* \(g_{n^{2}}\): The square gradient at time step \(n\).  
In this study, we call the structures in Table 1 the \"base models\" that we used as a reference to explore. This exploration involved testing and evaluating the performance of the proposed system for different numbers of neurons for the layers for each model, and thus, we discuss details in this regard in the next section where we explain the experiments we conducted.