# A Comparative Evaluation of Transformers and Deep Learning Models for Arabic Meter Classification
## 3 Materials and Methods
### Model Architectures
#### 3.3.2 Deep Learning Models  
Deep learning architectures such as BiLSTM and BiGRU are commonly used for handling sequential data, including natural language processing, time-series forecasting, and speech recognition. Both methods effectively capture long-terms dependencies and mitigate the vanishing gradient issue inherent in conventional RNNs. BiLSTM (Bidirectional Long Short-Term Memory) enhances contextual understanding by processing input sequences in both forward and reverse directions [14; 64]. The LSTM model includes three main gates: input, output, and forget gates. \[f_{t}=\sigma\Big{(}v_{f}I_{t}+w_{f}L_{t-1}+a_{f}\Big{)} \tag{1}\]  
where \(f_{t}\) is the forget gate at time stamp \(t\), \(\sigma\) is the sigmoid activation function, \(v_{f}\) and \(w_{f}\) denote the weight matrices for \(f_{t}\), \(I_{t}\) signifies the data input, \(a_{f}\) is the error vector, and \(L_{t-1}\) denotes the hidden state result from the previous time step. \[i_{t}=\sigma(v_{i}I_{t}+w_{i}L_{t-1}+a_{i}) \tag{2}\]  
Figure 8: Transformer model’s architecture. where \(i_{t}\) is the input gate, \(v_{i}\) and \(w_{i}\) denote the weight matrices for \(i_{t}\), and \(a_{i}\) is the error vector. \[\widetilde{CM}_{t}=tanh(vI_{t}+w_{c}L_{t-1}+a_{c}) \tag{3}\]  
where \(\widetilde{CM}_{t}\) is the candidate memory cell, \(a_{c}\) is the error vector, and \(v_{c}\) and \(w_{c}\) are the associated weight matrices. \[CS_{t}=f_{t}\cdot CS_{t-1}+i_{t}\cdot\widetilde{CM}_{t} \tag{4}\]  
where \(CS_{t}\) is the current cell state, \(CS_{t-1}\) represents the memory state from the previous timestamp, \(f_{t}\), \(i_{t}\), \(\widetilde{CM}_{t}\) is calculated from Equations (1), (2), (3), respectively. \[O_{t}=\sigma(v_{o}I_{t}+w_{o}L_{t-1}+a_{o}) \tag{5}\]  
where \(O_{t}\) is the output gate, \(v_{o}\) and \(w_{o}\) denote the weight matrices for \(O_{t}\), and \(a_{o}\) is the error vector. \[L_{t}=O_{t}\cdot\tanh(CS_{t}) \tag{6}\]  
where \(L_{t}\) is the hidden state at time \(O_{t}\), calculated from Equation (5), and \(CS_{t}\) from Equation (4). The BiLSTM analyzes the input sequence in both directions. \[L_{t}=\overrightarrow{L_{t}}\bigodot\ \overset{\leftarrow}{L_{t}} \tag{7}\]  
where the \(\overset{\rightarrow}{L_{t}}\) is the forward output and \(\overset{\leftarrow}{L_{t}}\) is the backward output of the LSTM. The symbol '\(\bigodot\)' denotes on operation such as multiplication, averaging, summation, or concatenation. In contrast, the BiGRU is a simplified variant of LSTM, with fewer parameters and enhanced computational efficiency. Like the BiLSTM, it processes sequences bidirectionally [26]. It includes two gates: the reset gate and the update gate. \[RS_{t}=\sigma(v_{r}I_{t}+w_{r}G_{t-1}+a_{r}) \tag{8}\]  
where \(RS_{t}\) is the reset gate, \(v_{r}\) and \(w_{r}\) are the weight matrices for \(RS_{t}\), \(G_{t-1}\) is the hidden state from the previous time step, and \(a_{r}\) is the bias vector. \[P_{t}=\sigma\big{(}v_{p}I_{t}+w_{p}G_{t-1}+a_{p}\big{)} \tag{9}\]  
where \(P_{t}\) is the update gate, \(v_{p}\) and \(w_{p}\) denote the weight matrices, and \(\text{a}_{\text{p}}\) is the error vector. \[i_{t}=\sigma(v_{i}I_{t}+w_{i}L_{t-1}+a_{i}) \tag{2}\]0  
where \(\widetilde{G}_{t}\) is the new hidden state, \(a_{h}\) is the error vector, and \(v_{h}\) and \(w_{h}\) are the weight matrices. \(RS_{t}\) is obtained from Equation (8). \[i_{t}=\sigma(v_{i}I_{t}+w_{i}L_{t-1}+a_{i}) \tag{2}\]1  
where \(G_{t}\) is the final updated hidden state, \(P_{t}\) is calculated from Equation (9), and \(\widetilde{G}_{t}\) is from Equation (10). \[i_{t}=\sigma(v_{i}I_{t}+w_{i}L_{t-1}+a_{i}) \tag{2}\]2  
where the \(\overset{\rightarrow}{G_{t}}\) is the forward output and \(\overset{\leftarrow}{G_{t}}\) is the backward output of the GRU. The symbol '\(\bigodot\)' denotes operations such as multiplication, averaging, summation, or concatenation.