## 5 Applying Recurrent Neural Network (RNN) for Gene Expression Data Classification  
The recurrent neural network (RNN) represents a neural network architecture designed to handle sequential data, including text, time series, and speech. The foundational concept of an RNN involves its ability to maintain connections to prior states, thereby enabling the model to preserve information about preceding sequence elements [10; 30; 31]. In general, the RNN architecture is composed of several pivotal components that facilitate the processing of sequential data and the uncovering of hidden dependencies in sequences, ultimately aiming to enhance the precision of identifying objects, the attributes of which are vectors of input data presented to the network:  
* Input layer: The recurrent neural network (RNN) receives a sequence of input data, which can be represented as a vector or a matrix. * Recurrent layer: The recurrent layer, being a pivotal component of an RNN, processes sequential data while retaining and updating a hidden state in each data processing step. It is noteworthy that a distinctive feature of RNNs is that the hidden state in step t encompasses information from both the preceding step t-1 and the current input signal. The recurrent layer applies an appropriate activation function to transform the combined input in each step. * Output layer: After the input sequence is processed through the recurrent layer, the final hidden state conveys the formulated information to the output layer to produce the desired outcome. Depending on the task, the output layer can have various structures. * Feedback among neurons of hidden layers: Upon obtaining the output, losses are computed by applying a loss function to compare the predicted output with the actual result. The error is then backpropagated to update the weight coefficients of the recurrent layer and optimize the model. Generally, the mathematical model of an RNN can be depicted as follows:  
\[h(t) =f_{act}^{h}(w_{ih}x(t)+w_{hh}f(t-1)) \tag{11}\] \[y(t) =f_{act}^{0}(w_{h0}f(t))\]  
where \(x(t)\) is the vector of input data; \(y(t)\) is the vector of output data (classes); \(f_{act}^{h}\) and \(f_{act}^{0}\) are the activation functions for the hidden and output layers, respectively; \(w_{ih}\), \(w_{hh}\), and \(w_{h0}\) are the weight coefficient matrices for the input layer to the first hidden layer, among hidden layers, and the last hidden layer to the output layer, respectively; \(h(t-1)\) and \(h(t)\) are the output values of the neurons in the hidden layers in steps \(t-1\) and \(t\), respectively. The primary drawback of a simple recurrent neural network (RNN) is the presence of the vanishing gradient problem, which complicates the processing of high-dimensional gene expression profiles for uncovering hidden patterns. To address this issue, more complex variants of RNNs have been developed, such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks, which incorporate specialized mechanisms for tackling the vanishing gradient problem and efficiently detecting hidden dependencies. Consequently, within the scope of the current research, LSTM and GRU RNNs are explored. It should be noted that compared with another type of neural network, likely a convolutional neural network (CNN), an RNN has a shorter list of hyperparameters, simplifying the formation of a list of optimal hyperparameters using grid search. The primary hyperparameters that determine the performance efficiency of RNNs include the following:  
* The number of recurrent (hidden) layers. * The number of neurons in the recurrent layers. * Activation functions for the recurrent and output layers. Typically, as with the previous type of network mentioned, the _softmax_ activation function is used for the neurons in the output layer. For the neurons in the recurrent layers, _sigmoid_, _tanh_, and _relu_ activation functions might be utilized.