## 5 Applying Recurrent Neural Network (RNN) for Gene Expression Data Classification
Results from preliminary modeling indicated that when using both models of RNNs (LSTM and GRU), the hyperbolic tangent (_tanh_) activation function is substantially more effective than the _relu_ and _sigmoid_ activation functions, based on sample classification criteria that comprise the experimental database. Therefore, the modeling process envisaged optimizing two RNN hyperparameters: the number of neurons in the recurrent layers and the number of recurrent layers. The procedure for forming the RNN optimal hyperparameter vector was carried out according to an algorithm, the implementation of which involves the following stages. **Stage I. **: Data formation and algorithm parameter adjustment. 1. Presenting the gene expression data as a matrix \(E=(e_{ij})_{n\times m}\), where \(n\) is the number of rows or samples under investigation; \(m\) is the number of genes, the expression values of which determine the state of the respective samples. 1. Forming the list of hyperparameters for optimization, their range, and step of change during the algorithm operation: \(layers=\overline{1,3}\) (number of neural layers in the recurrent layer of RNN); \(k=30,\ldots,80,\ dk=5\) (range and step change of the number of neurons in recurrent layers). 1. Dividing the set of gene expression data samples into two subsets in a 0.7/0.3 ratio, where the first subset, \(E_{train}\), is used for model training and the second one, \(E_{test}\), for testing. 1. Further splitting training subset \(E_{train}\) into two subsets in a 0.8/0.2 ratio, where the first subset, \(E^{\prime}_{train}\), is directly used for training and the second one, \(E_{valid}\), for model validation during training. Ensuring that the model does not overfit is controlled by monitoring the convergence nature of classification accuracy and the loss function values, calculated on training and validation subsets during model training. **Stage II. **: Algorithm operation within the hyperparameter adjustment range. 1. Initializing the number of neural layers in the recurrent layer: \(levels=1\). 2. Initializing the starting value of the number of neurons in the recurrent layers: \(k=30\). 2. Model training. At each training step, calculating the classification accuracy and the loss function value on the data subsets for training and validation. 2. Testing the model on the test data subset. Calculating the samples' classification accuracy, F1-score for each class. 2. If \(k<k_{max}\), increasing the number of neurons in the recurrent layers by 5 (k = k + 5), and escaping to step 2.3 of this procedure. Otherwise, calculating the integrated F1-score value; analyzing the obtained results; and forming the optimal decision regarding the number of neurons in the recurrent layers in this stage. 2. If the number of recurrent layers is less than the maximum number (\(layers<layers_{max}\)), increasing the number of layers by 1, and going to step 2.2 of this algorithm. Otherwise, proceeding to Stage III. **Stage III. **: Analysis of the obtained results and formulation of an optimal solution. Comparative analysis of the solutions obtained in the previous algorithm operation stage. Forming the optimal decision regarding the hyperparameter vector for the corresponding type of RNN.