## 3 Results  
The GA took about 70 days (approximately 5 h to train and evaluate a single model architecture) to complete the eleven generations (ten generations plus generation zero). The optimization process is a considerably slow process, as shown in Figure 4. However, as the generations advance, the population converges into more stable architectures, causing the populations of the last generations to be trained about 1.6 times faster. Figure 5 shows how the PM of the best chromosome increases along with the generations sideways with the reduction in diversity for the new generations. The best chromosome emerged in the seventh generation, while in the eighth, it registered an increase in diversity, which suggests there is potential for further optimization, as unseen architectures \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Hyperparameter** & **Locus** & \multicolumn{2}{c}{**Definition**} \\ \hline Direction of LSTM layers & 0 & 0: Unidirectional & 1: Bidirectional \\ \hline Number of stacked LSTM layers & 1–2 & 00: 1 & 01: 2 \\  & & 10: 3 & 11: 4 \\ \hline Position of self-attention layer & 3–4 & 00: No & 01: Before \\  & & 10: After & 11: Both \\ \hline Model dimension & 5–6 & 00: 300 & 01: 512 \\  & & 10: 768 & 11: 1024 \\ \hline Heads dimension & 7–8 & 00: 32 & 01: 64 \\  & & 10: 128 & 11: 256 \\ \hline Dropout percentage & 9–11 & 000: 5 & 010: 15 \\  & & 100: 25 & 110: 35 \\  & & 001: 10 & 011: 20 \\  & & 101: 30 & 111: 40 \\ \hline Operation before classification layer & 12–13 & 00: FFNN & 01: Pooling \\  & & 10: LSTM & 11: BiLSTM \\ \hline Classification layer shape factor & 14–15 & 00: 0.0 & 01: 0.5 \\  & & 10: 1.0 & 11: 1.5 \\ \hline Activation function & 16–17 & 00: ReLU & 01: Tanh \\  & & 10: Leaky-ReLU & 11: eLU \\ \hline Encoder replications & 18–19 & 00: 0 & 01: 1 \\  & & 10: 2 & 11: 3 \\ \hline Tokenizer & 20 & 0: Blankspace & 1: WordPiece \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Definition of the discrete optimization search space for the GA. Each feature is characterized by one to three loci. may now be explored. At the last fitted generation, the PM remained the same; therefore, the previous best chromosome was considered an elite member of the population. In addition to selecting the best architecture for the neural network, the intermediate architectures can be analyzed to find which patterns were beneficial for training the model. This is discussed in the following subsections. Furthermore, the best architecture is compared to the models found in the literature, with emphasis on the baseline model from [PERSON] et al. (2017), referred to as BaseModel, which is the BERTimbau model retrained with transfer learning with the same dataset from this work ([PERSON] et al., 2018). Each generation continuously converges into a set of the most desirable features. These features are the ones that allow the best fitness in the training environment and the characteristics of the dataset. Understanding the most important features of the latest populations enables one to disregard the hyperparameters or architectural characteristics that do not fit the dataset while providing insights into which hyperparameters the GA should focus on the most. Potentially, the top five provide the most significant information about each generation. Figure 6 reveals the evolution of each generation in time in terms of the LSTM and self-attention layers. As the number of generations increases, the overall number of LSTM layers tends to be reduced. Bidirectionality is preferred in later generations, although the number of unidirectional layers is larger for the top five chromosomes, due to chromosomes that replicate the encoder several times and increase their complexity. On the other hand,  
Figure 4: Durations of the optimization process: total duration (left axis) and average duration per chromosome (right axis). Figure 5: Performance of each generation over time, measured by the ROC-AUC score (right axis) and the diversity (left axis) of each population.