# GRAPE: Grammatical Algorithms in Python for Evolution
## 4 General Experiments
### Results and Discussion  
Figure 4 shows the average fitness of the best individual in the training sets across generations using GRAPE and PonyGE2 for each of the experiments. We run all experiments using PI Grow initialisation, tournament selection (size 7), the same parameters and the same grammars. In general, the results are quite similar. In the final generations, the curves are too close for each of the Pagie-1 and Heart Disease problems to distinguish, PonyGE2 is a slightly better with the Vladislavleva-4 problem and GRAPE is a little better on the Banknote problem. We show these results to compare the evolutionary process in both tools, but, of course, test performance is the key goal. Box plots for test fitness scores for all runs are shown in Figure 5, and their respective \(p\)-values calculated using the Mann-Whitney-Wilcoxon test. We used this test since we do not know _a priori_ what sort of distribution the results have, and the aim of this statistical analysis is to check if there is a significant difference between the results using PonyGE2 and GRAPE, which we compared in each problem independently. Given that the null hypothesis says that there is no difference between the results and that we need at most \(p=0.05\) to reject \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Initial Genome Length** & \\ \hline  & **Sensible Initialisation** & **PI Grow Initialisation** \\ \hline Banknote & 44.8 & 55.8 \\ Heart Disease & 38.1 & 46.2 \\ Pagie-1 & 21.6 & 27.1 \\ Vladislavleva-4 & 16.3 & 25.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average of the genome length in the first generation in 30 runs using sensible initialisation or PI Grow initialisation on GRAPE. \begin{table}
\begin{tabular}{l c} \hline \hline
**Parameter Type** & **Parameter Value** \\ \hline Number of runs & 30 \\ Number of generations & 200 \\ Population size & \(1000/1000/100/1000/500/500\) \\ Elitism ratio & 0.01 \\ Mutation probability & 0.01 \\ Crossover probability & 0.8 \\ Initial genome length & 21/24/50/42 \\ Maximum initial depth & 10 \\ Minimum initial depth & 1/1/1/2/1/1 \\ Maximum depth & 90 \\ Codon size & 255 \\ Wrapping & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Experimental hyperparameters. it with a confidence level of 95%, we can conclude that there is no significant difference between the results using PonyGE2 and GRAPE on these four problems. Figure 6 shows the average fitness of the best individual in the training set across generations when using GRAPE with the three initialisation methods. The results are similar in the first generations, but random initialisation presents the worst results for all problems once the evolution advances. This happens because this method cannot create a high-level diversity population. On the other hand, sensible initialisation and PI Grow initialisation present similar results in the final generations for the Pagie-1 and Banknote problems, while sensible initialisation is slightly better for the Vladislavleva-4 and Heart Disease problems. The average number of invalid individuals across generations when running GRAPE with distinct initialisation methods is shown in Figure 7. Both sensible initialisation and PI Grow initialisation start with zero invalid individuals, but in the second generation, both experience the peak of the curves. This is due to the randomness of the tails added to the initial individuals. On the other hand, random initialisation presents its peak of invalid individuals in the first generation. In general, the number of invalids decreases sharply in the first generations as the randomness of the initialisation loses influence on the current population. However, after some generations, this number slightly increases as the individuals in the population increase in size, and therefore become more complex. In this situation, applying crossover or mutation is more likely to invalidate the individuals than in a population with small individuals.