# Automated Design of Salient Object Detection Algorithms with Brain Programming
## 4 Experiments and Results
### Experiments with Benchmark's Score  
In the second round of experiments, we adapt the algorithm to use the proposed benchmark's score as a fitness function, as explained earlier; see Section 3.8. From now on, all report experiments consider this way of evaluation. Table 11 provides the results of the k-fold experimentation considering the GBVSBP algorithm with the FT database, while Table 12 provides the corresponding best individual. As can be seen, there is a decrease, but the ranking remains unaltered, as we verified with the solutions. The results considering the dataset IMGSAL are in Tables 13 and 14, while those of PASCAL-S are in Tables 15 and 16 respectively, while Table 17 provides partial results with the GBVSBP+MCG model to illustrate the performance. The experimental results with GBVSBP for the PASCAL-S database have higher similarity because the F-measure has greater stability than previous results. The results show a stable standard deviation between 1.60 and 3.47 for training and testing. We obtain similar behavior in the average results between 60.47% and 61.12% for the training stage. The best solution was reached in the fifth fold at the fourth run, scoring 63.38% during training, while in testing, the algorithm scored 61.94%. Table 16 shows the best trees. The experimental results with GBVSBP + MCG for the FT database show some loss of aptitude against its counterpart in the first block of experiments (5% in training and 3% in tests). However, the performance values are more stable with \(\sigma=0.56\) for training and \(\sigma=3.75\) for testing. In addition to stability, we must bear in mind that the results of this experiment will be much more consistent when the best individual is tested in the benchmark since we are using the same F-measure. As a result, the best results were 89.02% in training corresponding to the second fold of the first run and 89.02% at testing discovered in the first fold of the fifth run. Table 18 gives the best set of trees. \begin{table}
\begin{tabular}{c c} \hline \hline \multicolumn{2}{c}{_EVO\({}_{d}\) and \(EFI\) Operators_} & \multicolumn{1}{c}{**Fitness**} \\ \hline \(EVO_{O}=G_{(\sigma=1)}(G_{(\sigma=1)}(I_{y}))\) & Training = 0.6194 \\ \(EVO_{\text{C}}=I_{m}\) & Testing = 0.6194 \\ \(EVO_{\text{S}}=[\langle dilation_{disk}( dilation_{square}(erosion_{disk}(I_{y})))\rangle]\) \\ \(EFI=|G_{(\sigma=2)}(D_{y}(CM_{MM}))|\) \\ \hline \hline \end{tabular}
\end{table} 
Table 16: Program structure of the operators corresponding to the best solution for the PASCAL-S database. We select this solution from the 30 executions of Table 15. The final solution is similar to all others discovered by the system. This shows that the proposed programs are independent of the selected dataset. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Fold**} & \multicolumn{4}{c}{**PASCAL-S**} \\  & \multicolumn{2}{c}{**Run 1**} & \multicolumn{2}{c}{**Run 2**} & \multicolumn{2}{c}{**Run 3**} & \multicolumn{2}{c}{**Run 4**} & \multicolumn{2}{c}{**Run 5**} & \multicolumn{2}{c}{**Run 6**} \\ \hline  & Trng & Test & Trng & Test & Trng & Test & Trng & Test & Trng & Test & Trng & Test \\
1 & 62.03 & 59.75 & 61.39 & 57.53 & 61.88 & 57.98 & 60.76 & 56.06 & 61.62 & 58.75 & 60.89 & 58.19 \\
2 & 61.53 & 55.12 & 61.95 & 54.99 & 61.11 & 56.52 & 61.33 & 52.61 & 61.23 & 53.64 & 61.01 & 56.70 \\
3 & 59.28 & 58.83 & 60.08 & 61.08 & 59.84 & 60.32 & 60.60 & 59.57 & 59.82 & 61.33 & 59.71 & 61.78 \\
4 & 59.57 & 58.36 & 58.20 & 54.28 & 58.59 & 55.91 & 58.61 & 58.90 & 58.21 & 57.13 & 57.89 & 56.44 \\
5 & 63.20 & 61.40 & 61.94 & **61.94** & 63.08 & 59.12 & **63.38** & 59.01 & 62.93 & 59.00 & 62.87 & 61.34 \\ Average & **61.12** & 58.69 & 60.71 & 57.96 & 60.90 & 57.97 & 60.94 & 57.23 & 60.76 & 57.97 & **60.47** & 58.87 \\ \(\sigma\) & 1.67 & 2.31 & **1.60** & **3.47** & 1.75 & **1.82** & 1.71 & 2.92 & 1.81 & 2.85 & 1.83 & 2.54 \\ \hline \hline \end{tabular}
\end{table} 
Table 15: Performance of the best individuals of the GBVSBP model for all PASCAL-S database runs.