## 5 Experiments
### Algorithm Selection as a Function of Component Algorithms Training Epochs  
The next set of experiments evaluates Algorithm selection accuracy as a function of the number of training epochs of the component Algorithms. The Algorithm selectors were trained on either real and synthetic images, 270 real and 270 synthetic, respectively. Figure 10 shows the results of both the evaluation of the accuracy of the component Algorithms as well as the accuracy of Algorithm selection as a function of component Algorithms' training epochs. Figure 10a shows the accuracy of the individual component Algorithms. Figure 10b also shows the the accuracy of Algorithm selection when using images \(\mathbb{D}_{S}\) generated by GAN (in the case of SAS) or real images \(\mathbb{D}_{V}\) (in the case of RAS) for the training of Algorithm selectors. In Figure 10b, we consider for each epoch the most accurate Algorithm selection. In Figure 10a, the more the Algorithms are specifically trained to the dataset, the more they converge to individual accuracies with reduced variation. It can be concluded that that the component Algorithms' complementary performance becomes less relevant due to Algorithm relative overfitting or training. By relative overfitting, we mean that an Algorithm's accuracy \(\mathcal{A}\) tends to a much lower variance by matching the training dataset more and more precisely. In other words, as the performance overlap becomes larger, the Algorithm selection converges to select Algorithms with diminishing effect of improvement. Note that when component Algorithms are trained for one epoch, the difference in the average accuracy is larger than 7%, while, when trained for 30 epochs, their relative accuracy difference is <3%. In addition, their relative accuracy increases from the interval [84%-90%] at epoch 1 to [91%-93%] at 30 epochs. In addition, note that the highest and lowest accuracy of classification was achieved for 24 training epochs and for 4 training epochs by the DPN. Figure 10b displays a similar trend--it shows the accuracy of the best component Algorithm as well as the accuracy of Algorithm selection trained on real and on synthetic images. Similarly to Figure 10a, at epoch 1, the difference of accuracies is in interval [86%-90%], while at training epoch 30%, the interval is [91%-93%]. The best Algorithm from all five component Algorithms is taken for each epoch. First, observe that both the average synthetic and real Algorithm selection accuracy always under perform when compared to the best Algorithm. Figure 10: Algorithm selection and component Algorithms’ accuracy \(\mathcal{A}\) as a function of training epoch number of the component Algorithms. (**a**) Average Classification accuracy of Component Algorithms’ \(\mathcal{A}\) as a function of training epochs; (**b**) Average Classification accuracy \(\mathcal{A}\) when using the Algorithm selection as a function of training epochs. The best Algorithm accuracy (blue curve) shows the highest value for eleven data points. The SAS (orange line) shows the highest value for four data points. The RAS (red line) shows the highest value for three data points. The best Algorithm is more accurate for most of the training at lower epoch numbers. These epochs are [1; 2; 3; 4; 5; 8; 9; 10; 14; 15; 16; 22]. The Algorithm selection, on the other hand, is more accurate for most of the experiments with a higher number of training epochs; in particular, for epochs [5; 7; 12; 20; 24; 25; 26; 27; 28; 29; 30]. Interestingly, while the highest Algorithms' complementarity occurs in the early training epochs (due to highly randomized nature of under-trained Algorithms), the highest accuracy in the early training epochs is dominated by the best Algorithm (Figure 10b). This would imply that for Algorithms trained for few epochs only, the predictability of their behavior might be too difficult for the Algorithm selector despite their high complementary performance. It is only once the Algorithms are predictable enough that Algorithm selection improves the overall accuracy.