## 5 Experiments  
### Component Algorithm Accuracy  
The first set of experiments aims to verify the accuracy of the individual Algorithms. Table 2 displays how accurate each Algorithm is for classifying both the validation dataset \(\mathbb{D}_{V}\) and the test set \(\mathbb{D}_{T}\). The first column shows the Algorithm name, and the second column demonstrates the validation accuracy on the validation dataset of real images \(\mathbb{D}_{V}\), and the third column shows the accuracy on the test dataset \(\mathbb{D}_{T}\). The last column shows the accuracy of the validation dataset, which contains only synthetic images \(\mathbb{D}_{S}\). All the Algorithms were trained for five epochs on the train dataset \(\mathbb{D}_{t}\). In each column of Table 2, the results are ranked in descending order, from the most to the least accurate. In columns three and five the differences in relative ranks of each Algorithm on a given dataset are shown. For instance, the DPNWrapper obtained the highest accuracy on the validation dataset \(\mathbb{D}_{V}\) (column two, row two) but on the test set \(\mathbb{D}_{T}\), the same Algorithm's accuracy is one before the last one (column four), and thus the difference in rank, given the order introduced above, is 3. A rank of 0 means that the algorithm's classification accuracy does not change from one dataset to another with respect to the other algorithm's classification accuracy. Figure 3: A schematic representation of the various data preparation steps. The reason for these experiments is not only to determine the individual Algorithms' accuracy but also to demonstrate the fact that the accuracy of classification on the \(\mathbb{D}_{V}\) dataset is not a very reliable predictor for the accuracy obtained on the \(\mathbb{D}_{T}\) dataset. Even less, the accuracy from the validation dataset \(\mathbb{D}_{V}\) is not a very good predictor of the accuracy on the synthetic dataset \(\mathbb{D}_{S}\). However, the accuracy on the synthetic dataset \(\mathbb{D}_{S}\) is a better predictor for the accuracy on the test dataset \(\mathbb{D}_{T}\). Therefore, it should be possible to leverage this specific relation between the synthetic dataset and the test dataset to the Algorithm selection by training it on purely synthetic data. There are three main detailed observations from this experiment. First, note that when evaluated on \(\mathbb{D}_{S}\) that contains only synthetic images, only one Algorithm, the DenseNet, results in accuracy higher than 86%. The other Algorithms perform quite poorly, with VGG scoring nearly random result with accuracy 50.16%. The second observation is that on both the validation \(\mathbb{D}_{V}\) and test datasets \(\mathbb{D}_{T}\), all five Algorithms provide comparable accuracy \(\approx\)90% with only SqueezeNet evaluating at \(\approx\)83% on the test dataset \(\mathbb{D}_{T}\). The final observation is the discrepancy in Algorithm accuracy between the \(\mathbb{D}_{V}\) and \(\mathbb{D}_{T}\). The Table 2 has been colorized to indicate the relative accuracy of the individual Algorithms in descending order, starting from the highest by the following colors: red, green, blue, orange, and yellow. Observe that the evaluation on the \(\mathbb{D}_{V}\) results in a relatively poor generalization. If one would look for most efficient Algorithm based on the validation dataset, then both the \(\mathbb{D}_{S}\) and \(\mathbb{D}_{T}\) would result in poorer accuracy. To further analyze the performance of the individual Algorithms, Figure 4 shows the complementarity of the five used Algorithms' accuracy displayed as the correlation matrix. Each cell in Figure 4 shows the relative overlap between the result of classification between two Algorithms. For instance, the correlation between SqueezeNet and DPN92 is 0.7636. This scores means that 76.36% of their results are the same for both positive and negative labels. The notably lower accuracy on \(\mathbb{D}_{S}\) compared to real data is expected and informative for our study.