## 6 Discussion and Implications
Hence, usage of dataset specific features FC1 allowed to make a better decision on presence of a defect, in comparison with transformation specific features FC2. The model A0 uses the unaltered features, extracted by VGG16 and trained on ImageNet. These features are not customized for our dataset and demonstrate a baseline accuracy. Figures 7 and 8 demonstrate that the accuracy of models A1 and A2 changes with the grow of the number of training samples. At some level, the information extracted by RotNet will allows to outperform the baseline accuracy (ref. Figure 7). Referring to widely used statistical \"rule of thumb\" [38], which recommends for reliable estimates to have at least 40 to 60 times as many cases as variables that are relevant; in order to explain an outcome variable, we could speculate about the number of relevant features (variables). We could hypothesize that the number of relevant features to detect defect in solders should be more than 10 (as only when the number of samples is above 600, the fine-tuning starts showing better result than the baseline MCC-A0), and for PCB dataset the analysis of rather small number of features would allow detection of above 85 percent of defects (as MCC-A0 shows the accuracy above 85 percent based on only 78 samples). We must emphasize that these conclusions are purely speculative and are based on the mentioned \"rule of thumb\". PCB defects were internally divided into two groups: non-texture defect (scratch/abrasion/broken edge/missing and extra hole) vs. texture defects (washers and surface false alarms). Washers were included into texture defects, due to the observation that the presence or absence of a washer could be distinguished only by the texture of the rim (Figure 9). As it was mentioned before, texture defect detection was unsatisfactory due to the fact that training on a limited number of samples could not structure the expected way the information provided. The performance has been poor, due to the fact that the training set did not contain enough variety of \"normal\" texture/background samples; therefore, any \"normal\" sample is detected as \"defective\" if the pattern was not included in the training. The reason for this is in the inherent nature of unsupervised learning, which checks for any pattern not seen before (theoretically infinite), as opposed to supervised learning, which looks for a pattern already seen before (finite only). In some cases, there is a change of the shadows of the background (Figure 9). Limited number of training examples make network to ignore these features. We assume that the absence of some non-prominent feature (similar to the rather mild change of gradient of the background) in normal data makes the network not to be sensitive to small variations and, hence, ignore such manifestations. We also could assume that for unsupervised learning based on small number of samples, only the \"big\" features (high gradient image features, such as edges, corners, etc.) are considered, and not the texture of the image. One of the potential ways to overcome this problem is to increase the number of training cases to force the network to learn more about normal texture patterns, as shown in [39]. For a large number of normal cases (that is not usually a problem for mass production line, especially for six sigma processes in manufacturing) the variability of normal cases will be caught much better, and it is the way for better defect detection accuracy, especially for texture defects. A small number of normal cases for texture type of defect detection would result in a high number of false decisions. A usual requirement for process of learning is not to come to overfitted model. The learning should result in useful pattern extraction from data, rather than in \"memorizing\". It ensures a better prediction performance for unseen before data. If the image has a pre-defined design (e.g., technical image), the situation might be somehow different.