## 4 Materials and Methods
### Model Architecture are Experiment Description  
To perform defect analysis, the model should be able to distinguish between normal and defective patterns. As it is trained only on defect-free samples, the model should be able to identify correctly the repeatable normal patterns. This problem statement assumes learning by using a small number of normal samples only. Commonly accepted way to  
Figure 3: Defect-free printed circuit board. Figure 2: (**a**) normal (type 1); (**b**) normal (type2); (**c**) defect (type1); (**d**) defect (type 2). perform extraction of features from a new data without an exhaustive training is to use transfer learning. In our work, we used VGG16 [33] (and also conducted an experiment with ResNet50 [34], see Section 5), with network weights pre-trained on ImageNet dataset and pre-defined parameters. VGG16 has 16 layers (13 convolutional layers in 5 convolution blocks and 3 fully-connected layers, see Figure 4). It was trained on more than 1 million images to do classification into 1000 categories. Different approaches can be used to utilize a transfer learning network on a custom dataset; for example, in some cases, the initial layers (the corresponding coefficients) of VGG16 are frozen and the remaining layers weight coefficients are re-trained and adjusted using a custom training set. This approach is mainly used for supervised transfer learning. The number of classes in the dataset is equal to the number of neurons in the last fully-connected (fc) layer. The original VGG16 architecture (for the input 224 \(\times\) 224 \(\times\) 3) has a FC2 layer (Figure 4) of 4096 elements, and the output of flattened layer (after 13 convolutional layers) is a tensor of 25,088 elements. We compared three model architectures to identify which one is better under certain conditions. The first architecture, hereafter called A1, is comprised of two parts. The initial layers (i.e., convolutional layers) of the pre-trained VGG16 were frozen and used as the first part of feature extraction. Two additional fully-connected (FC, Figure 5) layers were added as the second part of the feature extractor, as it was also done in [35] that implements unsupervised learning network based on simple geometric transformations. We expect that this approach provides additional fine-tuning (according to our custom datasets) of the existing rich feature extraction capability VGG16 network without constructing a custom CNN model from the scratch. The RotNet model from [35] allows for the complex features' extraction from the input images without labelling the training data, and the author suggested using convolutional network to recognize 2D-rotations applied to the input data. For example, the input images could be rotated by 0, 90, 180, and 270 degrees, or as an option, can be flipped horizontally or vertically. Then, the ConvNet learns how to recognize geometric transformations. At the same time, by learning the transformation applied, representative features from the augmented dataset are extracted. As a result, the image representation is learned through artificial introduction of transformations classes [35], using an unsupervised learning  
Figure 4: VGG16 model. process by adding to the process the intermediate (semi-) supervised stage. The image understanding incorporated into this rotational ConvNet helps to extract the features essential for the image dataset under analysis, by using \"surrogate supervision signals\" [35]. After conducting several experiments, we chose the modification of VGG16 (Figure 6), which seems to work successfully in our task. The initial convolutional layers in VGG16 were frozen, and the subsequent fully-connected layers were removed. We have added two fully-connected layers (FC1 and FC2), each containing 1000 neurons (the number chosen arbitrarily), followed by a SoftMax layer with the number of outputs equal to the number of classes. The frozen VGG16 layers' output parameters are flattened, which serve as the input to FC1.