## 5 Deep Learning
### Elements of a CNN
#### 5.1.3 Convolutional Neural Network Models  
The different elements that build convolutional neural networks (CNN) have to be combined to obtain good solutions. Undoubtedly, the number of configurations is almost infinite. However, there is a good number of CNN configurations that have demonstrated excellent results. These CNNs are called backbone networks, since they are the basis of the different architectures that are implemented to solve, among others, the problem of defect detection and classification.  
In this section, an overview of some widely used CNNs will be provided, mentioning the works from our search that make use of such architectures. Again, some backbone networks that are not used in any of the works we have found during the search will be added to provide background for future projects.  
AlexNet  
AlexNet is a pioneer within CNN [53]. It is very simple in comparison to the latest models. AlexNet is composed of five convolutional layers, a pooling layer (max pooling), three FC layers and an activation layer using a ReLU function. It is not very common to find applications that use AlexNet today, but it is still very important because it was precisely that CNN, which inspired the rest of the models [52].  
VGGNet  
VGGNet won the localisation and classification tracks of the ILSVRC competition in 2014 [54]. Therefore, we are talking about a powerful network. It has two versions that are widely used, VGG-16 and VGG-19. VGG-16 is composed of 13 convolutional layers using \(3\times 3\) kernels, 5 pooling layers and 3 FC layers. VGG-19 includes three more convolutional layers. As the network structure is somewhat simple and its performance is great, it has been used as a backbone network to develop many different applications [50; 52]. In the search carried out in our scoping review, we found a couple of documents in which VGGNet was implemented for the task of defect detection and classification [55; 56].  
GoogleNet  
GoogleNet or Inception V1 [57] is an example of an inception network. It presents two main advantages. The first is a significant reduction in the parameters that the network must manage (about twelve times less than AlexNet). The second is the implementation of inception modules, which allow several convolution operations to be performed using different kernels on the same layer. It is necessary to consider that although the depth of the network is twenty-two convolutional layers, taking into account the layers within the inception modules, there would be more than fifty convolutional layers. It also replaces the max pooling algorithm with an average pooling algorithm in the last layers.  
A figure has been prepared to visualise how the inception modules operate. Figure 5 shows the parallel convolution operations that can be performed in the same layer with the help of the inception modules [52]. Moreover, there have been several advances from the initial Inception V1. Each new model is a step beyond the previous one in terms of precision and processing time, exceeding the previous model. Some of the new models are Inception V2, Inception V3, Inception V4, Inception-ResNet V1 and Inception-ResNet V2. An article has been found in our search implementing Inception V2 [58], another one with Inception V3 [59] and another one with Inception-ResNet V2 [55].  
ResNet  
The advent of ResNet revolutionised the world of CNN [60]. ResNet won the Best Paper Award at the Computer Vision and Pattern Recognition Conference in 2016. It has a depth of 152 layers, which is achieved through a novel idea. Each layer does not have to adjust every single weight of its neurons but only learns a residual correction of the previous layer. This depth allows for huge accuracy. In fact, ResNet is considered one of the most accurate models on CNN. Therefore, it is widely used for tasks that require detecting and classifying very small objects or, as in our case, defects [50]. Throughout our search, we have found one paper that implements the ResNet model for detecting and classifying defects [55; 59].