# Circuit Manufacturing Defect Detection Using VGG16 Convolutional Neural Networks

Authors: Sara A. Althubiti; Fayadh Alenezi; S. Shitharth; Sangeetha K.; Chennareddy Vijay Simha Reddy
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Manufacturing, one of the most valuable industries in the world, is boundlessly automatable yet still quite stuck in traditionally manual and slow processes. Industry 4.0 is racing to define a new era of digital manufacturing through Internet of Things- (IoT-) connected machines and factory systems, fully comprehensive data gathering, and seamless implementation of data-driven decision-making and action taking. Both academia and industry understand the tremendous value in modernizing manufacturing and are pioneering bleeding-edge strides every day to optimize one of the largest industries in the world. IoT production, functional testing, and fault detection equipment are already being used in today’s maturing smart factory paradigm to superintend intelligent manufacturing equipment and perform automated defect detection in order to enhance production quality and efficiency. This paper presents a powerful and precise computer vision model for automated classification of defect product from standard product. Human operators and inspectors without digital aid must spend inordinate amounts of time poring over visual data, especially in high volume production environments. Our model works quickly and accurately in sparing defective product from entering doomed operations that would otherwise incur waste in the form of wasted worker-hours, tardy disposition, and field failure. We use a convolutional neural network (CNN) with the Visual Geometry Group with 16 layers (VGG16) architecture and train it on the Printed Circuit Board (PCB) dataset with 3175 RBG images. The resultant trained model, assisted by finely tuned optimizers and learning rates, classifies defective product with 97.01% validating accuracy.


---

## Body

## 1. Introduction

In recent years, the world has moved towards the Fourth Industrial Revolution, which is also known as Industry 4.0. Its defining characteristic is to build the boosted productivity and increased efficiency that can be seen in factories and achieved by modern technologies [1] like the Internet of Things (IoT), Artificial Intelligence (AI), robotics, Cloud Computing (CC), sensors, and integrated systems. In Industry 4.0, large-scale IoT systems and machine-to-machine communications (M2M) are integrated to enhance automation, communication, and self-monitoring without the need for human intervention. The smart factory is a step beyond traditional automated manufacturing environments to factories where it may consider fully automotive systems. The machines are controlled by superior computational intelligence and are linked to sensors and other devices through wired or wireless networks [2]. In a study [3], the integration of all physical components and digital technologies into one system is called Cyber-Physical Systems (CPSs) and is analyzed in detail. In addition, the amount of sensors collecting data in smart factories is increasing exponentially [4]. Among the various kinds of sensor systems found in factories, vision-based systems are the most popular and effective when it comes to estimating and classifying product quality. A comprehensive review of automated vision-based defect detection approaches that look at numerous kinds of materials such as ceramics, textiles, and metals is introduced in [5].Smart manufacturing is a recent production paradigm in which machines are fully networked, monitored by sensors, and managed by better computer intelligence to boost system efficiency, product quality, and sustainability while cutting costs. The IoT, CC, and CPS are all-important supporting technologies for modern production [6–10]. These contemporary manufacturing technologies collect and analyze data from several stages of a product’s life cycle, including raw materials, machine operations, facility logistics, and also human operators [8]. Thanks to the proliferation of industrial data, data-driven intelligence combined with powerful analytics transforms massive volumes of data into relevant and insightful information for smart manufacturing.An important aspect of product quality control in the manufacturing environment is faulty product categorization on the product line. Currently, numerous methods are deployed to tackle this task. These defective product classification systems must match challenging requirements such as the need to work in real-time with high accuracy and robust performance in noisy environments like those seen in real-world factories. With the development of machine learning and sophisticated vision systems, feature-based defect classification algorithms are starting to be investigated and applied to classify defective products using various classifiers such as artificial neural networks (ANN), Bayesian network classifiers, and support vector machine (SVM) [11, 12]. When applied in real-world factories, these feature-based categorization methods might be susceptible to noise such as lighting fluctuations or shadows in photos. Moreover, some companies manufacture a large range of items across many product lines, making them inappropriate for the application of feature-based approaches.Deep learning-based algorithms have shown outstanding outcomes in many computer vision applications like object detection, picture categorization, and object identification during the last several years. These successes have displayed their great potential for application in the manufacturing environment [13, 14]. To combine wire defect region identification with faulty product categorization with a multitask CNN is devised [15, 16]. Other quality inspection tasks that use CNNs have been suggested to monitor various products such as PCBs [17, 18], metal surfaces [19], bottled wine, casting products, semiconductor fabrication, and light-emitting diode (LED) cup apertures, mobile phone screen, cover glass of display panels, bearings, optical film, and leather defect. Therefore, this research focuses on defect classification using a deep learning model (CNN) with VGG16 for better accuracy in classifying the model. Most researchers only concentrated on developing the software model and implementing it on a personal computer (PC) or server computer with graphics processing unit (GPU). The utilization of another computer for classifying products is not suitable with practical factories in terms of grade existed systems. Moreover, this research focuses on learning rate to generate a better-trained model by SGD optimizer which even produce better accuracy in large dataset. This proposed CNN with VGG16 assists to classify the defective PCB more accurately because of the better learning rate obtained by the pretrained model.The organization of the article is listed as follows: Section 2 suggests that the defective classification can be done through CNN and various defective product identification is discussed. Section 3 discusses the VGG16 architecture with CNN for a better training model, and an SGD optimizer for a high learning rate is illustrated. Section 4 has evaluated the defective detection classification from the normal product through CNN with VGG16 with other existing CNN and CNN with ResNet through various parameters like accuracy, precision, recall, and F1 score. Section 5 concludes that CNN with VGG16 has better accuracy in training model accuracy and evaluation with existing CNN shows that CNN with VGG16 has a high accuracy of 97%.


## 2. Related Work

This review is intended to study the approach of defective classification done on smart factories to enhance their level through various deep learning methods. Those are discussed by several researchers, and a few are discussed below. The image processing method and computer vision are focused on by Xie, who carefully reviews current improvements in surface identification. This research is mostly focused on industrial applications, while surface defect detection based on image processing needs high real-time performance by matching the results of prior studies [20]. From the standpoint of the textile industry’s defect management, Ngan et al. and Mahajan et al. investigated the use and development of flaw-detection systems frequently used in the manufacturing of textile for the textile industry in the advancement of defect detection [21, 22]. Thermal imaging technology is widely employed in a variety of industries. Aldave et al. [23] examined data from commercially available nonexperimental IR techniques to provide cameras in the field of nondestructive defect detection with references. In both business and academics, defect detection technology is a popular issue. Steel [24] and textile [25], which serve as crucial detection approaches, have yet to be classified by Pernkopf and Zhang et al. The description shows how defect-detection technology works, as well as current defect-detection equipment and alternative options. Furthermore, the assessment, review, and summary of the research status with major technologies in the US and abroad have yet to be finished.Ren et al. have examined the electrical resistance tomography (ERT) images for presenting the depictive identification through an evaluation approach based on color histogram [26]. Similarly, Song et al. devised a classification methodology for detecting flaws on wood surfaces based on the percentage color histogram feature and feature vector texture feature of picture blocks, which has been proved to be effective in trials, especially for junction type defects [27]. Prasitmeeboon and Yau proposed a two-step technical procedure for particleboard defect identification, involving the use of SVM and color histogram characteristics for defect detection and smoothing and threshold technology for defect localization [28, 29]. To differentiate keyboard light leakage faults from dust, Ren and Huang suggested an approach that combines Autoencoder and FCN with a deep neural network. In a test set of 1632 photos, the suggested technique is shown to reduce the false positive rate of light leaking defects from 6.27 percent to 2.37 percent [30]. Sampedro et al. suggested a method for the automated identification and diagnosis of insulator strings that contained one segmentation component for insulator strings and two diagnostic components for missing and damaged insulator disc units, respectively [31]. Balzategui et al. propose a method for segmenting defects in solar cell electroluminescence pictures. When compared to the method of continually performing CNN sliding windows, this methodology uses FCN with unique U-net architecture to generate the defect segmentation map in a single step [12, 32]. Gao et al. used FCN with faster RCNN to construct a deep learning model for tunnel defect detection based on FCN. This model can accurately and quickly detect flaws like stains, leaks, and pipeline blockages [33, 34]. Baumgartl et al. have developed a Laser Powder Bed Fusion (LPBF) device that incorporates off-axis in situ imaging and thermography. The photos were used as a source of data for the CNN model, which is used to identify printing problems. This model is used to detect thermographic in situ defects in LPBF processes, and it achieves a model accuracy of 96 percent. This CNN model, on the other hand, can only identify splatter and delamination faults in metal LPBF. As a result, other related flaws such as cracks, porosity, and unmelted powder, as well as alternative methods such as selective laser sintering (SLS), are not investigated [35, 36]. Jayanthi et al. illustrate that the application of Hough Circle Transform (HT) is to localize the targeted region of iris region efficiently through mask region-based CNN during the segmentation process. The proposed technique has presented with the performance of supreme iris recognition and resulted in maximum recognition accuracy of 99.14% which is comparatively higher than existing UniNet V2, AlexNet, Inception, ResNet, DenseNet, and VGGNet models [37, 38].This section discusses the various approaches of defect detection identification in various kinds of factories as well as various analyses from several researchers about machine learning methods and CNN in defect detection of products in factories. The defect detection through image processing can be done only through CNN, which shows better accuracy, and several researchers also justify that. Thus, the review support in focusing the CNN with VGG16 has assisted to identify the defective product classification with high accuracy.


## 3. Research Methodology

The goal of this research is to identify and classify defective product from the raw material of the manufacturing industries. This study focused on real-time datasets collected from the captured image through the network, and the images are preprocessed by region of interest (RoI) extraction. The purpose of RoI extraction is utilized for image segmentation which influences the performance of the preprocessing. Once the preprocessing is done by RoI extraction, then the dataset is progressed for the normalization which is done for splitting each pixel by 255. The entire pixel is represented in terms of 0 and 1 assist to learn the model better, and the dataset gets split into train dataset and test dataset.In general, the normalization is the initial process of all neural network, and the CNN works based on the deep forward neural network (FNN) that assist in learning invariant feature hierarchies. The distinctive attributes and invariant present in the CNN context are the CNN features that represent the same transformation applied at various locations of the model. CNN model is majorly used for training and data processing in grid topology or in a spatial relationship. The CNN model with VGG16 architecture working is performed as the training process and get evaluated for accomplishing the better pretrained model for identifying the exact defective product from manufactured product progressed from each unit of the smart manufacturing unit as shown in Figure1.Figure 1
Workflow of proposed classification method for a defect product.


### 3.1. Data Collection

This study used Peking University’s intelligent robot laboratory and a PCB defect dataset with 1245 photos and four different types of defects: false copper, mouse bite, missing hole, and open circuit. The various defect images have been fed for detecting and classifying them as the defective product from the normal image of the PCB dataset. Figure2 shows a categorized picture dataset with various defect kinds grouped into different groups.Figure 2
Input images with dissimilar defects on PCB: (a) missing hole; (b) spurious copper; (c) open circuit; (d) mouse bite.
(a)(b)(c)(d)Based on the measurements, the callout box average size for the defect available in the data set is about8×8 pixels as shown in Figure 2. The images can be rotated to a specific angle over the program for improving the model robustness and confirming the detection accuracy though the image gets rotated in the ensuing detection.


### 3.2. Data Preprocessing

The casting images are captured using special arrangements to ensure that the image is captured in stable lighting conditions. The lighting condition changes over time may lead to classification errors in the real-world manufacturing environment, and the vision system completely depends upon lighting conditions. The RoI extraction from the captured image is based on saturation channel details and hue saturation value (HSV) which discusses the color HSV and that the PCB color level gets differ from the background. Initially, the red, blue, and green (RBG) image is converted to HSV color space based on a general equation. Let us considerR,B,G∈0,1, the maximum region as MAX≕maxR,B,G and the minimum region asMIN≕MINR,B,G. Thus, the equation of H, S, and V is expressed as follows:
(1)H:0,ifR=B=G60°×0+B−GMAX−MIN,ifMAX=R60°×2+G−RMAX−MIN,ifMAX=B60°×4+R−BMAX−MIN,ifMAX=G,(2)S:0,ifR=B=GMAX−MINMAX,else,(3)V≔MAX.Eventually, the selection of the HSV threshold is based on pixel range values of the HSV color spacing concerning each histogram channel. Morphology is applied subsequently for removing the noise from the threshold images. Hence, the RoI extraction is considered to be the region of the PCB whereas this RoI is made to be located and cropped from original images.


### 3.3. Classification of Defective Product by CNN with VGG16

This portion of the research is devoted to determining which class the retrieved ROI picture belonged to. The defective product classification training method undergoes two major steps as follows:(i)
Step 1. Training CNN with VGG16(ii)
Step 2. Optimize and retrain the network


#### 3.3.1. Step 1: Training CNN with VGG16

To obtain better performance, CNN methods have been deployed in embedded systems. However, CNNs trained from scratch require a dataset with several labeled entries, more time consumption, and high power consumption in computation. Transfer learning (TL) is introduced to tackle these issues; TL is a technique for retraining a CNN model on a novel, small dataset through reprocessing its feature extractor portion. It gets trained using an existing large dataset, in this case the ImageNet dataset, and retrained only on the functionality of classification for saving training time. Based on TL, the pretrained model weights are fine-tuned for classifying a novel dataset. Moreover, this knowledge transfer can be significantly improved by learning performance without requiring efforts from work-intensive data labeling. The TL approach has accomplished advanced results on many datasets instead of a specified certain amount of training time. This proposed model VGG16 is familiar and mainly considered as one step up from AlexNet because of its deeper architecture and smaller kernel sizes.VGG16 has acquired more information from the AlexNet because it is a deep learning model represented in terms of layer numbers. The proposed VGG16 with CNN model intended on earlier model inadequacy and focused on better accuracy with high efficiency. According to this research, there are 6 variants with similar architecture that have been implemented and have layer numbers ranging from 11 to 19. In this proposed VGG16 architecture, the layers involved are 16 layers which include 13 convolutional layers along with max-pooling layer and 3 fully connected layers that are shown in Figure3. The main advantage present in VGG16 architecture while compared to AlexNet is usage of smaller filters placed on top of each other rather than one single large filter. The concept behind this approach is closely associated with sensitive fields of convolutional filters. However, each pixel receives information from 49 pixels of the previous layer through the kernel of 7×7. Hence, the higher sensitive fields can able to capture more motifs in a large area during missing of details whereas lesser receptive fields have failed to capture the usual patterns. Thus, the VGG16 architecture contains convolutional layers, max pooling layers, and fully connected layers. Even though there are further dissimilarities, the model which is implemented in this stake involves 3×3 and 2×2 convolutional filters. 1×1 fully connected convolutions only represented the linear combination of a pixel location over the layers. Indeed, the utilization of nonlinear activation and ReLU has been preferred in the AlexNet. Moreover, training of AlexNet on MNIST is not an issue in the earlier stack but VGG16 is a quite larger model and needs more data for highly trained. The MNIST dataset does not provide the preferred statistical variety to the model but the pretrained models are available in Keras and various deep learning frameworks and libraries. The adjustable weights and pretrained frameworks are the general practice to acquire certain layers or all layers from those networks have performed as the backbone of the feature extractor module. This method is said to be TL, and it speeds up deep learning models by eliminating the requirement of training huge models from scratch.Figure 3
VGG16 architecture with CNN.


#### 3.3.2. Step 2: Optimize and Retrain the Network

TensorRT-based applications have performed 40 times faster than CPU-only platforms during inference. Based on the TensorRT, the researchers were only required to focus on generating a new application with AI-powered instead of performance tuning for inference deployment. Then the pre-trained models are converted to the format of Open Neural Network Exchange (ONNX). These processes are completed on the server whereas the TensorRT model in the ONNX format can then be downloaded from the cloud and loaded onto the training dataset for the prediction phase. The experimental findings show that the deployed model can execute in real-time for all deep learning functions that run on the same objective functionfx for minimizing error and establishing fresh data input. This research focuses on the sample with a minibatch drawn consistently from the training set. The size of the minibatch is generally selected to be a comparatively less instance number that can be generated from one to a few hundred. However, the SGD performs as the machine-learning algorithm that executed better once it gets trained, though it reaches the local minimum in a reasonable amount of time. Hence, the significant parameter for SGD is the learning rate which is essential for decreasing the learning rate over time. The learning rate is represented with iteration k.Moreover, the pretrained model supports classifying the image through its loss functions and learning rates and assist in performing the classifier model. Therefore, the performed classifier CNN with VGG16 model has accomplished with better accuracy in identifying and classifying the defective product from normal along with the manufacturing shop ID.The experimental findings show that the deployed model can be execute in real-time applications for all deep learning functions that run on the same objective functionfx for minimizing error and establishing a fresh data input. This paper focuses on the sample with a minibatch, which draws consistently from the training set. The size of minibatch is generally selected to be a comparatively less instance number that can be generated from one to few hundreds.In a neural network, the loss of gradient over each parameter can be computed using back propagation networks. This will assist in initiating the model from the model loss and measured its derivative based on the weightage of last model layers. The derivatives have been selected in accordance with each layer that executed backward from last layer of previous model to first layer of previous model considered. However, the SGD performs as the DL algorithms that executed better once it gets trained, though it reaches the local minimum in a reasonable amount of time. Hence, the significant parameter for SGD is the learning rate which is essential for decreasing the learning rate over time. The learning rate has the ability in controlling the sizes of the updated steps along with gradient that is represented with iterationk. Thus, selecting the suitable learning rate is significant to train any neural network model whereas the better learning rate relies on every individual model and issues in the model.


## 3.1. Data Collection

This study used Peking University’s intelligent robot laboratory and a PCB defect dataset with 1245 photos and four different types of defects: false copper, mouse bite, missing hole, and open circuit. The various defect images have been fed for detecting and classifying them as the defective product from the normal image of the PCB dataset. Figure2 shows a categorized picture dataset with various defect kinds grouped into different groups.Figure 2
Input images with dissimilar defects on PCB: (a) missing hole; (b) spurious copper; (c) open circuit; (d) mouse bite.
(a)(b)(c)(d)Based on the measurements, the callout box average size for the defect available in the data set is about8×8 pixels as shown in Figure 2. The images can be rotated to a specific angle over the program for improving the model robustness and confirming the detection accuracy though the image gets rotated in the ensuing detection.


## 3.2. Data Preprocessing

The casting images are captured using special arrangements to ensure that the image is captured in stable lighting conditions. The lighting condition changes over time may lead to classification errors in the real-world manufacturing environment, and the vision system completely depends upon lighting conditions. The RoI extraction from the captured image is based on saturation channel details and hue saturation value (HSV) which discusses the color HSV and that the PCB color level gets differ from the background. Initially, the red, blue, and green (RBG) image is converted to HSV color space based on a general equation. Let us considerR,B,G∈0,1, the maximum region as MAX≕maxR,B,G and the minimum region asMIN≕MINR,B,G. Thus, the equation of H, S, and V is expressed as follows:
(1)H:0,ifR=B=G60°×0+B−GMAX−MIN,ifMAX=R60°×2+G−RMAX−MIN,ifMAX=B60°×4+R−BMAX−MIN,ifMAX=G,(2)S:0,ifR=B=GMAX−MINMAX,else,(3)V≔MAX.Eventually, the selection of the HSV threshold is based on pixel range values of the HSV color spacing concerning each histogram channel. Morphology is applied subsequently for removing the noise from the threshold images. Hence, the RoI extraction is considered to be the region of the PCB whereas this RoI is made to be located and cropped from original images.


## 3.3. Classification of Defective Product by CNN with VGG16

This portion of the research is devoted to determining which class the retrieved ROI picture belonged to. The defective product classification training method undergoes two major steps as follows:(i)
Step 1. Training CNN with VGG16(ii)
Step 2. Optimize and retrain the network


### 3.3.1. Step 1: Training CNN with VGG16

To obtain better performance, CNN methods have been deployed in embedded systems. However, CNNs trained from scratch require a dataset with several labeled entries, more time consumption, and high power consumption in computation. Transfer learning (TL) is introduced to tackle these issues; TL is a technique for retraining a CNN model on a novel, small dataset through reprocessing its feature extractor portion. It gets trained using an existing large dataset, in this case the ImageNet dataset, and retrained only on the functionality of classification for saving training time. Based on TL, the pretrained model weights are fine-tuned for classifying a novel dataset. Moreover, this knowledge transfer can be significantly improved by learning performance without requiring efforts from work-intensive data labeling. The TL approach has accomplished advanced results on many datasets instead of a specified certain amount of training time. This proposed model VGG16 is familiar and mainly considered as one step up from AlexNet because of its deeper architecture and smaller kernel sizes.VGG16 has acquired more information from the AlexNet because it is a deep learning model represented in terms of layer numbers. The proposed VGG16 with CNN model intended on earlier model inadequacy and focused on better accuracy with high efficiency. According to this research, there are 6 variants with similar architecture that have been implemented and have layer numbers ranging from 11 to 19. In this proposed VGG16 architecture, the layers involved are 16 layers which include 13 convolutional layers along with max-pooling layer and 3 fully connected layers that are shown in Figure3. The main advantage present in VGG16 architecture while compared to AlexNet is usage of smaller filters placed on top of each other rather than one single large filter. The concept behind this approach is closely associated with sensitive fields of convolutional filters. However, each pixel receives information from 49 pixels of the previous layer through the kernel of 7×7. Hence, the higher sensitive fields can able to capture more motifs in a large area during missing of details whereas lesser receptive fields have failed to capture the usual patterns. Thus, the VGG16 architecture contains convolutional layers, max pooling layers, and fully connected layers. Even though there are further dissimilarities, the model which is implemented in this stake involves 3×3 and 2×2 convolutional filters. 1×1 fully connected convolutions only represented the linear combination of a pixel location over the layers. Indeed, the utilization of nonlinear activation and ReLU has been preferred in the AlexNet. Moreover, training of AlexNet on MNIST is not an issue in the earlier stack but VGG16 is a quite larger model and needs more data for highly trained. The MNIST dataset does not provide the preferred statistical variety to the model but the pretrained models are available in Keras and various deep learning frameworks and libraries. The adjustable weights and pretrained frameworks are the general practice to acquire certain layers or all layers from those networks have performed as the backbone of the feature extractor module. This method is said to be TL, and it speeds up deep learning models by eliminating the requirement of training huge models from scratch.Figure 3
VGG16 architecture with CNN.


### 3.3.2. Step 2: Optimize and Retrain the Network

TensorRT-based applications have performed 40 times faster than CPU-only platforms during inference. Based on the TensorRT, the researchers were only required to focus on generating a new application with AI-powered instead of performance tuning for inference deployment. Then the pre-trained models are converted to the format of Open Neural Network Exchange (ONNX). These processes are completed on the server whereas the TensorRT model in the ONNX format can then be downloaded from the cloud and loaded onto the training dataset for the prediction phase. The experimental findings show that the deployed model can execute in real-time for all deep learning functions that run on the same objective functionfx for minimizing error and establishing fresh data input. This research focuses on the sample with a minibatch drawn consistently from the training set. The size of the minibatch is generally selected to be a comparatively less instance number that can be generated from one to a few hundred. However, the SGD performs as the machine-learning algorithm that executed better once it gets trained, though it reaches the local minimum in a reasonable amount of time. Hence, the significant parameter for SGD is the learning rate which is essential for decreasing the learning rate over time. The learning rate is represented with iteration k.Moreover, the pretrained model supports classifying the image through its loss functions and learning rates and assist in performing the classifier model. Therefore, the performed classifier CNN with VGG16 model has accomplished with better accuracy in identifying and classifying the defective product from normal along with the manufacturing shop ID.The experimental findings show that the deployed model can be execute in real-time applications for all deep learning functions that run on the same objective functionfx for minimizing error and establishing a fresh data input. This paper focuses on the sample with a minibatch, which draws consistently from the training set. The size of minibatch is generally selected to be a comparatively less instance number that can be generated from one to few hundreds.In a neural network, the loss of gradient over each parameter can be computed using back propagation networks. This will assist in initiating the model from the model loss and measured its derivative based on the weightage of last model layers. The derivatives have been selected in accordance with each layer that executed backward from last layer of previous model to first layer of previous model considered. However, the SGD performs as the DL algorithms that executed better once it gets trained, though it reaches the local minimum in a reasonable amount of time. Hence, the significant parameter for SGD is the learning rate which is essential for decreasing the learning rate over time. The learning rate has the ability in controlling the sizes of the updated steps along with gradient that is represented with iterationk. Thus, selecting the suitable learning rate is significant to train any neural network model whereas the better learning rate relies on every individual model and issues in the model.


## 3.3.1. Step 1: Training CNN with VGG16

To obtain better performance, CNN methods have been deployed in embedded systems. However, CNNs trained from scratch require a dataset with several labeled entries, more time consumption, and high power consumption in computation. Transfer learning (TL) is introduced to tackle these issues; TL is a technique for retraining a CNN model on a novel, small dataset through reprocessing its feature extractor portion. It gets trained using an existing large dataset, in this case the ImageNet dataset, and retrained only on the functionality of classification for saving training time. Based on TL, the pretrained model weights are fine-tuned for classifying a novel dataset. Moreover, this knowledge transfer can be significantly improved by learning performance without requiring efforts from work-intensive data labeling. The TL approach has accomplished advanced results on many datasets instead of a specified certain amount of training time. This proposed model VGG16 is familiar and mainly considered as one step up from AlexNet because of its deeper architecture and smaller kernel sizes.VGG16 has acquired more information from the AlexNet because it is a deep learning model represented in terms of layer numbers. The proposed VGG16 with CNN model intended on earlier model inadequacy and focused on better accuracy with high efficiency. According to this research, there are 6 variants with similar architecture that have been implemented and have layer numbers ranging from 11 to 19. In this proposed VGG16 architecture, the layers involved are 16 layers which include 13 convolutional layers along with max-pooling layer and 3 fully connected layers that are shown in Figure3. The main advantage present in VGG16 architecture while compared to AlexNet is usage of smaller filters placed on top of each other rather than one single large filter. The concept behind this approach is closely associated with sensitive fields of convolutional filters. However, each pixel receives information from 49 pixels of the previous layer through the kernel of 7×7. Hence, the higher sensitive fields can able to capture more motifs in a large area during missing of details whereas lesser receptive fields have failed to capture the usual patterns. Thus, the VGG16 architecture contains convolutional layers, max pooling layers, and fully connected layers. Even though there are further dissimilarities, the model which is implemented in this stake involves 3×3 and 2×2 convolutional filters. 1×1 fully connected convolutions only represented the linear combination of a pixel location over the layers. Indeed, the utilization of nonlinear activation and ReLU has been preferred in the AlexNet. Moreover, training of AlexNet on MNIST is not an issue in the earlier stack but VGG16 is a quite larger model and needs more data for highly trained. The MNIST dataset does not provide the preferred statistical variety to the model but the pretrained models are available in Keras and various deep learning frameworks and libraries. The adjustable weights and pretrained frameworks are the general practice to acquire certain layers or all layers from those networks have performed as the backbone of the feature extractor module. This method is said to be TL, and it speeds up deep learning models by eliminating the requirement of training huge models from scratch.Figure 3
VGG16 architecture with CNN.


## 3.3.2. Step 2: Optimize and Retrain the Network

TensorRT-based applications have performed 40 times faster than CPU-only platforms during inference. Based on the TensorRT, the researchers were only required to focus on generating a new application with AI-powered instead of performance tuning for inference deployment. Then the pre-trained models are converted to the format of Open Neural Network Exchange (ONNX). These processes are completed on the server whereas the TensorRT model in the ONNX format can then be downloaded from the cloud and loaded onto the training dataset for the prediction phase. The experimental findings show that the deployed model can execute in real-time for all deep learning functions that run on the same objective functionfx for minimizing error and establishing fresh data input. This research focuses on the sample with a minibatch drawn consistently from the training set. The size of the minibatch is generally selected to be a comparatively less instance number that can be generated from one to a few hundred. However, the SGD performs as the machine-learning algorithm that executed better once it gets trained, though it reaches the local minimum in a reasonable amount of time. Hence, the significant parameter for SGD is the learning rate which is essential for decreasing the learning rate over time. The learning rate is represented with iteration k.Moreover, the pretrained model supports classifying the image through its loss functions and learning rates and assist in performing the classifier model. Therefore, the performed classifier CNN with VGG16 model has accomplished with better accuracy in identifying and classifying the defective product from normal along with the manufacturing shop ID.The experimental findings show that the deployed model can be execute in real-time applications for all deep learning functions that run on the same objective functionfx for minimizing error and establishing a fresh data input. This paper focuses on the sample with a minibatch, which draws consistently from the training set. The size of minibatch is generally selected to be a comparatively less instance number that can be generated from one to few hundreds.In a neural network, the loss of gradient over each parameter can be computed using back propagation networks. This will assist in initiating the model from the model loss and measured its derivative based on the weightage of last model layers. The derivatives have been selected in accordance with each layer that executed backward from last layer of previous model to first layer of previous model considered. However, the SGD performs as the DL algorithms that executed better once it gets trained, though it reaches the local minimum in a reasonable amount of time. Hence, the significant parameter for SGD is the learning rate which is essential for decreasing the learning rate over time. The learning rate has the ability in controlling the sizes of the updated steps along with gradient that is represented with iterationk. Thus, selecting the suitable learning rate is significant to train any neural network model whereas the better learning rate relies on every individual model and issues in the model.


## 4. Result and Discussion

This experimental research utilized a high performance server along with the configuration of an Intel Core i7 DMI2 CPU, 16GB RAM, and a Quadro K600 GPU. The operating system used is Ubuntu 16.04.6 LST for executing the abovementioned GPU while training the dataset images. The optimizer and loss function utilized in this proposed model are SGD and cross entropy. The learning rate is set at 0.01 and the training epoch at 50 for optimal iteration. The casting dataset, a publicly accessible dataset relevant to product categorization, is used to assess and compare the performance of the suggested technique [37]. The PCB product data one contains 3175 augmented, 300×300 pixel RBG images for training and testing the system. The 3175 images consist of a normal product as well as defective product images shown in Table 1.Table 1
Dataset distributions for training, validation, and testing.
Product datasetProduct class statusImage countTraining imagesTesting imagesTrainingValidatingPCB datasetNormal19371317232387Defective1238842149248The original electrical wire and cropped electrical wire datasets are used to examine the effect of applying the ROI extraction step. The final number of cropped images is smaller than the number of original electrical wire images due to not including images when the ROI extraction process failed. For all datasets, we randomly choose 80% for training (70% for training and 10% for validation) and leave the remaining 20% for testing. The details of each dataset and the distributions used for training, validation, and testing are provided.Figure4 illustrates that the accuracy of the train model begins at 35% approximately at 2 epochs and improves until 94.8% at 10 epochs. A slight fluctuation occurs from 10 to 30 epochs and becomes steady with an accuracy of 96.2% after 50 epochs. In the case of the validation model, the accuracy is lower at the start and fluctuates until 10 epochs, but it steadies from 10 to 50 epochs with accuracy of 97.4%. This shows that the learning rate and the optimizer have reduced the loss function and improved the TL with better knowledge transformation through the pretrained model.Figure 4
Model accuracy for train and validation.Figure5 illustrates that the loss of the train model begins at 1.56 at 2 epochs and decreases as we approach 40 epochs. There is slight fluctuation occurs from 40 to 50 epochs which stabilizes at a loss of less than 0.16 at 50 epochs. In the case of the validation model, the loss begins at 1.52 and fluctuates until 30 epochs, but it stabilizes from 30 to 50 epochs with a loss of less than 0.1. This shows that the learning rate and the optimizer have reduced the loss function and improved the TL with better knowledge transformation through the pretrained model.Figure 5
Model loss for train and validation.In this study, accuracy, precision, recall, and F1 scores were used as assessment measures for the suggested CNN with VGG16 for product categorization. Table2 shows how these four assessment metrics were computed using true positive (TP), true negative (TN), false positive (FP), and false negative (FN) outcomes from a confusion matrix between the prediction and the reality. The ratio of correct predictions (TP+TN) to the total number of predictions (TP+TN+FP+FN) is known as accuracy. The number of accurately predicted samples (TP) divided by the number of positive samples anticipated (TP+FP) yields precision. The number of properly predicted (TP) samples divided by all actual positive samples (TP+FN) yields recall. The weighted average of accuracy and recall is the F1 score. The problem of defect classification in the product is binary. Therefore, F1 score more reliable statistical rate evaluation metric that considers both positive elements and negative elements is adopted for this study.Table 2
Evaluation of testing image by confusion matrix for various algorithm.
S. noAlgorithmConfusion matrix valuesTrue positive (TP)True negative (TN)False positive (FP)False negative (FN)1CNN33724910392CNN–VGG163912256133CNN-ResNet358241828Figure6 illustrates that the accuracy of CNN with VGG16 is high while compared to the existing CNN and CNN-ResNet. The performance of the proposed CNN-VGG16 is 97%, and it also determined that the TL is even better for large datasets than CNN and CNN-ResNet.Figure 6
Comparison of accuracy performance for various deep learning models.Figure7 illustrates the performance of precision, recall, and F1 score that describes the proposed model CNN-VGG16 that has high scores in all precision, recall, and F1 score that are 0.98, 0.97, and 0.98, correspondingly. The scores of CNN-VGG16 discuss that because of a better pretrained model using SGD optimizer and loss function as cross entropy. The positive is high in actual as well as prediction which assists in performing precision and recall that make the better weightage in F1 score. When comparing the performance of the proposed model CNN-VGG16, the CNN and CNN-ResNet have performed less in precision, recall, and F1 score.Figure 7
Comparison of precision, recall, and F1 score performance for various deep learning models.Moreover, the proposed CNN-VGG16 has better accuracy in classifying the defective product from the normal product in the smart factories through image processing. This initiation makes this Industry 4.0 technology reduces the waste as well as minimizes the cost of production.


## 5. Conclusion

As the manufacturing industry continues to propel into the modern age, intelligent data-driven systems are becoming ubiquitous in optimizing, replacing, and augmenting human labor. Improvements to the models behind these systems are becoming increasingly necessary. Our proposed model provides one such advance, in providing accurate classification for product defect in production. The model’s CNN-VGG16 architecture emphasizes layers that assist in capturing large pixels, and the SGD optimizer generates an optimal learning rate for learning PCB data. This pretraining model assists in implementing a better TL model to retrain CCN-VGG16 for PCB product dataset ROI extraction based on S-channel data. Each pretrained model is deployed for determining the defective PCB product with an accuracy of 97.3% in training. Moreover, the proposed CNN-VGG16 is evaluated using various deep learning models such as CNN and CNN-ResNet. Hence, the evaluation result through testing the image dataset of the proposed CNN-VGG16 has an accuracy of 97%. Thus, the proposed CNN-VGG16 can excel at determining defected PCBs before they reach further production stages. This will reduce scrap, increase yield, and minimize production cost across the board. As the Industry 4.0 paradigm encompasses production around the world, advancements such as ours will provide value for countless companies.



---

*Source: 1070405-2022-04-16.xml*