# Image Defect Recognition Method Based on Deep Learning Network

Authors: Xiaolian Di; Li Li
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Deep learning techniques are used to identify weld image defects in the process of image defect recognition. In this paper, a transfer learning method based on convolutional neural networks is proposed for the recognition problem of deep neural network models on weld flaw detection image data sets. Designing interdomain heterogeneous transfer learning with the pretrained model on the large data set, the interdomain heterogeneous transfer learning is used to transfer the pretrained model in the source data domain to the weld inspection image data set according to the difference of the content in the source and target data domains, and the effectiveness of the transfer learning in weld inspection image defect recognition is verified by fine-tuning the whole network by training the parameters of different layers using the frozen layer method. The effect of freezing different layers on the recognition performance of the model is also investigated.


---

## Body

## 1. Introduction

With the development of ray detection technology, inspection techniques with different imaging methods, such as radiographic inspection technology and ray real-time imaging inspection technology, have been more widely used. For some domestic heavy-equipment manufacturing enterprises, defect detection in large wall thickness of the welded parts still use radiographic film development technology, although there is a waste of time in the use of film, pollution to the environment, low efficiency, and other issues; however, the use of AB-level or B-level blackness of the film can be very sensitive to detect the presence of tiny defects in the weld information, which cannot be replaced by real-time image detection technology [1–3]. In addition to heavy-equipment manufacturers and other companies using film inspection, most domestic companies have developed to CR and DR digital ray detection imaging technology for welding quality inspection [4–6]. In the defect assessment stage, due to the unevenness and diversity of the nature of defects such as form, location, direction, and size, making it difficult to find a set of common means and methods to automatically identify weld defects in the actual production process [7], the weld defects are usually assessed manually in the assessment process. In the process of manual evaluation of the film, first is to determine whether there are defects on the weld flaw detection image, then determine the type of defects, followed by the determination of defect data, and finally the quality level assessment according to the quality acceptance criteria. This method is easily affected by the external conditions of radiation detection equipment and human subjective initiative, especially in strong light, which leads to eye fatigue, reduces the recognition ability of weld defects, and is easy to cause defect detection errors. In addition, the manual operation is less efficient, inconvenient to delete operations on the negative, and the negative classification and storage are time-consuming: all these problems greatly affect the efficiency of the radiographic inspection. Owing to the many shortcomings in the manual negative assessment process, it is increasingly unable to meet the needs of industrial automation and intelligent development [8]. Therefore, the detection and evaluation of weld defects began to gradually develop from manual methods to computerized automatic identification and evaluation.With the development of modern science and technology, computers and scanning technology have also gained greater progress and a wider range of applications. The traditional film becomes a digital image obtained through a scanner, and in real-time imaging, analog signals is also converted to digital signals, providing the possibility of computerized inspection and evaluation points. On this basis, researchers have decomposed the process of manual film evaluation into computer processing processes, such as preprocessing of weld inspection images, segmentation of weld areas, extraction of defect features, defect classification and identification, and displaying and saving the final classification results [9]. However, these systems are still inadequate in terms of full automation and cannot achieve complete automation and intelligent detection and require human-computer coupling in the weld defect detection and evaluation phase. However, these systems are still inadequate in terms of full automation and cannot achieve complete automation and intelligent inspection. The machine learning model in the weld assessment system involving machine learning cannot directly extract the image features but needs to manually design the geometric features, texture features, grayscale features, and so on in the image. Therefore, it cannot be automated in a real sense.With the development of computer technology and further improvement of performance in all aspects, as well as the emergence of Big Data, cloud computing, GPU, computer vision, speech recognition, natural language processing, and human-computer gaming based on deep learning have gained rapid development, opening up new paths for many experts and scholars. In image recognition, the emergence of deep learning has solved the inconvenience of manual extraction of features by researchers and is able to learn and recognize the deep features of input images autonomously with good results. In response to the aforementioned analysis, this paper attempts to apply deep learning to the field of weld inspection image recognition and evaluation and to achieve the goal of automatic identification of weld defects by establishing deep learning models for end-to-end training and learning of weld inspection images, so as to meet the requirements of modern enterprises for automation, intelligence, and efficiency in the field of weld inspection.


## 2. Related Work

With the continuous research, the methods for automatic detection and recognition of weld defects are being expanded, but from the results of the existing literature studies, the methods used in the process of weld defect recognition are mainly divided into a three-stage processing process, which are image segmentation, feature extraction, and defect classification [10]. Mao et al. [11] used median filtering technique to remove the noise in the image, followed by image enhancement technique, Ostu image segmentation method, edge detection technique, and Hough transform to calculate the region of interest of the X-ray image, and obtained a better segmentation effect without manually designing a suitable segmentation threshold. Abd El-aziz et al.[12] proposed an improved Ostu algorithm weighted object variance (WOV), which ensures that the threshold value is always the value located at the valley of two peaks or at the lower left edge of a single peak histogram, solving the problem of threshold selection in histograms in the case of single and double peaks, and the results show that the WOV algorithm outperforms Ostu, maximum entropy, valley-emphasis, and other algorithms. Wang et al. [13] used global thresholding and local thresholding segmentation methods on the preprocessed image to extract the weld region and the defect region, respectively, which can quickly and effectively segment the defects present in the weld image, but the effectiveness of the method is not obvious when applied to the segmentation of weld defects of complex shapes. Cheng and Yu [14] proposed a defect segmentation algorithm based on the removal of the background, which has good results in the segmentation detection of defect areas that cope with defect sizes greater than six pixels without considering the type of defect.The common image features include shape features, texture features, gray scale symbiotic matrix, and so on. Gao et al. [15] used a combination of geometric and texture features to form 43 feature descriptors and conducted a pattern recognition experiment with multiple classifications. Li et al. [16] proposed a weld defect recognition method based on gray scale-gradient symbiotic matrix, and cluster analysis, which comprehensively considered the combined distribution of pixel-level grayscale and edge-gradient size and added the change information of the image into the statistical matrix of grayscale information to avoid the complexity and diversity of traditional analysis of weld defect information and effectively identify and analyze the weld defect information.In recent years, the advantages of deep learning in image recognition and classification are more and more prominent, and its application is more and more wide. By increasing the number of layers of the convolutional neural network model and the width of each layer of the network, the ability of the neural network in extracting and learning image features is further improved, which can avoid the subjectivity and inefficiency in the process of manual extraction of features and obtain better recognition and classification results, and the ability to transfer the features can be migrated and applied to different recognition domains. Mei et al. [17] applied principal component analysis (PCA) to the classification of weld defects, analyzed the image feature vector, identified the components with greater influence, improved the linear independence between the input variables, reduced the repetitive description of features in the image, and then used support vector machines to classify and identify different defects. Musleh et al. [2] proposed a defect classification and recognition method based on direct multiclass support vector machines, which transformed the defect-type recognition problem into an optimization problem under constraints and described the defects by feature vectors composed of defect edges and regional features, solving the problem of low defect recognition rate in the case of low sample capacity. Niu et al. [5] segmented and extracted defect features based on denoising and enhancement of weld flaw detection images, established classification rules for weld defect features using decision tree methods, and obtained a high recognition rate using these rules in image classification recognition. Caggiano et al. [6] proposed an adaptive SVM decision algorithm based on the degree of separation for the problem of low accuracy of weld defect recognition. First, the mathematical morphological reconstruction of the filtered image is performed, and the defect category with the maximum separation degree is separated, and an adaptive binomial tree SVM classifier is constructed, which can reduce the accumulation error of binomial trees and obtain good classification results. Zhang et al. [18] proposed a BP neural network-based weld defect image recognition method, which first performs image processing to extract defects and calculates feature parameters and uses the high fault tolerance of neural network to solve the problems of noise and partial loss of input image in the digitized image of weld, which can greatly improve the accuracy of weld defect image recognition.


## 3. Weld Image Defect Classification and Image Enhancement

### 3.1. Image Classification

There are many kinds of defects commonly found in the weld, and understanding the type and classification of weld defects is the basis for detection and identification. There are two common classification methods, one is to classify the defects according to their different locations in the weld, which can be divided into external and internal defects, where external defects contain weld tumors, biting, burn-through, slag, surface porosity, and shape defects, and internal defects contain porosity, cracks, failure to weld through, failure to fuse, and so on. The second is the classification according to GB 6417-86 “Classification and description of metal fusion weld defects” [4], including cracks, porosity, slag, failure to weld through, failure to fuse, and poor dimensional defects in six categories. Figure 1 shows the X-ray inspection images of porosity, cracking, nonfusion, and nonpenetration.Figure 1
Typical X-ray defect image.Whether it is based on manual or machine learning, it is necessary to identify the types of weld defects. The lack of welding quality grade can be divided into four categories.


### 3.2. X-Ray Image Denoising and Enhancement

In this paper, one X-ray image shown in Figure2 was randomly selected from the weld flaw image data, and the images were denoised using the five aforementioned denoising methods. The peak signal-to-noise ratio (PSNR) [18] can be used to compare and analyze the denoising effect of the five noise reduction methods, which are calculated as shown in (1) and (2)(1)PSNR=10log102552MSE,(2)MSE=∑1M∑1Nfi,j−f0i,j2MN,where fi,j is the pixel gray value of the image after noise removal, f0i,j is the pixel gray value of the input image, and the image pixels are M∗N.Figure 2
Results of filtering effect.The five images containing pores, cracks, unfused, unbroken, and no defects were randomly selected from the weld X-ray image data, and five filtering methods, namely mean filter, median filter, Gaussian filter, bilateral filter, and wavelet filter, were used to filter and reduce the noise of the weld images, and the peak signal-to-noise ratio of each filtering method was calculated after noise reduction.After the statistical comparison and analysis of the gray value of the image, it can be found that the gray value of the low-contrast image appears to be concentrated in one region, and its gray histogram shows a single-peaked type, which needs to be stretched.Using the average gray value as the segmentation criterion, the grayness of the region larger than the average gray increases and the grayness of the region smaller than the average gray decreases. The average gray value is used as the segmentation criterion, so that the grayness of the regions larger than the average grayness increases and the grayness of the regions smaller than the average grayness decreases. Based on this feature, the Sin function can be used. The nonlinear transformation of the Sin function is characterized by a gentle change in the upper and lower peak regions and a large gradient in the middle. The transformation equation of Sin function is shown in equation (3):(3)fx,y=1271+sinπ.f0x,yb−a−π.a+b2.b−a.


## 3.1. Image Classification

There are many kinds of defects commonly found in the weld, and understanding the type and classification of weld defects is the basis for detection and identification. There are two common classification methods, one is to classify the defects according to their different locations in the weld, which can be divided into external and internal defects, where external defects contain weld tumors, biting, burn-through, slag, surface porosity, and shape defects, and internal defects contain porosity, cracks, failure to weld through, failure to fuse, and so on. The second is the classification according to GB 6417-86 “Classification and description of metal fusion weld defects” [4], including cracks, porosity, slag, failure to weld through, failure to fuse, and poor dimensional defects in six categories. Figure 1 shows the X-ray inspection images of porosity, cracking, nonfusion, and nonpenetration.Figure 1
Typical X-ray defect image.Whether it is based on manual or machine learning, it is necessary to identify the types of weld defects. The lack of welding quality grade can be divided into four categories.


## 3.2. X-Ray Image Denoising and Enhancement

In this paper, one X-ray image shown in Figure2 was randomly selected from the weld flaw image data, and the images were denoised using the five aforementioned denoising methods. The peak signal-to-noise ratio (PSNR) [18] can be used to compare and analyze the denoising effect of the five noise reduction methods, which are calculated as shown in (1) and (2)(1)PSNR=10log102552MSE,(2)MSE=∑1M∑1Nfi,j−f0i,j2MN,where fi,j is the pixel gray value of the image after noise removal, f0i,j is the pixel gray value of the input image, and the image pixels are M∗N.Figure 2
Results of filtering effect.The five images containing pores, cracks, unfused, unbroken, and no defects were randomly selected from the weld X-ray image data, and five filtering methods, namely mean filter, median filter, Gaussian filter, bilateral filter, and wavelet filter, were used to filter and reduce the noise of the weld images, and the peak signal-to-noise ratio of each filtering method was calculated after noise reduction.After the statistical comparison and analysis of the gray value of the image, it can be found that the gray value of the low-contrast image appears to be concentrated in one region, and its gray histogram shows a single-peaked type, which needs to be stretched.Using the average gray value as the segmentation criterion, the grayness of the region larger than the average gray increases and the grayness of the region smaller than the average gray decreases. The average gray value is used as the segmentation criterion, so that the grayness of the regions larger than the average grayness increases and the grayness of the regions smaller than the average grayness decreases. Based on this feature, the Sin function can be used. The nonlinear transformation of the Sin function is characterized by a gentle change in the upper and lower peak regions and a large gradient in the middle. The transformation equation of Sin function is shown in equation (3):(3)fx,y=1271+sinπ.f0x,yb−a−π.a+b2.b−a.


## 4. Weld Defect Identification Application

### 4.1. Transfer Learning

Formalizing the theoretical problem is a prerequisite for all research, and for transfer learning, Qiang Yang et al. [12] define transfer learning as follows: there exists a certain type of learning task T = {Y, f(∗)} in the source data D = {X, P(X)}, and transfer learning can use the knowledge on the source data domain DS = {XS, P(XS)} and the source task domain TS = {YS, fS(−)} to improve the target domain DT = {XT, P(XT)} and the target task TT = {YT, fT(−)} so that ft() is minimized. {YT, fT(−)} on the prediction function ft() such that ft() is minimized, that is, argmin (fT(XT), YT), where DT ≠ DS or TT ≠ TS, X denotes the feature space over domain D, Y denotes the category space, f(−) denotes the model prediction, and p(−) denotes the edge probability distribution [1, 2]. It can be seen that in the deep learning domain, more models have similar learning ability in some specific learning tasks, and transfer can be accomplished between the source and target domains by two types of target tasks with certain correlation. The difference between transfer learning and traditional machine learning can be visualized through Figures 3 and 4.Figure 3
Traditional machine learning.Figure 4
Transfer learning.As can be seen from the figure, in the traditional machine learning model, training different models requires data from each domain. The data in different domains do not have transfer learning performance on the same model. In contrast, transfer learning is able to use existing knowledge to learn new knowledge [6].


### 4.2. Interdomain Heterogeneous Transfer Learning

According to the degree of association between the source and target domain image data, transfer learning can be divided into intradomain transfer learning and interdomain transfer learning [18]. If the sample features of the source domain DS and the target domain DT are highly correlated, the intradomain transfer learning model based on convolutional neural network is constructed for target domain image recognition, that is, DS = DT, PS ≠ PT and on the heterogeneous space learning task where the images in the source and target domains have large differences, that is, DS ≠ DT and PS ≠ PT. Owing to the differences in image features between the data sets, the recognition ability using the recognition ability of the convolutional neural network model trained in the source domain becomes weak, and interdomain transfer learning can be used at this time. In the weld image defect recognition problem studied in this paper, the correlation between the ImageNet source domain data images and the target domain weld inspection images is small, and interdomain heterogeneous transfer learning is required. The source and target domain data sets in this task differ between each other, and the underlying features are universal in the image feature extraction process, but the higher-level features will differ in the integration process, so the pretrained model cannot be used directly [19].In this paper, we use the easy-to-implement VGG16 model proposed by Wang et al. [13] as the source domain pretraining model, which contains five consecutive convolutional layers named C1, C2, and so on, and five pooling layers named P1, P2, and so on, and three fully connected layers named FC6, FC7, and FC8, respectively, with the input image size of 224 × 224.For the image recognition problem in this paper, in order to effectively perform heterogeneous spatial feature transfer, the full connectivity layer in the VGG16 model needs to be modified based on the VGG16 model. The fully connected layer in the VGG16 model needs to be modified on the basis of the VGG16 model. In order to reduce the computational overhead, the number of neurons in the last two fully connected layers is reduced to 512 and 5, and the last fully connected layer corresponds to the five types of weld inspection data: porous, cracked, unfused, unperforated, and defect free [20].The VGG16 model is first trained on the ImageNet source domain data set to obtain the weight parameters of convolutional layers C1–C4, pooling layers P1–P4, and fully connected layers FC6–FC8. The model is then migrated to the weld inspection image data set for recognition, and the new image data features are extracted using the convolutional and pooling layers, and finally, the model is fine-tuned using the gradient descent algorithm and layer freezing techniques in the fully connected layer, and the image recognition results are output by a Softmax regression classifier. The detailed model framework and structural configuration are shown in Figure5.Figure 5
Interdomain heterogeneous network model.The VGG16 model used in this study for transfer learning, in which the pretrained model is trained on the ImageNet data set, which is the world’s largest database for image recognition tasks and contains 15 million images, basically covering images of objects in life, and in a large-scale computer vision based on the ImageNet database challenge, some of the images in the ImageNet database are used, and the images contain 1000 different image types, including various plants and animals, buildings, and scratches and defects on the surface of some objects. Although these images are diverse and different, they have certain correlation in low-dimensional features, and in deep learning kind can accomplish the classification and recognition tasks of various objects through the correlation of low-dimensional features building blocks of high-dimensional abstract features and thus. Since the model trained with ImageNet image data has a strong underlying feature representation capability during transfer learning and can cope well with the same type of image recognition tasks, this data set is used to pretrain the VGG16 model and then perform transfer learning training to complete the defect recognition task for weld inspection images.In the task of deep-learning-based X-ray weld inspection image recognition, the initially collected images are expanded using image data enhancement methods in order to fully and effectively extract and learn image features and improve the generalization ability of the model in the image recognition process. The sample size is increased to 5200 by randomly flipping, panning angles, contrast changes, and changing the hue of the weld inspection images, and the 5200 images are normalized to a 68 × 68 size region of interest using bilinear interpolation in order to make the images meet the input requirements of the improved convolutional neural network model. In the improved model based on VGG16, the input size of the image is required to be 224 × 224 × 3, so the size of the image needs to be expanded. The image size expansion will change the macroscopic features of the image to a certain extent, but the overall defective features of the image will not be distorted, which will enable the model to classify and identify the defects from various types of features in the secondary school. The OpenCV image processing toolkit is used to expand the size of five types of weld inspection images, that is, the image size is normalized to 224 × 224 pixels. The sample data are also divided into training and test samples, where the validation samples are half the number of training samples, as shown in Figure6 for the transfer learning data sample size.Figure 6
Transfer learning sample data.In this paper, the pretraining data set of the heterogeneous transfer model is ImageNet, and the target domain is weld inspection image data. The experiment is based on Linux Ubuntu16.04 operating system, Inter(R) Core(TM) I5-2400 CPU @3.10 GHz processor, and implemented in Tensorflow framework using python language. Considering the model accuracy and training time, the number of iterations is set to 250, the stochastic gradient descent algorithm is used, the momentum parameter is 0.9, the batch_size size is 20, and the learning rate is set toe−4.After pretraining, parameter transfer, model fine-tuning, and model prediction by the afoementioned heterogeneous migrating convolutional neural network model, end-to-end learning of heterogeneous space can be realized, and automatic extraction of image features and defects can be completed.


## 4.1. Transfer Learning

Formalizing the theoretical problem is a prerequisite for all research, and for transfer learning, Qiang Yang et al. [12] define transfer learning as follows: there exists a certain type of learning task T = {Y, f(∗)} in the source data D = {X, P(X)}, and transfer learning can use the knowledge on the source data domain DS = {XS, P(XS)} and the source task domain TS = {YS, fS(−)} to improve the target domain DT = {XT, P(XT)} and the target task TT = {YT, fT(−)} so that ft() is minimized. {YT, fT(−)} on the prediction function ft() such that ft() is minimized, that is, argmin (fT(XT), YT), where DT ≠ DS or TT ≠ TS, X denotes the feature space over domain D, Y denotes the category space, f(−) denotes the model prediction, and p(−) denotes the edge probability distribution [1, 2]. It can be seen that in the deep learning domain, more models have similar learning ability in some specific learning tasks, and transfer can be accomplished between the source and target domains by two types of target tasks with certain correlation. The difference between transfer learning and traditional machine learning can be visualized through Figures 3 and 4.Figure 3
Traditional machine learning.Figure 4
Transfer learning.As can be seen from the figure, in the traditional machine learning model, training different models requires data from each domain. The data in different domains do not have transfer learning performance on the same model. In contrast, transfer learning is able to use existing knowledge to learn new knowledge [6].


## 4.2. Interdomain Heterogeneous Transfer Learning

According to the degree of association between the source and target domain image data, transfer learning can be divided into intradomain transfer learning and interdomain transfer learning [18]. If the sample features of the source domain DS and the target domain DT are highly correlated, the intradomain transfer learning model based on convolutional neural network is constructed for target domain image recognition, that is, DS = DT, PS ≠ PT and on the heterogeneous space learning task where the images in the source and target domains have large differences, that is, DS ≠ DT and PS ≠ PT. Owing to the differences in image features between the data sets, the recognition ability using the recognition ability of the convolutional neural network model trained in the source domain becomes weak, and interdomain transfer learning can be used at this time. In the weld image defect recognition problem studied in this paper, the correlation between the ImageNet source domain data images and the target domain weld inspection images is small, and interdomain heterogeneous transfer learning is required. The source and target domain data sets in this task differ between each other, and the underlying features are universal in the image feature extraction process, but the higher-level features will differ in the integration process, so the pretrained model cannot be used directly [19].In this paper, we use the easy-to-implement VGG16 model proposed by Wang et al. [13] as the source domain pretraining model, which contains five consecutive convolutional layers named C1, C2, and so on, and five pooling layers named P1, P2, and so on, and three fully connected layers named FC6, FC7, and FC8, respectively, with the input image size of 224 × 224.For the image recognition problem in this paper, in order to effectively perform heterogeneous spatial feature transfer, the full connectivity layer in the VGG16 model needs to be modified based on the VGG16 model. The fully connected layer in the VGG16 model needs to be modified on the basis of the VGG16 model. In order to reduce the computational overhead, the number of neurons in the last two fully connected layers is reduced to 512 and 5, and the last fully connected layer corresponds to the five types of weld inspection data: porous, cracked, unfused, unperforated, and defect free [20].The VGG16 model is first trained on the ImageNet source domain data set to obtain the weight parameters of convolutional layers C1–C4, pooling layers P1–P4, and fully connected layers FC6–FC8. The model is then migrated to the weld inspection image data set for recognition, and the new image data features are extracted using the convolutional and pooling layers, and finally, the model is fine-tuned using the gradient descent algorithm and layer freezing techniques in the fully connected layer, and the image recognition results are output by a Softmax regression classifier. The detailed model framework and structural configuration are shown in Figure5.Figure 5
Interdomain heterogeneous network model.The VGG16 model used in this study for transfer learning, in which the pretrained model is trained on the ImageNet data set, which is the world’s largest database for image recognition tasks and contains 15 million images, basically covering images of objects in life, and in a large-scale computer vision based on the ImageNet database challenge, some of the images in the ImageNet database are used, and the images contain 1000 different image types, including various plants and animals, buildings, and scratches and defects on the surface of some objects. Although these images are diverse and different, they have certain correlation in low-dimensional features, and in deep learning kind can accomplish the classification and recognition tasks of various objects through the correlation of low-dimensional features building blocks of high-dimensional abstract features and thus. Since the model trained with ImageNet image data has a strong underlying feature representation capability during transfer learning and can cope well with the same type of image recognition tasks, this data set is used to pretrain the VGG16 model and then perform transfer learning training to complete the defect recognition task for weld inspection images.In the task of deep-learning-based X-ray weld inspection image recognition, the initially collected images are expanded using image data enhancement methods in order to fully and effectively extract and learn image features and improve the generalization ability of the model in the image recognition process. The sample size is increased to 5200 by randomly flipping, panning angles, contrast changes, and changing the hue of the weld inspection images, and the 5200 images are normalized to a 68 × 68 size region of interest using bilinear interpolation in order to make the images meet the input requirements of the improved convolutional neural network model. In the improved model based on VGG16, the input size of the image is required to be 224 × 224 × 3, so the size of the image needs to be expanded. The image size expansion will change the macroscopic features of the image to a certain extent, but the overall defective features of the image will not be distorted, which will enable the model to classify and identify the defects from various types of features in the secondary school. The OpenCV image processing toolkit is used to expand the size of five types of weld inspection images, that is, the image size is normalized to 224 × 224 pixels. The sample data are also divided into training and test samples, where the validation samples are half the number of training samples, as shown in Figure6 for the transfer learning data sample size.Figure 6
Transfer learning sample data.In this paper, the pretraining data set of the heterogeneous transfer model is ImageNet, and the target domain is weld inspection image data. The experiment is based on Linux Ubuntu16.04 operating system, Inter(R) Core(TM) I5-2400 CPU @3.10 GHz processor, and implemented in Tensorflow framework using python language. Considering the model accuracy and training time, the number of iterations is set to 250, the stochastic gradient descent algorithm is used, the momentum parameter is 0.9, the batch_size size is 20, and the learning rate is set toe−4.After pretraining, parameter transfer, model fine-tuning, and model prediction by the afoementioned heterogeneous migrating convolutional neural network model, end-to-end learning of heterogeneous space can be realized, and automatic extraction of image features and defects can be completed.


## 5. Experimental Results and Analysis

According to the model parameters set in the previous section, the convolutional neural network model is trained in this paper with and without parameter freezing. The average accuracy is shown in Table1, and the training loss of different methods is shown in Figure 7.Table 1
Experimental results on the weld defect data set.
Experimental methodAverage accuracyOver fitting ratioCNN-without_TF——CNN-Train_All96.341.0194CNN-Frozen_C1C295.131.0357CNN-Frozen_C1C2C393.861.0142CNN-Frozen_C1C2C3C491.971.0761Figure 7
Cross-entropy loss for different experimental methods.In Table1, the average accuracy and overfitting ratio of the model with 250 iterations under different experimental methods, where CNN-without_TF directly uses the VGG16 initial model without training and transfer learning on the ImageNet data set, CNN-Train_All indicates that the VGG16 model is trained with sufficient data and transfer learning is performed, and CNN-Forzen_Cx indicates the experimental method of freezing the training parameters of x layers in the heterogeneous transfer model and fine-tuning the model. The analysis of Table 1 shows that the CNN-without_TF method lacks sufficient data to train the model in the transfer recognition task of the weld inspection image data set, the feature extraction and expression ability is relatively weak, the recognition effect is low, and there is a very serious overfitting problem and the performance is poor, so the accuracy and overfitting ratio of the method are not counted.From the experimental results, it can be seen that when the source domain model features and training parameters are completely mounted to fine-tune the whole model into, it will make the learning ability of the target domain improve rapidly. As the adjustment process from beginning to end gradually refines the underlying features of the original input, the feature expression ability between layers is stronger, which makes the abstract features of the image better integrated and then shows better results. The more the number of frozen convolutional layers, the more the accuracy of the model decreases and the overfitting ratio of the model increases. The reason for this situation is that the more the convolutional layers are frozen, the less the parameters need to be trained in the model, and the convolutional layers from the bottom to the top reduce the computation process, the learning ability of the data is gradually weakened, and the recognition accuracy is gradually reduced, which shows that the computation and feature extraction ability on the source domain data set is weakened, the shared features among the layers are weakened due to the weakened ability to influence each other. The original bottom layer features cannot be relearned when propagated layer by layer, resulting in a gradual decrease in the feature transfer ability of the top layer. That is, the model only migrates and learns the features of the top layer but cannot realize the gradual abstraction, portrayal, and extraction of the features from the bottom layer to the top layer, so the recognition rate of the model will gradually decrease at the end.In order to test the accuracy and generalization ability of the models on the weld inspection images, 20 unlearned weld inspection defect images of each type were taken as a test set to test the generalization ability of the different freeze Table2 layer models.Table 2
Test set classification results.
Test method/defect typeGPFLLOPLOFCKRecognition accuracy/%CNN-Train_All202020191897CNN-Frozen_C1C2202018181894CNN-Frozen_C1C2C3202018171792CNN-Frozen_C1C2C3C4201917171689Recognition accuracy/%10098.7591.2588.7586.250It can be seen from the results in Table 2 that the test results in the comparison of different experimental methods are basically consistent with the results obtained from the training and verification data. When our method makes full use of the characteristics of the source domain model and training parameters to fine tune the whole model, our method has better generalization ability and the best recognition effect, and the test accuracy increases with the increase of the number of freezing layers. As the number of frozen layers increases, the test accuracy of the model decreases and the test accuracy of the weld probe image also decreases gradually. Second, a comparative analysis of different weld flaw detection image defect types shows that the correct test rate for porosity and defect-free weld flaw detection images is higher than other defect types, while the lowest correct test rate is for cracks, mainly due to the low amount of crack defect data. The test results are not ideal for unfused and unwelded defect images, mainly because their defect shapes and sizes. The image features are easily confused during the training process because of their similarity in shape, size, color, and so on, and thus are prone to errors.In summary, the convolutional neural network model based on transfer learning has good performance in the classification of small sample weld flaw images. The model significantly enhances the robustness of the convolutional neural network for classification and recognition of small samples with inadequate training. The model significantly enhances the robustness of the convolutional neural network and has good generalization ability for classification recognition of small sample images with insufficient training, and the recognition accuracy can reach more than 90%.


## 6. Conclusion

In this paper, we propose an interdomain heterogeneous transfer learning method combining CNN and transfer learning for the problem of defect recognition in weld inspection images. The effect [21] of freezing different [11] convolutional layers on the feature expression ability of the image and the [6] recognition performance of the model is investigated, and the final results show that the interdomain heterogeneous transfer learning method based on convolutional neural network not only effectively overcomes the problem of small amount of data but also effectively improves the recognition performance and generalization ability of the model, increases the average recognition accuracy of the model to more than 90%, and can effectively achieve the [22] defect recognition of weld inspection image task.



---

*Source: 5696334-2022-03-01.xml*