Neural network-based defect detection method for gluing quality on aircraft skin

Disclosed in the present invention is a neural network-based defect detection method for gluing quality on aircraft skin. The method includes: data acquisition: taking photos of aircraft skin by using a camera to acquire image data; preprocessing the acquired image data; annotating the data by using annotation software to acquire a data set for network training; establishing a defect detection network model based on feature erasure and boundary refinement, where the defect detection network model includes a feature extraction network, a semantic-guided feature erasure module, a multi-scale feature fusion network, and a defect prediction network based on boundary refinement, which are sequentially connected, the data set is used for training the network model, and trained model parameters are saved; and detecting a directly collected skin gluing image by using the trained network model and outputting detection results.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to Chinese Patent Application No. 202310676359X, filed with the China National Intellectual Property Administration on Jun. 8, 2023 and entitled “NEURAL NETWORK-BASED DEFECT DETECTION METHOD FOR GLUING QUALITY ON AIRCRAFT SKIN”, which is incorporated herein by reference in its entirety or part.

TECHNICAL FIELD

The present invention belongs to the technical field of defect detection of aircraft skin, and in particular relates to a neural network-based defect detection method for gluing quality on aircraft skin.

BACKGROUND

With rapid development of science and technology in China, aircraft play a crucial role in various fields such as military, transportation, and agriculture. As an important part of aircraft, ensuring manufacturing quality of aircraft skin is a crucial factor in determining overall performance and safe operation of the aircraft.

The primary cause of surface damage and defects on the aircraft skin lies in cyclic pressurization during takeoff and depressurization during landing, causing periodic expansion and contraction of a skin surface, thereby resulting in micro cracks in materials around rivets on the surface of the aircraft. Especially, harsh flight conditions can further accelerate crack propagation and induce corrosion. Such defects not only can affect the aesthetic surface of the aircraft skin, but also can destroy the surface integrity of the aircraft skin to a certain extent, leading to a reduction in structural strength that critically endangers the lives and property security of pilots, and passengers.

Traditional aircraft skin defect detection is commonly realized through visual inspection by technicians, which is closely related to the experience, sense of responsibility, and the like of the technicians, such that the conventional method exhibits significant limitations, is prone to problems such as missed defects, false defects and oversight defects, and is low in detection efficiency. With the continuous improvement of performance of aircraft equipment, accelerating development of corresponding detection technologies towards smart, integrated, digital, and online-enabled solutions is urgently needed. At present, most aviation manufacturing enterprises in China have widely adopted digital measurement equipment for surface defect detection of the aircraft skin, such as laser radars, laser trackers, and total stations. While transitioning from traditional detection methods dependent on tooling like mold lines and templates, the industry remains predominantly reliant on manual inspections by the technicians. In order to solve the prominent problems of poor consistency, low efficiency, and the like due to heavy reliance on manual labor for the acquisition of a detection technology, the neural network-based defect detection method for gluing quality on aircraft skin is proposed.

SUMMARY

In view of the technical problem, the present invention provides a neural network-based defect detection method for gluing quality on aircraft skin.

The present invention adopts the following technical solution to solve the technical problem.

The neural network-based defect detection method for gluing quality on aircraft skin includes the following steps:

Preferably, in S300, the feature extraction network being configured to extract the multi-scale feature map, and the semantic-guided feature erasure module being configured to process the multi-scale feature map to enable the predefined region of the feature map to have the predefined probability of being set to zero, include:

Preferably, in S300, the defect prediction network based on boundary refinement being configured to perform prediction on the basis of the fused multi-scale feature map to obtain classification prediction results and Bbox prediction results, includes:

Preferably, each of the coarse classification branches and each of the coarse Bbox prediction branches both include 4 3×3 convolutional layers and 1 1×1 convolutional layer, and S322 includes:

Preferably, the calculation formula of S32222 is as follows:

Preferably, a predefined network loss function includes classification loss Focal Loss and Bbox prediction loss GIoU Loss, where the classification loss includes coarse classification loss Losscls_coa and final refined classification loss Losscls_ref, and the Bbox prediction loss GIoU Loss includes coarse prediction loss Lossreg_coa and refined prediction lossLossreg_ref;

DETAILED DESCRIPTION OF THE EMBODIMENTS

In order to provide a better understanding of the technical solution of the present invention for those skilled in the art, the present invention will be described below in detail with reference to the accompanying drawings.

In an embodiment, as shown in FIG. 1, a neural network-based defect detection method for gluing quality on aircraft skin includes the following steps:

Specifically, a schematic structural diagram of the defect detection network model is shown as FIG. 2.

According to the neural network-based defect detection method for gluing quality on aircraft skin, the defect detection network model based on feature erasure and boundary refinement can quickly and accurately achieve non-destructive testing of gluing defects of the aircraft skin, thereby promoting the high-quality intelligent manufacturing process of the skin.

In an embodiment, as shown in FIG. 3, in S300, the feature extraction network being configured to extract the multi-scale feature map, and the semantic-guided feature erasure module being configured to process the multi-scale feature map to enable the predefined region of the feature map to have the predefined probability of being set to zero, include:

Specifically, after defect images are extracted by means of the residual network, three feature maps F1, F2 and F3 with different scale sizes are obtained. In order to enhance the robustness of the network, the semantic-guided feature erasure module is adopted for feature processing, such that some regions of the feature map have a certain probability of being set to zero. In the embodiment, the probability of DropOut is set to 0.4, i.e., each element in the fk feature has the probability of 0.4 being set to 0, which can enhance the feature extraction ability of the neural network and enable the extracted features to have more robustness. Due to high similarity between fk and global semantic information, the fk feature is more discriminative compared to other features.

Further, the feature fusion module often includes a Feature Pyramid Network (FPN), which exists to acquire feature maps with high-level semantic information and low-level position information, and then deeply fuse features of different scales. Due to different sizes of the feature maps in different layers of ResNet, receptive fields of the feature maps mapped back to original images are also different, and usually high-level features are more semantic, while low-level features belong to pixel-level position information. By using horizontal connection and vertical connection of the FPN and other manners for feature fusion, high-level semantic features and low-level pixel features can be effectively fused. Due to small surface defects on the aircraft skin and unclear features between the defects and the background, the high-level feature maps with semantic information are fused through a top-down feature fusion module to enable bottom-level pixel level features to have the high-level semantic information, thereby improving detection accuracy. To this end, a top-down feature pyramid structure is adopted to fuse foreign object features. In the method of the present invention, instead of a five-layer FPN structure of a classic target detection algorithm RetinaNet, only four layers of feature maps with different scale sizes are selected to construct the feature pyramid structure, with number of channels being 256, 512, 1024, and 2048, respectively. After the 1×1 convolution, the number of the channels is unified to 256 dimensions. This can reduce the number of parameters during a detection process while ensuring defect detection accuracy for the aircraft skin, thereby optimizing the detected network structure, reducing computational power consumption, accelerating the detection speed to a certain extent, and achieving the purpose of saving training time.

In an embodiment, as shown in FIG. 2, in S300, the defect prediction network based on boundary refinement being configured to perform prediction on the basis of the fused multi-scale feature map to obtain classification prediction results and Bbox prediction results, includes:

In an embodiment, as shown in FIG. 4, S321 includes:

Specifically, since most of gluing defects for the skin are slender and barely visible, morphological features are easily ignored by the network model, and the defect feature enhancement network is used to enhance gluing defects for the skin, thereby ensuring effective enhancement of the morphological features.

In an embodiment, as shown in FIG. 2, each of the coarse classification branches and each of the coarse Bbox prediction branches both include 4 3×3 convolutional layers and 1 1×1 convolutional layer, and S322 includes:

In an embodiment, as shown in FIG. 5, S3222 includes:

Further, the calculation formula of S32222 is as follows:

Further, according to a residual learning idea, the obtained features Fbbox_g or Fcls_g with boundary awareness or the shape enhanced features Fs are subjected to element summation, and then inputted into the 1×1 convolutional layer to obtain a final refined classification score Clsrefine or a Bbox bias prediction result Bboxrefine.

Finally, losses among coarse classification, coarse Bbox prediction, final classification prediction, final Bbox prediction, and true labels are calculated, respectively.

In an embodiment, a predefined network loss function includes classification loss Focal Loss and Bbox prediction loss GIoU Loss, where the classification loss includes coarse classification loss Losscls_coa and final refined classification loss Losscls_ref, and the Bbox prediction loss GIoU Loss includes coarse prediction loss Lossreg_coa and refined prediction loss Lossreg_ref;

the classification loss Focal Loss is calculated as follows:

Specifically, the classification loss includes two aspects: coarse classification loss Losscls_coa and final refined classification loss Losscls_ref In order to alleviate the problem of imbalance between positive and negative samples, Focal Loss is used as the classification loss; and the Bbox prediction loss also correspondingly includes two types: coarse prediction loss Lossreg_coa and refined prediction loss Lossreg_ref, both of which use GIoU loss as the Bbox prediction loss.

Further, the neural network is trained by using a back propagation algorithm and a stochastic gradient descent algorithm, and training weights are saved.

Firstly, the back propagation algorithm is used to calculate the gradient of the loss function relative to each parameter, and a chain rule is used to traverse the network in a reverse order (namely, from an output layer to an input layer) to calculate the gradient. The back propagation algorithm will repeatedly use intermediate values saved in forward propagation to avoid duplicate calculations and save computation time.

If the gradient descent algorithm is used, the computation cost for each independent variable iteration is O(n), which increases linearly with n (the number of samples). Therefore, when a training data set is larger, the computation cost for the gradient descent in each iteration will be higher, and the computation cost generated during iterations can be reduced by using stochastic gradient descent. In each iteration of the stochastic gradient descent, the algorithm randomly selects a part of samples and updates model parameters by calculating the gradients of the samples, gradually approaching an optimal solution. We randomly and uniformly sample an index i from a data sample, where iϵ1, . . . , n, and calculate the gradient ∇J(θ) to update the weights θ:

In a repeated training process, the steps of forward propagation, calculating loss, back propagation, and updating weights and biases are repeated until the model converges, and whether the model converges is determined by comparing the changes in the loss function values.

The training weights are saved, once training of the model is complete, the training weights are saved. These weights can be used for prediction. The weights are saved to a file for reloading when needed.

According to the neural network-based defect detection method for gluing quality on aircraft skin, the defect detection network model based on feature erasure and boundary refinement can quickly and accurately achieve non-destructive testing of gluing defects of the aircraft skin, thereby promoting the high-quality intelligent manufacturing process of the skin. The prominent problems in the prior art of poor consistency and low efficiency due to heavy reliance on manual labor for the acquisition of a detection technology, are solved.

The above are preferred embodiments of the present invention. It should be noted that, for those of ordinary skill in the art, a plurality of improvements and modifications may be made without departing from the principle of the present invention, and the improvements and modifications are also regarded to be within the protection scope of the present invention.