# Defect Detection in Composite Products Based on Sparse Moving Window Principal Component Thermography

Authors: Jing Jie; Shiqing Dai; Beiping Hou; Miao Zhang; Le Zhou
Journal: Advances in Polymer Technology
 (2020)

---

## Abstract

As a nondestructive testing (NDT) technology, pulsed thermography (PT) has been widely used in the defect detection of the composite products due to its efficiency and large detection range. To enhance the distinction between defective and defect-free region and eliminate the influence of the measurement noise and nonuniform background of the thermal image generated by PT, a number of thermographic data analysis approaches have been proposed. However, these traditional methods only consider the correlations among the pixel while leave the time series correlations unmodeled. In this paper, a sparse moving window principal component thermography (SMWPCT) method is proposed to incorporate several thermal images using the moving window strategy. Also, the sparse trick is used to provide clearer and more interpretable results because of the structure sparsity. The effectiveness of the method is verified by the defect detection experiment of carbon fiber-reinforced plastic specimens.


---

## Body

## 1. Introduction

The nondestructive testing (NDT) [1, 2] is a method to detect the presence of defects or unevenness in the tested objects by utilizing the characteristics of light, heat, magnetism, or electricity, which do not affect the performance of the testing objects. The nondestructive nature of the inspection makes NDT more and more popular, such as the detection of defects on the surface and subsurface of composite materials. Among different types of NDT methods, pulsed thermography has been widely used and studied owing to its fast detection speed, noncontact, and nonpollution [3]. Through scanning, recording, or observing the change of the surface temperature field, which is caused by the difference of heat transfer to the deep layer, the NDT will realize the detection of the surface and internal workpiece defects or analyze the internal structure [4–6].For enhancing the detection efficiency and visibility, several signal processing and data analysis methods have been proposed. Among them, the most straightforward approach is to reduce the measurement noises and eliminate the nonuniform background. Thermographic signal reconstruction (TSR) method [7] converts time domain signals into frequency domain signals, which separated the nonuniform noise and significantly reduce noise interference by removing noise part in the data reconstruction. Besides, the differential absolute contrast (DAC) [8] and mathematical morphology (MM) [9] are also proposed for the same purpose, which are both based on the reconstruction of defect-free images so that the nonuniform background has been eliminated by extracting the features of the original thermal images. However, both TSR and DAC have processed the time domain data while they ignored the spatial information. Hence, penalized least square (PELS) [10] has been developed to utilize the time series thermal images and their spatial information simultaneously. Furthermore, Chang et al. have decomposed the thermal images into high-frequency noises, low-frequency background, and signal information using the signal decomposition, which is called the multimaintenance ensemble empirical mode decomposition (MEEMD) method [11]. Even though PELS and MEEMD have been proven the effective signal processing methods for NDT, they are still quite time-consuming, and the model selection is hard to achieve due to a large number of model parameters. In addition, the noise reduction and background removal methods are only suitable for defect detection of single temperature attenuation curve or single thermal images.Another kind of approach is the thermographic data analysis method, which extracts the principal features from multiple thermal images and automatically recognize the defects using these features or loading matrixes. Recently, the data analysis and feature extraction technologies have been widely used in process modeling, monitoring, and optimization areas [12–17]. Utilizing their advantages, the main information can be maintained with few features and the minimum reconstruction errors [18–20]. The higher-order statistics (HOS) [21] extracts the features of red-hot image sequences and compresses the feature information into a unique image for defect detection. The pulsed phase thermography (PPT) method separates one-dimensional Fourier for each pixel of the thermal imaging sequence and judge defects according to amplitude and phase [22]. Besides, principal component thermography (PCT) [23] applied principal component analysis (PCA) [24] to the thermal image data processing, which has the advantages of feature extraction, data compression, and noise reduction. Compared with the original data, the obtained PCT feature map has a significant improvement in defect significance. Similar to PCT, several extension works have been made for improvement of the defect detection performance, such as stable principal component pursuit (SPCP) [25], sparse principal component thermography (SPCT) [26, 27], and independent component thermography (ICT) [28].Among them, SPCT is an improved algorithm for PCT, which can obtain the sparse principal components by applyingL1 constraint [29]. Indeed, the components containing few features will enhance the interpretability and visibility of the detection results. Moreover, an improved method called CCIPCT is proposed to use a shorter computational cost to estimate the covariance matrix and singular value decomposition (SVD) when calculating the principal component. The main advantage of CCIPCT is its faster performance in higher sequential acquisition [30]. However, PCT, SPCT, and CCIPCT only considered the correlations within the signal image and ignored the dynamics of heat transfer. The pulse thermal imaging defect detection method mainly determines the defect location based on the inconsistent heat transfer rates of different areas inside the object. Hence, there are strong cross-correlations among the adjacent temporal thermal images. In this paper, sparse moving window principal component thermal imaging (SMWPCT) is proposed to extract both cross-correlations from the temporal and spatial scales. For this purpose, the moving window strategy is utilized to cover several adjacent temporal thermal images within a period of time. The principal features of these moving windows are further extracted by sparse PCT, which will provide clearer and more interpretable detection results.The rest of this paper is structured as follows. In the second section, the structure of the data collected from the pulse thermal imaging is briefly introduced. Then, the moving window strategy-based SPCT is proposed with the detailed algorithm in the third section. Next, the feasibility and effectiveness of the proposed method are demonstrated by the carbon fiber-reinforced plastic (CFRP) specimen. Finally, some conclusions are made.


## 2. Thermographic Data Preprocessing

The pulse thermal imaging data are collected as follows. Firstly, the tested object is firstly heated by the flash lamps using the pulse signals. After that, it is cooled naturally. During the whole heating and cooling period, the thermal images are acquired by the infrared camera, which is shown in Figure1. The collected thermal data are the grayscale image and the pixel value of the image which represents the heating degree of the corresponding position. If there is a defect area within the object and the material is nonuniform, it will result in the discontinuity of heat conduction inside the object. Hence, the temperature of the defect area will be abnormally higher or lower than that of the surrounding area, which results that the pixel value of the defect area is different from that of the surrounding area on the thermal image.Figure 1
Thermal imaging data acquisition.For some obvious defects inside the testing object, it is also apparent to find out the defect areas directly from the original thermal images. However, it is born to be an unattractive work to check every image on visual observations. Also, some tiny defects are more difficult to distinguish by sight. On the contrary, the uneven heating will lead to uneven background, and the presence of measurement noise is inevitable, which further increases the detection difficulty. Finally, suppose there aren frames of thermal images have been collected, in which each image consists of h×w pixels. For most data analysis models, the three-dimensional data cannot be directly applied. Therefore, the thermal image data processing methods are needed to reduce the number of thermal images detected by vision and improve the significance of defects.For the three-dimensional matrixh×w×n, the common data processing method is to expand the original data to a two-dimensional matrix. According to the chronological order, each thermal image can be converted to a row vector of length h×w, which is shown in Figure 2. Then, a two-dimensional matrix with n×hw size can be obtained, in which each row represents the original thermal image data, and each element represents an image pixel value. Finally, the normalization of the processing data is made, which firstly subtracts the average value of each row and then divides it by the standard deviation.Figure 2
Three-dimensional expansion.


## 3. Sparse Moving Window Principal Component Thermography

In this section, the traditional PCT method is firstly extended using the moving window strategy. After that, the sparse moving window principal component thermography (SMWPCT) method is proposed, which follows the detailed model parameter derivation. In PCT, the original thermal image data are projected along the maximum orthogonal direction, in which the principal components are extracted. Since PCT is built based on the preprocessing data two-dimensional matrix, it indicates that the thermal image at a sampling interval is independent with its historical thermal values. When the sampling interval is large, such assumption is valid. However, the pulse thermography sampling interval is extremely short. Hence, it becomes necessary to consider both cross-correlations between the pixels in different regions of the single thermal image and the autocorrelations of pixels in the same region at different sampling intervals.For this purpose, the moving window strategy is introduced before PCT is used, which is named as moving window PCT (MWPCT). In MWPCT, the augmented matrix containing several past values is constructed. Using the moving window, the time series correlations among the thermal images can be effectively extracted, and it is helpful to distinguish the defect region and the normal region. Assume that the preprocessed thermal image data matrixX contains n thermal images, and each image is composed of m=h×w pixel values. Hence, X∈Rn×m can be expressed as follows:(1)X=x11x12⋯x1mx21x22⋯x2m⋮⋮⋱⋮xn1xn2⋯xnm=x1x2⋮xn,where xi∈Rmi=1,2,…,n is the m-dimensional row vector, and it indicates that a single thermal image with m pixel values.Using the moving window strategy, the augmented matrix of the original observationX is constructed as follows:(2)X¯=xlxl−1⋯x1xl+sxl+s−1⋯xs+1⋮⋮⋱⋮xnxn−1⋯xn−l+1,in which l is the moving window length and s is the moving window step. Usually, these two parameters make a trade-off between the model accuracy and calculation complexity. When the window size l is large and moving step s is small, the model accuracy will improve while the calculation complexity increases and vice versa. Based on the augmented matrix X¯, the first principal component (PC) is obtained by(3)maxX¯p2Subject top2≤1,in which p is the principal eigenvector using the singular value decomposition, and it is also called the loading vector. ⋅2 denotes the L2 norm. The first PC can be calculated using t=X¯p, which can be treated as the linear combination of X¯. By calculating the first PC, the maximal correlated feature is obtained. Next, replacing X¯ using the model reconstruction error can obtain PCs orthogonal to each other in a similar manner [31]. Usually, the primary information of X¯ is mainly concentrated on several components. Therefore, few PCs will represent the original thermal images, and the defects can be visualized using these PCs. Assuming that the number of selected PCs is k, the size of the extracted feature matrix is hwl×k. To visualize the final results, each PC can be reshaped into a two-dimensional matrix of h×w. Hence, a total of l×k loading images can be generated, in order to achieve data compression and reduce the number of images to be observed. Due to the difference of features between defects and background, they are separated into different PCs and appear in different loading images. In addition, the measurement noise is retained in the residual subspace to achieve the purpose of noise suppression.Even though MWPCT is able to extract both autocorrelations and cross-correlations in the 2-D expanded matrix, it still inherits some disadvantages of PCT. At first, each PC is linearly weighted by the original data, which means that all the elements in the loading vectors are usually nonzero, which brings out the difficulty and disturbance in final detection. To highlight the visual results and reduce the detection noise, a sparsity penalty can be added on the loadings to use few features and enhance the interpretability of the final results. Hence, the sparse moving window principal component thermography (SMWPCT) method is further developed next. The optimization problem of the first PC extracted by SMWPCT is usually expressed as follows:(4)maxp˜X¯p˜22−γp˜0Subject top˜2≤1,ormaxp˜X¯p˜2−γX¯p˜1Subject top˜2≤1,where γ is the tuning parameter that controls the sparsity of p˜, ⋅0 denotes the L0 norm, which is calculated as the total number of nonzero elements in the vector and it represents the vector sparsity, and ⋅1 denotes the L1 norm, which represents the sum of the absolute values of the elements in the vector, and it is usually used to obtain the sparse result. Similar to MWPCT, the other PCs that are orthogonal to each other can be obtained repeatedly by performing the definition step and replacing X¯ with the current errors.The sparse approach is able to select the most related PCs and restrict the feature variables that are not closely related to zero. In such manner, noise is further eliminated, and the dimension reduction becomes more condensed and easier to be interpreted. To make the model parameter estimation procedure more effective, the optimization problem can be transformed to be a Lasso regression problem; Zou et al. have introduced anL2 norm regularization term in the objective function of the optimization function [32], which is given as(5)minP,ΩX¯−X¯PΨT2+γ1∑i=1Iψi22+γ2∑i=1Iψi1Subject toPTP=Ι,where P is the loading matrix and Ψ=ψ1ψ2…ψI is a sparse approximation of P with I PCs. γ1 and γ2 are model tuning parameters. The combination of L1 and L2 constitutes an elastic net penalty and encouraging grouping which indicates that the variables of strong correlations will appear or disappear together [27].Finally, the entire algorithms for the SMWPCT-based thermographic data analysis method are summarized as follows:(1)
Collect the thermal image data based on the pulsed thermography technology(2)
Rearrange the three-dimensional matrix to a two-dimensional form and normalize the measurement(3)
Select the appropriate moving window size and step size, and perform a moving window on the augmented matrixX¯(4)
Estimate the model parameterP and the sparse matrix Q by solving the optimization problem of equation (5)(5)
Reconstruct the thermal images based onQ(6)
Observe the reconstructed thermal image to obtain the defect detection result


## 4. Case Study

In this section, a tested carbon fiber-reinforced polymer (CFRP) with subsurface defects is demonstrated to validate the feasibility of the proposed method. CFRP is obtained by pressing and drawing a plurality of continuous fibers and resin. It has been widely used in military, aerospace, racing, and other fields [9].In this case, several Teflon strips were inserted into the fiberboard before the resin transfer molding to mimic the defective areas. In the tested CFRP board, there are totally ten defect areas, where the left bottom rectangle is a surface defect and the other nine rectangles are subsurface defects. The depth of these subsurface ones is different. Three defects on the left side were covered by one layer of fiber sheet while the middle column was beneath two layers and three on the right were covered by three layers. The thickness of each fiber sheet is about 0.26 cm. Besides, three kinds of defects are designed, where their sizes are 1.6 cm × 1.6 cm, 0.8 cm × 0.8 cm, and 0.4 cm × 0.4 cm, respectively (Figure3).Figure 3
Illustration of defective regions.To obtain the pulsed thermal imaging data, two 3000W flashes are used to heat the acquisition in the form of thermal pulse, and the heating time is about 3 ms. An infrared camera (TGS-G100EXD, NEC) is installed to collect thermal images in the reflection mode. The resolution of this camera is 320 × 240 pixels, and the sampling rate is 30 frames per second. A computer is connected to the camera for thermal imaging data processing. The process is cut to get the thermal images of the region of interest (ROI). Finally, there are 54 thermal images with 105 × 120 pixels which are collected. In Figure4, the 1st, 10th, 20th, 30th, 40th, and 50th thermal images of the thermal images are illustrated. The color bars indicate the pixel values. From the original thermal images, it can be seen that it is difficult to infer the defect locations owing to the nonuniform background and noises.Figure 4
Illustration of defective regions. (a) Img-1. (b) Img-10. (c) Img-20. (d) Img-30. (e) Img-40. (f) Img-50.
(a)
(b)
(c)
(d)
(e)
(f)In order to evaluate the performance of the proposed SMWPCT method, several alternative methods PCT, SPCT, and MWPCT are also tested using the same thermal images. The experimental hardware platform is Window10, Intel i5-7500 CPU@3.40 GHz and 16 GBRAM. Also, the visibility of the defect area is measured by the signal-to-noise ratio (SNR) indicator, which refers to the ratio of signal to noise in an electronic device or system [33, 34]. In this paper, three kinds of SNR considered in [33, 34] are tested, which can be expressed as(6)SNR1=Μdef−Μnσn,SNR2=10log10Μdef−Μn2σn2,SNR3=Μdef−Μnσdef2+σn2/2,where Μdef is the average pixel value of the defect area and Μn is the average pixel value of the nondefect area. σdef is the standard deviation of the pixel in the defect area and σn is the standard deviation of the pixel in the nondefect area. SNR is dimensionless, which reflects the contrast relationship between the defective area and the nondefective area. The larger the value of SNR, the more significant the defect.The traditional PCT method is built based on all 54 thermal image samples. The detection results using the first six loading images of PCT are given in Figure5(a). The loading values are represented by the red and blue color bars, in which the redder one indicates the larger value while the bluer one represents the lower value. From the final results, it can be seen that the PCT loading of the first several PCs can better reveal the position of the defect areas compared to the original thermal image. Since the heat transfer efficiency of the defect area is lower than that of the normal one, the loading value is smaller. And most of the defect information exists in the first loading image. However, PCT still has certain limitations. On the one hand, there are still more background or noise information in the first loading image, which cause some disturbances in defect detection and recognition. On the other hand, the defect information can still be observed more or less in the remaining loading images, which brings out difficulty on the selection of number of PCs.Figure 5
Results: (a) PCT, (b) SPCT, (c) MWPCT, and (d) SMWPCT.
(a)
(b)
(c)
(d)By applying sparsity penalties to the loadings, SPCT is able to reduce the background and noise interference of the result, which makes it more compact and interpretable.In SPCT, the sparsity parameter is designed to penalize the loadings of different principal components, and it corresponds to an upper bound on the L1-norm of the BETA coefficients. Due to the nature of the L1 penalty, some coefficients will be shrunk to zero if the sparsity parameter is large enough. By contrast, SPCA results in a regular PCA when it equals to 0. Zou et al. [32] have specified the choice of penalty parameter, and it is set to 500 in this article. The results are shown in Figure 5(b). Compared with PCT, the loading image of SPCT is much sparse, in which the shallower defects are highlighted in the first loading image. It can be also seen that the deepest defect is better displayed in the second loading image. In the third and subsequent loading images, the loading image becomes very sparse due to much useful information retained. Therefore, the defect detection process for the thermal image can ignore these loading images, which also reduces the number of images that needs to be observed.Next, MWPCT is compared with the moving window strategy. For the moving window technology, the window size and the step size are two model parameters to make a trade-off between the accuracy and computational load [35]. To select the proper values for these model parameters, several simulations have been made. Finally, the moving window size is set to 7, and the step size is 5 in MWPCT in this case. The experiment result is shown in Figure 5(c). Compared to PCT, the defect areas in the first loading image of MWPCT are more prominent and are greatly reduced in the second loading image. It indicates that more accurate features have been captured using MWPCT. Besides, MWPCT concentrates the defect information in the first loading image so that we can only pay attention to the first loading image to identify the defect. Due to the inclusion of more defect information, the defect area is significantly improved, and the image appears that the defect area is darker in color.Finally, the proposed SMWPCT method is tested, which combines the advantages of SPCT and MWPCT. It extracts the defect features based on the moving window strategy with sparsely loading images. The detection results are shown in Figure5(d). It can be seen that both the concentration of defect features and the sparseness of irrelevant information are reflected in SMWPCT, and the saliency of defect areas has also been improved. The result is also quite reasonable since the proposed SMWPCT method extracts both cross-correlations from the temporal and spatial scales, and more accurate and compact features are obtained. Based on that, the defect detection procedure becomes straightforward and easy to be operated.Moreover, the defect visibility is quantified by the signal-to-noise ratio indicator. Since the three defects in the last column are covered by three layers of fiberboard, when the heat conduction reaches the thickness of three-layer fiberboard, the loss of energy leads to poor visibility of the thermal image. Therefore, the calculation of the SNR index does not include the three deepest defects. Three kinds of different SNR values based on PCT, SPCT, MWPCT, and SMWPCT are listed in Tables1–3. It can be seen that the performance of the proposed SMWPCT is superior to several alternatives in most cases even based on different SNR indicators. Also, the results have verified the effectiveness and reliability of SMWPCT.Table 1
SNR1 of different methods.
Methods/SNR1/defect
Def_1
Def_2
Def_3
Def_4
Def_5
Def_6
Def_7
Sum
PCT
3.35
2.48
2.76
2.66
1.10
0.79
0.90
14.04
SPCT
3.67
2.70
3.01
2.89
1.21
0.88
0.98
15.32
MWPCT
4.32
2.94
3.19
3.11
1.14
0.77
0.94
16.42
SMWPCT
4.45
3.21
3.62
3.37
1.39
0.99
1.11
18.15Table 2
SNR2 of different methods.
Methods/SNR2/defect
Def_1
Def_2
Def_3
Def_4
Def_5
Def_6
Def_7
Sum
PCT
10.50
7.88
8.81
8.48
0.85
−2.00
−0.87
39.39
SPCT
11.28
8.62
9.56
9.23
1.64
−1.15
−0.20
41.68
MWPCT
12.71
9.38
10.08
9.85
1.17
−2.27
−0.53
46.00
SMWPCT
12.97
10.13
11.18
10.55
2.86
−0.05
0.90
48.65Table 3
SNR3 of different methods.
Methods/SNR3/defect
Def_1
Def_2
Def_3
Def_4
Def_5
Def_6
Def_7
Sum
PCT
1.84
1.36
1.52
1.46
0.61
0.44
0.50
7.73
SPCT
1.87
1.38
1.54
1.48
0.62
0.45
0.50
7.83
MWPCT
2.08
1.42
1.54
1.50
0.55
0.37
0.45
7.90
SMWPCT
2.00
1.45
1.63
1.52
0.63
0.45
0.50
8.17


## 5. Conclusion

Due to the existence of noises and nonuniform background, the artificial detection of the defects inside the composite products is difficult to achieve. Hence, it is necessary to apply the data analysis approaches based on the thermal imaging data. In this paper, a SMWPCT method is proposed, which combines SPCT with the moving window strategy. In SMWPCT, both dynamic and static information of the thermal imaging data can be captured under the sparse structure. The experimental results have illustrated the effectiveness of the proposed method. It reveals that not only the correlations between the pixels in different regions of the single thermal image but also the correlations of pixels in the same region at different sampling intervals are extracted in SMWPCT. Hence, the performance of the proposed method is superior to several alternatives.



---

*Source: 4682689-2020-02-07.xml*