# Blowhole Detection Based on Bidirectional Enhancement and Omnidirectional Analysis for X-Ray Inspection of Castings

Authors: Xinyue Zhao; Jingjing Liang; Zaixing He; Shuyou Zhang
Journal: Mathematical Problems in Engineering
 (2019)

---

## Abstract

Small blurred blowholes are difficult to detect, especially those in castings having a complex structure. In this paper, we propose a blowhole detection method based on bidirectional enhancement and omnidirectional analysis for X-ray inspection of castings. After obtaining the X-ray images of a casting from the X-ray inspection equipment, there are two main processes to detect the blowholes. First, in the extraction process for the potential defect region, we propose a method based on bidirectional enhancement that highlights the blurred defects and models the nondefective structure. Second, in the precise defect identification process, we propose a method to distinguish real defects by 3D model fitting-based omnidirectional analysis. The experimental results indicate that the proposed method has a higher performance in blowhole detection than comparable methods.


---

## Body

## 1. Introduction

Various unintended internal defects, including blowholes, fractures, inclusions, or slag formation, can occur during the metal casting process [1, 2]. Some internal defects can negatively affect the strength of the product. Thus, it is necessary to check metal castings to ensure that they will be safe when the pieces are put into use [3]. Compared with other types of defects, blowhole defects are smaller in size and distributed in a scattered manner, which makes it difficult to detect blowholes in blurred radiographic images, especially those in castings having a complex structure. Castings having a complex structure are those with an uneven thickness, which appears as a number of different light and dark areas in the X-ray images. Existing technology has difficulty dealing with such cases. In this research, we focus on addressing this issue.Nondestructive testing encompasses a wide group of techniques that analyze the inner parts of materials, components, or a system without causing damage [4–6]. X-ray testing is one such nondestructive testing technique [7]. It works through visual or computer-aided analysis of X-ray images [8]. Compared with visual analysis, automatic detection is objective and reproducible. In industry, X-ray testing has been accepted as a way to detect the inner defects in castings via image processing techniques. Blowhole defects always show up as bright ellipses in a digital X-ray image. The existing approaches for detecting blowholes are almost always based on the local discontinuity feature to classify each image pixel as defective or not [9].Early defect detection methods were based on a considerable amount of reference data or preparatory work. The reference images are prepared before detection and are then compared with the test images. The significant differences would be identified as defects [8]. The distribution of gray values in the reference image needs to correlate with the current image, which can be difficult to realize. The MODAN-Filter [10] is a median filter with adapted filter masks, which is designed to differentiate the structural contours of the casting piece from the casting defects. The distribution of the mask should coincide with each part of the structure, so the size of the filter masks and the values for their coefficients have to be chosen [8].The related works in defect detection involve cases of no prior knowledge of the piece’s structure and a great deal of preparatory work [8]. Model-based methods [11] refer to methods that are based on comparing the target image with a model image, which is estimated using a filter. Thresholding methods [12–14] select adaptive threshold values to determine if a pixel is part of a defect region. Zhao et al. [15] proposed a method based on the grayscale arranging pairs (GAP) feature for defect detection. Zhao et al. [16] proposed a method to discriminate defects using a randomly distributed triangle (RDT) feature, which was extracted to represent the geometric characteristic of each defect. Kumar et al. [17] presented a novel approach for multiclass weld flaw classification by means of a gray-level co-occurrence matrix- (GLCM-) based texture feature extraction technique. Previous work on a multiple-view detection method was conducted by Mery et al. who introduced a method for precisely detecting defects by tracking and analyzing the correspondence between the different views [18–20]. Girshick et al. [21] proposed a method for detection that extracted proposed regions from the input image and computed CNN features for classification.Although many defect detection methods have been proposed, the problem is far from completely solved. Modern industry has escalated to higher requirements for detection accuracy, especially for blurred defects. It is still a challenge to detect small blurred blowholes precisely. It is hard to segment the low-contrast defects from the background. In addition, complex structures always disturb defect segmentation. The edges of structures may break the edge of the defects, and some defect-like structures lead to false positives. Furthermore, the noise in X-ray images is interference in detection efforts.To solve these problems, we propose a blowhole detection method based on bidirectional enhancement and omnidirectional analysis for X-ray inspection of castings. The procedure in our method is classic in nature: (1) generate a reference image, (2) extract potential defects by subtraction, and (3) identify the real defects from the potential defects. The first and third steps are crucial. In both steps, we have proposed new methods to improve the final detection accuracy. Therefore, our main contributions are twofold. The first is a bidirectional enhancement-based extraction technique for potential defect regions. For image enhancement, the commonly used methods have involved one-way enhancement, which strengthens not only the structure but also the defect, leading to difficulties in detection. Unlike those enhancement methods, bidirectional enhancement can strengthen the defect and weaken the structure simultaneously, thus improving the accuracy of potential defect region extraction. The second contribution is omnidirectional analysis-based precise defect identification. With respect to defect recognition, the existing methods mainly analyze potential defects by analyzing the grayscale curves along several directions. These methods can easily misjudge false defects and consider them to be real defects because some false defects have curves similar to real defects along some directions, but curves along other directions that may be different. Omnidirectional analysis is used to analyze all directions of the defects using a 3D model, which can reduce the false detection ratio with the help of the proposed multithreshold AdaBoost classifier.The rest of the paper is organized as follows: first potential defect region extraction is explained in Section2. Section 3 describes the precise defect identification method. The experiments and the results that were obtained are presented and discussed in Section 4. Finally, some conclusions are given in Section 5.


## 2. Potential Defect Region Extraction via Bidirectional Enhancement

With respect to image enhancement, the commonly used methods involve one-way enhancement, which strengthens the structure while strengthening the defect, making detection difficult. In this section, we introduce a process for extracting potential defect regions based on bidirectional enhancement. Unlike the commonly used enhancement methods, the bidirectional enhancement method strengthens the defect while simultaneously weakening the structure. Forward enhancement refers to improving the contrast between the defects and the background, that is, highlighting the blurred defects. Reverse enhancement means enhancing the casting structure and achieving nondefective structure modeling.Both forward and reverse enhancements involve a process of image sharpening. Image sharpening is performed by acquiring the high-frequency component of the image, multiplying it by a sharpening factor, and adding the enhanced image to the original image. The sharpening factor determines the sharpening level. Image sharpening is used twice. In a process that performs two enhancements, we need to use two different sharpening factors to generate images with two sharpening levels. With one sharpening factor, we get a sharpening image as the result of forward enhancement. With the other sharpening factor, we get another sharpening image to be the input for reverse enhancement. Because the latter image requires a higher sharpening level, we set the latter factor to be larger than the former. We call the former factor a low sharpening factor and the latter one a high sharpening factor. The larger the low sharpening factor is, the more obvious the defect area is in the image, but there is greater noise. The larger the difference between the two factors, the fewer false defects derived from the structure, but real defects that are too close to the structure boundaries would be affected and become less noticeable. Overall, these two sharpening factors are not very sensitive and have a wide range of choices. Based on a substantial number of numerical studies, high performance can be achieved with a low sharpening factor in the range of (1, 3), and the difference between the two sharpening factors is in the range of (4, 6).


### 2.1. Highlighting Blurred Defects Based on Forward Enhancement

Some blowhole defects in the original images can be blurred, which increases the difficulty in detecting defects. Therefore, the goal of the forward enhancement operation is to highlight the defects. The defects in the radiographic images are mainly composed of high-frequency components [22]. In this paper, we use the unsharp masking method [23] to highlight the blurred defects.First, the original image (Figure1(a)) is filtered with a Gaussian low-pass filter. The result is a blurred image (Figure 1(b)). Compared with the original image, the high-frequency component of the blurred image representing the defect is suppressed and the low-frequency component representing the structure remains. Then, the blurred image is subtracted from the original image to obtain an image with the high-frequency component (shown in Figure 1(c)). Finally, the grayscale of the image with the high-frequency component is enlarged with the low sharpening factor and the resulting image is superimposed onto the original image. In this way, the blurred defects are highlighted. The resulting image after forward enhancement is shown in Figure 1(d).Figure 1
An example of highlighting a blurred defect based on forward enhancement: (a) original image; (b) the blurred image; (c) an image with the high-frequency component; (d) the image after forward enhancement.
(a)
(b)
(c)
(d)


### 2.2. Nondefective Structure Modeling Based on Reverse Enhancement

The reverse enhancement aims at achieving nondefective structure modeling including enhancement of the casting structure and elimination of defects. The resulting image is useful in resisting the interference from the structure in the segmentation process.Using the same process in the above section with a high sharpening factor, we can get an image with a higher sharpening level, which is a high sharpening image. We set the high sharpening image as the input for reverse enhancement and then apply the median filtering algorithm in the high sharpening image. The median filtering algorithm is a classic algorithm which can smooth and remove small connected areas of the images. Thus, the defects and small structures that are of similar size to the defects can be eliminated. The kernel in the median filtering is a circular kernel. The kernel radius is set as 25 pixels. Because the diameters of the blowholes generally range from 6 pixels to 22 pixels, median filtering can achieve high performance as long as it is larger than the size of the blowholes. The original image is shown in Figure2(a). And the reference image that is obtained after reverse enhancement is shown in Figure 2(b).Figure 2
An example of nondefective structure modeling based on reverse enhancement: (a) original image and (b) the image after reverse enhancement.
(a)
(b)


### 2.3. Potential Defect Region Segmentation

In this section, we extract the potential defect regions for later analysis. The potential defects include real defects and defect-like false defects. The false defects are X-ray projections of the normal casting structure. Some of the false defects that are significantly different from the blowhole defects were eliminated in the subtraction step. The “defect-like false defects” refer to false defects that are similar to defects in size and are difficult to eliminate in the subtraction step. They are essentially X-ray projections of the local structure.The segmentation method is mainly comprised of three steps: subtraction, edge detection, and region selection, as shown in Figure3. We obtained a low sharpening image in Section 3.1 and reference image in Section 3.2. In order to eliminate interference from the structures, we subtract the reference image from the low sharpening image. The subtraction result in Figure 3 mainly contains potential defects. Then, we detect the edge of the remaining part from the subtraction result. The Canny edge detection method [24] is used to detect the edges, through which the edge is more likely to be continuous. The edges of the regular structure are thickened in order to ensure the edge of the defect is closed. We then apply mathematical dilation morphology of binary images to the edge image in order to connect the adjacent line segments and extract the image skeleton to thin the edges (shown in Figure 3). We finally search the connected areas in the binary image and retain the segments with the appropriate area (shown in Figure 3). Actually, because the grayscale near edges does not change abruptly, the extracted regions are smaller than the corresponding defects. According to the size of the blowhole defects in the image coordinate system, we set the thresholds for the potential defect regions as 10 pixels and 300 pixels. The regions with an area between these two thresholds are retained as potential defect regions.Figure 3
The defect region segmentation process. It comprises three steps: subtraction, edge detection, and region selection. First, the reference image is subtracted from the low sharpening image. Then, the edge of the remaining part is detected from the subtraction result. Finally, the connected areas in the binary image with the appropriate areas are selected.


## 2.1. Highlighting Blurred Defects Based on Forward Enhancement

Some blowhole defects in the original images can be blurred, which increases the difficulty in detecting defects. Therefore, the goal of the forward enhancement operation is to highlight the defects. The defects in the radiographic images are mainly composed of high-frequency components [22]. In this paper, we use the unsharp masking method [23] to highlight the blurred defects.First, the original image (Figure1(a)) is filtered with a Gaussian low-pass filter. The result is a blurred image (Figure 1(b)). Compared with the original image, the high-frequency component of the blurred image representing the defect is suppressed and the low-frequency component representing the structure remains. Then, the blurred image is subtracted from the original image to obtain an image with the high-frequency component (shown in Figure 1(c)). Finally, the grayscale of the image with the high-frequency component is enlarged with the low sharpening factor and the resulting image is superimposed onto the original image. In this way, the blurred defects are highlighted. The resulting image after forward enhancement is shown in Figure 1(d).Figure 1
An example of highlighting a blurred defect based on forward enhancement: (a) original image; (b) the blurred image; (c) an image with the high-frequency component; (d) the image after forward enhancement.
(a)
(b)
(c)
(d)


## 2.2. Nondefective Structure Modeling Based on Reverse Enhancement

The reverse enhancement aims at achieving nondefective structure modeling including enhancement of the casting structure and elimination of defects. The resulting image is useful in resisting the interference from the structure in the segmentation process.Using the same process in the above section with a high sharpening factor, we can get an image with a higher sharpening level, which is a high sharpening image. We set the high sharpening image as the input for reverse enhancement and then apply the median filtering algorithm in the high sharpening image. The median filtering algorithm is a classic algorithm which can smooth and remove small connected areas of the images. Thus, the defects and small structures that are of similar size to the defects can be eliminated. The kernel in the median filtering is a circular kernel. The kernel radius is set as 25 pixels. Because the diameters of the blowholes generally range from 6 pixels to 22 pixels, median filtering can achieve high performance as long as it is larger than the size of the blowholes. The original image is shown in Figure2(a). And the reference image that is obtained after reverse enhancement is shown in Figure 2(b).Figure 2
An example of nondefective structure modeling based on reverse enhancement: (a) original image and (b) the image after reverse enhancement.
(a)
(b)


## 2.3. Potential Defect Region Segmentation

In this section, we extract the potential defect regions for later analysis. The potential defects include real defects and defect-like false defects. The false defects are X-ray projections of the normal casting structure. Some of the false defects that are significantly different from the blowhole defects were eliminated in the subtraction step. The “defect-like false defects” refer to false defects that are similar to defects in size and are difficult to eliminate in the subtraction step. They are essentially X-ray projections of the local structure.The segmentation method is mainly comprised of three steps: subtraction, edge detection, and region selection, as shown in Figure3. We obtained a low sharpening image in Section 3.1 and reference image in Section 3.2. In order to eliminate interference from the structures, we subtract the reference image from the low sharpening image. The subtraction result in Figure 3 mainly contains potential defects. Then, we detect the edge of the remaining part from the subtraction result. The Canny edge detection method [24] is used to detect the edges, through which the edge is more likely to be continuous. The edges of the regular structure are thickened in order to ensure the edge of the defect is closed. We then apply mathematical dilation morphology of binary images to the edge image in order to connect the adjacent line segments and extract the image skeleton to thin the edges (shown in Figure 3). We finally search the connected areas in the binary image and retain the segments with the appropriate area (shown in Figure 3). Actually, because the grayscale near edges does not change abruptly, the extracted regions are smaller than the corresponding defects. According to the size of the blowhole defects in the image coordinate system, we set the thresholds for the potential defect regions as 10 pixels and 300 pixels. The regions with an area between these two thresholds are retained as potential defect regions.Figure 3
The defect region segmentation process. It comprises three steps: subtraction, edge detection, and region selection. First, the reference image is subtracted from the low sharpening image. Then, the edge of the remaining part is detected from the subtraction result. Finally, the connected areas in the binary image with the appropriate areas are selected.


## 3. Precise Defect Identification via 3D Model Fitting-Based Omnidirectional Analysis

In the X-ray image, the casting defect appears as a small highlighted area. Many studies are based on this feature to describe defects and realize detection. The existing methods mainly analyze potential defects by analyzing the grayscale curves along several directions. If the curves present a small “mountain” shape, the corresponding region is considered to be a defect. For example, He et al. [25] located defects by analyzing the grayscale curves along two directions of the potential defect region. Mery et al. [26] analyzed the grayscale curves along eight directions to judge defects. However, these methods can easily misidentify false defects as real defects because some false defects have curves similar to real defects in some directions, but curves along other directions may be different. Therefore, in this paper, we do not analyze grayscale curves in several directions but rather analyze grayscale change along all directions, that is, we perform an omnidirectional analysis. In practice, unlike traditional methods, we do not analyze the curve along many directions separate from the potential defect regions. The reason is that too many directions lead to problems of complicated calculation and difficulty in the statistical analysis. We set a 3D model in advance and then fit the 3D surface plot of potential defect regions to this model. From the fitting results, several features can be extracted. Finally, we use the classifier to classify potential defect regions into two categories based on these features. The omnidirectional analysis based on 3D model fitting greatly simplifies the computation.


### 3.1. 3D Model Fitting for Omnidirectional Defect Analysis

Each kind of defect in a casting has a similar distribution of gray values and can fit the same kind of 3D model with different coefficients. The omnidirectional analysis method is appropriate for most kinds of defect detection. In this paper, we only describe the analysis for blowhole defects. In this section, we analyze the 3D surface plots of the potential blowhole defects, fit the 3D surface plots with the corresponding preset model, and discuss the features.Blowhole defects commonly have the following two characteristics. They are in the shape of a circle and their brightness is greater than their background. The grayscale images and the 3D surface plot of typical segments are shown in Figure4. The images in Figures 4(a)–4(d) show the characteristics of real defects. From the 3D surface plot of real defects, we found that actual blowholes are different from the standard defect model due to blurring and unclear contrast. The images in Figures 4(e)–4(h) show the characteristics of false defects. Most of the false defects are derived from noise, edges, and boundaries. Some of the 3D surface plots of real defects and false defects have a similar appearance. It is difficult to distinguish them through the 3D surface plots directly.Figure 4
The grayscale images and the 3D surface plots of typical segments: (a–d) images of the real defects and (e–h) images of the false defects. The images on the left of each part are the grayscale images. The images on the right of each part are the 3D surface plot.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)To make the defects clearer, we cropped the potential defect regions from the low sharpening image. The horizontal size of the region window is double the horizontal coordinate size of the segments. The vertical size of the region window is double the vertical coordinate size of the segments. The center of the region window coincides with the segment center. The cropped regions are resized to bek∗k, where k is the size of the length of the reshaped region. Figure 4 shows the grayscale images and the 3D surface plots of the potential defects. Figures 4(a)–4(d) show the real defects. It can be seen that the real defects in the 3D surface plots appear as hill-like protrusions which are close to a Gaussian surface shape even if the entire surface is inclining. Figures 4(e)–4(h) show the false defects. The 3D surface plot of the false defects is clutter and cannot match the Gaussian surface shape well. Thus, we can use the Gaussian surface as the preset 3D model of the blowholes for fitting the 3D surface plot of the defects. Although real blowholes are different in detail and a simple Gaussian surface can hardly fit all of them well, this does not affect the final defect detection much, since the purpose of the fitting process is to produce discrimination features between real and false defects, rather than to build a digital model for each defect.The real defects are located near the boundaries of the structure, which leads to the sloping of the 3D surface plots. We need to eliminate the structural background for the follow-up 3D model fitting. We get the defect image without structural background by subtracting the reference images. First, we use median filtering to obtain the reference image from the low sharpening image. Then, the small size reference images corresponding to the defects are cropped from the large size reference image. We subtract the small size reference images from the defect images and finally get the defect image without structural background:(1)gdefect_partx,y≈fdefect_backgroundx,y−ftemplatex,y=gdefect_partx,y+gbackground_partx,y−ftemplatex,y.We use the preset 3D model to fit the defect with the background eliminated. The preset 3D model for fitting is defined as(2)z=d0+a0×−x−b122×c12−x−b222×c22,where d0, a0, b1, c1, b2, and c2 are the coefficients, x and y are the independent variables, and z is the dependent variable. To improve the convergence speed and get a reasonable fitting result, we set the value range for the coefficients. The lower bound of a0 is set as 0 because the defect center is brighter than its periphery. We set the lower bounds of b1 and b2 as k/4 and the upper bounds as 3k/4. The lower bounds of c1 and c2 are set as 1. In this way, we can finally get d0, a0, b1, c1, b2, and c2. In addition, the goodness-of-fit statistics including adjusted R-square and RMSE are useful for judging the goodness of fit. R-square is described as(3)R_square=1−∑z_actualx,y−z_predictx,y2∑z_actualx,y−z_mean2,where z_actual (x, y) is the actual grayscale at the (x, y) coordinate, z_predict (x, y) is the 3D model value at the (x, y) coordinate, and z_mean is the mean of all the z_actuals. The adjusted R-square is described as(4)AdjustedR_square=1−1−Rsquare2n−1n−p−1,where n is the number of coordinates and p is the number of the independent variables and we set n=k∗k and p=2. The adjusted R-square can characterize a good or bad fit. The normal range of the values is [0, 1], where the closer the value is to 1, the better the 3D model fits the data. RMSE reflects the fitting error. RMSE is described as(5)RMSE=∑z_actualx,y−z_predictx,y2n.By analyzing the value of the coefficients and the goodness-of-fit statistics, we can set a threshold to distinguish real defects and false defects. Figure5 shows the 3D surface plots of potential defects with the background eliminated and the fitting result images. Figures 5(a)–5(e) are images of the real defects. It can be seen from the fitting results that the 3D surface plots of real defects fit the preset model well. Figures 5(e)–5(h) are images of the false defects. The fitting error is large for the false defects.Figure 5
The 3D surface plots with the background eliminated and the 3D model fitting result images of typical segments: (a–d) images of the real defects and (e–h) images of the false defects. The images on the left of each part are the 3D surface plot with the background eliminated. The images on the right of each part are the 3D model fitting result images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)We obtained the coefficients and the goodness-of-fit statistics from the 3D model fitting. Some of the features can help to screen for real defects while there is little difference in some features between the two classes. We analyzed the characters of the features with graphs that distributed the two classes of features. The graphs were drawn with the fitting results of 109 real defects and 238 false defects.a0: there is a positive correlation between the intensity of a defect region and a0 which reflects the height of the 3D model. The graph in Figure 6(a) shows the distribution of the two classes for the a0. By analyzing the graph, we can summarize that a large number of false defects have low intensity, which could help distinguish false defects. The false defects with low a0 are mostly extracted as potential defects for noise and have low intensity compared with real defects and false defects extracted for structure.pd: according to the character of the 3D Gaussian surface, the values ofb1 and b2 reflect the location of the surface peak. The surface peaks of real defects are close to the center compared to the defect-like false defects because we made the region centers coincide with the segment centers when cropping the regions. The peak deviation (pd) is defined as the sum of the off-center distances of b1 and b2, which can be shown as(6)pd=b1−k2+b2−k2.The graph in Figure6(b) shows the distribution of the two classes for the peak deviation. Most real defects have low peak deviation and a few of the false defects have high peak deviation. Thus, pd can help distinguish false defects and real defects.max (c1, c2)/min (c1, c2): the values of c1 and c2 reflect the width of the hill-like 3D Gaussian surface. Most blowhole defects have similar c1 and c2 while some false defects extracted from the structure do not. The distribution of the two classes of max (c1, c2)/min (c1, c2) is shown in the graph in Figure 6(c). However, the two curves are similar and we can barely distinguish the false defects from the real defects from this figure. The deviation between c1 and c2 of real defects may come from noise disturbing the fitting result.AdjustedR-square: adjusted R-square can characterize a good or bad fit. A low close degree of the adjusted R-square to 1 reflects a good fit. Thus, real defects always have a fitting result of adjusted R-square near 1. The distribution of the two classes of adjusted R-squares is shown in Figure 6(d). The graph is in line with the conjecture and adjusted R-square is useful for distinguishing false defects and real defects.ef: RMSE reflects the fitting error. There is a positive correlation between RMSE anda0, so the effect of a0 should be eliminated when we use RMSE to measure the fitting error. The fitting error (ef) is defined as the result of dividing RMSE by a0, which can be shown as(7)ef=RMSEa0.Figure 6
The analysis graphs of the coefficients and the goodness-of-fit statistics. The distribution of the two classes for (a)a0, (b) peak deviation, (c) max (c1, c2)/min (c1, c2), (d) adjusted R-square, and (e) fitting error.
(a)
(b)
(c)
(d)
(e)A real defect always has a low fitting error, and a few real defects may have some fitting error for the influence of noise and structure. This conclusion can be obtained from Figure6(e) which shows the distribution of the two classes for fitting error. The fitting error (ef) is useful for distinguishing false defects and real defects.


### 3.2. Defect Identification Based on Multithreshold AdaBoost

To classify potential defects, we built threshold classifiers for each feature and then trained a strong ensemble classifier consisting of the threshold classifiers. In order to obtain the feature that yields the best detection performance, the receiver operation characteristic [1] (ROC) curve is analyzed and the AdaBoost algorithm [21] is applied to produce the weight for each threshold classifier.The ROC curve is a plot of the sensitivity (Sn) against the 1−specificity (1 − Sp) defined as(8)Sn=TPTP+FN,1−Sp=FPTN+FP,where TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, and FN is the number of false negative. The points of the ROC curve show the detection performance at different thresholds. The point (0, 1), that is, FPR = 0 and TPR = 1, means that FN (false negative) = 0 and FP (false positive) = 0, and the classifier can correctly classify all the potential defects. We can learn from the analysis that the closer the ROC curve is to the upper left corner, the better the classifier performance is. We analyzed 100 potential defects including 49 real defects and 51 false defects to obtain the ROC curves. The ROC curves are shown in Figure 7. The curves show that when one of the five features is big enough or small enough, the category of the region can be determined. Based on this phenomenon, we propose determining three thresholds for each feature. The first threshold of each feature has a corresponding point closest to (0, 1) on the ROC curve. The classifier with the first threshold has a low false-positive rate (FPR) and high true-positive rate (TPR) which is commonly used. We have calculated the standard error for the five threshold classifiers. The standard errors of classifiers corresponding to a0, ef, pd, adjusted R-square, and max (c1, c2)/min (c1, c2) are, respectively, 0.316, 0.283, 0.469, 0.316, and 0.574. The second threshold corresponds to TPR = 0.99 and the third threshold corresponds to FPR = 0.01. Taking the ROC of a0 in Figure 7 as an example, if a0 is smaller than the second threshold, the possibility that the potential defect is a false defect is very high. If a0 is bigger than the third threshold, the possibility that the potential defect is a real defect is very high. These two threshold classifiers are accurate in judging potential defects with big enough or small enough features. Using three thresholds for each feature can increase the accuracy of the ensemble classifier.Figure 7
The ROC curve of different features. The curves show the detection performance with different threshold classifiers.The AdaBoost algorithm is applied to produce the weight of each threshold classifier for the ensemble classifier. AdaBoost is a commonly used ensemble classifier method. The main idea behind this algorithm is that it focuses more on the data that are difficult to classify to determine the weight of each weak classifier and in turn form a strong classifier. The input for constructing the ensemble classifier includes the training set in thek-th x1k,y1,…,xlk,yl and the thresholds th1,…,th3k. Then, initialization is performed, where the weights of all training data are equal, that is wi1=1/l,i=1,…,l. Then, the training process is performed. For data classified correctly, the value of weights would be reduced, and for data classified incorrectly, the value of weights would be increased. The next iteration focuses more on the data classified incorrectly. Furthermore, the weights of each weak classifier are updated during every iteration and are negatively correlated with the classification error of each weak classifier. Finally, the ensemble classifier is formed through a process of iteration [27].


## 3.1. 3D Model Fitting for Omnidirectional Defect Analysis

Each kind of defect in a casting has a similar distribution of gray values and can fit the same kind of 3D model with different coefficients. The omnidirectional analysis method is appropriate for most kinds of defect detection. In this paper, we only describe the analysis for blowhole defects. In this section, we analyze the 3D surface plots of the potential blowhole defects, fit the 3D surface plots with the corresponding preset model, and discuss the features.Blowhole defects commonly have the following two characteristics. They are in the shape of a circle and their brightness is greater than their background. The grayscale images and the 3D surface plot of typical segments are shown in Figure4. The images in Figures 4(a)–4(d) show the characteristics of real defects. From the 3D surface plot of real defects, we found that actual blowholes are different from the standard defect model due to blurring and unclear contrast. The images in Figures 4(e)–4(h) show the characteristics of false defects. Most of the false defects are derived from noise, edges, and boundaries. Some of the 3D surface plots of real defects and false defects have a similar appearance. It is difficult to distinguish them through the 3D surface plots directly.Figure 4
The grayscale images and the 3D surface plots of typical segments: (a–d) images of the real defects and (e–h) images of the false defects. The images on the left of each part are the grayscale images. The images on the right of each part are the 3D surface plot.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)To make the defects clearer, we cropped the potential defect regions from the low sharpening image. The horizontal size of the region window is double the horizontal coordinate size of the segments. The vertical size of the region window is double the vertical coordinate size of the segments. The center of the region window coincides with the segment center. The cropped regions are resized to bek∗k, where k is the size of the length of the reshaped region. Figure 4 shows the grayscale images and the 3D surface plots of the potential defects. Figures 4(a)–4(d) show the real defects. It can be seen that the real defects in the 3D surface plots appear as hill-like protrusions which are close to a Gaussian surface shape even if the entire surface is inclining. Figures 4(e)–4(h) show the false defects. The 3D surface plot of the false defects is clutter and cannot match the Gaussian surface shape well. Thus, we can use the Gaussian surface as the preset 3D model of the blowholes for fitting the 3D surface plot of the defects. Although real blowholes are different in detail and a simple Gaussian surface can hardly fit all of them well, this does not affect the final defect detection much, since the purpose of the fitting process is to produce discrimination features between real and false defects, rather than to build a digital model for each defect.The real defects are located near the boundaries of the structure, which leads to the sloping of the 3D surface plots. We need to eliminate the structural background for the follow-up 3D model fitting. We get the defect image without structural background by subtracting the reference images. First, we use median filtering to obtain the reference image from the low sharpening image. Then, the small size reference images corresponding to the defects are cropped from the large size reference image. We subtract the small size reference images from the defect images and finally get the defect image without structural background:(1)gdefect_partx,y≈fdefect_backgroundx,y−ftemplatex,y=gdefect_partx,y+gbackground_partx,y−ftemplatex,y.We use the preset 3D model to fit the defect with the background eliminated. The preset 3D model for fitting is defined as(2)z=d0+a0×−x−b122×c12−x−b222×c22,where d0, a0, b1, c1, b2, and c2 are the coefficients, x and y are the independent variables, and z is the dependent variable. To improve the convergence speed and get a reasonable fitting result, we set the value range for the coefficients. The lower bound of a0 is set as 0 because the defect center is brighter than its periphery. We set the lower bounds of b1 and b2 as k/4 and the upper bounds as 3k/4. The lower bounds of c1 and c2 are set as 1. In this way, we can finally get d0, a0, b1, c1, b2, and c2. In addition, the goodness-of-fit statistics including adjusted R-square and RMSE are useful for judging the goodness of fit. R-square is described as(3)R_square=1−∑z_actualx,y−z_predictx,y2∑z_actualx,y−z_mean2,where z_actual (x, y) is the actual grayscale at the (x, y) coordinate, z_predict (x, y) is the 3D model value at the (x, y) coordinate, and z_mean is the mean of all the z_actuals. The adjusted R-square is described as(4)AdjustedR_square=1−1−Rsquare2n−1n−p−1,where n is the number of coordinates and p is the number of the independent variables and we set n=k∗k and p=2. The adjusted R-square can characterize a good or bad fit. The normal range of the values is [0, 1], where the closer the value is to 1, the better the 3D model fits the data. RMSE reflects the fitting error. RMSE is described as(5)RMSE=∑z_actualx,y−z_predictx,y2n.By analyzing the value of the coefficients and the goodness-of-fit statistics, we can set a threshold to distinguish real defects and false defects. Figure5 shows the 3D surface plots of potential defects with the background eliminated and the fitting result images. Figures 5(a)–5(e) are images of the real defects. It can be seen from the fitting results that the 3D surface plots of real defects fit the preset model well. Figures 5(e)–5(h) are images of the false defects. The fitting error is large for the false defects.Figure 5
The 3D surface plots with the background eliminated and the 3D model fitting result images of typical segments: (a–d) images of the real defects and (e–h) images of the false defects. The images on the left of each part are the 3D surface plot with the background eliminated. The images on the right of each part are the 3D model fitting result images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)We obtained the coefficients and the goodness-of-fit statistics from the 3D model fitting. Some of the features can help to screen for real defects while there is little difference in some features between the two classes. We analyzed the characters of the features with graphs that distributed the two classes of features. The graphs were drawn with the fitting results of 109 real defects and 238 false defects.a0: there is a positive correlation between the intensity of a defect region and a0 which reflects the height of the 3D model. The graph in Figure 6(a) shows the distribution of the two classes for the a0. By analyzing the graph, we can summarize that a large number of false defects have low intensity, which could help distinguish false defects. The false defects with low a0 are mostly extracted as potential defects for noise and have low intensity compared with real defects and false defects extracted for structure.pd: according to the character of the 3D Gaussian surface, the values ofb1 and b2 reflect the location of the surface peak. The surface peaks of real defects are close to the center compared to the defect-like false defects because we made the region centers coincide with the segment centers when cropping the regions. The peak deviation (pd) is defined as the sum of the off-center distances of b1 and b2, which can be shown as(6)pd=b1−k2+b2−k2.The graph in Figure6(b) shows the distribution of the two classes for the peak deviation. Most real defects have low peak deviation and a few of the false defects have high peak deviation. Thus, pd can help distinguish false defects and real defects.max (c1, c2)/min (c1, c2): the values of c1 and c2 reflect the width of the hill-like 3D Gaussian surface. Most blowhole defects have similar c1 and c2 while some false defects extracted from the structure do not. The distribution of the two classes of max (c1, c2)/min (c1, c2) is shown in the graph in Figure 6(c). However, the two curves are similar and we can barely distinguish the false defects from the real defects from this figure. The deviation between c1 and c2 of real defects may come from noise disturbing the fitting result.AdjustedR-square: adjusted R-square can characterize a good or bad fit. A low close degree of the adjusted R-square to 1 reflects a good fit. Thus, real defects always have a fitting result of adjusted R-square near 1. The distribution of the two classes of adjusted R-squares is shown in Figure 6(d). The graph is in line with the conjecture and adjusted R-square is useful for distinguishing false defects and real defects.ef: RMSE reflects the fitting error. There is a positive correlation between RMSE anda0, so the effect of a0 should be eliminated when we use RMSE to measure the fitting error. The fitting error (ef) is defined as the result of dividing RMSE by a0, which can be shown as(7)ef=RMSEa0.Figure 6
The analysis graphs of the coefficients and the goodness-of-fit statistics. The distribution of the two classes for (a)a0, (b) peak deviation, (c) max (c1, c2)/min (c1, c2), (d) adjusted R-square, and (e) fitting error.
(a)
(b)
(c)
(d)
(e)A real defect always has a low fitting error, and a few real defects may have some fitting error for the influence of noise and structure. This conclusion can be obtained from Figure6(e) which shows the distribution of the two classes for fitting error. The fitting error (ef) is useful for distinguishing false defects and real defects.


## 3.2. Defect Identification Based on Multithreshold AdaBoost

To classify potential defects, we built threshold classifiers for each feature and then trained a strong ensemble classifier consisting of the threshold classifiers. In order to obtain the feature that yields the best detection performance, the receiver operation characteristic [1] (ROC) curve is analyzed and the AdaBoost algorithm [21] is applied to produce the weight for each threshold classifier.The ROC curve is a plot of the sensitivity (Sn) against the 1−specificity (1 − Sp) defined as(8)Sn=TPTP+FN,1−Sp=FPTN+FP,where TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, and FN is the number of false negative. The points of the ROC curve show the detection performance at different thresholds. The point (0, 1), that is, FPR = 0 and TPR = 1, means that FN (false negative) = 0 and FP (false positive) = 0, and the classifier can correctly classify all the potential defects. We can learn from the analysis that the closer the ROC curve is to the upper left corner, the better the classifier performance is. We analyzed 100 potential defects including 49 real defects and 51 false defects to obtain the ROC curves. The ROC curves are shown in Figure 7. The curves show that when one of the five features is big enough or small enough, the category of the region can be determined. Based on this phenomenon, we propose determining three thresholds for each feature. The first threshold of each feature has a corresponding point closest to (0, 1) on the ROC curve. The classifier with the first threshold has a low false-positive rate (FPR) and high true-positive rate (TPR) which is commonly used. We have calculated the standard error for the five threshold classifiers. The standard errors of classifiers corresponding to a0, ef, pd, adjusted R-square, and max (c1, c2)/min (c1, c2) are, respectively, 0.316, 0.283, 0.469, 0.316, and 0.574. The second threshold corresponds to TPR = 0.99 and the third threshold corresponds to FPR = 0.01. Taking the ROC of a0 in Figure 7 as an example, if a0 is smaller than the second threshold, the possibility that the potential defect is a false defect is very high. If a0 is bigger than the third threshold, the possibility that the potential defect is a real defect is very high. These two threshold classifiers are accurate in judging potential defects with big enough or small enough features. Using three thresholds for each feature can increase the accuracy of the ensemble classifier.Figure 7
The ROC curve of different features. The curves show the detection performance with different threshold classifiers.The AdaBoost algorithm is applied to produce the weight of each threshold classifier for the ensemble classifier. AdaBoost is a commonly used ensemble classifier method. The main idea behind this algorithm is that it focuses more on the data that are difficult to classify to determine the weight of each weak classifier and in turn form a strong classifier. The input for constructing the ensemble classifier includes the training set in thek-th x1k,y1,…,xlk,yl and the thresholds th1,…,th3k. Then, initialization is performed, where the weights of all training data are equal, that is wi1=1/l,i=1,…,l. Then, the training process is performed. For data classified correctly, the value of weights would be reduced, and for data classified incorrectly, the value of weights would be increased. The next iteration focuses more on the data classified incorrectly. Furthermore, the weights of each weak classifier are updated during every iteration and are negatively correlated with the classification error of each weak classifier. Finally, the ensemble classifier is formed through a process of iteration [27].


## 4. Experimental Results

In this section, we test the performance of the algorithm for extracting potential defect regions described in Section3 and the algorithm for precise defect identification proposed in Section 4.The original images are from the GDXray database. The GDXray database includes five groups of X-ray images: castings, welds, baggage, natural objects, and settings. We use images from the “castings” group in the database. The coordinates of the bounding box of the object of interest, which are ground truth, are available in the database and they are manually annotated.In the field of defect detection, there are mainly three indicators to measure the detection effects of the methods. They are precision (Pr), recall (Re), and accuracy (Ac). They are described as(9)Pr=TPTP+FP,Re=TPTP+FN,Ac=TP+TNTP+TN+FP+FN,where TP is the number of real defects correctly classified, TN is the number of false defects correctly classified, and FP is the sum of false defects classified as real defects; these are known as “false alarms.” FN is the sum of real defects classified as false defects. If all the real defects were detected and no false defects were detected, Pr, Re, and Ac would be 1.


### 4.1. Potential Defect Region Extraction Result

The first experiment aims at testing the performance of the algorithm in extracting potential defect regions in Section3. The method in [26] first applied the Laplacian-of-Gaussian (LoG) kernel and a zero-crossing algorithm to detect the edges and then thicken the edges of the regular structure to enclose some edges.Precision (Pr) and recall (Re) are used to evaluate the performance, and they are shown in Table1. The Re results indicate that 94.12 real defect regions were extracted using the proposed method and 83.53 real defects were segmented using the method in [26] when there are 100 real defects. This means the proposed method can extract more real defect regions. In addition, the Pr results indicate that 8.15 potential regions extracted using the proposed method are real defect regions and 0.16 potential regions extracted using the method in [26] are real defect regions when there are 100 potential regions extracted. It can be seen that the proposed method can reduce the extraction of false defects well.Table 1
The performance of the methods for extraction of potential defect regions in the GDXray database.
Method
TP
FP
FN
Pr
Re
Method in [26]
71
43108
14
0.0016
0.8353
Proposed
80
902
5
0.0815
0.9412


### 4.2. Precise Defect Identification Results

The second experiment tested three methods: the proposed method, the crossing line profile- (CLP-) based method [26], and the high-contrast pixel- (HP-) based method [28].Using the method described in Section3, the cropped X-ray images of the potential defects were extracted from the castings group of the GDXray database. We disjointed the learning and testing data which is a technique called hold-out validation. Thus, the above potential defects of two classes were split into learning and testing subsets. After gathering the defect features, we saved the defect features in random order. The first 100 potential defects included 49 real defects and 51 false defects which were set as the training set and the remaining 120 defects included 60 real defects and 60 false defects which were set as the testing set.The classifiers applied were the ensemble classifiers in Section4. For the CLP-based method and the HP-based method, we determined three thresholds with their ROC curves and combined the three threshold classifiers with AdaBoost described in Section 4. For the proposed method, we determined three thresholds for each feature with the ROC curve and combined the fifteen threshold classifiers with AdaBoost described in Section 4.We tested the three methods and calculated Pr, Re, and Ac to evaluate the performance. The experiment results are summarized in Table2. The proposed method has Pr = 0.906, Re = 0.967, and Ac = 0.933 which means there were 48.3 real defects detected and there were 5 “false alarms” when trying to detect 50 real defects from 100 potential regions. Figure 7 shows the ROC curve of the proposed method, the CLP-based method, and the HP-based method. The area under the ROC curve (AUC) is used to measure the performance of the threshold classifiers. The bigger the AUC is, the higher the performance is. Table 2 and Figure 8 indicate that the proposed method performs well compared with the other two methods. The original images and the enlargements of real defects are shown in Figure 9 in order to present the defects clearly. The results for detecting images with the proposed method and methods based on CLP and HP are shown in Figure 10. The original images are shown in Figure 10(a). The images in Figure 10(b) are the processing results of the CLP-based method, which indicates that the method could easily produce a few “false alarms”; meanwhile, most real defects can be detected. The images in Figure 10(c) are the processing results of the HP-based method, which indicates that the method did not work well. The processing results of the proposed method are shown in Figure 10(d) and demonstrate that it was effective. The images in Figure 10(e) show the ground truth. The detection results especially for the images in groups 4 and 5 show that the proposed method could classify the false defects from the structure accurately. The images in 10(c) and 10(d) in groups 3 and 5 show that the proposed method has a higher performance in detecting blurred defects.Table 2
The performance of the methods for detecting defects in the GDXray database.
Method
Pr
Re
Ac
CLP
0.781
0.950
0.842
HP
0.754
0.767
0.758
Proposed
0.906
0.967
0.933Figure 8
The ROC curve of the proposed method, the crossing line profile- (CLP-) based method, and the high-contrast pixel- (HP-) based method. The curve shows that the proposed method performs well compared with the other two methods.Figure 9
The original images and the enlargements of real defects. The original defects in the images are magnified.
(a)
(b)
(c)
(d)
(e)Figure 10
The detection results of the proposed method and methods based on CLP and HP: (a) original images; (b) processing results of the CLP-based method; (c) processing results of the HP-based method; (d) processing results of the proposed method; (e) ground truth.
(a)
(b)
(c)
(d)
(e)The results show that the proposed method significantly outperformed the other two methods.


## 4.1. Potential Defect Region Extraction Result

The first experiment aims at testing the performance of the algorithm in extracting potential defect regions in Section3. The method in [26] first applied the Laplacian-of-Gaussian (LoG) kernel and a zero-crossing algorithm to detect the edges and then thicken the edges of the regular structure to enclose some edges.Precision (Pr) and recall (Re) are used to evaluate the performance, and they are shown in Table1. The Re results indicate that 94.12 real defect regions were extracted using the proposed method and 83.53 real defects were segmented using the method in [26] when there are 100 real defects. This means the proposed method can extract more real defect regions. In addition, the Pr results indicate that 8.15 potential regions extracted using the proposed method are real defect regions and 0.16 potential regions extracted using the method in [26] are real defect regions when there are 100 potential regions extracted. It can be seen that the proposed method can reduce the extraction of false defects well.Table 1
The performance of the methods for extraction of potential defect regions in the GDXray database.
Method
TP
FP
FN
Pr
Re
Method in [26]
71
43108
14
0.0016
0.8353
Proposed
80
902
5
0.0815
0.9412


## 4.2. Precise Defect Identification Results

The second experiment tested three methods: the proposed method, the crossing line profile- (CLP-) based method [26], and the high-contrast pixel- (HP-) based method [28].Using the method described in Section3, the cropped X-ray images of the potential defects were extracted from the castings group of the GDXray database. We disjointed the learning and testing data which is a technique called hold-out validation. Thus, the above potential defects of two classes were split into learning and testing subsets. After gathering the defect features, we saved the defect features in random order. The first 100 potential defects included 49 real defects and 51 false defects which were set as the training set and the remaining 120 defects included 60 real defects and 60 false defects which were set as the testing set.The classifiers applied were the ensemble classifiers in Section4. For the CLP-based method and the HP-based method, we determined three thresholds with their ROC curves and combined the three threshold classifiers with AdaBoost described in Section 4. For the proposed method, we determined three thresholds for each feature with the ROC curve and combined the fifteen threshold classifiers with AdaBoost described in Section 4.We tested the three methods and calculated Pr, Re, and Ac to evaluate the performance. The experiment results are summarized in Table2. The proposed method has Pr = 0.906, Re = 0.967, and Ac = 0.933 which means there were 48.3 real defects detected and there were 5 “false alarms” when trying to detect 50 real defects from 100 potential regions. Figure 7 shows the ROC curve of the proposed method, the CLP-based method, and the HP-based method. The area under the ROC curve (AUC) is used to measure the performance of the threshold classifiers. The bigger the AUC is, the higher the performance is. Table 2 and Figure 8 indicate that the proposed method performs well compared with the other two methods. The original images and the enlargements of real defects are shown in Figure 9 in order to present the defects clearly. The results for detecting images with the proposed method and methods based on CLP and HP are shown in Figure 10. The original images are shown in Figure 10(a). The images in Figure 10(b) are the processing results of the CLP-based method, which indicates that the method could easily produce a few “false alarms”; meanwhile, most real defects can be detected. The images in Figure 10(c) are the processing results of the HP-based method, which indicates that the method did not work well. The processing results of the proposed method are shown in Figure 10(d) and demonstrate that it was effective. The images in Figure 10(e) show the ground truth. The detection results especially for the images in groups 4 and 5 show that the proposed method could classify the false defects from the structure accurately. The images in 10(c) and 10(d) in groups 3 and 5 show that the proposed method has a higher performance in detecting blurred defects.Table 2
The performance of the methods for detecting defects in the GDXray database.
Method
Pr
Re
Ac
CLP
0.781
0.950
0.842
HP
0.754
0.767
0.758
Proposed
0.906
0.967
0.933Figure 8
The ROC curve of the proposed method, the crossing line profile- (CLP-) based method, and the high-contrast pixel- (HP-) based method. The curve shows that the proposed method performs well compared with the other two methods.Figure 9
The original images and the enlargements of real defects. The original defects in the images are magnified.
(a)
(b)
(c)
(d)
(e)Figure 10
The detection results of the proposed method and methods based on CLP and HP: (a) original images; (b) processing results of the CLP-based method; (c) processing results of the HP-based method; (d) processing results of the proposed method; (e) ground truth.
(a)
(b)
(c)
(d)
(e)The results show that the proposed method significantly outperformed the other two methods.


## 5. Conclusions

In this paper, we presented a blowhole detection method based on bidirectional enhancement and omnidirectional analysis for X-ray inspection of castings. The method is mainly based on bidirectional enhancement and omnidirectional analysis. The bidirectional enhancement refers to highlighting of blurred defects and nondefective structure modeling. It plays an important role in the potential defect region extraction process. In the precise defect identification process, we extracted features by omnidirectional analysis and then classified defects with an ensemble classifier. In the experiment part, we compared the proposed method with other methods. The experimental results showed that the proposed method achieves a higher performance in blowhole detection.



---

*Source: 2468505-2019-10-07.xml*