# Defect Detection and Localization of Nonlinear System Based on Particle Filter with an Adaptive Parametric Model

Authors: Jingjing Wu; Shujuan Song; Wei An; Deqiang Zhou; Hong Zhang
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

A robust particle filter (PF) and its application to fault/defect detection of nonlinear system are investigated in this paper. First, an adaptive parametric model is exploited as the observation model for a nonlinear system. Second, by incorporating the parametric model, particle filter is employed to estimate more accurate hidden states for the nonlinear stochastic system. Third, by formulating the problem of defect detection within the hypothesis testing framework, the statistical properties of the proposed testing are established. Finally, experimental results demonstrate the effectiveness and robustness of the proposed detector on real defect detection and localization in images.


---

## Body

## 1. Introduction

Fault detection, as a subfield of control engineering, is to monitor a stochastic system and identify when a fault occurred and the information of fault such as the type and its location. Fault detection approaches play a fundamental role to improve manufacturing process and application process in a variety of industry contexts [1–7], for example, quality control in process monitoring [1–3], product manufacturing [4], medium restoration [5], and facilities maintenance [8]. Advantages of low costs, high automation, and high quality of defect detection techniques resulted in growing interests in recent years. Fault/defect detection for quality inspection includes the use of cameras, eddy current, ultrasonic, X-ray sensors and other sensors, which offer the measurements to be analyzed for extraction of the information about the fault. With the high demands for quality control using cameras in industry, visual inspection systems attract more attention in recent years. In a similar vein to fault detection of linear or nonlinear systems, defect detection in visual inspection systems can be taken as monitoring the variation of the measurements from cameras. In this paper, we devote ourselves to solving the visual defect detection problems from an unusual perspective of fault detection.Currently, two rough categories of approaches are available for defect detection, that is, signal processing-based and model-based methods. The signal processing algorithms perform mathematical and statistical analysis tools on the measurements to extract faults [8–13]. The developed methods like Gabor filter and wavelet transform have been proven to be effective solutions to locating the defects with less prior information in nonlinear systems. In recent years, defect detectors on the basis of Gabor filter [10] and wavelet transform [11] are efficient to find defects for web fabrics with stable repetition of textures, since it is easier to find defects in frequency domain. Techniques of the signal processing also cover a number of computational intelligence approaches like fuzzy logic and neural network [13], which provide some effective solutions for fault detection in various industrial problems. However, the data-driven methods suffer from large computational load and storage space.The second category is an evolving collection of methodologies aiming to exploit the models of the system in temporal and spatial space to decide the occurrence of fault/detection [3, 5–7, 14]. Taking advantages of temporal or spatial models with the prior information, the model-based method can achieve robustness even in a scene with heavy amount of noises. This paper focuses on techniques of the defect detection of this category.Various model-based approaches for defect/fault detection have been proposed in the past decades [3, 5, 6, 15]. Since the system state completely represents the system’s internal (hidden) status and condition containing the fault information, the intrinsic problem in the model-based methodologies is the estimation of the system states from measurements of sensors. One of the most popular components of model-based algorithms is the Bayesian method [16], which yields the posterior distribution of the system state containing the hidden state such as the occurrence and the fault type. Bayesian filter and its variations have been effective solutions to visual inspection of surface defects of materials and fault detection in process control [3], for example, chemical process. For the linear Gaussian system, Kalman filter [15–17] has been exploited to detect surface defects in visual inspection systems. Particle filter, as an implementation algorithm of Bayesian filter for nonlinear and non-Gaussian systems, has been used for fault detection of chemical process [3, 18]. However, state estimation and fault detection still remain a challenge due to the absence of suitable models for the actual application context. The linear Gaussian models in [15] cannot meet the requirements of the defect detection and localization in the actual visual inspection systems, due to the complicated intensity distributions of background pixels.In this paper, a new intelligent defect detection algorithm without prior knowledge based on PF is presented. We formulate the visual defect detection problem to estimate the hidden states using PF and decide the occurrence or locations of defects in 2D images by chi-square test. In our method, intensity of the 2D image (the same vein as the control system) along each row or column line is assumed as a time series or a random process{xk}k=1N. To detect faults/defects in the image (the system), reasonable state and measurement models are proposed firstly for particle filter. Then, with the proposed models, particle filters along rows and columns are implemented to estimate states and the measurement innovations (residuals). Finally, abrupt changes in the measurement innovations are used to locate defects in the inspection images by chi-square test. Tests on the real database demonstrate the effectiveness of the proposed algorithm.


## 2. Problem Formulation of Defect Detection

The defect detection in a visual inspection system is to identify the occurrence of the defect and its location from the measurements of camera sensors, that is, the digital image of the inspected object. A measurementzk in a digital image is the gray level or intensity of the pixel. Then the pixel intensity along each row or column line in an image of M×N pixels is assumed as a time series or a random process {zk}k=1N or {zk}k=1M, where k is one-dimensional coordinate of the relevant pixel on the scan line (i.e., the time step of the random process).When the inspected target is free from defects, the intensity of pixels along each scan line takes on small variations. Therefore, the gray level along a scan line in a defect-free image can be defined as a white Gaussian state series{xk}k=1N or {xk}k=1M. Due to subtle intensity changes for defect-free areas, the state sequence can be modeled by the following discrete time linear Gaussian dynamical model; that is,(1)xk=Fkxk-1+vk, where xk is the state in the time series, Fk is the state transition model, and vk is the zero-mean white Gaussian noise sequences with variance σv,k2. For defect inspection problem in this paper, the dynamical mode uses the random walk model with Fk=1. Each state in a Gaussian series can also be written as(2)xk~NFkxk-1,σv,k2,where N(A,B) denotes a Gaussian distribution with expectation A and variance B.However, due to the influence of defects, uneven illumination, and the geometrical structure of the inspected object, the actual measurements usually follow nonlinear model, which can be formulated by a transition function approximating the Markov transition relation between the state estimatexk of the defect-free and the actual measurement zk of a pixel (defect-free or defective). Therefore, the measurement model for an arbitrary pixel can be defined as (3)zk=gkxk+wk,where gk(·) denotes the transition function from the state estimate xk to the measurement zk and wk is zero-mean white Gaussian noise sequences with variance σw,k2.In terms of the established models, Bayesian filter can be exploited to yield the state estimate ofx^k for each pixel along a scan line of an image by Bayes recursion below [17]:(4)pk∣k-1xk∣z1:k-1=∫fk∣k-1xk∣xpk-1x∣z1:k-1dx,(5)pkxk∣z1:k=gkzk∣xkpk∣k-1xk∣z1:k-1∫gkzk∣xpk∣k-1xk∣z1:k-1dx,where fk∣k-1(·∣·) is the transition density defined by (1) or (2) and gk(·∣·) is the likelihood function defined by (3). The posterior density pk(xk∣z1:k) includes all information of the state at time k and the state estimate x^k of the kth pixel in a scan line can be obtained by maximum a posteriori (MAP) criterion.Once the state estimatex^k-1 at time step k-1 (i.e., intensity of the k-1th pixel along the scan line) is obtained, the difference between the predicted measurement zk∣k-1 and the actual measurement zk is defined as residual νk∗, which can be calculated by(6)νk∗=zk-zk∣k-1.Here, zk∣k-1 can be derived by (1) and (3) with the form(7)zk∣k-1=gkxk∣k-1=gkFk∗x^k-1,where xk∣k-1 is the predicted state using (1). It can be seen from (6) and (7) that a residual νk∗ reveals the difference between the actual measurement of a pixel and the defect-free pixel, which can be used to decide the occurrence of a fault/defect. When the measurement zk is a Gaussian variable, the statistic νk∗Σ-12=νk∗′Σ-1νk∗ is chi-square distributed. Then a chi-square test [17] will declare detection of a defect if(8)νk∗Σ-12>λ=χn2α,where 1-α is the level of confidence, n indicates the degrees of freedom, and λ is the detection threshold.


## 3. Particle Filter Based on Adaptive Parametric Model

### 3.1. Adaptive Parametric Measurement Model

As explained in Section2, it is important to warrant that the defect information is revealed by residual νk∗ of the state and the measurement. To attain this purpose, the state model about the defect-free object based on random walk model is presented in (1) and (2). Furthermore, it is also necessary to design an accurate measurement model which can closely approximate the real measurement contaminated by noise, defects, and other disturbances. As shown in Figure 1, the measurements along a scan line in a complex visual inspection system follow multimodel distribution.Figure 1
An image for inspecting the surface defect on the base desk of a hard disk and its histogram. (a) The image of the inspected object. (b) The histogram of gray levels of pixels along a column line in (a).
(a)
(b)In order to obtain an accurate model, the Gaussian mixture model is exploited to approximate the multimodel distribution of measurements in the image of the inspected object. Assume that at timek-1 (i.e., the k-1th pixel in a scan line), the probability density of the measurement can be approximated by a Gaussian mixture model (GMM) as(9)zk-1~∑m=1Gπk-1,mNzk-1∣μk-1,m,σk-1,m,where πk-1,m is the weight of the mth Gaussian component in the mixture with ∑m=1Gπk-1,m=1, μk-1,m is the mean, σk-1,m is the variance, and G is the Gaussian component number with G= 3~5. Then, the measurement likelihood function can be designed as(10)pzk∣xk=∑m=1Gπk,mNzk;μk,m,σk,m2.All the parameters{πk,m,μk,m,σk,m2}m=1G of Gaussian items in (10) can be automatically updated by the model (9) at time k-1 using k-means approximation [19] in Algorithm 1. The number of Gaussian distributions G is manually set by making a tradeoff among the complexity of the pixel intensity model, the available memory, and the computational power. On the one hand, due to more ideal illumination and regular geometrical structure of the inspection condition, subtle changes of illumination of the desk base surface and only repetitive changes of scene elements may occur in the real inspected images (see Figure 1(a)). Thus, it is reasonable to simply model the values of a particular pixel along a scan line as a mixture of a fixed number of Gaussian distributions. On the other hand, less Gaussian distributions can alleviate the computation load. Considering the above elements, the value of G should be as small as possible, on the premise that the number of distributions is able to represent the value of each pixel along each scan line of the inspected image. For example, in this paper, G is set to be 3 in the application of desk base inspection.Algorithm 1:The update of GMM in the likelihood by k-means approximation.
Input: zk, {πm,k-1,μm,k-1,σm,k-1}m=1G
Output: {πm,k,μm,k,σm,k}m=1G
Step  1. Update weights of the G Gaussian items by πm,k=(1-α)πm,k-1+α(Mm,k-1),
whereα is the learning rate, Mm,k-1 is 1 for the model matched by zk and 0 for the
remaining models. If none of theG distributions matches zk, the least probable distribution
is replaced with a distribution withzk as its mean, a high variance, and low weight.
Step  2. Update parameters of the Gaussian item: ∀zk satisfies zk-μk,m<2.5σk,m, compute
zm,k=zk
μm,k=(1-ρ)μm,k-1+ρzm,k,
σm,k2=(1-ρ)σm,k-12+ρzm,k-μm,kT(zm,k-μm,k)
withρ=αN(zm,k;μm,k,σm,k).In addition,πk,m, μk,m, and σk,m can be computed by the k-means approximation in Algorithm 1. It should be noticed that the weight πk,m of a Gaussian distribution is updated according to(11)πm,k=1-απm,k-1+αMm,k-1,where α is the learning rate and Mm,k-1i is 1 for the model matched by zk (i.e., zk-μk,m<2.5σk,m) and 0 for the remaining models otherwise. If none of the G distributions matches zk, the least probable distribution is replaced with a distribution with zk as its mean, a high variance, and a low weight.Once parameters{πk,m,μk,m,σk,m2}m=1G at time k are obtained, the likelihood function (10) can be updated and be used to implement PF or Bayes recursion instead of gk(·). Due to the better ability of fitting the complex distribution of pixel values, the proposed adaptive measurement model can help to represent the system more correctly.


### 3.2. Particle Filter with the Adaptive Model

As analyzed in Section2, state is required to be estimated for defect detection. Since the proposed measurement model in Section 3.1 is Gaussian and nonlinear, Bayesian filter in (4) and (5) can be solved by particle filter [18]. Particle filter is based on Monte Carlo approximation, which is able to effectively deal with nonlinear and non-Gaussian problems [18]. In terms of the proposed state model in (1) and the measurement likelihood in (10), particle filter with adaptive model is summarized below.Step 1 (initializing).
DrawN particles from prior density of states p(x0). Assign weights ω=1/N with N particles for filtering.Step 2 (important sampling).
Firstly, the particlesxk-1(i)i=1N of p(xk-1∣z1:k-1) in (4) are propagated by the state model p(xk(i)∣xk-1(i)), which can be obtained using (1) or (2) with the form(12)pxki∣xk-1i~N0,σv,k2.Secondly, given a new observationzk, each sampled particle xk(j) from (12) is weighted by the proposed adaptive likelihood in (10) with the form (13)wki∝pzk∣xki.Finally, the weights are normalized by(14)ωtj=ωtj∑j=1Nωtj.Step 3 (resampling).
Resample new particles from the particlesxk(i),ωk(i)i=1N in Step 1 according to roulette method, where the probability of choosing a particle is proportional to its weight. After resampling, the resampled particles x~k(i),1/Ni=1N are with equal weights, which can approximate the posterior distribution pk(xk∣z1:k) in (5) at time k.Step 4 (state extraction).
The state estimatex^k can be extracted by using either the minimum mean square error criterion:(15)x^k≈1N∑i=1Nx~kior the maximum a posteriori (MAP) criterion:(16)x^k=argmaxxkpxk∣z1:k≈argmaxxkωki.


## 3.1. Adaptive Parametric Measurement Model

As explained in Section2, it is important to warrant that the defect information is revealed by residual νk∗ of the state and the measurement. To attain this purpose, the state model about the defect-free object based on random walk model is presented in (1) and (2). Furthermore, it is also necessary to design an accurate measurement model which can closely approximate the real measurement contaminated by noise, defects, and other disturbances. As shown in Figure 1, the measurements along a scan line in a complex visual inspection system follow multimodel distribution.Figure 1
An image for inspecting the surface defect on the base desk of a hard disk and its histogram. (a) The image of the inspected object. (b) The histogram of gray levels of pixels along a column line in (a).
(a)
(b)In order to obtain an accurate model, the Gaussian mixture model is exploited to approximate the multimodel distribution of measurements in the image of the inspected object. Assume that at timek-1 (i.e., the k-1th pixel in a scan line), the probability density of the measurement can be approximated by a Gaussian mixture model (GMM) as(9)zk-1~∑m=1Gπk-1,mNzk-1∣μk-1,m,σk-1,m,where πk-1,m is the weight of the mth Gaussian component in the mixture with ∑m=1Gπk-1,m=1, μk-1,m is the mean, σk-1,m is the variance, and G is the Gaussian component number with G= 3~5. Then, the measurement likelihood function can be designed as(10)pzk∣xk=∑m=1Gπk,mNzk;μk,m,σk,m2.All the parameters{πk,m,μk,m,σk,m2}m=1G of Gaussian items in (10) can be automatically updated by the model (9) at time k-1 using k-means approximation [19] in Algorithm 1. The number of Gaussian distributions G is manually set by making a tradeoff among the complexity of the pixel intensity model, the available memory, and the computational power. On the one hand, due to more ideal illumination and regular geometrical structure of the inspection condition, subtle changes of illumination of the desk base surface and only repetitive changes of scene elements may occur in the real inspected images (see Figure 1(a)). Thus, it is reasonable to simply model the values of a particular pixel along a scan line as a mixture of a fixed number of Gaussian distributions. On the other hand, less Gaussian distributions can alleviate the computation load. Considering the above elements, the value of G should be as small as possible, on the premise that the number of distributions is able to represent the value of each pixel along each scan line of the inspected image. For example, in this paper, G is set to be 3 in the application of desk base inspection.Algorithm 1:The update of GMM in the likelihood by k-means approximation.
Input: zk, {πm,k-1,μm,k-1,σm,k-1}m=1G
Output: {πm,k,μm,k,σm,k}m=1G
Step  1. Update weights of the G Gaussian items by πm,k=(1-α)πm,k-1+α(Mm,k-1),
whereα is the learning rate, Mm,k-1 is 1 for the model matched by zk and 0 for the
remaining models. If none of theG distributions matches zk, the least probable distribution
is replaced with a distribution withzk as its mean, a high variance, and low weight.
Step  2. Update parameters of the Gaussian item: ∀zk satisfies zk-μk,m<2.5σk,m, compute
zm,k=zk
μm,k=(1-ρ)μm,k-1+ρzm,k,
σm,k2=(1-ρ)σm,k-12+ρzm,k-μm,kT(zm,k-μm,k)
withρ=αN(zm,k;μm,k,σm,k).In addition,πk,m, μk,m, and σk,m can be computed by the k-means approximation in Algorithm 1. It should be noticed that the weight πk,m of a Gaussian distribution is updated according to(11)πm,k=1-απm,k-1+αMm,k-1,where α is the learning rate and Mm,k-1i is 1 for the model matched by zk (i.e., zk-μk,m<2.5σk,m) and 0 for the remaining models otherwise. If none of the G distributions matches zk, the least probable distribution is replaced with a distribution with zk as its mean, a high variance, and a low weight.Once parameters{πk,m,μk,m,σk,m2}m=1G at time k are obtained, the likelihood function (10) can be updated and be used to implement PF or Bayes recursion instead of gk(·). Due to the better ability of fitting the complex distribution of pixel values, the proposed adaptive measurement model can help to represent the system more correctly.


## 3.2. Particle Filter with the Adaptive Model

As analyzed in Section2, state is required to be estimated for defect detection. Since the proposed measurement model in Section 3.1 is Gaussian and nonlinear, Bayesian filter in (4) and (5) can be solved by particle filter [18]. Particle filter is based on Monte Carlo approximation, which is able to effectively deal with nonlinear and non-Gaussian problems [18]. In terms of the proposed state model in (1) and the measurement likelihood in (10), particle filter with adaptive model is summarized below.Step 1 (initializing).
DrawN particles from prior density of states p(x0). Assign weights ω=1/N with N particles for filtering.Step 2 (important sampling).
Firstly, the particlesxk-1(i)i=1N of p(xk-1∣z1:k-1) in (4) are propagated by the state model p(xk(i)∣xk-1(i)), which can be obtained using (1) or (2) with the form(12)pxki∣xk-1i~N0,σv,k2.Secondly, given a new observationzk, each sampled particle xk(j) from (12) is weighted by the proposed adaptive likelihood in (10) with the form (13)wki∝pzk∣xki.Finally, the weights are normalized by(14)ωtj=ωtj∑j=1Nωtj.Step 3 (resampling).
Resample new particles from the particlesxk(i),ωk(i)i=1N in Step 1 according to roulette method, where the probability of choosing a particle is proportional to its weight. After resampling, the resampled particles x~k(i),1/Ni=1N are with equal weights, which can approximate the posterior distribution pk(xk∣z1:k) in (5) at time k.Step 4 (state extraction).
The state estimatex^k can be extracted by using either the minimum mean square error criterion:(15)x^k≈1N∑i=1Nx~kior the maximum a posteriori (MAP) criterion:(16)x^k=argmaxxkpxk∣z1:k≈argmaxxkωki.


## 4. Defect Detection and Localization Based on PF

When the state estimates{x^k}k=1N or {x^k}k=1M of pixels along each row or column scan line are generated by the proposed PF in Section 3, the state estimates can be employed to establish the statistic for detecting and locating defects. To yield the statistic for chi-square test, the residual νk∗ for each pixel should be computed firstly using (6). Under the Gaussian assumption, since the residual sequences {νk∗}k=1N and {νk∗}k=1M are Gaussian-distributed, the normalized residual squared statistic (NRSS) εk is approximately chi-square distributed with the form(17)εk=νk∗′Sk-1νk∗,where Sk-1 denotes the covariance of νk∗. In terms of the principle of a check of the goodness of fit, εk is employed as the statistic and the fault/defect can be located along the scan lines by the chi-square test: (18)εk>λ=χn2α,where 1-α denotes the level of confidence, n is the degrees of freedom with n=dim⁡(zk), and the detection threshold is λ. Here, λ can be referred to in the chi-square distribution table in terms of α and n. If the statistic εk for a pixel satisfies (18), an alarm indicating a defect point is given.When the chi-square tests along all row lines are completed, the potential defect points are stored in the detection maskΘ(M×N), which is an M×N matrix with value 1 for an edge point and with value 0 for a defect-free point. Similarly, the mask Θ′(M×N) along columns can be obtained. Considering different sensitivity of both tests along row and column directions, the final mask Θ^(M×N) (see Figure 3(b)) can be calculated by(19)Θ^i,j=Θi,j∣Θ′i,j,where A(i,j) denotes detection result of the ith row jth pixel in the image and “∣” denotes logical OR operator. Based on all of the above, the proposed defect detector using PF with adaptive model is described in Figure 2. In addition, in order to locate accurately the occurrence of a defect with lower false alarms, the cumulative sum of the statistic εk is calculated to distinguish the defect from the image (see Figure 3(a)).Figure 2
Procedure of the proposed detector.Figure 3
A sample result of defect detection. (a) Defect locations using the accumulative sum of NSSRs. (b) Defect locations using NSSR.
(a)
(b)


## 5. Experimental Results

### 5.1. Experimental Setup

The VACE protocol [20] is introduced here to evaluate the detection performance of the proposed algorithm, which uses two scores, namely, multiple object detection accuracy (MODA) and multiple object detection precision (MODP). The MODP and MODA measures are directly proportional to the area overlap between the system output and the ground truth. Higher scores of MODP and MODA indicate better performance in defect detection precision and accuracy, respectively. Denote Git and Dit as the ith ground truth object and its associated detected object (defect) in the tth frame, respectively. MODP and MODA are defined as(20)MODP=∑i=1Nframes∑t=1NmappedGit∩Dit/Git∪Dit∑j=1NframesNmappedj,where ∑i=1Nmapped∑t=1NframesGit∩Dit/Git∪Dit quantifies the spatial overlap ratio between the ground truth and the detected object in a sequence of frames, Nmapped is the number of mapped objects over the entire sequence, and Nmappedj refers to the number of mapped objects in the jth frame. Consider(21)MODA=1-∑i=1Nframescmmi+cffpi∑i=1NframesNGi,where mi is the number of missed detects, fpi is the total number of false positives, cm(·) and cf(·) are the cost functions for mi and fpi, and NGi is the number of ground truth objects in the ith frame. In our experiments, the ground truth detects are annotated manually according to annotation guidelines for VACE [15]. The best possible match between ground truth and detected objects in a global sense can be solved by Hungarian algorithm.


### 5.2. Experimental Results and Discussion

#### 5.2.1. Defect Detection Experiment

The proposed detector is tested by the defect image database, which contains multiple kinds of defects on the surface of the desk base of hard disks.A sample result using the proposed defect detector is shown in Figure3. Figure 3(a) plots the accumulative sum of NSSRs for row and column scan lines and marks the defect location with red circles, whose centers are calculated by abrupt changes in the response curve of NSSRs. Figure 3(b) shows another defect location result of the same inspected part by a mask image in (19) directly using NSSR of each pixel. Figure 3 indicates that both methods on the NSSRs and their accumulative sum are effective to locate defects in an image. In the following experiments, each of the location methods is chosen arbitrarily.Figure4 shows the detection results of various types of defects, for example, corrosion, hard blister, pin holes, and crack. It is shown that the proposed detector is able to accurately and automatically identify and locate defective areas from complex background with much less false alarms. As well known, uneven illumination (see Figures 4(a) and 4(b)) and geometrical structure (see Figure 4(a)) lead to scattered discontinuities in the defect-free areas, which make defect detection more difficult. To alleviate it, a more accurate adaptive measurement model is proposed for PF, which can inhibit the influence of discontinuities in the defect-free pixels.Figure 4
Application of the proposed method to inspect various defects.
(a)
(b)
(c)
(d)


#### 5.2.2. Comparison Experiment

To evaluate the performance of the proposed detector further, we compare the proposed detector (DET_PF) with the detector in [15] (DET_KAL) and Otsu thresholding (DET_THR). Figure 5 shows the comparison result of DET_PF and DET_KAL. Since DET_KAL employs a single Gaussian model as the measurement model, it cannot accurately approximate the measurements of multimodel distribution. It can be seen from Figure 5(b) that DET_KAL is not able to locate defects in a complicated background (i.e., complicated intensity distribution of pixels in defect-free areas). Figure 5 indicates that the proposed detector based on PF with a robust model outperforms DET_KAL in fault detection for nonlinear system.Figure 5
Sample comparison detection results of detectors. (a) Detection results of DET_PF. (b) Detection results of DET_KAL.
(a)
(b)To quantify the performance difference of the three methods, we compute their performance scores according to evaluation method presented in Section5.1. Figure 6 shows the accuracy and the precision scores (i.e., MODA and MODP scores) of the three defect detectors applied to the test database. It can be observed that the proposed detector achieves higher performance scores than common thresholding DET_THR method and DET_KAL.Figure 6
Comparison result of evaluation score for three defect detectors.


#### 5.2.3. Sensitivity Analysis of Parameters

The parameters involved in the proposed detector, such as Gaussian model noise sequenceswk and vk in (1), (2), and (3) and the decision threshold λ in (17), can be preset manually according to the statistical characteristics of the gray level or the intensity of the inspected image. As mentioned in Section 2 of this paper, wk and vk are zero-mean white Gaussian sequences and are proportional to their standard deviations σw and σv.Figure7 illustrates the accuracy and precision transitions of defect detection performance when the parameters σw, σv, and λ of the detector are changed. When the value of one parameter with log2 scale at a time are changed and the other parameter values are fixed, then the average values of the corresponding MODP and MODA measures are obtained and plotted in Figure 7. As shown in Figures 7(a) and 7(b), the detector is sensitive to σw and σv. Both σw and σv represent the model errors between the chosen system models (i.e., σw for dynamical models and σv for the measurement model) and the truth. Smaller and larger σw generally leads to divergence of the particle filter and a degraded detection precision, namely, decreasing values of MODA and MODP, respectively. Similarly, choices of σv depend on the measurement model error, which can be determined practically in terms of image quality, and a smaller σv would be chosen for an image of higher quality. The theoretical value of the decision threshold λ can be computed according to [17]. As shown in Figure 7(c), a small variation of the detection performance scores is related to theoretical value of the decision threshold λ. It can be seen that, in a real application of the proposed detector, quantities of σw, σv, and λ can be manually heuristically chosen from several tests.Figure 7
Sensitivity analysis of parameters used in the detector. (a) Average evaluation scores for the standard deviation of the dynamical model. (b) Average evaluation scores for the standard deviation of the observation model. (c) Average evaluation scores for the decision threshold.
(a)
(b)
(c)


#### 5.2.4. Computational Complexity Analysis and Comparison

The computational complexity of the proposed method depends on two main steps: thek-means approximation based adaptive measurement model learning and particle filter. Indeed, at time k, the k-means approximation step requires calculation of cost:(22)Ca,k=Gk∗Jk,where G is the number of Gaussian items and Jk is the number of pixels in the inspected image.The particle filter requires(23)Cb,k=43pk3+2pk2+4pk2+Pk∗Nk∗Jk,where Pk denotes the state dimension in the PF and Nk denotes the number of particles. In the application to the desk base of hard disk, Pk is 1 and Nk is 200. Thus, in the experiment, the computational load is Cb,k≈5∗Nk∗Jk=1000Jk.Furthermore, in our experiments, the proposed defect detection algorithm was carried out by an Intel 2.2 GHz CPU PC. Detection was done at an average rate of 20 frames per second (i.e., 50 ms per image) for images with size of1024∗768 without code optimization. In comparison with the similar method (MR) in [15], MR is able to process the a test image in about 60 ms. MR suffers more intensively computational load, which may be induced by the matrix inversion operation in Kalman filter with constant velocity state model and multidimensional vector of the state. From the above analysis, with code optimization, the proposed method will be a promising solution to defect detection in real time.


## 5.1. Experimental Setup

The VACE protocol [20] is introduced here to evaluate the detection performance of the proposed algorithm, which uses two scores, namely, multiple object detection accuracy (MODA) and multiple object detection precision (MODP). The MODP and MODA measures are directly proportional to the area overlap between the system output and the ground truth. Higher scores of MODP and MODA indicate better performance in defect detection precision and accuracy, respectively. Denote Git and Dit as the ith ground truth object and its associated detected object (defect) in the tth frame, respectively. MODP and MODA are defined as(20)MODP=∑i=1Nframes∑t=1NmappedGit∩Dit/Git∪Dit∑j=1NframesNmappedj,where ∑i=1Nmapped∑t=1NframesGit∩Dit/Git∪Dit quantifies the spatial overlap ratio between the ground truth and the detected object in a sequence of frames, Nmapped is the number of mapped objects over the entire sequence, and Nmappedj refers to the number of mapped objects in the jth frame. Consider(21)MODA=1-∑i=1Nframescmmi+cffpi∑i=1NframesNGi,where mi is the number of missed detects, fpi is the total number of false positives, cm(·) and cf(·) are the cost functions for mi and fpi, and NGi is the number of ground truth objects in the ith frame. In our experiments, the ground truth detects are annotated manually according to annotation guidelines for VACE [15]. The best possible match between ground truth and detected objects in a global sense can be solved by Hungarian algorithm.


## 5.2. Experimental Results and Discussion

### 5.2.1. Defect Detection Experiment

The proposed detector is tested by the defect image database, which contains multiple kinds of defects on the surface of the desk base of hard disks.A sample result using the proposed defect detector is shown in Figure3. Figure 3(a) plots the accumulative sum of NSSRs for row and column scan lines and marks the defect location with red circles, whose centers are calculated by abrupt changes in the response curve of NSSRs. Figure 3(b) shows another defect location result of the same inspected part by a mask image in (19) directly using NSSR of each pixel. Figure 3 indicates that both methods on the NSSRs and their accumulative sum are effective to locate defects in an image. In the following experiments, each of the location methods is chosen arbitrarily.Figure4 shows the detection results of various types of defects, for example, corrosion, hard blister, pin holes, and crack. It is shown that the proposed detector is able to accurately and automatically identify and locate defective areas from complex background with much less false alarms. As well known, uneven illumination (see Figures 4(a) and 4(b)) and geometrical structure (see Figure 4(a)) lead to scattered discontinuities in the defect-free areas, which make defect detection more difficult. To alleviate it, a more accurate adaptive measurement model is proposed for PF, which can inhibit the influence of discontinuities in the defect-free pixels.Figure 4
Application of the proposed method to inspect various defects.
(a)
(b)
(c)
(d)


### 5.2.2. Comparison Experiment

To evaluate the performance of the proposed detector further, we compare the proposed detector (DET_PF) with the detector in [15] (DET_KAL) and Otsu thresholding (DET_THR). Figure 5 shows the comparison result of DET_PF and DET_KAL. Since DET_KAL employs a single Gaussian model as the measurement model, it cannot accurately approximate the measurements of multimodel distribution. It can be seen from Figure 5(b) that DET_KAL is not able to locate defects in a complicated background (i.e., complicated intensity distribution of pixels in defect-free areas). Figure 5 indicates that the proposed detector based on PF with a robust model outperforms DET_KAL in fault detection for nonlinear system.Figure 5
Sample comparison detection results of detectors. (a) Detection results of DET_PF. (b) Detection results of DET_KAL.
(a)
(b)To quantify the performance difference of the three methods, we compute their performance scores according to evaluation method presented in Section5.1. Figure 6 shows the accuracy and the precision scores (i.e., MODA and MODP scores) of the three defect detectors applied to the test database. It can be observed that the proposed detector achieves higher performance scores than common thresholding DET_THR method and DET_KAL.Figure 6
Comparison result of evaluation score for three defect detectors.


### 5.2.3. Sensitivity Analysis of Parameters

The parameters involved in the proposed detector, such as Gaussian model noise sequenceswk and vk in (1), (2), and (3) and the decision threshold λ in (17), can be preset manually according to the statistical characteristics of the gray level or the intensity of the inspected image. As mentioned in Section 2 of this paper, wk and vk are zero-mean white Gaussian sequences and are proportional to their standard deviations σw and σv.Figure7 illustrates the accuracy and precision transitions of defect detection performance when the parameters σw, σv, and λ of the detector are changed. When the value of one parameter with log2 scale at a time are changed and the other parameter values are fixed, then the average values of the corresponding MODP and MODA measures are obtained and plotted in Figure 7. As shown in Figures 7(a) and 7(b), the detector is sensitive to σw and σv. Both σw and σv represent the model errors between the chosen system models (i.e., σw for dynamical models and σv for the measurement model) and the truth. Smaller and larger σw generally leads to divergence of the particle filter and a degraded detection precision, namely, decreasing values of MODA and MODP, respectively. Similarly, choices of σv depend on the measurement model error, which can be determined practically in terms of image quality, and a smaller σv would be chosen for an image of higher quality. The theoretical value of the decision threshold λ can be computed according to [17]. As shown in Figure 7(c), a small variation of the detection performance scores is related to theoretical value of the decision threshold λ. It can be seen that, in a real application of the proposed detector, quantities of σw, σv, and λ can be manually heuristically chosen from several tests.Figure 7
Sensitivity analysis of parameters used in the detector. (a) Average evaluation scores for the standard deviation of the dynamical model. (b) Average evaluation scores for the standard deviation of the observation model. (c) Average evaluation scores for the decision threshold.
(a)
(b)
(c)


### 5.2.4. Computational Complexity Analysis and Comparison

The computational complexity of the proposed method depends on two main steps: thek-means approximation based adaptive measurement model learning and particle filter. Indeed, at time k, the k-means approximation step requires calculation of cost:(22)Ca,k=Gk∗Jk,where G is the number of Gaussian items and Jk is the number of pixels in the inspected image.The particle filter requires(23)Cb,k=43pk3+2pk2+4pk2+Pk∗Nk∗Jk,where Pk denotes the state dimension in the PF and Nk denotes the number of particles. In the application to the desk base of hard disk, Pk is 1 and Nk is 200. Thus, in the experiment, the computational load is Cb,k≈5∗Nk∗Jk=1000Jk.Furthermore, in our experiments, the proposed defect detection algorithm was carried out by an Intel 2.2 GHz CPU PC. Detection was done at an average rate of 20 frames per second (i.e., 50 ms per image) for images with size of1024∗768 without code optimization. In comparison with the similar method (MR) in [15], MR is able to process the a test image in about 60 ms. MR suffers more intensively computational load, which may be induced by the matrix inversion operation in Kalman filter with constant velocity state model and multidimensional vector of the state. From the above analysis, with code optimization, the proposed method will be a promising solution to defect detection in real time.


## 5.2.1. Defect Detection Experiment

The proposed detector is tested by the defect image database, which contains multiple kinds of defects on the surface of the desk base of hard disks.A sample result using the proposed defect detector is shown in Figure3. Figure 3(a) plots the accumulative sum of NSSRs for row and column scan lines and marks the defect location with red circles, whose centers are calculated by abrupt changes in the response curve of NSSRs. Figure 3(b) shows another defect location result of the same inspected part by a mask image in (19) directly using NSSR of each pixel. Figure 3 indicates that both methods on the NSSRs and their accumulative sum are effective to locate defects in an image. In the following experiments, each of the location methods is chosen arbitrarily.Figure4 shows the detection results of various types of defects, for example, corrosion, hard blister, pin holes, and crack. It is shown that the proposed detector is able to accurately and automatically identify and locate defective areas from complex background with much less false alarms. As well known, uneven illumination (see Figures 4(a) and 4(b)) and geometrical structure (see Figure 4(a)) lead to scattered discontinuities in the defect-free areas, which make defect detection more difficult. To alleviate it, a more accurate adaptive measurement model is proposed for PF, which can inhibit the influence of discontinuities in the defect-free pixels.Figure 4
Application of the proposed method to inspect various defects.
(a)
(b)
(c)
(d)


## 5.2.2. Comparison Experiment

To evaluate the performance of the proposed detector further, we compare the proposed detector (DET_PF) with the detector in [15] (DET_KAL) and Otsu thresholding (DET_THR). Figure 5 shows the comparison result of DET_PF and DET_KAL. Since DET_KAL employs a single Gaussian model as the measurement model, it cannot accurately approximate the measurements of multimodel distribution. It can be seen from Figure 5(b) that DET_KAL is not able to locate defects in a complicated background (i.e., complicated intensity distribution of pixels in defect-free areas). Figure 5 indicates that the proposed detector based on PF with a robust model outperforms DET_KAL in fault detection for nonlinear system.Figure 5
Sample comparison detection results of detectors. (a) Detection results of DET_PF. (b) Detection results of DET_KAL.
(a)
(b)To quantify the performance difference of the three methods, we compute their performance scores according to evaluation method presented in Section5.1. Figure 6 shows the accuracy and the precision scores (i.e., MODA and MODP scores) of the three defect detectors applied to the test database. It can be observed that the proposed detector achieves higher performance scores than common thresholding DET_THR method and DET_KAL.Figure 6
Comparison result of evaluation score for three defect detectors.


## 5.2.3. Sensitivity Analysis of Parameters

The parameters involved in the proposed detector, such as Gaussian model noise sequenceswk and vk in (1), (2), and (3) and the decision threshold λ in (17), can be preset manually according to the statistical characteristics of the gray level or the intensity of the inspected image. As mentioned in Section 2 of this paper, wk and vk are zero-mean white Gaussian sequences and are proportional to their standard deviations σw and σv.Figure7 illustrates the accuracy and precision transitions of defect detection performance when the parameters σw, σv, and λ of the detector are changed. When the value of one parameter with log2 scale at a time are changed and the other parameter values are fixed, then the average values of the corresponding MODP and MODA measures are obtained and plotted in Figure 7. As shown in Figures 7(a) and 7(b), the detector is sensitive to σw and σv. Both σw and σv represent the model errors between the chosen system models (i.e., σw for dynamical models and σv for the measurement model) and the truth. Smaller and larger σw generally leads to divergence of the particle filter and a degraded detection precision, namely, decreasing values of MODA and MODP, respectively. Similarly, choices of σv depend on the measurement model error, which can be determined practically in terms of image quality, and a smaller σv would be chosen for an image of higher quality. The theoretical value of the decision threshold λ can be computed according to [17]. As shown in Figure 7(c), a small variation of the detection performance scores is related to theoretical value of the decision threshold λ. It can be seen that, in a real application of the proposed detector, quantities of σw, σv, and λ can be manually heuristically chosen from several tests.Figure 7
Sensitivity analysis of parameters used in the detector. (a) Average evaluation scores for the standard deviation of the dynamical model. (b) Average evaluation scores for the standard deviation of the observation model. (c) Average evaluation scores for the decision threshold.
(a)
(b)
(c)


## 5.2.4. Computational Complexity Analysis and Comparison

The computational complexity of the proposed method depends on two main steps: thek-means approximation based adaptive measurement model learning and particle filter. Indeed, at time k, the k-means approximation step requires calculation of cost:(22)Ca,k=Gk∗Jk,where G is the number of Gaussian items and Jk is the number of pixels in the inspected image.The particle filter requires(23)Cb,k=43pk3+2pk2+4pk2+Pk∗Nk∗Jk,where Pk denotes the state dimension in the PF and Nk denotes the number of particles. In the application to the desk base of hard disk, Pk is 1 and Nk is 200. Thus, in the experiment, the computational load is Cb,k≈5∗Nk∗Jk=1000Jk.Furthermore, in our experiments, the proposed defect detection algorithm was carried out by an Intel 2.2 GHz CPU PC. Detection was done at an average rate of 20 frames per second (i.e., 50 ms per image) for images with size of1024∗768 without code optimization. In comparison with the similar method (MR) in [15], MR is able to process the a test image in about 60 ms. MR suffers more intensively computational load, which may be induced by the matrix inversion operation in Kalman filter with constant velocity state model and multidimensional vector of the state. From the above analysis, with code optimization, the proposed method will be a promising solution to defect detection in real time.


## 6. Conclusions

This paper presents a novel methodology for automatic fault/defect detection based on PF for nonlinear systems. The defect/fault detecting framework in a stochastic dynamical system based on PF and chi-square test is proposed first. To alleviate the scattered discontinuities from the uneven illumination and geometrical structure of the inspected objects, an adaptive measurement model is designed and estimation method of the hidden states for nonlinear system based on PF is presented in detail. Third, the statistical properties of the hidden states are employed to establish an effective statistic for defect/fault defection by chi-square test. Finally, the improved PF with adaptive parametric model and the fault detection framework are applied to identify the defects in visual inspection system. Experiments demonstrate that this proposed methodology can detect most defects efficiently with high quality in nonlinear system, which can be exploited to deal with fault detection problems in other industry fields.



---

*Source: 759035-2015-12-24.xml*