# Infrared Small Target Detection with Total Variation and Reweightedℓ1 Regularization

Authors: Houzhang Fang; Min Chen; Xiyang Liu; Shoukui Yao
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

Infrared small target detection plays an important role in infrared search and tracking systems applications. It is difficult to perform target detection when only a single image with complex background clutters and noise is available, where the key is to suppress the complex background clutters and noise while enhancing the small target. In this paper, we propose a novel model for separating the background from the small target based on nonlocal self-similarity for infrared patch-image. A total variation-based regularization term for the small target image is incorporated into the model to suppress the residual background clutters and noise while enhancing the smoothness of the solution. Furthermore, a reweighted sparse constraint is imposed for the small target image to remove the nontarget points while better highlighting the small target. For higher computational efficiency, an adapted version of the alternating direction method of multipliers is employed to solve the resulting minimization problem. Comparative experiments with synthetic and real data demonstrate that the proposed method is superior in detection performance to the state-of-the-art methods in terms of both objective measure and visual quality.


---

## Body

## 1. Introduction

Infrared small target detection has been receiving considerable attention due to their wide applications in many fields, such as remote sensing [1], surveillance [2], environment monitoring [3], aerospace [4], and unmanned aerial vehicles [2]. The robust detection of infrared small target is still a challenging problem. The main difficulties of infrared small target detection lie in the following facts: (1) in realistic applications, the infrared imaging system is easily disturbed by such factors as the climate, atmospheric turbulence, and vibrating, as a result of which the observed small target often suffers complex background clutters and noise, thus a low signal-to-noise and signal-to-clutter ratio; (2) small target pixels occupy a smaller proportion of the whole observed image due to the long imaging distance; and (3) the infrared small target has no obvious texture and shape features.A large number of methods have been developed to address the issues of small target detection. These methods can be roughly classified into two categories: single-frame detection [5–16] and sequential multiframe detection [17–19]. Recently, Gao et al. [17] employed the mixture of the Gaussians model [20] with the Markov random field to model the complex noise of which the target is assumed as a component. Then the small targets are separated from noise by using the spatiotemporal information. The sequential methods usually utilize the temporal information of the multiframe image sequence which requires some strong assumptions of consistent information of targets and background between frames. However, these assumptions and prior knowledge can hardly be attained in applications. As such, single-frame detection has been attracting the attention of many researchers. In this research, we consider single-frame detection. In the conventional single-frame detection methods, background clutter suppression-based methods are the representative type for their implementation simplicity and computational efficiency, such as Top-hat filter [7], Max-mean and Max-median filter [5], and multilevel filter [6], which first suppressed the background clutters and noise using the filter and then extract the small target with an intensity threshold. Moon et al. [6] utilized a set of filters (multilevel filter) to enhance the target and suppress background clutters. However, these methods will lead to high false alarm rates and poor detection performance when the signal-to-clutter ratio is low. To better suppress the background clutters and noise while enhancing the small target, some methods [8, 9] based on local contrast enhancement have been proposed. Chen et al. [8] proposed a local contrast detection method by defining a new contrast measure inspired by the biological visual mechanism. Han et al. [9] proposed an improved difference of Gabor filter method based on human visual system to suppress the background edges and lower the false alarm rates. These methods try to improve the ability to suppress the clutters and noise as well as enhance the target. Shi et al. [16] proposed an enhanced high-boost-based multiscale local contrast measure (HB-MLCM) method to enhance the true target and suppress background clutters, and the experimental results verified the effectiveness of this method.Different from the background clutter suppression-based methods, another representative method tackles the infrared small target detection based on the robust principal component analysis (RPCA) [21]. The core idea of this kind of approach is to seek the best sparse and low-rank approximation of the given observation matrix, where the target is regarded as the sparse component and the background is low-rank component. RPCA framework has been widely applied to infrared small target detection [22–26] for its generalization and computation efficiency and can be extended to cope with large-scale problems. In realistic applications, sparse and low-rank recovery normally cannot be directly applied to small target detection because the decomposed background image matrix usually does not satisfy the low-rank characteristics. To deal with this issue, Gao et al. [22] proposed a more generic method by constructing a new infrared patch-image (IPI) model using the repeated local patches based on the nonlocal self-correlation property of the infrared image. The small target detection task is converted into an optimization problem of recovering low-rank and sparse matrices, which can be solved efficiently using existing optimization solution techniques. Wang et al. [23] introduced a total variation (TV) constraint on the background into the RPCA framework for better preserving the edges and corners of background region so as to improve the background estimation and achieve good detection results. Yao et al. [25] tackled infrared small target detection using a coarse-to-fine two-phase method. In the coarse phase, the background is separated by exploiting the nonlocal self-similarity properties under the RPCA framework, and then the target is separated from the unstructured clutters and noise in the unstructured layer in the fine phase. In this method, it is still very difficult to separate the small target from the unstructured clutters and noise using only the local prior when the clutters and noise are heavy. Therefore, the detection performance for the coarse-to-fine method is still limited. Zhang et al. [26] proposed to constrain the small target with nonconvex ℓp norm so as to restore sparser target images, and the method achieved an impressive detection performance.Recently, several convolution neural network- (CNN-) based infrared small target detection methods have been proposed [27–30]. Fan et al. [27] simulated the training data using the MNIST database (http://yann.lecun.com/exdb/mnist/) containing handwritten images and labels, and then the target and background are estimated based on the convolution kernels in the first layer of the CNN architecture, which are learned from the input data. The main disadvantage of applying the deep learning method to infrared small target detection is that feature learning will be very difficult for the CNN-based method as the infrared small target has no remarkable texture and shape features. In addition, because of insufficient training data, the visible datasets [27] or simulated infrared datasets [29, 30] are used to train the detection model. As a result, generalizability of the detection method is also limited.In practical applications, the observed image acquired by the infrared thermal imaging often suffers from evolving background clutters and heavy noise, especially in poor imaging conditions. Such background clutters and random noise are the main interferences to target-background separation. To obtain a more stable detection, an appropriate formulation under the RPCA framework is employed to decompose the constructed infrared patch-image into the sparse target component, the low-rank background component, and the noise component [21]. With complex background clutters and noise, the simple sparse constraint cannot distinguish the small target from the heavy clutters and noise well. And the heavy residual clutters and noise in the small target component result in high false alarm rates, thus poor robustness of small target detection. To resolve this problem, we apply the TV regularization on the small target to suppress the residual clutters and noise in the target image. TV regularization is originally introduced by Rudin et al. [31] to deal with the image denoising problem. TV regularization and its variants are also successfully extended to handle other issues, such as clutter suppression in radar target detection [32], blind image deconvolution [33, 34], and image destriping [35]. Zhang et al. [32] proposed a radar extended target detection method by combining the sparse constraint and total variation for suppressing the clutters and noise. The rational of using total variation as a regularizer for removing the clutters and noise is that the total variation of the interfering image is significantly larger than that of the clean image, and the minimization of the total variation can remove the clutters and noise [31]. Existing research work [31–35] shows that TV-based regularization has good performances on removing the clutters and noise from images without rich textures and details, which is suitable for small target images. Thus, incorporation of TV regularization can prevent the clutters and noise from remaining in the target image. Additionally, we integrate a weighted ℓ1 norm into the sparse constraint to better suppress the nontarget sparse points [24]. Since the size of the infrared object is small, the introduction of the weighting factor into the ℓ1 norm makes the target image sparser. Experiments show that the TV regularization and weighted mechanism constraints added in the RPCA formulation are suitable for small target detection.The contributions of this paper are listed as follows:(1)
We propose a new variant of low-rank matrix decomposition based on the RPCA framework for the separation of the target from the background. The TV regularization and weightedℓ1 norm are combined to constrain the small target. The proposed model enforces the spatial smoothing constraint on the target image, thus effectively suppressing the residual clutters and noise with TV regularization. Furthermore, the model accurately characterizes the sparse small target while suppressing the nontarget points via weighted ℓ1 norm constraint, which leads to a lower false alarm rate.(2)
The resulting constructed model contains the nuclear norm, TV norm, and weightedℓ1 norm, which is a challenging nonconvex optimization problem. To effectively solve this problem, an efficient numerical optimization algorithm based on the alternating direction method of multipliers is applied to solve the proposed minimization problem.The remainder of this paper is organized as follows. Section2 gives some notations and briefly introduces the IPI model. The proposed model is described in Section 3. Section 4 provides extensive experiments on synthetic and real infrared images to validate the effectiveness of the proposed method. We conclude this work in Section 5.


## 2. Notations and IPI Model

The mathematical notations we use in this paper are described in Table1.Table 1
Mathematical notations.
Notations
Explanation
X
Matrix
x
Scalar
TV
⋅
TV norm, which is defined as = ∑i=1m−1∑j=1n−1xi,j−xi+1,j2+xi,j−xi,j+12=∑i=1m−1xi,n−xi+1,n+∑j=1n−1xm,j−xm,j+1
∇
X
∇
x
X
,
∇
y
X
T
X
,
Y
Inner product between matrixX and Y, which is defined as ∑i=1m∑j=1nXijYij
X
0
ℓ
0 norm of X, which is the number of nonzero elements
X
1
ℓ
1 norm of X, which is the sum of all Xij, i.e., ∑ijXij
X
F
The Frobenius norm ofX, which is defined as ∑ijXij2
X
∗
Nuclear norm ofX, which is the sum of singular valuesThe original RPCA formulation decomposes the observation matrix into a low-rank and a sparse matrix [21]. Generally, the stable RPCA formulation for the infrared image model can be formulated as follows [21, 22]:(1)fD=fT+fB+fN,where fD, fT, fB, and fN are the original infrared image, the target image, the background image, and the random noise image, respectively. Then, we need to formulate appropriate constraints for the target, the background, and the noise based on some prior knowledge.In equation (1), the low-rank properties of the background image fB typically do not hold true in practice. The seminal work of [22] first proposed to employ the nonlocal self-similarity of the background to construct a new infrared patch-image (IPI) model. To calculate three components, the IPI model is transformed into the following form:(2)D=T+B+N,where D, T, B, and N are the corresponding patch-images constructed from fD, fT, fB, and fN by sliding the window of size a×a from left to right and top to bottom with a fixed step size. The IPI model can be extended to practical application very well.In practical infrared images, the small target usually has only several pixels or dozens of pixels compared with the whole image, so the target patch-imageT can be described as a sparse matrix:(3)T0<k,where k is determined by the number of small targets and their sizes. ⋅0 represents the ℓ0 norm.The background imagefB is usually considered to change slowly, so there are strong correlation properties between its local and nonlocal patches. And the low-rank property in B can be depicted as(4)rankB≤r,where r is an integer determined by the rank of B.The noise is assumed to be Gaussian random noise, andN can be described as(5)D−B−TF≤δ,where δ is a very small positive constant.According to the constructed IPI model, the infrared small target detection can be transformed into an optimization problem of recovering low-rank and sparse matrices, which can be formulated as(6)minT,BB∗+λT1s.t.D−T−BF≤δ,D=T+B+N,where ⋅∗ is introduced to constrain the low-rank property in B. ⋅1 is used to replace ⋅0 to simplify the solution of this model. λ is a weighting parameter.


## 3. Proposed Method

In this section, we first present the proposed detection model and analyze the rational behind the model. Secondly, the solution of the proposed model is carefully designed. Finally, we give the target detection procedure.


### 3.1. Proposed Model

The small target detection based on the IPI model in (6) does work well in suppressing the clutters, noise, and nontarget points only using the ℓ1 norm constraint for the target image. In order to better separate the small target from the constructed observation image, the TV regularization term and the adaptive weighted ℓ1 norm on the target image are introduced into equation (6). The proposed model can be written as(7)minT,BBwB,∗+λ1TWT,1+λ2∇T1s.t.D−T−BF≤δ,D=T+B+N,where λ1 and λ2 are compromising parameters. The vector wB and the matrix WT are the weights for the background B and the target T, respectively. The weight wB is incorporated into the nuclear norm term for better preserving the strong edges of the background image [24], which is defined as(8)wB,j=1σB,j+εB,where σB,j is the jth singular value of B and wB=wB,1,wB,2,…,wB,nT is a weight for σB,j. εB is a positive value. BwB,∗=∑jwB,jσB,j is the reweighted nuclear norm of B.The weightWT is employed to suppress the nontarget points while enhancing the small target, and each element of WT is defined as(9)WT,ij=CTij+εT,where Tij is the pixel value of position i,j in the target image T. C is a compromising constant and εT is a small positive constant to avoid division by zero. The advantage of the weighted ℓ1 norm in equation (7) is to give every single element a different weight, thus better depicting the small target component. Because the small targets are a little brighter than other nontarget parts, this allows the small target to be better separated as the iteration proceeds. In equation (7), the TV regularization has good performance in removing the background clutters and the noise. The two main interferences can be effectively suppressed, and thus, the small target can be further highlighted. By balancing the sparsity prior TWT,1 and the TV prior ∇T1 using parameters λ1 and λ2, the proposed regularization constraints on the small target yield a sound target detection result. Therefore, the combination of the TV regularization and the weighted ℓ1 norm is a reasonable design for small target detection.


### 3.2. Solution of the Proposed Model

The minimization problem (7) with the equality and inequality constraints can be solved using the fast optimization algorithm. The difficulties in finding T are that TWT,1 is nonquadratic and nonseparable and that the ℓ1 norm term ∇T1 is nonsmooth and nonseparable. The nuclear norm BwB,∗ is nonquadratic and nonseparable. We can see from the above analysis that the objective function and constraints in equation (7) have different structures. Therefore, it is useful and necessary to split and solve them separately, which is exactly the forte of the alternating direction method of multipliers (ADMM) [36]. The idea is to convert the minimization problem on T,B in (7) into a constrained one by introducing three auxiliary variables Z1, Z2, and Z3, and four Lagrangian multiplier variables Y1, Y2, Y3, and Y4.In each iteration, the ADMM updates the splitting variables separately and alternatively by solving the partial augmented Lagrangian of (7). Problem (7) can be further converted into an unconstrained problem:(10)minT,BZ1wB,∗+λ1Z2WT,1+λ2Z31+Y1,Z1−B+Y2,Z2−T+Y3,Z3−∇T+Y4,D−T−B+β2Z1−BF2+Z2−TF2+Z3−∇TF2+D−T−BF2,where ⋅,⋅ denotes the inner product operation. β is the positive penalty parameter. To solve the above minimization problem about several variables, we utilize the alternating minimization method. Its idea is to solve only one subminimization problem while fixing other variables in each step of the iterative procedure.(1)
ForZ1 subproblem, it can be transformed to(11)minZ1Z1wB,∗+β2Z1−B+1βY1F2.The above problem is a nuclear norm minimization problem, which can be readily solved by singular value thresholding [37], so the solution of Z1 can be written as(12)Z1=SVTwB/βB−1βY1,whereSVTμ⋅ is the singular thresholding operator.(13)SVTμY=Udiagσ−μ+VT,σ−μ+=σ−μ,σ>μ,0,otherwise.Here,X=UΣVT is the singular value decomposition of matrix X, and Σ=diagσi1≤i≤r are the first r largest singular values of matrix X.(2)
ForZ2 subproblem, it can be translated to(14)minZ2λ1Z2WT,1+β2Z2−T+1βY2F2.The optimization problem (14) is convex and can be solved by using the well-known soft thresholding operator:(15)Z2=softThλ1WT/βT−1βY2,where(16)softThεX=signXmax0,X−ε,where⋅ denotes the absolute value, and all operations are implemented by componentwise. For t∈ℝ, the signum function signt is defined as(17)signt=+1,t>0,0,t=0,−1,t<0.For matrixX, we let signX be defined by componentwise.(3)
ForZ3 subproblem, it can be translated to(18)minZ3λ2Z31+β2Z3−∇T+1βY3F2.Optimization problem (18) can also be solved by using the soft thresholding operator. Because ∇T=∇xX,∇yXT, equation (18) can be solved by two equations:(19)Z3x=softThλ2/β∇xT−1βY3,(20)Z3y=softThλ2/β∇yT−1βY3,where∇x and ∇y are the horizontal and vertical first-order forward finite difference operators, respectively.(4)
ForB subproblem, it can be translated to(21)minBZ1−B+1βY1F2+D−T−B+1βY4F2.It is a quadratic programming problem and has a analytical solution:(22)B=12Z1+D−T+1βY1+Y4.(5)
ForT subproblem, it can be translated to(23)minTZ2−T+1βY2F2+∇T−Z3+1βY3F2+T−D−B+1βY4F2.Optimization problem (23) is also a quadratic programming problem. The equivalent liner system of the model (23) can be expressed as(24)2I+∇T∇T=Z2+∇TZ3+1βY2+∇TY3+Y4+D−B,where the superscriptT denotes the transpose operation.Problem (23) has a closed solution and can be solved readily using the fast Fourier transform (FFT):(25)T=ℱ−1ℱβζ1+∇Tζ2+ζ32β+βℱ∇T∇,whereℱ−1 and ℱ denote the inverse FFT and FFT operators, respectively, and ζ1=Z2+D+B, ζ2=βZ3+Y3, and ζ3=Y2+Y4.(6)
Finally, we update the Lagrangian multiplier variablesY1, Y2, Y3x, Y3y, and Y4 in the following manner:(26)Y1=Y1+βZ1−B,Y2=Y2+βZ2−T,Y3x=Y3x+βZ3x−∇xT,Y3y=Y3y+βZ3y−∇yT,Y4=Y4+βD−T−B.Taking into account the above equations, we obtain the complete iteration used in our algorithm for the target and background separation, then the optimization algorithm is shown in Algorithm1.Algorithm 1:Target and background separation solver via ADMM.
(1)
Input: infrared patch-image matrix D∈ℝm×n, λ1, λ2, C, β.
(2)
Output: infrared patch-image matrix B,T.
(3)
Initialization: B0=zerosm,n,T0=zerosm,n,Y10=Y2=Y3x0=Y3y0=Y40=zerosm,n,βmax=107,k=0.
(4)
while not converged do do
(5)
% Updating procedure for Z1k+1
(6)
Z
1
k
+
1
=
SVT
w
B
k
/
β
k
B
k
−
1
/
β
k
Y
1
k;
(7)
% Updating procedure for Z2k+1
(8)
Z
2
k
+
1
=
softTh
λ
1
w
T
k
/
β
k
T
k
−
1
/
β
k
Y
2
k;
(9)
% Updating procedure for Z3k+1
(10)
Z
3
x
k
+
1
=
softTh
λ
2
/
β
k
∇
x
T
k
−
1
/
β
k
Y
3
k; Z3yk+1=softThλ2/βk∇yTk−1/βkY3k;
(11)
% Updating procedure for Bk+1
(12)
B
k
+
1
=
1
/
2
Z
1
k
+
1
+
D
−
T
k
+
1
/
β
k
Y
1
k
+
Y
4
k;
(13)
% Updating procedure for Tk+1
(14)
T
k
+
1
=
ℱ
−
1
ℱ
β
ζ
1
k
+
1
+
∇
T
ζ
2
k
+
1
+
ζ
3
k
/
2
β
+
β
ℱ
∇
T
∇; ζ1k+1=Z2k+1+D+Bk+1, ζ2k+1=βZ3k+1+Y3k, ζ3k=Y2k+Y4k;
(15)
ComputewBk+1 and WTk+1 using (8) and (9), respectively;
(16)
% Update Lagrangian multipliers Y1k+1, Y2k+1, Y3xk+1, Y3yk+1, Y4k+1
(17)
Y
1
k
+
1
=
Y
1
k
+
β
k
Z
1
k
+
1
−
B
k
+
1; Y2k+1=Y2k+βkZ2k+1−Tk+1; Y3xk+1=Y3xk+βkZ3xk+1−∇xTk+1; Y3yk+1=Y3yk+βkZ3yk+1−∇yTk+1; Y4k+1=Y4k+βkD−Tk+1−Bk+1;
(18)
% Updating procedure for βk+1
(19)
β
k
+
1
=
min
ρ
∗
β
k
,
β
max;
(20)
% Check the convergence conditions
(21)
D
−
T
k
+
1
−
B
k
+
1
F
/
D
F
<
ε;
(22)
Updatek≔k+1.
(23)
end While


### 3.3. Detection Procedure

The diagram of the proposed detection approach is given in Figure1. The detailed detection procedure is described as follows:(1)
The patch-imageD∈ℝa2×t (we set m=a2,n=t) is constructed from the original image fD∈ℝM×N by sliding the window of size a×a from left to right and top to bottom with a fixed step size, where t is the number of window slips.(2)
The target patch-imageT and the background patch-image B are obtained from the patch-image D by applying Algorithm 1.(3)
The target imagefT and background image fB are reconstructed from the patch-image T and B, respectively.(4)
The final small target detection result is obtained from the target imagefT by the adaptive threshold tup, which is defined as(27)tup=μ+kσ,where µ and σ are the mean and standard deviation of the target image fT, respectively. k is constant determined experimentally. In our experiments, we empirically set the parameter ranging as 1,10. A pixel at x,y can be segmented as the target pixel if fTx,y>tup; otherwise, it is a background pixel.Figure 1
The various steps of the proposed algorithm.


## 3.1. Proposed Model

The small target detection based on the IPI model in (6) does work well in suppressing the clutters, noise, and nontarget points only using the ℓ1 norm constraint for the target image. In order to better separate the small target from the constructed observation image, the TV regularization term and the adaptive weighted ℓ1 norm on the target image are introduced into equation (6). The proposed model can be written as(7)minT,BBwB,∗+λ1TWT,1+λ2∇T1s.t.D−T−BF≤δ,D=T+B+N,where λ1 and λ2 are compromising parameters. The vector wB and the matrix WT are the weights for the background B and the target T, respectively. The weight wB is incorporated into the nuclear norm term for better preserving the strong edges of the background image [24], which is defined as(8)wB,j=1σB,j+εB,where σB,j is the jth singular value of B and wB=wB,1,wB,2,…,wB,nT is a weight for σB,j. εB is a positive value. BwB,∗=∑jwB,jσB,j is the reweighted nuclear norm of B.The weightWT is employed to suppress the nontarget points while enhancing the small target, and each element of WT is defined as(9)WT,ij=CTij+εT,where Tij is the pixel value of position i,j in the target image T. C is a compromising constant and εT is a small positive constant to avoid division by zero. The advantage of the weighted ℓ1 norm in equation (7) is to give every single element a different weight, thus better depicting the small target component. Because the small targets are a little brighter than other nontarget parts, this allows the small target to be better separated as the iteration proceeds. In equation (7), the TV regularization has good performance in removing the background clutters and the noise. The two main interferences can be effectively suppressed, and thus, the small target can be further highlighted. By balancing the sparsity prior TWT,1 and the TV prior ∇T1 using parameters λ1 and λ2, the proposed regularization constraints on the small target yield a sound target detection result. Therefore, the combination of the TV regularization and the weighted ℓ1 norm is a reasonable design for small target detection.


## 3.2. Solution of the Proposed Model

The minimization problem (7) with the equality and inequality constraints can be solved using the fast optimization algorithm. The difficulties in finding T are that TWT,1 is nonquadratic and nonseparable and that the ℓ1 norm term ∇T1 is nonsmooth and nonseparable. The nuclear norm BwB,∗ is nonquadratic and nonseparable. We can see from the above analysis that the objective function and constraints in equation (7) have different structures. Therefore, it is useful and necessary to split and solve them separately, which is exactly the forte of the alternating direction method of multipliers (ADMM) [36]. The idea is to convert the minimization problem on T,B in (7) into a constrained one by introducing three auxiliary variables Z1, Z2, and Z3, and four Lagrangian multiplier variables Y1, Y2, Y3, and Y4.In each iteration, the ADMM updates the splitting variables separately and alternatively by solving the partial augmented Lagrangian of (7). Problem (7) can be further converted into an unconstrained problem:(10)minT,BZ1wB,∗+λ1Z2WT,1+λ2Z31+Y1,Z1−B+Y2,Z2−T+Y3,Z3−∇T+Y4,D−T−B+β2Z1−BF2+Z2−TF2+Z3−∇TF2+D−T−BF2,where ⋅,⋅ denotes the inner product operation. β is the positive penalty parameter. To solve the above minimization problem about several variables, we utilize the alternating minimization method. Its idea is to solve only one subminimization problem while fixing other variables in each step of the iterative procedure.(1)
ForZ1 subproblem, it can be transformed to(11)minZ1Z1wB,∗+β2Z1−B+1βY1F2.The above problem is a nuclear norm minimization problem, which can be readily solved by singular value thresholding [37], so the solution of Z1 can be written as(12)Z1=SVTwB/βB−1βY1,whereSVTμ⋅ is the singular thresholding operator.(13)SVTμY=Udiagσ−μ+VT,σ−μ+=σ−μ,σ>μ,0,otherwise.Here,X=UΣVT is the singular value decomposition of matrix X, and Σ=diagσi1≤i≤r are the first r largest singular values of matrix X.(2)
ForZ2 subproblem, it can be translated to(14)minZ2λ1Z2WT,1+β2Z2−T+1βY2F2.The optimization problem (14) is convex and can be solved by using the well-known soft thresholding operator:(15)Z2=softThλ1WT/βT−1βY2,where(16)softThεX=signXmax0,X−ε,where⋅ denotes the absolute value, and all operations are implemented by componentwise. For t∈ℝ, the signum function signt is defined as(17)signt=+1,t>0,0,t=0,−1,t<0.For matrixX, we let signX be defined by componentwise.(3)
ForZ3 subproblem, it can be translated to(18)minZ3λ2Z31+β2Z3−∇T+1βY3F2.Optimization problem (18) can also be solved by using the soft thresholding operator. Because ∇T=∇xX,∇yXT, equation (18) can be solved by two equations:(19)Z3x=softThλ2/β∇xT−1βY3,(20)Z3y=softThλ2/β∇yT−1βY3,where∇x and ∇y are the horizontal and vertical first-order forward finite difference operators, respectively.(4)
ForB subproblem, it can be translated to(21)minBZ1−B+1βY1F2+D−T−B+1βY4F2.It is a quadratic programming problem and has a analytical solution:(22)B=12Z1+D−T+1βY1+Y4.(5)
ForT subproblem, it can be translated to(23)minTZ2−T+1βY2F2+∇T−Z3+1βY3F2+T−D−B+1βY4F2.Optimization problem (23) is also a quadratic programming problem. The equivalent liner system of the model (23) can be expressed as(24)2I+∇T∇T=Z2+∇TZ3+1βY2+∇TY3+Y4+D−B,where the superscriptT denotes the transpose operation.Problem (23) has a closed solution and can be solved readily using the fast Fourier transform (FFT):(25)T=ℱ−1ℱβζ1+∇Tζ2+ζ32β+βℱ∇T∇,whereℱ−1 and ℱ denote the inverse FFT and FFT operators, respectively, and ζ1=Z2+D+B, ζ2=βZ3+Y3, and ζ3=Y2+Y4.(6)
Finally, we update the Lagrangian multiplier variablesY1, Y2, Y3x, Y3y, and Y4 in the following manner:(26)Y1=Y1+βZ1−B,Y2=Y2+βZ2−T,Y3x=Y3x+βZ3x−∇xT,Y3y=Y3y+βZ3y−∇yT,Y4=Y4+βD−T−B.Taking into account the above equations, we obtain the complete iteration used in our algorithm for the target and background separation, then the optimization algorithm is shown in Algorithm1.Algorithm 1:Target and background separation solver via ADMM.
(1)
Input: infrared patch-image matrix D∈ℝm×n, λ1, λ2, C, β.
(2)
Output: infrared patch-image matrix B,T.
(3)
Initialization: B0=zerosm,n,T0=zerosm,n,Y10=Y2=Y3x0=Y3y0=Y40=zerosm,n,βmax=107,k=0.
(4)
while not converged do do
(5)
% Updating procedure for Z1k+1
(6)
Z
1
k
+
1
=
SVT
w
B
k
/
β
k
B
k
−
1
/
β
k
Y
1
k;
(7)
% Updating procedure for Z2k+1
(8)
Z
2
k
+
1
=
softTh
λ
1
w
T
k
/
β
k
T
k
−
1
/
β
k
Y
2
k;
(9)
% Updating procedure for Z3k+1
(10)
Z
3
x
k
+
1
=
softTh
λ
2
/
β
k
∇
x
T
k
−
1
/
β
k
Y
3
k; Z3yk+1=softThλ2/βk∇yTk−1/βkY3k;
(11)
% Updating procedure for Bk+1
(12)
B
k
+
1
=
1
/
2
Z
1
k
+
1
+
D
−
T
k
+
1
/
β
k
Y
1
k
+
Y
4
k;
(13)
% Updating procedure for Tk+1
(14)
T
k
+
1
=
ℱ
−
1
ℱ
β
ζ
1
k
+
1
+
∇
T
ζ
2
k
+
1
+
ζ
3
k
/
2
β
+
β
ℱ
∇
T
∇; ζ1k+1=Z2k+1+D+Bk+1, ζ2k+1=βZ3k+1+Y3k, ζ3k=Y2k+Y4k;
(15)
ComputewBk+1 and WTk+1 using (8) and (9), respectively;
(16)
% Update Lagrangian multipliers Y1k+1, Y2k+1, Y3xk+1, Y3yk+1, Y4k+1
(17)
Y
1
k
+
1
=
Y
1
k
+
β
k
Z
1
k
+
1
−
B
k
+
1; Y2k+1=Y2k+βkZ2k+1−Tk+1; Y3xk+1=Y3xk+βkZ3xk+1−∇xTk+1; Y3yk+1=Y3yk+βkZ3yk+1−∇yTk+1; Y4k+1=Y4k+βkD−Tk+1−Bk+1;
(18)
% Updating procedure for βk+1
(19)
β
k
+
1
=
min
ρ
∗
β
k
,
β
max;
(20)
% Check the convergence conditions
(21)
D
−
T
k
+
1
−
B
k
+
1
F
/
D
F
<
ε;
(22)
Updatek≔k+1.
(23)
end While


## 3.3. Detection Procedure

The diagram of the proposed detection approach is given in Figure1. The detailed detection procedure is described as follows:(1)
The patch-imageD∈ℝa2×t (we set m=a2,n=t) is constructed from the original image fD∈ℝM×N by sliding the window of size a×a from left to right and top to bottom with a fixed step size, where t is the number of window slips.(2)
The target patch-imageT and the background patch-image B are obtained from the patch-image D by applying Algorithm 1.(3)
The target imagefT and background image fB are reconstructed from the patch-image T and B, respectively.(4)
The final small target detection result is obtained from the target imagefT by the adaptive threshold tup, which is defined as(27)tup=μ+kσ,where µ and σ are the mean and standard deviation of the target image fT, respectively. k is constant determined experimentally. In our experiments, we empirically set the parameter ranging as 1,10. A pixel at x,y can be segmented as the target pixel if fTx,y>tup; otherwise, it is a background pixel.Figure 1
The various steps of the proposed algorithm.


## 4. Experimental Results and Analysis

In this section, we perform the experiments on simulated and real infrared images to validate the performance of the proposed method. We firstly introduce the experimental setup, including the dataset, the baseline methods, and the evaluation metric indexes. Secondly, we analyze how the key parameters influence the performance of the proposed method. Then, we compare the proposed method with the baseline methods. Finally, an analysis is made of the computational complexity of the proposed method.


### 4.1. Experimental Settings

#### 4.1.1. Datasets

(1) Real Datasets. Firstly, we exploit six real infrared images, as shown in Figure 2, to validate the background suppression and target detection ability of the proposed method. The backgrounds in these images are varying from uniform to complex. The size of the targets is small, and some of the targets are also very dim. The specific description of the selected images is shown in Table 2.Figure 2
Six real infrared images.
(a)
(b)
(c)
(d)
(e)
(f)Table 2
The specific description of six real images.
Image
Target type
Image
Target characteristic
Background characteristic
Image (a)
A helicopter
200× 150
Very small
Uniform backgrounds to heavy noise
Image (b)
A ship
280× 228
Low SCR value, small and dim buried in background
Blurred sea-sky background and almost unchanged
Image (c)
An airplane
250× 200
Low SCR value
Heavy cloudy-sky background clutters
Image (d)
A helicopter
281× 240
A small size, low SCR value
Heavy cloudy clutters
Image (e)
A ship
220× 140
A small size, low SCR value
Heavy sea clutters background almost unchanged
Image (f)
A ship
320× 196
Low SCR value, close to the sea-sky-line
Heavy sea clutters of sea-sky-line(2) Simulated Datasets. The synthesized images fD are obtained by embedding four simulated targets fT in the background image fB. The synthesized image can be generated by(28)fDx,y=maxrfTx−x0,y−y0,fBx,y,x∈1+x0,n+x0,y∈1+y0,m+y0,fBx,y,otherwise.The simulated targetfT is obtained by resizing the target with size of M×N by using the bicubic interpolation. x0,y0 is the left upper corner coordinates of the target embedding, which is randomly generated. r is a randomly generated value from h,255, where h is the maximum pixel value of the background image. Finally, a Gaussian filter is used to blur the synthesized image. We generate 100 frames of simulation sequences for each of the six background images. In each frame, four simulated small targets are randomly embedded. Then, we select the representative image of each sequence as shown in Figure 3.Figure 3
Synthesized multiple target images of six sequences.
(a)
(b)
(c)
(d)
(e)
(f)


#### 4.1.2. Evaluation Metrics

To qualitatively evaluate the performance of the proposed method, we introduce several numerical metrics. Next, we define these metrics.The signal-to-clutter ration (SCR) indicates the difficulty of target detection in a local area, and the computation formula can be written as(29)SCR=μt−μbσb,where μt and μb represent the average pixel value of the target region and the neighboring area around the target, respectively. σb is the standard deviation of the pixel value in the neighboring area around the target. As shown in Figure 4, we assume the size of the small target is a×b, and then the size of its background region is a+2d×b+2d, where d is the pixel width of neighboring area. In our experiment, we set d=20 pixels.Figure 4
The external rectangle and neighboring background rectangle of a small target.The signal-to-clutter ratio gain (SCRG) is a widely used numerical metric to evaluate the performance of small target detection, which can be defined as the ratio of SCR in the original image to that in the processed image:(30)SCRG=SCRoutSCRin.The background suppression factor (BSF) can also be used to describe how difficult the small target detection is. It can be defined as follows:(31)BSF=σinσout,where σin and σout denote the standard deviations of pixel value in the original and processed images except the target region, respectively. In general, the higher the SCR and BSF values of a small target image are, the easier it is for the target to be detected.Since the calculation of SCRG and BSF involves the standard deviation of the background area, both indexes can obtain a zero value if the background is too clean. To handle this problem, we adopt the contrast gain (CG) to assess the ability of clutters and noise suppression and small target enhancement. CG is defined as follows:(32)CG=CONoutCONin.In the above equation,CONin and CONout denote the contrasts (CON) of the original and processed images, respectively, and CONs can be calculated as follows:(33)CON=μt−μb,where μt and μb are the same as those in equation (29).The detection probability (Pd) and false alarm rate Fa are the two frequently used metrics for evaluating the detection performance, which are defined as follows:(34)Pd=#true targets detected#total bmtrue targets,Fa=#false pixels detected#total pixels in images.If a method has a good detection performance, then it has high (Pd) and low Fa values. The detected result is correct if it simultaneously meets two conditions: (i) the detected result and a ground truth have overlap pixels; (ii) the pixel distance between centers of the detection result and the ground truth is less than a threshold (3 pixels in our research). The receiver operating characteristic (ROC) curve shows the dynamic relationship between the detection probability (Pd) and false alarm rate Fa quantitatively.


#### 4.1.3. Baseline Methods and Parameter Setting

We compare the proposed method with six baseline methods: Top-hat [7], Max-mean [5], Max-median [5], IPI [22], TV-PCP [23], and MLMC [16]. Table 3 presents the specific parameter setting for each method.Table 3
Detailed parameter setting for the test methods.
Methods
Acronyms
Parameter setting
Top-hat method
Top-hat
Structure size: 12× 12
Max-mean filter
Max-mean
Filter size: 15× 15
Max-median filter
Max-median
Filter size: 15× 15
Infrared patch-image model
IPI
Patch size: 50× 50, sliding step: 10, λ=1/maxm,n, ε=10−6
Total variation regularization and principal component pursuit
TV-PCP
Patch size:50×50, sliding step: 14 maxIter = 250, ε=5×10−6
MLMC
Scale numberK = 4
Ours
Ours
Patch size:50×50, sliding step: 10, ε=10−7, λ1=1/maxm,n, λ2=0.006, maxIter = 500The proposed model has several parameters which influence the performance of the proposed method. The regularization parametersλ1 and λ2 control the trade-off among three components in the proposed model (7). Specifically, the two parameters control the sensitivity level of the model to the nuclear norm, reweighted ℓ1 norm, and TV term. We set the parameter λ1 to be 1/maxm,n the same as in PCP problem [38]. For λ2, a large value of λ2 means that the influence of TV regularization becomes more significant and that the clutters and noise can be suppressed remarkably. A small value of λ2 may cause more clutters and noise residual in the target image. In our implementation, we empirically set the parameter range as λ2∈1/maxm,n,1.5/maxm,n. We find that the proposed method can work very well in such a range. β is a penalization parameter and is updated by βk+1=ρβk, where k is the number of iterations. ρ=1.25 and the initial β0=1.25/D, where D denotes the spectral norm of matrix D. The stopping criterion of the proposed method for the iteration is “normalized step difference energy” (NSDE): NSDE=D−Tk+1−Bk+1F/DF<ε for each iteration, where ε=10−7. We set the maximum number of iterations to maxIter = 500.Both the proposed method and the methods for comparison are implemented using MATLAB 2018a running on a computer server with an Intel Xeon E5-2620 2.10 GHz CPU, 142 GB of RAM, and Linux system.


### 4.2. Parameter Analysis

The performance of the proposed method is influenced by several important parameters, such as the patch size, sliding step, and trade-off parameterC. In this subsection, we analyze the influence of these parameters on the performance of our method and discuss the appropriate selection of these parameters. The ROC curves corresponding to the patch size and sliding step on sequences 1–6 are shown in Figures 5 and 6, respectively.Figure 5
ROC curves for sequences 1–6 with respect to different patch sizes: (a) sequence 1; (b) sequence 2; (c) sequence 3; (d) sequence 4; (e) sequence 5; (f) sequence 6.
(a)
(b)
(c)
(d)
(e)
(f)Figure 6
ROC curves for sequences 1–6 with respect to different sliding steps: (a) sequence 1; (b) sequence 2; (c) sequence 3; (d) sequence 4; (e) sequence 5; (f) sequence 6.
(a)
(b)
(c)
(d)
(e)
(f)


#### 4.2.1. Patch Size

Patch size is a very important parameter in our experiments. A small patch size usually increases the correlation in a local area and prevents the background and target parts from being well separated from the original image, thus a lower detection probability. A large patch size reduces the correlation between the nonlocal areas and causes a large background residue in the target, thus a larger false alarm rate. Large patch sizes also increase the computational complexity. In order to choose the appropriate patch size, we fix the sliding step as 10 and vary the patch size from 20 to 80 with an interval of 10. And then our method is tested in six simulated image sequences and the ROC curves are shown in Figure5. We can observe from the results that the proposed method has a similar performance when the patch size is 50, 60, and 70. From Figures 5(a), 5(b), 5(d), and 5(f), we can see that a small patch size yields a low detection probability. In Figures 5(c) and 5(e), it is observed that a large patch size results in a high false alarm rate. In order to balance the detection probability and false alarm rate, we choose the patch size as 50.


#### 4.2.2. Sliding Step

Sliding step is also a parameter that affects the size of the patch-image matrix. A large sliding step reduces the correlation between the nonlocal areas, leaving a large background component in the target, which causes higher false alarm rates. In order to get a suitable sliding step, we fix the patch size as 50× 50 and vary the patch size from 2 to 16 with an interval of 2 and present the ROC curves in Figure 6. In Figures 6(b) and 6(f), it can be seen that a large sliding step brings about a high false alarm rate. In order to balance the detection probability and false alarm rate, we choose the sliding step of 10.


#### 4.2.3. Trade-Off Parameter

C is a trade-off parameter introduced in the adaptive weight factor. A large value of C tends to yield better background suppression effect. A small value of C results in more background and clutters and noise residual in the small target image. For a better balance between the preservation of the small target pixels and the suppression of the background clutters and noise, the parameter C needs to be chosen carefully. Empirically, a reasonable C for an infrared image typically lies in the range [0.01, 0.3]. Figure 7 shows the processed results by using different values of C and we chose 0.1 as the final C value.Figure 7
The detection results using different choicesC: (a) C=0.001; (b) C=0.01; (c) C=0.1; (d) C=0.3; (e) C=0.5.
(a)
(b)
(c)
(d)
(e)


### 4.3. Comparison with the Baseline Methods

Table4 presents three index values of seven methods for six real images. As shown in Table 4, our method consistently achieves the highest values for the three evaluating metrics as compared with other five methods. For each measure of SCR, BSF, and CG, a higher score implies better performance. High SCR, BSF, and CG values mean that the background clutters and noise in the processed images are suppressed better, thus easier detection of the small targets. In this table, it is shown that the value of the metric SCRG approaches infinity in some cases. It happens when the computation of the standard deviation in equation (30) is performed based on a clean local background area.Table 4
SCRG, BSF, and CG comparisons of seven methods using six test images.
Methods
Evaluation indexes
Figure7(a)
Figure7(b)
Figure7(c)
Figure7(d)
Figure7(e)
Figure7(f)
Top-hat
SCRG
0.7293
2.8136
1.2570
1.2127
1.0951
1.2586
BSF
4.3850
12.4920
2.3549
2.5420
1.6498
1.5829
CG
0.9145
1.0303
0.6520
0.9672
1.0954
0.8683
Max-mean
SCRG
3.1266
8.7743
9.1730
2.3488
2.0967
4.5015
BSF
7.5985
5.5402
3.9490
4.3879
2.4770
2.4768
CG
1.5546
3.1316
2.5962
1.6843
1.1479
1.1179
Max-median
SCRG
2.9499
9.7824
7.6414
3.2046
2.1385
4.4410
BSF
9.0372
6.2069
4.2723
6.3041
2.7539
2.5966
CG
1.5748
2.5862
1.6389
1.8157
1.2600
1.2297
IPI
SCRG
6.2432
Inf
36.9539
Inf
12.4616
41.3657
BSF
44.0414
79.7198
14.9939
31.3549
32.6376
27.5367
CG
1.6242
4.4338
2.3021
1.1791
2.8150
1.7741
TV-PCP
SCRG
10.7821
Inf
61.4484
2270.5115
16.0146
48.1482
BSF
96.4571
64.2095
28.7416
57.4668
57.0914
30.2031
CG
1.4729
3.6194
2.0528
1.0971
2.5456
1.7436
MLMC
SCRG
4.0888
25.9482
17.5429
8.1452
2.0800
11.6160
BSF
97.8868
27.6753
10.6644
5.2934
12.3820
39.9455
CG
1.0410
1.7478
1.7112
1.2009
1.1074
0.5745
Ours
SCRG
15.3347
Inf
109.9528
Inf
284.3204
168.5884
BSF
133.3089
130.7386
31.8744
57.5505
61.5704
77.4739
CG
1.6911
5.2929
2.5061
1.2011
3.7795
1.9793Figures8 and 9 show the detected results of the seven methods for six real infrared images. We also give the close-up of the small target for each detected result. As shown in Figure 8 (a1)–(a8), the Top-hat method has the enhancement effect to some extent; however, the background is not suppressed well and thus this method suffers a high false alarm rate. It can be seen from Figure 8 (b1)–(b8) and Figure 8 (c1)–(c8) that the Max-mean and Max-median methods have a better ability to enhance the small target than the Top-hat method, but the background suppression for these two methods is not satisfactory. The background clutters and noise residuals are still obvious, and thus, it is still difficult for the dim targets to be detected. Figure 9 (d1)–(d8) shows that the IPI method achieves an obvious improvement in enhancing the small target and suppressing the background clutters and noise. But the background suppression is not enough: the target images still have a bit of clutters and noise residuals. As shown in Figure 9 (e1)–(e8), the TV-PCP method gets a good background suppression and target enhancement performance under the relatively weak background clutters. However, there is still room for improvement in the strong background clutters. As shown in Figure 9 (f7), the detection result obtained with the TV-PCP method still has the sea clutters and noise residual. From Figure 9 (f1)–(f8), it can be seen that compared with the IPI method and the TV-PCP method, the proposed method has better background and noise suppression ability when the target pixels are similar in the detected results of the three methods, and thus, the target can be easily detected in the subsequent segmentation. The reason may be that the proposed method introduces the total variation constraint to small target component and that the background clutters and noise in the small target part are suppressed better. Due to the introduction of the weighted ℓ1 norm, some nontarget points are also suppressed. In a word, the combination of the TV regularization and the weighted ℓ1 norm is an effective approach in suppressing the clutters and noise while enhancing the small target.Figure 8
The detected results of the seven methods for three real infrared images and the close-up of the small target for each detected result. (a1)-(c1) are the real infrared images corresponding to Figures2(a)–2(c). (a2)-(c2), (a3)-(c3), (a4)-(c4), (a5)-(c5), (a6)-(c6), (a7)-(c7), and (a8)-(c8) are the detected results obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.Figure 9
The detected results of the seven methods for three real infrared images and the close-up of the small target for each detected result. (d1)-(f1) are the real infrared images corresponding to Figures2(d)–2(f). (d2)-(f2), (d3)-(f3), (d4)-(f4), (d5)-(f5), (d6)-(f6), (d7)-(f7), and (d8)-(f8) are the detected results obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.In order to intuitively present the ability of the clutter suppression and target enhancement, the 3D gray maps of the detected results of the seven methods for six real infrared images are also given in Figures10 and 11. We use the blue-green circle to mark the obvious background clutters residual generated by other methods. We can see from Figure 10 (a1)-(c1) and Figure 11 (d1)–(f1) that there exist the background components whose gray levels are close to or higher than those of the targets, which means that it is very challenging to detect dim and weak target. The detection effect with the Top-hat, Max-mean, and Max-median methods as shown in Figures 10 and 11 is consistent with those in Figures 8 and 9. The IPI and TV-PCP methods present a relatively good ability in background suppression and target enhancement in most cases, but in the detection results of both methods many clutters and noise residuals still exist in the scene with strong clutters and noise. In general, the background clutters and random noise are the main obstructions to the separation of target and background. From the Figure 10 (a7), (b7), and (c7) and Figure 11 (d7), (e7), and (f7), we can observe that the MLMC method still exists many clutters and noise residuals. The detected results produced by our method in Figures 10 and 11 display a flatter background and more prominent target than those by other methods, which implies that our method is capable of removing the interferences with a consistent robustness in background suppression and target enhancement in different scenes by combining the TV regularization and reweighted ℓ1 norm sparse constraint.Figure 10
3D gray maps of the detected results of seven methods for three real infrared images. (a1)-(c1) are the 3D gray maps of real infrared images corresponding to Figures7(a)–7(c). (a2)-(c2), (a3)-(c3), (a4)-(c4), (a5)-(c5), (a6)-(c6), (a7)-(c7), and (a8)-(c8) are the 3D gray maps obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.Figure 11
3D gray maps of the detected results of seven methods for three real infrared images. (d1)-(f1) are the 3D gray maps of real infrared images corresponding to Figures7(d)–7(f). (d2)-(f2), (d3)-(f3), (d4)-(f4), (d5)-(f5), (d6)-(f6), (d7)-(f7), and (e8)-(f8) are the 3D gray maps obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.Figure 12
ROC curves of the detection results of six simulated sequences: (a) sequence 1; (b) sequence 2; (c) sequence 3; (d) sequence 4; (e) sequence 5; (f) sequence 6.
(a)
(b)
(c)
(d)
(e)
(f)To further verify the performance of our method, we make a comparison of the ROC curves produced by six methods for the six simulated sequences in Figure12. It is found that the Top-hat method is not satisfactory, for it has a low detection probability. Both the Max-mean and Max-median methods possess higher detection probability than the Top-hat method for the sequences except sequence 2. The IPI and TV-PCP methods have better performance than the above three baseline methods for all sequences. In general, the proposed method consistently achieves the highest detection probability in almost all cases, implying that the proposed method has a better detection performance than the baseline methods.


### 4.4. Convergence Rate of the Algorithm

We investigated the convergence behavior of the proposed algorithm. Figure13 illustrates the evolutional curve of the objective function and NSDE versus the iterations for the real infrared image object detection in Figure 7(a). It can be observed clearly that the proposed method achieves good convergence and converges after about 20 iterations.Figure 13
Evolution of the objective function and NSDE versus number of iterations for the proposed method.
(a)
(b)


### 4.5. Computational Complexity and Computation Time of the Algorithm

In this subsection, we compare the computational costs of seven methods. The computational complexity and computation time corresponding to the six images in Figure7 by the seven methods are shown in Table 5. Suppose the size of the original image is M×N. The size of the patch-image constructed from the original image is m×n (for simplicity, we assume m>n). The computational complexity of the Top-hat method is OMNL2logL2, where L2 is the size of the structure element. For the Max-median method, the complexity is dominated by the median filtering, its computational cost being OL2logL2. Considering the image size, the final cost of Max-median is OMNL2logL2. As for the Max-mean method, the total cost is OMNL2. For the TV-PCP method, the total cost of each iteration is Omn2 operations. The proposed model contains an optimization problem: the objective function includes the nuclear norm, ℓ1 norm, and total variation norm and is solved by ADMM. For the proposed method, the computational complexities of updating the variables Z1, Z2, Z3x, Z3y, B, and T in (12), (15), (19), (20), (22), and (24) for each iteration are Omn2, Omn, Omn, Omn, Omn, and Omn, respectively. Therefore, the total cost of each iteration for the proposed method is Omn2 operations.Table 5
Comparison of computational complexity and computation time (in seconds) of seven methods.
Method
Complexity
Figure7(a)
Figure7(b)
Figure7(c)
Figure7(d)
Figure7(e)
Figure7(f)
Top-hat
O
M
N
L
2
log
L
2
0.273
0.280
0.262
0.275
0.264
0.239
Max-mean
O
M
N
L
2
1.290
2.211
2.186
2.273
1.319
11.553
Max-median
O
M
N
L
2
log
L
2
1.251
2.408
1.934
2.713
1.264
11.948
IPI
O
m
n
2
1.548
5.825
4.138
5.311
1.673
22.848
TV-PCP
O
m
n
2
15.154
40.589
31.039
45.586
12.617
201.407
MLMC
O
M
N
L
3
0.034
0.028
0.026
0.029
0.025
0.028
Ours
O
m
n
2
6.319
23.242
11.695
18.743
6.498
88.440


## 4.1. Experimental Settings

### 4.1.1. Datasets

(1) Real Datasets. Firstly, we exploit six real infrared images, as shown in Figure 2, to validate the background suppression and target detection ability of the proposed method. The backgrounds in these images are varying from uniform to complex. The size of the targets is small, and some of the targets are also very dim. The specific description of the selected images is shown in Table 2.Figure 2
Six real infrared images.
(a)
(b)
(c)
(d)
(e)
(f)Table 2
The specific description of six real images.
Image
Target type
Image
Target characteristic
Background characteristic
Image (a)
A helicopter
200× 150
Very small
Uniform backgrounds to heavy noise
Image (b)
A ship
280× 228
Low SCR value, small and dim buried in background
Blurred sea-sky background and almost unchanged
Image (c)
An airplane
250× 200
Low SCR value
Heavy cloudy-sky background clutters
Image (d)
A helicopter
281× 240
A small size, low SCR value
Heavy cloudy clutters
Image (e)
A ship
220× 140
A small size, low SCR value
Heavy sea clutters background almost unchanged
Image (f)
A ship
320× 196
Low SCR value, close to the sea-sky-line
Heavy sea clutters of sea-sky-line(2) Simulated Datasets. The synthesized images fD are obtained by embedding four simulated targets fT in the background image fB. The synthesized image can be generated by(28)fDx,y=maxrfTx−x0,y−y0,fBx,y,x∈1+x0,n+x0,y∈1+y0,m+y0,fBx,y,otherwise.The simulated targetfT is obtained by resizing the target with size of M×N by using the bicubic interpolation. x0,y0 is the left upper corner coordinates of the target embedding, which is randomly generated. r is a randomly generated value from h,255, where h is the maximum pixel value of the background image. Finally, a Gaussian filter is used to blur the synthesized image. We generate 100 frames of simulation sequences for each of the six background images. In each frame, four simulated small targets are randomly embedded. Then, we select the representative image of each sequence as shown in Figure 3.Figure 3
Synthesized multiple target images of six sequences.
(a)
(b)
(c)
(d)
(e)
(f)


### 4.1.2. Evaluation Metrics

To qualitatively evaluate the performance of the proposed method, we introduce several numerical metrics. Next, we define these metrics.The signal-to-clutter ration (SCR) indicates the difficulty of target detection in a local area, and the computation formula can be written as(29)SCR=μt−μbσb,where μt and μb represent the average pixel value of the target region and the neighboring area around the target, respectively. σb is the standard deviation of the pixel value in the neighboring area around the target. As shown in Figure 4, we assume the size of the small target is a×b, and then the size of its background region is a+2d×b+2d, where d is the pixel width of neighboring area. In our experiment, we set d=20 pixels.Figure 4
The external rectangle and neighboring background rectangle of a small target.The signal-to-clutter ratio gain (SCRG) is a widely used numerical metric to evaluate the performance of small target detection, which can be defined as the ratio of SCR in the original image to that in the processed image:(30)SCRG=SCRoutSCRin.The background suppression factor (BSF) can also be used to describe how difficult the small target detection is. It can be defined as follows:(31)BSF=σinσout,where σin and σout denote the standard deviations of pixel value in the original and processed images except the target region, respectively. In general, the higher the SCR and BSF values of a small target image are, the easier it is for the target to be detected.Since the calculation of SCRG and BSF involves the standard deviation of the background area, both indexes can obtain a zero value if the background is too clean. To handle this problem, we adopt the contrast gain (CG) to assess the ability of clutters and noise suppression and small target enhancement. CG is defined as follows:(32)CG=CONoutCONin.In the above equation,CONin and CONout denote the contrasts (CON) of the original and processed images, respectively, and CONs can be calculated as follows:(33)CON=μt−μb,where μt and μb are the same as those in equation (29).The detection probability (Pd) and false alarm rate Fa are the two frequently used metrics for evaluating the detection performance, which are defined as follows:(34)Pd=#true targets detected#total bmtrue targets,Fa=#false pixels detected#total pixels in images.If a method has a good detection performance, then it has high (Pd) and low Fa values. The detected result is correct if it simultaneously meets two conditions: (i) the detected result and a ground truth have overlap pixels; (ii) the pixel distance between centers of the detection result and the ground truth is less than a threshold (3 pixels in our research). The receiver operating characteristic (ROC) curve shows the dynamic relationship between the detection probability (Pd) and false alarm rate Fa quantitatively.


### 4.1.3. Baseline Methods and Parameter Setting

We compare the proposed method with six baseline methods: Top-hat [7], Max-mean [5], Max-median [5], IPI [22], TV-PCP [23], and MLMC [16]. Table 3 presents the specific parameter setting for each method.Table 3
Detailed parameter setting for the test methods.
Methods
Acronyms
Parameter setting
Top-hat method
Top-hat
Structure size: 12× 12
Max-mean filter
Max-mean
Filter size: 15× 15
Max-median filter
Max-median
Filter size: 15× 15
Infrared patch-image model
IPI
Patch size: 50× 50, sliding step: 10, λ=1/maxm,n, ε=10−6
Total variation regularization and principal component pursuit
TV-PCP
Patch size:50×50, sliding step: 14 maxIter = 250, ε=5×10−6
MLMC
Scale numberK = 4
Ours
Ours
Patch size:50×50, sliding step: 10, ε=10−7, λ1=1/maxm,n, λ2=0.006, maxIter = 500The proposed model has several parameters which influence the performance of the proposed method. The regularization parametersλ1 and λ2 control the trade-off among three components in the proposed model (7). Specifically, the two parameters control the sensitivity level of the model to the nuclear norm, reweighted ℓ1 norm, and TV term. We set the parameter λ1 to be 1/maxm,n the same as in PCP problem [38]. For λ2, a large value of λ2 means that the influence of TV regularization becomes more significant and that the clutters and noise can be suppressed remarkably. A small value of λ2 may cause more clutters and noise residual in the target image. In our implementation, we empirically set the parameter range as λ2∈1/maxm,n,1.5/maxm,n. We find that the proposed method can work very well in such a range. β is a penalization parameter and is updated by βk+1=ρβk, where k is the number of iterations. ρ=1.25 and the initial β0=1.25/D, where D denotes the spectral norm of matrix D. The stopping criterion of the proposed method for the iteration is “normalized step difference energy” (NSDE): NSDE=D−Tk+1−Bk+1F/DF<ε for each iteration, where ε=10−7. We set the maximum number of iterations to maxIter = 500.Both the proposed method and the methods for comparison are implemented using MATLAB 2018a running on a computer server with an Intel Xeon E5-2620 2.10 GHz CPU, 142 GB of RAM, and Linux system.


## 4.1.1. Datasets

(1) Real Datasets. Firstly, we exploit six real infrared images, as shown in Figure 2, to validate the background suppression and target detection ability of the proposed method. The backgrounds in these images are varying from uniform to complex. The size of the targets is small, and some of the targets are also very dim. The specific description of the selected images is shown in Table 2.Figure 2
Six real infrared images.
(a)
(b)
(c)
(d)
(e)
(f)Table 2
The specific description of six real images.
Image
Target type
Image
Target characteristic
Background characteristic
Image (a)
A helicopter
200× 150
Very small
Uniform backgrounds to heavy noise
Image (b)
A ship
280× 228
Low SCR value, small and dim buried in background
Blurred sea-sky background and almost unchanged
Image (c)
An airplane
250× 200
Low SCR value
Heavy cloudy-sky background clutters
Image (d)
A helicopter
281× 240
A small size, low SCR value
Heavy cloudy clutters
Image (e)
A ship
220× 140
A small size, low SCR value
Heavy sea clutters background almost unchanged
Image (f)
A ship
320× 196
Low SCR value, close to the sea-sky-line
Heavy sea clutters of sea-sky-line(2) Simulated Datasets. The synthesized images fD are obtained by embedding four simulated targets fT in the background image fB. The synthesized image can be generated by(28)fDx,y=maxrfTx−x0,y−y0,fBx,y,x∈1+x0,n+x0,y∈1+y0,m+y0,fBx,y,otherwise.The simulated targetfT is obtained by resizing the target with size of M×N by using the bicubic interpolation. x0,y0 is the left upper corner coordinates of the target embedding, which is randomly generated. r is a randomly generated value from h,255, where h is the maximum pixel value of the background image. Finally, a Gaussian filter is used to blur the synthesized image. We generate 100 frames of simulation sequences for each of the six background images. In each frame, four simulated small targets are randomly embedded. Then, we select the representative image of each sequence as shown in Figure 3.Figure 3
Synthesized multiple target images of six sequences.
(a)
(b)
(c)
(d)
(e)
(f)


## 4.1.2. Evaluation Metrics

To qualitatively evaluate the performance of the proposed method, we introduce several numerical metrics. Next, we define these metrics.The signal-to-clutter ration (SCR) indicates the difficulty of target detection in a local area, and the computation formula can be written as(29)SCR=μt−μbσb,where μt and μb represent the average pixel value of the target region and the neighboring area around the target, respectively. σb is the standard deviation of the pixel value in the neighboring area around the target. As shown in Figure 4, we assume the size of the small target is a×b, and then the size of its background region is a+2d×b+2d, where d is the pixel width of neighboring area. In our experiment, we set d=20 pixels.Figure 4
The external rectangle and neighboring background rectangle of a small target.The signal-to-clutter ratio gain (SCRG) is a widely used numerical metric to evaluate the performance of small target detection, which can be defined as the ratio of SCR in the original image to that in the processed image:(30)SCRG=SCRoutSCRin.The background suppression factor (BSF) can also be used to describe how difficult the small target detection is. It can be defined as follows:(31)BSF=σinσout,where σin and σout denote the standard deviations of pixel value in the original and processed images except the target region, respectively. In general, the higher the SCR and BSF values of a small target image are, the easier it is for the target to be detected.Since the calculation of SCRG and BSF involves the standard deviation of the background area, both indexes can obtain a zero value if the background is too clean. To handle this problem, we adopt the contrast gain (CG) to assess the ability of clutters and noise suppression and small target enhancement. CG is defined as follows:(32)CG=CONoutCONin.In the above equation,CONin and CONout denote the contrasts (CON) of the original and processed images, respectively, and CONs can be calculated as follows:(33)CON=μt−μb,where μt and μb are the same as those in equation (29).The detection probability (Pd) and false alarm rate Fa are the two frequently used metrics for evaluating the detection performance, which are defined as follows:(34)Pd=#true targets detected#total bmtrue targets,Fa=#false pixels detected#total pixels in images.If a method has a good detection performance, then it has high (Pd) and low Fa values. The detected result is correct if it simultaneously meets two conditions: (i) the detected result and a ground truth have overlap pixels; (ii) the pixel distance between centers of the detection result and the ground truth is less than a threshold (3 pixels in our research). The receiver operating characteristic (ROC) curve shows the dynamic relationship between the detection probability (Pd) and false alarm rate Fa quantitatively.


## 4.1.3. Baseline Methods and Parameter Setting

We compare the proposed method with six baseline methods: Top-hat [7], Max-mean [5], Max-median [5], IPI [22], TV-PCP [23], and MLMC [16]. Table 3 presents the specific parameter setting for each method.Table 3
Detailed parameter setting for the test methods.
Methods
Acronyms
Parameter setting
Top-hat method
Top-hat
Structure size: 12× 12
Max-mean filter
Max-mean
Filter size: 15× 15
Max-median filter
Max-median
Filter size: 15× 15
Infrared patch-image model
IPI
Patch size: 50× 50, sliding step: 10, λ=1/maxm,n, ε=10−6
Total variation regularization and principal component pursuit
TV-PCP
Patch size:50×50, sliding step: 14 maxIter = 250, ε=5×10−6
MLMC
Scale numberK = 4
Ours
Ours
Patch size:50×50, sliding step: 10, ε=10−7, λ1=1/maxm,n, λ2=0.006, maxIter = 500The proposed model has several parameters which influence the performance of the proposed method. The regularization parametersλ1 and λ2 control the trade-off among three components in the proposed model (7). Specifically, the two parameters control the sensitivity level of the model to the nuclear norm, reweighted ℓ1 norm, and TV term. We set the parameter λ1 to be 1/maxm,n the same as in PCP problem [38]. For λ2, a large value of λ2 means that the influence of TV regularization becomes more significant and that the clutters and noise can be suppressed remarkably. A small value of λ2 may cause more clutters and noise residual in the target image. In our implementation, we empirically set the parameter range as λ2∈1/maxm,n,1.5/maxm,n. We find that the proposed method can work very well in such a range. β is a penalization parameter and is updated by βk+1=ρβk, where k is the number of iterations. ρ=1.25 and the initial β0=1.25/D, where D denotes the spectral norm of matrix D. The stopping criterion of the proposed method for the iteration is “normalized step difference energy” (NSDE): NSDE=D−Tk+1−Bk+1F/DF<ε for each iteration, where ε=10−7. We set the maximum number of iterations to maxIter = 500.Both the proposed method and the methods for comparison are implemented using MATLAB 2018a running on a computer server with an Intel Xeon E5-2620 2.10 GHz CPU, 142 GB of RAM, and Linux system.


## 4.2. Parameter Analysis

The performance of the proposed method is influenced by several important parameters, such as the patch size, sliding step, and trade-off parameterC. In this subsection, we analyze the influence of these parameters on the performance of our method and discuss the appropriate selection of these parameters. The ROC curves corresponding to the patch size and sliding step on sequences 1–6 are shown in Figures 5 and 6, respectively.Figure 5
ROC curves for sequences 1–6 with respect to different patch sizes: (a) sequence 1; (b) sequence 2; (c) sequence 3; (d) sequence 4; (e) sequence 5; (f) sequence 6.
(a)
(b)
(c)
(d)
(e)
(f)Figure 6
ROC curves for sequences 1–6 with respect to different sliding steps: (a) sequence 1; (b) sequence 2; (c) sequence 3; (d) sequence 4; (e) sequence 5; (f) sequence 6.
(a)
(b)
(c)
(d)
(e)
(f)


### 4.2.1. Patch Size

Patch size is a very important parameter in our experiments. A small patch size usually increases the correlation in a local area and prevents the background and target parts from being well separated from the original image, thus a lower detection probability. A large patch size reduces the correlation between the nonlocal areas and causes a large background residue in the target, thus a larger false alarm rate. Large patch sizes also increase the computational complexity. In order to choose the appropriate patch size, we fix the sliding step as 10 and vary the patch size from 20 to 80 with an interval of 10. And then our method is tested in six simulated image sequences and the ROC curves are shown in Figure5. We can observe from the results that the proposed method has a similar performance when the patch size is 50, 60, and 70. From Figures 5(a), 5(b), 5(d), and 5(f), we can see that a small patch size yields a low detection probability. In Figures 5(c) and 5(e), it is observed that a large patch size results in a high false alarm rate. In order to balance the detection probability and false alarm rate, we choose the patch size as 50.


### 4.2.2. Sliding Step

Sliding step is also a parameter that affects the size of the patch-image matrix. A large sliding step reduces the correlation between the nonlocal areas, leaving a large background component in the target, which causes higher false alarm rates. In order to get a suitable sliding step, we fix the patch size as 50× 50 and vary the patch size from 2 to 16 with an interval of 2 and present the ROC curves in Figure 6. In Figures 6(b) and 6(f), it can be seen that a large sliding step brings about a high false alarm rate. In order to balance the detection probability and false alarm rate, we choose the sliding step of 10.


### 4.2.3. Trade-Off Parameter

C is a trade-off parameter introduced in the adaptive weight factor. A large value of C tends to yield better background suppression effect. A small value of C results in more background and clutters and noise residual in the small target image. For a better balance between the preservation of the small target pixels and the suppression of the background clutters and noise, the parameter C needs to be chosen carefully. Empirically, a reasonable C for an infrared image typically lies in the range [0.01, 0.3]. Figure 7 shows the processed results by using different values of C and we chose 0.1 as the final C value.Figure 7
The detection results using different choicesC: (a) C=0.001; (b) C=0.01; (c) C=0.1; (d) C=0.3; (e) C=0.5.
(a)
(b)
(c)
(d)
(e)


## 4.2.1. Patch Size

Patch size is a very important parameter in our experiments. A small patch size usually increases the correlation in a local area and prevents the background and target parts from being well separated from the original image, thus a lower detection probability. A large patch size reduces the correlation between the nonlocal areas and causes a large background residue in the target, thus a larger false alarm rate. Large patch sizes also increase the computational complexity. In order to choose the appropriate patch size, we fix the sliding step as 10 and vary the patch size from 20 to 80 with an interval of 10. And then our method is tested in six simulated image sequences and the ROC curves are shown in Figure5. We can observe from the results that the proposed method has a similar performance when the patch size is 50, 60, and 70. From Figures 5(a), 5(b), 5(d), and 5(f), we can see that a small patch size yields a low detection probability. In Figures 5(c) and 5(e), it is observed that a large patch size results in a high false alarm rate. In order to balance the detection probability and false alarm rate, we choose the patch size as 50.


## 4.2.2. Sliding Step

Sliding step is also a parameter that affects the size of the patch-image matrix. A large sliding step reduces the correlation between the nonlocal areas, leaving a large background component in the target, which causes higher false alarm rates. In order to get a suitable sliding step, we fix the patch size as 50× 50 and vary the patch size from 2 to 16 with an interval of 2 and present the ROC curves in Figure 6. In Figures 6(b) and 6(f), it can be seen that a large sliding step brings about a high false alarm rate. In order to balance the detection probability and false alarm rate, we choose the sliding step of 10.


## 4.2.3. Trade-Off Parameter

C is a trade-off parameter introduced in the adaptive weight factor. A large value of C tends to yield better background suppression effect. A small value of C results in more background and clutters and noise residual in the small target image. For a better balance between the preservation of the small target pixels and the suppression of the background clutters and noise, the parameter C needs to be chosen carefully. Empirically, a reasonable C for an infrared image typically lies in the range [0.01, 0.3]. Figure 7 shows the processed results by using different values of C and we chose 0.1 as the final C value.Figure 7
The detection results using different choicesC: (a) C=0.001; (b) C=0.01; (c) C=0.1; (d) C=0.3; (e) C=0.5.
(a)
(b)
(c)
(d)
(e)


## 4.3. Comparison with the Baseline Methods

Table4 presents three index values of seven methods for six real images. As shown in Table 4, our method consistently achieves the highest values for the three evaluating metrics as compared with other five methods. For each measure of SCR, BSF, and CG, a higher score implies better performance. High SCR, BSF, and CG values mean that the background clutters and noise in the processed images are suppressed better, thus easier detection of the small targets. In this table, it is shown that the value of the metric SCRG approaches infinity in some cases. It happens when the computation of the standard deviation in equation (30) is performed based on a clean local background area.Table 4
SCRG, BSF, and CG comparisons of seven methods using six test images.
Methods
Evaluation indexes
Figure7(a)
Figure7(b)
Figure7(c)
Figure7(d)
Figure7(e)
Figure7(f)
Top-hat
SCRG
0.7293
2.8136
1.2570
1.2127
1.0951
1.2586
BSF
4.3850
12.4920
2.3549
2.5420
1.6498
1.5829
CG
0.9145
1.0303
0.6520
0.9672
1.0954
0.8683
Max-mean
SCRG
3.1266
8.7743
9.1730
2.3488
2.0967
4.5015
BSF
7.5985
5.5402
3.9490
4.3879
2.4770
2.4768
CG
1.5546
3.1316
2.5962
1.6843
1.1479
1.1179
Max-median
SCRG
2.9499
9.7824
7.6414
3.2046
2.1385
4.4410
BSF
9.0372
6.2069
4.2723
6.3041
2.7539
2.5966
CG
1.5748
2.5862
1.6389
1.8157
1.2600
1.2297
IPI
SCRG
6.2432
Inf
36.9539
Inf
12.4616
41.3657
BSF
44.0414
79.7198
14.9939
31.3549
32.6376
27.5367
CG
1.6242
4.4338
2.3021
1.1791
2.8150
1.7741
TV-PCP
SCRG
10.7821
Inf
61.4484
2270.5115
16.0146
48.1482
BSF
96.4571
64.2095
28.7416
57.4668
57.0914
30.2031
CG
1.4729
3.6194
2.0528
1.0971
2.5456
1.7436
MLMC
SCRG
4.0888
25.9482
17.5429
8.1452
2.0800
11.6160
BSF
97.8868
27.6753
10.6644
5.2934
12.3820
39.9455
CG
1.0410
1.7478
1.7112
1.2009
1.1074
0.5745
Ours
SCRG
15.3347
Inf
109.9528
Inf
284.3204
168.5884
BSF
133.3089
130.7386
31.8744
57.5505
61.5704
77.4739
CG
1.6911
5.2929
2.5061
1.2011
3.7795
1.9793Figures8 and 9 show the detected results of the seven methods for six real infrared images. We also give the close-up of the small target for each detected result. As shown in Figure 8 (a1)–(a8), the Top-hat method has the enhancement effect to some extent; however, the background is not suppressed well and thus this method suffers a high false alarm rate. It can be seen from Figure 8 (b1)–(b8) and Figure 8 (c1)–(c8) that the Max-mean and Max-median methods have a better ability to enhance the small target than the Top-hat method, but the background suppression for these two methods is not satisfactory. The background clutters and noise residuals are still obvious, and thus, it is still difficult for the dim targets to be detected. Figure 9 (d1)–(d8) shows that the IPI method achieves an obvious improvement in enhancing the small target and suppressing the background clutters and noise. But the background suppression is not enough: the target images still have a bit of clutters and noise residuals. As shown in Figure 9 (e1)–(e8), the TV-PCP method gets a good background suppression and target enhancement performance under the relatively weak background clutters. However, there is still room for improvement in the strong background clutters. As shown in Figure 9 (f7), the detection result obtained with the TV-PCP method still has the sea clutters and noise residual. From Figure 9 (f1)–(f8), it can be seen that compared with the IPI method and the TV-PCP method, the proposed method has better background and noise suppression ability when the target pixels are similar in the detected results of the three methods, and thus, the target can be easily detected in the subsequent segmentation. The reason may be that the proposed method introduces the total variation constraint to small target component and that the background clutters and noise in the small target part are suppressed better. Due to the introduction of the weighted ℓ1 norm, some nontarget points are also suppressed. In a word, the combination of the TV regularization and the weighted ℓ1 norm is an effective approach in suppressing the clutters and noise while enhancing the small target.Figure 8
The detected results of the seven methods for three real infrared images and the close-up of the small target for each detected result. (a1)-(c1) are the real infrared images corresponding to Figures2(a)–2(c). (a2)-(c2), (a3)-(c3), (a4)-(c4), (a5)-(c5), (a6)-(c6), (a7)-(c7), and (a8)-(c8) are the detected results obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.Figure 9
The detected results of the seven methods for three real infrared images and the close-up of the small target for each detected result. (d1)-(f1) are the real infrared images corresponding to Figures2(d)–2(f). (d2)-(f2), (d3)-(f3), (d4)-(f4), (d5)-(f5), (d6)-(f6), (d7)-(f7), and (d8)-(f8) are the detected results obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.In order to intuitively present the ability of the clutter suppression and target enhancement, the 3D gray maps of the detected results of the seven methods for six real infrared images are also given in Figures10 and 11. We use the blue-green circle to mark the obvious background clutters residual generated by other methods. We can see from Figure 10 (a1)-(c1) and Figure 11 (d1)–(f1) that there exist the background components whose gray levels are close to or higher than those of the targets, which means that it is very challenging to detect dim and weak target. The detection effect with the Top-hat, Max-mean, and Max-median methods as shown in Figures 10 and 11 is consistent with those in Figures 8 and 9. The IPI and TV-PCP methods present a relatively good ability in background suppression and target enhancement in most cases, but in the detection results of both methods many clutters and noise residuals still exist in the scene with strong clutters and noise. In general, the background clutters and random noise are the main obstructions to the separation of target and background. From the Figure 10 (a7), (b7), and (c7) and Figure 11 (d7), (e7), and (f7), we can observe that the MLMC method still exists many clutters and noise residuals. The detected results produced by our method in Figures 10 and 11 display a flatter background and more prominent target than those by other methods, which implies that our method is capable of removing the interferences with a consistent robustness in background suppression and target enhancement in different scenes by combining the TV regularization and reweighted ℓ1 norm sparse constraint.Figure 10
3D gray maps of the detected results of seven methods for three real infrared images. (a1)-(c1) are the 3D gray maps of real infrared images corresponding to Figures7(a)–7(c). (a2)-(c2), (a3)-(c3), (a4)-(c4), (a5)-(c5), (a6)-(c6), (a7)-(c7), and (a8)-(c8) are the 3D gray maps obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.Figure 11
3D gray maps of the detected results of seven methods for three real infrared images. (d1)-(f1) are the 3D gray maps of real infrared images corresponding to Figures7(d)–7(f). (d2)-(f2), (d3)-(f3), (d4)-(f4), (d5)-(f5), (d6)-(f6), (d7)-(f7), and (e8)-(f8) are the 3D gray maps obtained by using the Top-hat, Max-mean, Max-median, IPI, TV-PCP, MLMC, and proposed methods, respectively.Figure 12
ROC curves of the detection results of six simulated sequences: (a) sequence 1; (b) sequence 2; (c) sequence 3; (d) sequence 4; (e) sequence 5; (f) sequence 6.
(a)
(b)
(c)
(d)
(e)
(f)To further verify the performance of our method, we make a comparison of the ROC curves produced by six methods for the six simulated sequences in Figure12. It is found that the Top-hat method is not satisfactory, for it has a low detection probability. Both the Max-mean and Max-median methods possess higher detection probability than the Top-hat method for the sequences except sequence 2. The IPI and TV-PCP methods have better performance than the above three baseline methods for all sequences. In general, the proposed method consistently achieves the highest detection probability in almost all cases, implying that the proposed method has a better detection performance than the baseline methods.


## 4.4. Convergence Rate of the Algorithm

We investigated the convergence behavior of the proposed algorithm. Figure13 illustrates the evolutional curve of the objective function and NSDE versus the iterations for the real infrared image object detection in Figure 7(a). It can be observed clearly that the proposed method achieves good convergence and converges after about 20 iterations.Figure 13
Evolution of the objective function and NSDE versus number of iterations for the proposed method.
(a)
(b)


## 4.5. Computational Complexity and Computation Time of the Algorithm

In this subsection, we compare the computational costs of seven methods. The computational complexity and computation time corresponding to the six images in Figure7 by the seven methods are shown in Table 5. Suppose the size of the original image is M×N. The size of the patch-image constructed from the original image is m×n (for simplicity, we assume m>n). The computational complexity of the Top-hat method is OMNL2logL2, where L2 is the size of the structure element. For the Max-median method, the complexity is dominated by the median filtering, its computational cost being OL2logL2. Considering the image size, the final cost of Max-median is OMNL2logL2. As for the Max-mean method, the total cost is OMNL2. For the TV-PCP method, the total cost of each iteration is Omn2 operations. The proposed model contains an optimization problem: the objective function includes the nuclear norm, ℓ1 norm, and total variation norm and is solved by ADMM. For the proposed method, the computational complexities of updating the variables Z1, Z2, Z3x, Z3y, B, and T in (12), (15), (19), (20), (22), and (24) for each iteration are Omn2, Omn, Omn, Omn, Omn, and Omn, respectively. Therefore, the total cost of each iteration for the proposed method is Omn2 operations.Table 5
Comparison of computational complexity and computation time (in seconds) of seven methods.
Method
Complexity
Figure7(a)
Figure7(b)
Figure7(c)
Figure7(d)
Figure7(e)
Figure7(f)
Top-hat
O
M
N
L
2
log
L
2
0.273
0.280
0.262
0.275
0.264
0.239
Max-mean
O
M
N
L
2
1.290
2.211
2.186
2.273
1.319
11.553
Max-median
O
M
N
L
2
log
L
2
1.251
2.408
1.934
2.713
1.264
11.948
IPI
O
m
n
2
1.548
5.825
4.138
5.311
1.673
22.848
TV-PCP
O
m
n
2
15.154
40.589
31.039
45.586
12.617
201.407
MLMC
O
M
N
L
3
0.034
0.028
0.026
0.029
0.025
0.028
Ours
O
m
n
2
6.319
23.242
11.695
18.743
6.498
88.440


## 5. Conclusion

For better detection of infrared small targets with complex backgrounds clutters and noise, a novel method integrating the TV regularization with reweightedℓ1 norm constraints is proposed to describe the sparse small target component more accurately. Furthermore, a low-rank constraint item is adopted to depict the background component. The TV regularization is incorporated into the model for suppressing the residual background clutters and noise and a sparsity reweighted ℓ1 norm on the target is employed to enhance the small target. Finally, the task of small target detection is transformed into an optimization problem, which can be efficiently solved by invoking ADMM. The experiments on both synthesized and real small target images demonstrate that our method is more efficient and effective in detecting small targets under complex background clutters and noise than the conventional baseline methods. The computational time of the proposed method can be improved with efficient C/C++ implementation.The proposed method also has some limitations. The regularization parameterλ2 is not updated adaptively and need to be manually tuned. Although the parameter setting in the proposed method can be empirically determined in a range, it is still more preferred for some applications to have a completely automatic parameter estimation method.



---

*Source: 1529704-2020-01-27.xml*