# Hierarchical Recognition System for Target Recognition from Sparse Representations

Authors: Zongyong Cui; Zongjie Cao; Jianyu Yang; Hongliang Ren
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

A hierarchical
recognition system (HRS) based on constrained Deep Belief Network
(DBN) is proposed for SAR Automatic Target Recognition (SAR ATR). As
a classical Deep Learning method, DBN has shown great
performance on data reconstruction, big data mining, and
classification. However, few works have been carried out to solve
small data problems (like SAR ATR) by Deep Learning method. In HRS,
the deep structure and pattern classifier are combined to solve small
data classification problems. After building the DBN with multiple
Restricted Boltzmann Machines (RBMs), hierarchical features can be
obtained, and then they are fed to classifier directly. To obtain more
natural sparse feature representation, the Constrained RBM (CRBM) is
proposed with solving a generalized optimization problem. Three RBM
variants,L1-RNM, L2-RBM, and L1/2-RBM, are presented and
introduced to HRS in this paper. The experiments on MSTAR public
dataset show that the performance of the proposed HRS with CRBM
outperforms current pattern recognition methods in SAR ATR, like
PCA + SVM, LDA + SVM, and NMF + SVM.


---

## Body

## 1. Introduction

Synthetic Aperture Radar (SAR) Automatic Target Recognition (ATR) plays an important role in military and civil applications, such as social security, environmental monitoring, and national defense [1–5]. Most current researches focus on the pattern features [6–8] or pattern classifiers [9, 10]. The pattern recognition methods have shown the excellent ability on classifying the small data. However, if the samples number is huge, the pattern recognition methods are slow and inefficient. With the development of SAR imaging ability, more data can be captured. The data dimensionality is increasing, which means more powerful algorithms are needed.Since Hintion and Salakhutdinov proposed the Deep Auto-Encoder networks [11], Deep Leaning has been a research hot spot in recent years. Deep Learning algorithms have shown the great performance on big data reconstruction, data mining, and classification [12], as they can learn hierarchical representations of high-dimensional data and have been applied in many fields, such as handwritten digit recognition [13, 14], object detection [15–17], and scene classification [18–20]. Thus introducing Deep Learning to SAR ATR is necessary and urgent. Few researchers have started such work. The Auto-Encoder is applied to SAR ATR directly in [21], but with few theoretical problems solution. In [22], the features of SAR target and shadow are extracted based on the multilayer Auto-Encoder; then the combined features are fed to Synergetic Neural Network (SNN) for recognition. However, the recognition performance is not very prominent.For current SAR image database, a hierarchical recognition system (HRS) with combining Deep Belief Network (DBN) and pattern classifier is proposed in this paper. The proposed HRS has both advantages of deep structure and pattern recognition. Based on the great reconstruction ability of DBN, the features can be obtained in each layer. These features can be fed to classifier for high performance recognition.Meanwhile, in order to obtain sparse feature representation, the Constrained Restricted Boltzmann Machine (CRBM) is defined based on a generalized optimization problem. Unlike the Sparse RBM (SRBM) constrains, the expectation of the hidden units to a certain value, the constraint in CRBM is performed on the probability density of hidden units directly to obtain more sparse solution. Three RBM variants with norm constraint,L1-RNM, L2-RBM, and L1/2-RBM, are presented. Stacked CRBMs are used to built Constrained DBN (CDBN), which can be introduced to the proposed HRS.From the performance on MSTAR public dataset, the proposed HRS with CRBM can effectively solve the small dataset recognition problem and outperforms current pattern recognition methods in SAR ATR, like PCA + SVM, LDA + SVM, and NMF + SVM [23] (“PCA + SVM” means extracting the target feature by PCA, and using the SVM for classification; “LDA + SVM” and “NMF + SVM” have the similar meanings).The contribution of this paper includes two aspects: one is a hierarchical recognition system built for SAR ATR, which can obtain hierarchical features for recognition. The other is the CRBM proposed to obtain more natural sparse feature representation and introduced to HRS for better performance.The rest of this paper is organized as follows. Section2 introduces the framework of the proposed HRS and the hierarchical features representation in HRS. Section 3 describes the Constrained RBM with a generalized optimization problem and presents three specific RBM variants. The recognition experiments based on MSTAR database are performed and the results are analyzed in Section 4. Finally in Section 5 the conclusion and future work are stated.


## 2. Hierarchical Recognition System

The purpose of the the proposed Hierarchical Recognition System is to solve small data classification problems by combining the deep structure and the pattern classifiers. The framework of HRS is shown in Figure1.Figure 1
The framework of proposd hierarchical recognition system based on deep structure.


### 2.1. Deep Structure

Suppose the deep structure in HRS hasN layers. In each layer, the features can be obtained by certain feature exacting algorithm. Then the features are fed to pattern classifiers for recognition. The features in each layer can be the same or different, and the classifier in each layer can be the same or not. For the convenience of measurement and comparison, the features and classifier in each layer both remain the same.The deep structure of Deep Belief Networks (DBN) is mainly discussed in this paper. The DBN is stacked by Restricted Boltzmann Machines (RBMs). In Figure1, the left part can be seen as a DBN with N layers. In each layer, the reconstruction work is done by RBM.Actually, the deep structure is not only from DBN but can be from Stacked Auto-Encoder [24] or Convolutional Neural Networks (CNN) [25, 26] also.


### 2.2. Hierarchical Features Representation

Just like the feed forward perception in neural network, the features in LayerN are obtained by the following way:(1)FEA_LN=sigmVW1TN=1sigmFEA_LN-1WNTN>1,where FEA_LN means the feature obtained in Layer N, V stands for the samples, sigm is a function corresponding to f(x)=sigm(x)=1/(1+e-x), and the WNT indicates the transform of weight basis matrix WN in Layer N.The hierarchical features of the training and test samples obtained by DBN can be treated as the pattern features and fed to the pattern classifier for recognition work directly.


## 2.1. Deep Structure

Suppose the deep structure in HRS hasN layers. In each layer, the features can be obtained by certain feature exacting algorithm. Then the features are fed to pattern classifiers for recognition. The features in each layer can be the same or different, and the classifier in each layer can be the same or not. For the convenience of measurement and comparison, the features and classifier in each layer both remain the same.The deep structure of Deep Belief Networks (DBN) is mainly discussed in this paper. The DBN is stacked by Restricted Boltzmann Machines (RBMs). In Figure1, the left part can be seen as a DBN with N layers. In each layer, the reconstruction work is done by RBM.Actually, the deep structure is not only from DBN but can be from Stacked Auto-Encoder [24] or Convolutional Neural Networks (CNN) [25, 26] also.


## 2.2. Hierarchical Features Representation

Just like the feed forward perception in neural network, the features in LayerN are obtained by the following way:(1)FEA_LN=sigmVW1TN=1sigmFEA_LN-1WNTN>1,where FEA_LN means the feature obtained in Layer N, V stands for the samples, sigm is a function corresponding to f(x)=sigm(x)=1/(1+e-x), and the WNT indicates the transform of weight basis matrix WN in Layer N.The hierarchical features of the training and test samples obtained by DBN can be treated as the pattern features and fed to the pattern classifier for recognition work directly.


## 3. Constrained RBM

Due to the textural characters of SAR images, sparse representation is beneficial for SAR ATR [27, 28]. In order to obtain the sparse representation and improve the performance of HRS, the sparse constraint is introduced to DBN, and the constraint is forced on RBM. In this section, simple introduction about RBM and the Sparse RBM proposed by Lee et al. in 2007 are presented at first. Then the Constrained RBM is proposed based on a generalized optimal problem. The CRBM can be introduced to DBN to build Constrained DBN (CDBN).


### 3.1. Restricted Boltzmann Machine

The DBN is stacked by multiple RBMs. The RBM is a particular type of Markov random field that has a two-layer architecture [29, 30]. The structure of RBM is shown in Figure 2.Figure 2
RBM model.An RBM has one visual layer and one hidden layer. The visible unitsV={v1,v2,…,vi,…,vD} in the visual layer are connected to the hidden binary stochastic units H={h1,h2,…,hj,…,hF}(H∈{0,1}) in the hidden layer by a weight basis matrix W={wij}. The energy of the state {V,H} is defined as follows:(2)EV,H;θ=-VTWH-BTV-ATH=-∑i=1D‍∑j=1F‍wijvihj-∑i=1D‍bivi-∑j=1F‍cjhj,where θ={W,B,A} are the model parameters, wij(1⩽i⩽D,1⩽j⩽F) represents the symmetric interaction term between visible unit i and hidden unit j, B={b1,b2,…,bi,…,bD}, C={c1,c2,…,cj,…,cF} in which bi and cj are bias terms, hj means the jth hidden unit in H, and vi means the ith visual unit in V.The update of parametersθ={W,B,A} can be obtained by Gradient Descent (GD) method:(3)wij⟵wij+α·vihjdata-vihjrecon,bi⟵bi+α·vidata-virecon,cj⟵cj+α·hjdata-hjrecon,where ·data stands for the expectation over all samples and ·recon is the expectation over the reconstruction data obtained by Contrastive Divergence (CD) learning. For detailed information of CD learning, please refer to [31].


### 3.2. Sparse RBM

It is believed that solving the reconstruction minimal error optimization problem by sparse constraint can obtain better performance in feature representation. For sparse representation, the SRBM constrains the activation of the hidden units at a fixed levelρ. To achieve this purpose, a regularization term is added on the log-likelihood cost function of RBM. The optimization problem in SRBM can be described as follows [13]:(4)minimizewij,bi,cj-∑l=1m‍ln∑h‍PVl,Hl;θ+λ∑j=1F‍ρ-1m∑l=1m‍Ehjl∣Vl2,where {V1,…,Vl,…,Vm} stands for the training set including m examples, λ is a regularization constant, and ρ is a parameter to control the sparseness of the hidden units hj.The updating of the log-likelihood term can be computed by CD learning. The right-hand side of (4) is updated by gradient descent method. In the gradient step, SRBM only update the bias term cj instead of updating all the parameters. The update of SRBM just adds one additional update rule of cj following the last rule in (3).


### 3.3. Constrained RBM

The SRBM constrains the expectation of the hidden units values on RBM for sparse representation but does not constrain the probability density function of hidden units directly. In this paper, Constrained RBM (CRBM) is proposed by extending (4) to be a generalized optimization problem for more sparse representation. The constraint is performed on the probability density of hidden units, which can include the case of constraining the expectation. The optimization problem can be described as(5)minimizewij,bi,cj-∑l=1m‍ln∑h‍PVl,Hl;θ+λ∑l=1m‍fPhjl∣Vl,where f(·) is a function about Phjl∣Vl, and P={Ph1l∣Vl,Ph2l∣Vl,…,Phjl∣Vl,…,PhFl∣Vl}.Different from SRBM, which constrains the average activation probability expectation of the hidden units values, the constraint to RBM in (5) is performed on the probability density of hidden units directly. The purpose of this generalized optimization problem is to obtain more sparse representation by increasing the probability of hj=0 along with reducing the probability of hj=1.In (5), the f(·) can be the functions including norms, the combination of norms, or the composite functions about Phjl∣Vl. Thus, the SRBM can be seen a special case of CRBM. The norm constraints are mainly discussed in this paper:(6)fx=Xp=∑i=1n‍xip1/p.For the convenient calculations, (6) can be modified to f(x)=(1/p)Xpp. Three RBM variants, L1-RBM, L2-RBM, and L1/2-RBM, corresponding to three common norms, L1-norm, L2-norm, and L1/2-norm, are specifically applied to SAR target recognition and shown the performance in the following sections.Introducing CRBM to HRS will build the Constrained HRS (CHRS). The CHRS withL1-norm, L2-norm, and L1/2-norm constraints are named HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM) correspondingly.


## 3.1. Restricted Boltzmann Machine

The DBN is stacked by multiple RBMs. The RBM is a particular type of Markov random field that has a two-layer architecture [29, 30]. The structure of RBM is shown in Figure 2.Figure 2
RBM model.An RBM has one visual layer and one hidden layer. The visible unitsV={v1,v2,…,vi,…,vD} in the visual layer are connected to the hidden binary stochastic units H={h1,h2,…,hj,…,hF}(H∈{0,1}) in the hidden layer by a weight basis matrix W={wij}. The energy of the state {V,H} is defined as follows:(2)EV,H;θ=-VTWH-BTV-ATH=-∑i=1D‍∑j=1F‍wijvihj-∑i=1D‍bivi-∑j=1F‍cjhj,where θ={W,B,A} are the model parameters, wij(1⩽i⩽D,1⩽j⩽F) represents the symmetric interaction term between visible unit i and hidden unit j, B={b1,b2,…,bi,…,bD}, C={c1,c2,…,cj,…,cF} in which bi and cj are bias terms, hj means the jth hidden unit in H, and vi means the ith visual unit in V.The update of parametersθ={W,B,A} can be obtained by Gradient Descent (GD) method:(3)wij⟵wij+α·vihjdata-vihjrecon,bi⟵bi+α·vidata-virecon,cj⟵cj+α·hjdata-hjrecon,where ·data stands for the expectation over all samples and ·recon is the expectation over the reconstruction data obtained by Contrastive Divergence (CD) learning. For detailed information of CD learning, please refer to [31].


## 3.2. Sparse RBM

It is believed that solving the reconstruction minimal error optimization problem by sparse constraint can obtain better performance in feature representation. For sparse representation, the SRBM constrains the activation of the hidden units at a fixed levelρ. To achieve this purpose, a regularization term is added on the log-likelihood cost function of RBM. The optimization problem in SRBM can be described as follows [13]:(4)minimizewij,bi,cj-∑l=1m‍ln∑h‍PVl,Hl;θ+λ∑j=1F‍ρ-1m∑l=1m‍Ehjl∣Vl2,where {V1,…,Vl,…,Vm} stands for the training set including m examples, λ is a regularization constant, and ρ is a parameter to control the sparseness of the hidden units hj.The updating of the log-likelihood term can be computed by CD learning. The right-hand side of (4) is updated by gradient descent method. In the gradient step, SRBM only update the bias term cj instead of updating all the parameters. The update of SRBM just adds one additional update rule of cj following the last rule in (3).


## 3.3. Constrained RBM

The SRBM constrains the expectation of the hidden units values on RBM for sparse representation but does not constrain the probability density function of hidden units directly. In this paper, Constrained RBM (CRBM) is proposed by extending (4) to be a generalized optimization problem for more sparse representation. The constraint is performed on the probability density of hidden units, which can include the case of constraining the expectation. The optimization problem can be described as(5)minimizewij,bi,cj-∑l=1m‍ln∑h‍PVl,Hl;θ+λ∑l=1m‍fPhjl∣Vl,where f(·) is a function about Phjl∣Vl, and P={Ph1l∣Vl,Ph2l∣Vl,…,Phjl∣Vl,…,PhFl∣Vl}.Different from SRBM, which constrains the average activation probability expectation of the hidden units values, the constraint to RBM in (5) is performed on the probability density of hidden units directly. The purpose of this generalized optimization problem is to obtain more sparse representation by increasing the probability of hj=0 along with reducing the probability of hj=1.In (5), the f(·) can be the functions including norms, the combination of norms, or the composite functions about Phjl∣Vl. Thus, the SRBM can be seen a special case of CRBM. The norm constraints are mainly discussed in this paper:(6)fx=Xp=∑i=1n‍xip1/p.For the convenient calculations, (6) can be modified to f(x)=(1/p)Xpp. Three RBM variants, L1-RBM, L2-RBM, and L1/2-RBM, corresponding to three common norms, L1-norm, L2-norm, and L1/2-norm, are specifically applied to SAR target recognition and shown the performance in the following sections.Introducing CRBM to HRS will build the Constrained HRS (CHRS). The CHRS withL1-norm, L2-norm, and L1/2-norm constraints are named HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM) correspondingly.


## 4. Experimental Results and Analysis

To verify the performance of the proposed HRS, in this section, the HRS are compared with DBN and some pattern recognition methods, like PCA + SVM, LDA + SVM, and NMF + SVM. The experiments are performed on a “small” dataset.


### 4.1. Experiment Data

The SAR images data are taken from MSTAR public database [32]. Conventional 3-class recognition problem uses three targets, BMP2, BTR70, and T72, with depression angles 17° and 15° for training and test, respectively. BMP2 has three types, and only the type sn-c21 is used for training. Meanwhile, T72 has three types, and only the type sn-132 is used for test. The raw images of targets BMP2, BTR70, and T72 have 128 × 128 pixels. For convenience, all the images are only cropped by extracting 64 × 64 patches from the center of the image. The statistics of these three targets are listed in Table 1. From Table 1, it can be seen that the training set has 698 samples and the test set has 1365 samples.Table 1
Training and test data from MSTAR.
Class
Training
Test
Type
Size
Type
Size
BMP2
sn-9563
sn-9563
195
sn-9566
sn-9566
196
sn-c21
233
sn-c21
196
BTR70
sn-c71
233
sn-c71
196
T72
sn-132
232
sn-132
196
sn-812
sn-812
195
sn-s7
sn-s7
191
Total
698
1365The sample number in MSTAR is in hundreds level. Compared to the standard databases for Deep Learning algorithms, MNIST, CIFAR, and ImageNet databases, which have tens of thousand samples [25, 33], the MSTAR database only have hundreds samples and can be definitely seen as the “small” dataset.


### 4.2. Initialization

To build the proposed HRS, the DBN and HRS are stacked by two RBMs or CRBMs layers. Thus, the DBN has five variants: DBN(RBM), DBN(SRBM), DBN(L1-RBM), DBN(L2-RBM), and DBN(L1/2-RBM). Meanwhile, the proposed HRS has five variants: HRS(RBM), HRS(SRBM), HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM).The SVM is chosen for pattern classifier. The HRS built for experiments in this section is shown is Figure3.Figure 3
The structure of HRS with two RBMs or CRBMs layers used in the experiments.Both of the two layers in DBN and HRS have 300 hidden units. The input sample has 4096 (64×64) pixels and the visual layer has 4096 units. For RBM, the learning rate α in (3) is set to 0.0045. For BPNN, the learning rate is set to 1. The parameter λ in (5) is set to 0.00001.


### 4.3. Experiments Analysis

The experiments mainly include two aspects. One is to show the performance of features in each DBN layer the other one is to compare the performance of different recognition methods.Table2 lists the performance of the hierarchical features in DBN two layers with respect to iteration. The second and third columns indicate the recognition rates by features FEA_L1 and FEA_L2 using SVM classifier. The last column shows the performance of the common DBN algorithm which uses Softmax for classification.Table 2
The performance of HRS and DBN on MSTAR with respect to iteration.
Iteration
FEA_L1 + SVM
FEA_L2 + SVM
DBN + feedforward
50
92.97%
92.82%
91.50%
100
94.21%
92.53%
92.31%
150
94.87%
93.11%
92.53%
200
94.43%
93.41%
92.89%
300
94.36%
93.85%
93.41%
400
94.87%
94.29%
94.07%
500
94.58%
94.07%
93.99%Form Table2, it can be seen that, compared to common DBN, only using the feature in one layer can have better performance. The FFNN obtains the results by fusing the hierarchical features. However, the fusion may reduce the performance because of less texture information in small dataset.Besides, the performance of featureFEA_L1 outperforms the feature FEA_L2, which means that the feature obtained in the first layer has the best performance for MSATR 3-class recognition problem, in part because the texture information in SAR images is relatively small and the features in higher layers correspond with more reconstruction loss.Please note that, in Table2, when the iteration is 500, the recognition rates obtained by DBN and HRS are both lower than when the iteration is 400. It can be seen that more iterations do not mean better recognition performance. That is partly because the MSTAR database can be seen as “small” dataset, and too many iterations may lead to overfitting.The comparison between pattern recognition methods and HRS is shown in Table3. In this part, only feature FEA_L1 is used, and the iteration for NMF, DBN, and HRS is set to 200.Table 3
The recognition rates obtained by different methods.
Methods
BMP2
BTR70
T72
Average
PCA + SVM
88.42%
94.39%
92.96%
91.21%
LDA + SVM
79.56%
96.94%
85.91%
84.76%
NMF + SVM
90.12%
99.49%
93.81%
93.04%
DBN (RBM)
86.88%
99.49%
96.74%
92.89%
DBN (SRBM)
86.88%
99.49%
97.08%
93.04%
DBN (L1-RBM)
87.05%
99.49%
97.25%
93.19%
DBN (L2-RBM)
87.05%
99.49%
97.08%
93.11%
DBN (L1/2-RBM)
89.78%
99.49%
97.60%
94.51%
HRS (RBM)
89.61%
99.49%
97.60%
94.43%
HRS (SRBM)
89.95%
99.49%
98.11%
94.73%
HRS (L1-RBM)
90.12%
100%
98.45%
95.09%
HRS (L2-RBM)
90.12%
99.49%
98.28%
94.95%
HRS (L1/2-RBM)
90.46%
100%
98.80%
95.31%From Table3, it can be seen that, for targets BMP2 and BTR70, the proposed HRS has similar performance with the pattern methods. But for target T72, the proposed HRS has an obvious improvement. From the average recognition rates, it can be seen that the performance of DBN is better than PCA + SVM and LDA + SVM, but little worse than NMF + SVM. The proposed HRS outperforms DBN and all three pattern methods. Moreover, the proposed three constrained HRS variants, HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM) have better performance than DBN, HRS(RBM), and HRS(SRBM), in which the HRS(L1/2-RBM) has the best performance.Comparing the five DBN variants, it can be seen that the DBN with sparse constraint can obtain better performance than DBN. The DBN(L1/2-RBM) especially has the best performance. Comparing the five HRS variants, the HRS with sparse constraint outperforms the HRS without sparse constraint. The HRS (L1/2-RBM) especially can obtian the best recognition rate. Thus, the feasibility of the proposed CRBM can be verified.Comparing the HRS variants to DBN variants, it can be seen that the proposed HRS has better performance than DBN. Thus, the effectiveness of the proposed HRS can be verified.Overall, the results in Tables2 and 3 verify the feasibility and effectiveness of the proposed HRS, and adding sparse constraint on HRS can improve the recognition performance.


## 4.1. Experiment Data

The SAR images data are taken from MSTAR public database [32]. Conventional 3-class recognition problem uses three targets, BMP2, BTR70, and T72, with depression angles 17° and 15° for training and test, respectively. BMP2 has three types, and only the type sn-c21 is used for training. Meanwhile, T72 has three types, and only the type sn-132 is used for test. The raw images of targets BMP2, BTR70, and T72 have 128 × 128 pixels. For convenience, all the images are only cropped by extracting 64 × 64 patches from the center of the image. The statistics of these three targets are listed in Table 1. From Table 1, it can be seen that the training set has 698 samples and the test set has 1365 samples.Table 1
Training and test data from MSTAR.
Class
Training
Test
Type
Size
Type
Size
BMP2
sn-9563
sn-9563
195
sn-9566
sn-9566
196
sn-c21
233
sn-c21
196
BTR70
sn-c71
233
sn-c71
196
T72
sn-132
232
sn-132
196
sn-812
sn-812
195
sn-s7
sn-s7
191
Total
698
1365The sample number in MSTAR is in hundreds level. Compared to the standard databases for Deep Learning algorithms, MNIST, CIFAR, and ImageNet databases, which have tens of thousand samples [25, 33], the MSTAR database only have hundreds samples and can be definitely seen as the “small” dataset.


## 4.2. Initialization

To build the proposed HRS, the DBN and HRS are stacked by two RBMs or CRBMs layers. Thus, the DBN has five variants: DBN(RBM), DBN(SRBM), DBN(L1-RBM), DBN(L2-RBM), and DBN(L1/2-RBM). Meanwhile, the proposed HRS has five variants: HRS(RBM), HRS(SRBM), HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM).The SVM is chosen for pattern classifier. The HRS built for experiments in this section is shown is Figure3.Figure 3
The structure of HRS with two RBMs or CRBMs layers used in the experiments.Both of the two layers in DBN and HRS have 300 hidden units. The input sample has 4096 (64×64) pixels and the visual layer has 4096 units. For RBM, the learning rate α in (3) is set to 0.0045. For BPNN, the learning rate is set to 1. The parameter λ in (5) is set to 0.00001.


## 4.3. Experiments Analysis

The experiments mainly include two aspects. One is to show the performance of features in each DBN layer the other one is to compare the performance of different recognition methods.Table2 lists the performance of the hierarchical features in DBN two layers with respect to iteration. The second and third columns indicate the recognition rates by features FEA_L1 and FEA_L2 using SVM classifier. The last column shows the performance of the common DBN algorithm which uses Softmax for classification.Table 2
The performance of HRS and DBN on MSTAR with respect to iteration.
Iteration
FEA_L1 + SVM
FEA_L2 + SVM
DBN + feedforward
50
92.97%
92.82%
91.50%
100
94.21%
92.53%
92.31%
150
94.87%
93.11%
92.53%
200
94.43%
93.41%
92.89%
300
94.36%
93.85%
93.41%
400
94.87%
94.29%
94.07%
500
94.58%
94.07%
93.99%Form Table2, it can be seen that, compared to common DBN, only using the feature in one layer can have better performance. The FFNN obtains the results by fusing the hierarchical features. However, the fusion may reduce the performance because of less texture information in small dataset.Besides, the performance of featureFEA_L1 outperforms the feature FEA_L2, which means that the feature obtained in the first layer has the best performance for MSATR 3-class recognition problem, in part because the texture information in SAR images is relatively small and the features in higher layers correspond with more reconstruction loss.Please note that, in Table2, when the iteration is 500, the recognition rates obtained by DBN and HRS are both lower than when the iteration is 400. It can be seen that more iterations do not mean better recognition performance. That is partly because the MSTAR database can be seen as “small” dataset, and too many iterations may lead to overfitting.The comparison between pattern recognition methods and HRS is shown in Table3. In this part, only feature FEA_L1 is used, and the iteration for NMF, DBN, and HRS is set to 200.Table 3
The recognition rates obtained by different methods.
Methods
BMP2
BTR70
T72
Average
PCA + SVM
88.42%
94.39%
92.96%
91.21%
LDA + SVM
79.56%
96.94%
85.91%
84.76%
NMF + SVM
90.12%
99.49%
93.81%
93.04%
DBN (RBM)
86.88%
99.49%
96.74%
92.89%
DBN (SRBM)
86.88%
99.49%
97.08%
93.04%
DBN (L1-RBM)
87.05%
99.49%
97.25%
93.19%
DBN (L2-RBM)
87.05%
99.49%
97.08%
93.11%
DBN (L1/2-RBM)
89.78%
99.49%
97.60%
94.51%
HRS (RBM)
89.61%
99.49%
97.60%
94.43%
HRS (SRBM)
89.95%
99.49%
98.11%
94.73%
HRS (L1-RBM)
90.12%
100%
98.45%
95.09%
HRS (L2-RBM)
90.12%
99.49%
98.28%
94.95%
HRS (L1/2-RBM)
90.46%
100%
98.80%
95.31%From Table3, it can be seen that, for targets BMP2 and BTR70, the proposed HRS has similar performance with the pattern methods. But for target T72, the proposed HRS has an obvious improvement. From the average recognition rates, it can be seen that the performance of DBN is better than PCA + SVM and LDA + SVM, but little worse than NMF + SVM. The proposed HRS outperforms DBN and all three pattern methods. Moreover, the proposed three constrained HRS variants, HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM) have better performance than DBN, HRS(RBM), and HRS(SRBM), in which the HRS(L1/2-RBM) has the best performance.Comparing the five DBN variants, it can be seen that the DBN with sparse constraint can obtain better performance than DBN. The DBN(L1/2-RBM) especially has the best performance. Comparing the five HRS variants, the HRS with sparse constraint outperforms the HRS without sparse constraint. The HRS (L1/2-RBM) especially can obtian the best recognition rate. Thus, the feasibility of the proposed CRBM can be verified.Comparing the HRS variants to DBN variants, it can be seen that the proposed HRS has better performance than DBN. Thus, the effectiveness of the proposed HRS can be verified.Overall, the results in Tables2 and 3 verify the feasibility and effectiveness of the proposed HRS, and adding sparse constraint on HRS can improve the recognition performance.


## 5. Conclusion

The hierarchical recognition system (HRS) based on Deep Belief Network (DBN) is proposed to solve SAR Automatic Target Recognition (SAR ATR) problem. In HRS, the deep structure of DBN is combined with pattern classifier to solve small data classification problems. The hierarchical features are obtained by the multiple RBMs which is stacked in DBN, and then are fed to pattern classifier directly. To obtain more natural sparse feature representation, the Constrained RBM (CRBM) is proposed with solving a generalized optimization problem. Three RBM variants,L1-RNM, L2-RBM, and L1/2-RBM, are introduced to HRS, corresponding to three HRS variants, HRS(L1-RBM), HRS(L2-RBM), and HRS(L1/2-RBM) which are presented in this paper. The experiments on MSTAR public dataset show the performance of the proposed HRS with CRBM outperforms the DBN and current pattern recognition methods in SAR ATR, like PCA + SVM, LDA + SVM, and NMF + SVM.



---

*Source: 527095-2015-09-16.xml*