# Object-Oriented Semisupervised Classification of VHR Images by Combining MedLDA and a Bilateral Filter

Authors: Shi He; Hong Tang; Jing Li; Yang Shu; Li Shen
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

A Bayesian hierarchical model is presented to classify very high resolution (VHR) images in a semisupervised manner, in which both a maximum entropy discrimination latent Dirichlet allocation (MedLDA) and a bilateral filter are combined into a novel application framework. The primary contribution of this paper is to nullify the disadvantages of traditional probabilistic topic models on pixel-level supervised information and to achieve the effective classification of VHR remote sensing images. This framework consists of the following two iterative steps. In the training stage, the model utilizes the central labeled pixel and its neighborhood, as a squared labeled image object, to train the classifiers. In the classification stage, each central unlabeled pixel with its neighborhood, as an unlabeled object, is classified as a user-provided geoobject class label with the maximum posterior probability. Gibbs sampling is adopted for model inference. The experimental results demonstrate that the proposed method outperforms two classical SVM-based supervised classification methods and probabilistic-topic-models-based classification methods.


---

## Body

## 1. Introduction

With the development of imaging technology, many airborne and satellite sensors, for example, QuickBird, IKONOS, and WorldView, can provide very high resolution (VHR) images. Pixel-based supervised classification methods that have been successfully applied to low or moderate resolution remote sensing images do not have desirable results when applied to VHR images, because the spatial relationship among pixels is neglected when these methods are used and the “pepper and salt” effect is often observed in classification results of VHR remote sensing images. To solve this problem, object-based classification methods are often used to classify VHR images. For example, Wang et al. integrated the pixel and object-based methods for mapping mangroves with IKONOS imagery [1]. Salehi et al. developed an object-based classification framework for QuickBird imagery coupled with a layer of height points to classify a complex urban environment [2]. Kim et al. investigated the use of a geographic object-based image analysis approach with the incorporation of object-specific grey-level cooccurrence matrix texture measures from a multispectral IKONOS image for mapping forest type [3]. These methods often consist of two sequential steps, that is, segmentation and classification [4]. Although this two-step procedure works well in some cases, several problems still remain. First, the classification results depends heavily on the segmentation algorithm. Second, training objects must be labeled before the image analysis in these classification methods. However supervised information is often obtained at a pixel-level. Thus a contradiction is noted between image objects and pixel-level supervised information, and labels cannot be used directly to train object classifiers [4].In natural language processing, probabilistic topic models can be applied to find the latent topic representations of documents in a corpus [5, 6]. When these models are used to model remote sensing images, the images are often partitioned into a set of image tiles (i.e., documents) and the characteristics of pixels are often treated as visual words [4–9]. These probabilistic topic models have been used to discover semantic structures from very high resolution (VHR) remote sensing images, such as in target detection [7], image clustering [8, 10, 11], and image annotation [12]. Probabilistic topic models can also use supervised information for discovering latent topic representations, such as supervised latent Dirichlet allocation (sLDA) [13], discriminative latent Dirichlet allocation (DiscLDA) [14], semisupervised latent Dirichlet allocation (ssLDA) [4], and maximum entropy discrimination latent Dirichlet allocation (MedLDA) [15]. Differences among these models are noted in the classification: the sLDA, DiscLDA, and ssLDA are learned under likelihood-driven objective functions, which are fully generative models. MedLDA employs the discriminative max-margin principle into the process of topic learning, which is more suitable for classification tasks. Additionally, the supervised information in these models is associated with documents or tiles of images rather than individual pixels except for the ssLDA. Thus, a contradiction is noted between classification results of individual pixels and tile-level supervised information in these models, and tile-level labels cannot be used directly to train individual pixels classifiers.To address the aforementioned problems in object-based supervised classification methods and probabilistic topic models, we present an object-oriented semisupervised classification method [4, 16] for VHR remote sensing images based on the MedLDA model [15, 17] and bilateral filter [18] in a novel framework, which is referred to as the semisupervised MedLDA (ssMedLDA). Each image object is defined as a squared image block with a neighborhood for pixels without image segmentation [10]. The main contribution of our proposed model is the combination of a probabilistic topic model with pixel-level supervised information to achieve effective object-oriented semisupervised classification of VHR remote sensing images. The remainder of this paper is organized as follows. In Section 2, the proposed approach is presented in details. The experimental results are given in Section 3. Finally, the conclusions are presented in Section 4.


## 2. Methodology

First, the problem between the supervised information for objects and pixel-level is discussed in this section. Second the proposed method ssMedLDA for classification processing is introduced. Third, the algorithm is presented.


### 2.1. Problem Statement

LetS
=
s
=
i
,
j
∣
1
≤
i
≤
I
,
1
≤
j
≤
J be the set of lattice sites in the given satellite image, where I and J are width and height of the image, respectively. A random field indexed by S is given by W
=
{
W
s
=
w
s
∣
w
s
∈
V
,
s
∈
S
}, where a random variable W
s at site s takes a value w
s in its state space V. The set w
=
{
w
s
∣
s
∈
S
} is drawn from the state space X with the joint probability P
(
W
=
w
). In this paper, a linear discriminative function, for example, y
s
=
η
T
z
s
-, is used to bridge the user-provided geoobject class label y
s and an observed image object O
s via an inferred latent image feature z
s
-, where η is learned under the max-margin principle.As shown in Figure1, the latent image feature z
s
- in probabilistic topic models is an expected measurement for the sth image object O
s, which consists of m
=
h
×
h pixels, that is, w
s
→
=
{
w
s
1
,
w
s
2
,
…
,
w
s
m
}. As for image object O
s, its latent image feature z
s
- is a k-dimension vector with element z
s
k
-,(1)
z
s
k
-
=
1
m
∑
n
=
1
m
I
z
s
n
k
=
1
=
∑
n
=
1
m
1
m
I
z
s
n
k
=
1
,where I
(
·
) is an indicator function that equals 1 when the predicate holds; otherwise it is 0. Because latent topic assignments of pixels in image object O
s, that is, {
z
s
1
,
z
s
2
,
…
,
z
s
m
}, are assumed to be independent and identically distributed in the MedLDA, the contribution or distance coefficientof each latent topic assignment to the image object O
s is assumed to be equal, that is, 1
/
m. When image objects are given geoobject class labels, the parameter of linear discriminative function η can be learned using the inferred latent features.Figure 1
Supervised informationy
s on pixel w
s and image object O
s and the latent image feature.As shown in Figure1, the user-provided geoobject class labels are often given for partially labelled pixels. Thus a contradiction is noted between image objects O
s and pixel-level supervised information y
s: the supervised information has different influences on each pixel in the image object O
s and cannot be used directly to train object-oriented classifiers. Thus the problem is how to use labeled pixels directly to train object oriented classifiers.To solve the above-mentioned problems, an edge-preserving filtering (EPF) [19], for example, the bilateral filter [18], is used to construct the relationship between the central labeled pixel and the other pixels in the image object. The pixel-level supervised information can be diffused to object-level supervised information based on the distance which includes both the spatial distance or gap g
s
n
s
p
a and the spectral distance g
s
n
s
p
e between the center pixel w
s and the nth pixel w
s
n in object O
s. f
s
p
a and f
s
p
e are defined as the spatial and spectral decay functions, respectively. Therefore, the pixel-level supervised information diffuses to object-level semisupervised information in this paper by a
s
n: (2)
a
s
n
=
f
Spa
g
s
n
Spa
·
f
Spe
g
s
n
Spe
=
1
2
π
σ
spa
exp
⁡
-
L
s
-
L
n
2
σ
spa
2
∗
1
2
π
σ
spe
exp
⁡
-
G
s
-
G
n
2
σ
spe
2
∝
exp
⁡
-
L
s
-
L
n
2
σ
spa
2
-
G
s
-
G
n
2
σ
spe
2
,where L
s is the space location of w
s, L
n is the space location of w
s
n, G
s is the spectral location of w
s, G
n is the spectral location of w
s
n, σ
s
p
a is a space decay parameter, σ
s
p
e is a decay spectral decay parameter, and σ
s
p
a
>
0 and σ
s
p
e
>
0.


### 2.2. ssMedLDA

Letw
s
→
=
{
w
s
1
,
w
s
2
,
…
,
w
s
m
} be a vector of m pixels appearing in the sth image object O
s in the remote sensing image, and let K be the number of latent topics in the model. The vector of the response of discrete variable y
=
{
y
s
∣
s
∈
S
,
y
s
∈
(
1,2
,
…
,
B
)
} in the remote sensing image is the geoobject class labels, and B is the number of class labels. Different from the model of the MedLDA, the variable y
s of the geoobject class label for the central pixels w
s
c in the objects O
s is shown in Figure 2. The half hashed pattern denotes that the class node is partially observed. The generative process of the ssMedLDA model is as follows:(1)
Sample topic proportionsθ
∣
α
~
D
i
r
i
c
h
l
e
t
(
α
).
(2)
For each of then pixels w
s
n in sth image object O
s,
(a)
sample a topic assignmentz
s
n
∣
θ
~
M
u
l
t
i
n
o
m
i
a
l
(
θ
);
(b)
sample a wordw
s
n from P
(
w
s
n
∣
z
s
n
,
β
), a multinomial probability conditioned on z
s
n, namely, w
s
n
∣
z
s
n
,
β
1
:
K
~
M
u
l
t
i
-
G
a
u
s
s
i
a
n
d
i
s
t
r
i
b
u
t
i
o
n
(
β
z
n
).
(3)
Sampley
s
~
N
(
η
T
z
s
-
), where z
s
- is a vector with element z
s
k
-
=
∑
n
=
1
m
a
s
n
·
I
(
z
s
n
k
=
1
)
, and I
(
·
) is an indicator function that equals 1 when predicate holds; otherwise it is 0 in the image object O
s.Figure 2
Graphic model. The shaded nodes including the pixel valuew
s
n and the set of distance coefficients in the sth image object a
s
→ represent the observed random variables, and the hashed fill node including y
s denotes the partially observed random variable class. The intuitional explanations of the model are shown at the end of the red arrows.The distance coefficienta
s
n is integrated into ssMedLDA through conditional topic random fields [20] as shown in Figure 2.


### 2.3. Algorithm

As shown in Figure3, Gibbs sampling is used to estimate the model inference and approximate the posterior distribution of latent variables in the model. The algorithm of ssMedLDA can be summarized as follows.Figure 3
Algorithm of the ssMedLDA.Step 1 (initialization).
In this step, the parameters in the model are initialized. Two types of parameters must be set, including the number of latent topics,K; the Dirichlet hyperparameter, α; the neighborhood size, h; the positive regularization parameter, c; and the cost of making a wrong prediction, l. In addition, the matrix of topicszis initialized by assigning random topics for all pixels.Step 2 (inference).
In the ssMedLDA, topics are learned in two ways based on whether the supervised information of the central pixel is available or not.
(a) When the central pixelw
s
c in sth image object O
s is unlabeled, analogous to the deduction in previous studies [5, 21], the conditional distribution of z
s
c
k is given by the following:(3)
q
z
s
c
k
∣
z
¬
s
c
,
w
s
c
∝
p
w
s
c
∣
μ
k
,
Σ
k
C
s
,
¬
s
c
k
+
α
k
,where (
μ
k
,
Σ
k
) are parameters of the kth Gaussian distribution. C
s
,
¬
s
c
k is the number of times that terms being associated with topic k within the object O
s are without z
s
c; and α
k is the value of the Dirichlet hyperparameter associated with topic k.
(b) When the central pixelw
s
c is labeled, analogous to the deduction previously presented [17], the formulas of the conditional posterior of z
s
c
k change to the following:(4)
q
z
s
c
k
∣
z
¬
s
c
,
w
s
c
,
η
,
λ
∝
p
w
s
c
∣
μ
k
,
Σ
k
C
s
,
¬
s
c
k
+
α
k
·
∏
i
=
1
L
exp
⁡
a
s
c
y
s
i
c
l
+
λ
s
i
η
i
k
λ
s
i
-
c
2
a
s
c
2
η
i
k
2
+
2
a
s
c
η
i
k
Λ
s
c
i
2
λ
s
i
,where Λ
s
c
i
=
1
/
m
-
1
Σ
k
′
η
i
k
′
C
s
,
¬
s
c
k
′, ∀
a
s
n
≥
0, ∑
n
m
a
s
n
=
1, λ is the augmented variables, η is the classification model, and y
s
i equals 1 when y
s
=
i; otherwise it is equivalent to −1; L is the number of classes of interest.
The augmented variablesλ can be given by the following:(5)
p
λ
s
i
-
1
∣
z
,
η
=
I
ς
λ
s
i
-
1
;
1
c
ξ
s
i
,
1
,where I
ς indicates the inverse Gaussian, λ
s
i
-
1 obeys the inverse Gaussian, and ξ
s
i
=
l
-
y
s
i
η
T
z
s
-.
The linear discriminative functionη can be given by the following:(6)
q
η
i
∣
z
,
λ
∝
N
η
;
μ
,
Σ
,where μ
i
=
Σ
i
(
c
∑
s
=
1
S
′
y
s
i
c
l
+
λ
s
i
/
λ
s
i
z
s
-
), Σ
i
=
1
/
v
2
I
+
c
2
∑
s
=
1
S
′
z
s
-
z
s
-
T
/
λ
s
i
-
1, and S
′ indicates the number of labeled pixels.Step 3 (classification).
Each unlabeled pixely
s is classified as a user-provided geoobject class label with the maximum posterior probability(7)
y
s
=
argmax
i
=
1
,
…
,
L
⁡
η
i
T
z
s
-
.


## 2.1. Problem Statement

LetS
=
s
=
i
,
j
∣
1
≤
i
≤
I
,
1
≤
j
≤
J be the set of lattice sites in the given satellite image, where I and J are width and height of the image, respectively. A random field indexed by S is given by W
=
{
W
s
=
w
s
∣
w
s
∈
V
,
s
∈
S
}, where a random variable W
s at site s takes a value w
s in its state space V. The set w
=
{
w
s
∣
s
∈
S
} is drawn from the state space X with the joint probability P
(
W
=
w
). In this paper, a linear discriminative function, for example, y
s
=
η
T
z
s
-, is used to bridge the user-provided geoobject class label y
s and an observed image object O
s via an inferred latent image feature z
s
-, where η is learned under the max-margin principle.As shown in Figure1, the latent image feature z
s
- in probabilistic topic models is an expected measurement for the sth image object O
s, which consists of m
=
h
×
h pixels, that is, w
s
→
=
{
w
s
1
,
w
s
2
,
…
,
w
s
m
}. As for image object O
s, its latent image feature z
s
- is a k-dimension vector with element z
s
k
-,(1)
z
s
k
-
=
1
m
∑
n
=
1
m
I
z
s
n
k
=
1
=
∑
n
=
1
m
1
m
I
z
s
n
k
=
1
,where I
(
·
) is an indicator function that equals 1 when the predicate holds; otherwise it is 0. Because latent topic assignments of pixels in image object O
s, that is, {
z
s
1
,
z
s
2
,
…
,
z
s
m
}, are assumed to be independent and identically distributed in the MedLDA, the contribution or distance coefficientof each latent topic assignment to the image object O
s is assumed to be equal, that is, 1
/
m. When image objects are given geoobject class labels, the parameter of linear discriminative function η can be learned using the inferred latent features.Figure 1
Supervised informationy
s on pixel w
s and image object O
s and the latent image feature.As shown in Figure1, the user-provided geoobject class labels are often given for partially labelled pixels. Thus a contradiction is noted between image objects O
s and pixel-level supervised information y
s: the supervised information has different influences on each pixel in the image object O
s and cannot be used directly to train object-oriented classifiers. Thus the problem is how to use labeled pixels directly to train object oriented classifiers.To solve the above-mentioned problems, an edge-preserving filtering (EPF) [19], for example, the bilateral filter [18], is used to construct the relationship between the central labeled pixel and the other pixels in the image object. The pixel-level supervised information can be diffused to object-level supervised information based on the distance which includes both the spatial distance or gap g
s
n
s
p
a and the spectral distance g
s
n
s
p
e between the center pixel w
s and the nth pixel w
s
n in object O
s. f
s
p
a and f
s
p
e are defined as the spatial and spectral decay functions, respectively. Therefore, the pixel-level supervised information diffuses to object-level semisupervised information in this paper by a
s
n: (2)
a
s
n
=
f
Spa
g
s
n
Spa
·
f
Spe
g
s
n
Spe
=
1
2
π
σ
spa
exp
⁡
-
L
s
-
L
n
2
σ
spa
2
∗
1
2
π
σ
spe
exp
⁡
-
G
s
-
G
n
2
σ
spe
2
∝
exp
⁡
-
L
s
-
L
n
2
σ
spa
2
-
G
s
-
G
n
2
σ
spe
2
,where L
s is the space location of w
s, L
n is the space location of w
s
n, G
s is the spectral location of w
s, G
n is the spectral location of w
s
n, σ
s
p
a is a space decay parameter, σ
s
p
e is a decay spectral decay parameter, and σ
s
p
a
>
0 and σ
s
p
e
>
0.


## 2.2. ssMedLDA

Letw
s
→
=
{
w
s
1
,
w
s
2
,
…
,
w
s
m
} be a vector of m pixels appearing in the sth image object O
s in the remote sensing image, and let K be the number of latent topics in the model. The vector of the response of discrete variable y
=
{
y
s
∣
s
∈
S
,
y
s
∈
(
1,2
,
…
,
B
)
} in the remote sensing image is the geoobject class labels, and B is the number of class labels. Different from the model of the MedLDA, the variable y
s of the geoobject class label for the central pixels w
s
c in the objects O
s is shown in Figure 2. The half hashed pattern denotes that the class node is partially observed. The generative process of the ssMedLDA model is as follows:(1)
Sample topic proportionsθ
∣
α
~
D
i
r
i
c
h
l
e
t
(
α
).
(2)
For each of then pixels w
s
n in sth image object O
s,
(a)
sample a topic assignmentz
s
n
∣
θ
~
M
u
l
t
i
n
o
m
i
a
l
(
θ
);
(b)
sample a wordw
s
n from P
(
w
s
n
∣
z
s
n
,
β
), a multinomial probability conditioned on z
s
n, namely, w
s
n
∣
z
s
n
,
β
1
:
K
~
M
u
l
t
i
-
G
a
u
s
s
i
a
n
d
i
s
t
r
i
b
u
t
i
o
n
(
β
z
n
).
(3)
Sampley
s
~
N
(
η
T
z
s
-
), where z
s
- is a vector with element z
s
k
-
=
∑
n
=
1
m
a
s
n
·
I
(
z
s
n
k
=
1
)
, and I
(
·
) is an indicator function that equals 1 when predicate holds; otherwise it is 0 in the image object O
s.Figure 2
Graphic model. The shaded nodes including the pixel valuew
s
n and the set of distance coefficients in the sth image object a
s
→ represent the observed random variables, and the hashed fill node including y
s denotes the partially observed random variable class. The intuitional explanations of the model are shown at the end of the red arrows.The distance coefficienta
s
n is integrated into ssMedLDA through conditional topic random fields [20] as shown in Figure 2.


## 2.3. Algorithm

As shown in Figure3, Gibbs sampling is used to estimate the model inference and approximate the posterior distribution of latent variables in the model. The algorithm of ssMedLDA can be summarized as follows.Figure 3
Algorithm of the ssMedLDA.Step 1 (initialization).
In this step, the parameters in the model are initialized. Two types of parameters must be set, including the number of latent topics,K; the Dirichlet hyperparameter, α; the neighborhood size, h; the positive regularization parameter, c; and the cost of making a wrong prediction, l. In addition, the matrix of topicszis initialized by assigning random topics for all pixels.Step 2 (inference).
In the ssMedLDA, topics are learned in two ways based on whether the supervised information of the central pixel is available or not.
(a) When the central pixelw
s
c in sth image object O
s is unlabeled, analogous to the deduction in previous studies [5, 21], the conditional distribution of z
s
c
k is given by the following:(3)
q
z
s
c
k
∣
z
¬
s
c
,
w
s
c
∝
p
w
s
c
∣
μ
k
,
Σ
k
C
s
,
¬
s
c
k
+
α
k
,where (
μ
k
,
Σ
k
) are parameters of the kth Gaussian distribution. C
s
,
¬
s
c
k is the number of times that terms being associated with topic k within the object O
s are without z
s
c; and α
k is the value of the Dirichlet hyperparameter associated with topic k.
(b) When the central pixelw
s
c is labeled, analogous to the deduction previously presented [17], the formulas of the conditional posterior of z
s
c
k change to the following:(4)
q
z
s
c
k
∣
z
¬
s
c
,
w
s
c
,
η
,
λ
∝
p
w
s
c
∣
μ
k
,
Σ
k
C
s
,
¬
s
c
k
+
α
k
·
∏
i
=
1
L
exp
⁡
a
s
c
y
s
i
c
l
+
λ
s
i
η
i
k
λ
s
i
-
c
2
a
s
c
2
η
i
k
2
+
2
a
s
c
η
i
k
Λ
s
c
i
2
λ
s
i
,where Λ
s
c
i
=
1
/
m
-
1
Σ
k
′
η
i
k
′
C
s
,
¬
s
c
k
′, ∀
a
s
n
≥
0, ∑
n
m
a
s
n
=
1, λ is the augmented variables, η is the classification model, and y
s
i equals 1 when y
s
=
i; otherwise it is equivalent to −1; L is the number of classes of interest.
The augmented variablesλ can be given by the following:(5)
p
λ
s
i
-
1
∣
z
,
η
=
I
ς
λ
s
i
-
1
;
1
c
ξ
s
i
,
1
,where I
ς indicates the inverse Gaussian, λ
s
i
-
1 obeys the inverse Gaussian, and ξ
s
i
=
l
-
y
s
i
η
T
z
s
-.
The linear discriminative functionη can be given by the following:(6)
q
η
i
∣
z
,
λ
∝
N
η
;
μ
,
Σ
,where μ
i
=
Σ
i
(
c
∑
s
=
1
S
′
y
s
i
c
l
+
λ
s
i
/
λ
s
i
z
s
-
), Σ
i
=
1
/
v
2
I
+
c
2
∑
s
=
1
S
′
z
s
-
z
s
-
T
/
λ
s
i
-
1, and S
′ indicates the number of labeled pixels.Step 3 (classification).
Each unlabeled pixely
s is classified as a user-provided geoobject class label with the maximum posterior probability(7)
y
s
=
argmax
i
=
1
,
…
,
L
⁡
η
i
T
z
s
-
.


## 3. Experimental Result and Discussion

In this section, the experimental data and the quantitative evaluation methods for the experimental results are described. The performance of the ssMedLDA is then compared with that of the pixel-based SVM, spectral-spatial SVM, and ssLDA methods. Finally the influence of different sizes of labeled pixels on the methods is analyzed. The proposed ssMedLDA algorithm and other methods are coded and implemented in a MATLAB environment.


### 3.1. Experimental Data

The VHR data was collected by the ROSIS optical sensor over the urban area of the University of Pavia, Italy. The image data with a size of 610 × 340 pixels and a high spatial resolution of 1.3 m/pixel was used in the experiment. Figure4(a) shows a color composite of the image, whereas Figure 4(b) presents the ground truth map, which includes nine geoobject classes of interest, that is, asphalt, meadows, gravel, trees, metal sheets, bare soil, bitumen, bricks, and shadows, for which each geoclass is represented by a color.Figure 4
The initial VHR image and its ground truth.
(a)
Remote sensing image
(b)
Ground truth


### 3.2. Comparison with Existing Methods and Parameters Setting

To evaluate the effectiveness of the proposed method, the performance of the ssMedLDA was compared to three classification methods: (1) the pixel-based SVM; (2) the spectral-spatial SVM: pixel-based SVM classification was generated and a majority voting within the neighborhoods was defined by image segmentation (termed as SVM + MV) [22]; and (3) the ssLDA which is the supervised probabilistic topic models that used the pixel-level supervised information. The segmentation map for the three existing classification methods used the entropy rate superpixel segmentation algorithm, which is effective and efficient [23]. The optimal number of segments D
=
4000 was experimentally derived.In the two SVMs, a multiclass one-versus-one SVM classification with a RBF kernel from a LIBSVM was used [24] and the other parameters were determined by parameter optimization. In the ssLDA, the number of latent topics K was 80 and the Dirichlet hyperparameter α was 1 + 50/K. In the ssMedLDA, the parameters K and α were identical to the ssLDA. The positive regularization parameter c and the cost of making a wrong prediction l were 1, the number of classes of interests L is 9 according to the ground truth, the neighborhood size h was 11, the space decay parameter σ
s
p
a was 5, and spectral decay parameter σ
s
p
e was the variance of the VHR image.In total, 10% of the labeled pixels were selected for the four methods at equal interval for training purposes and the remaining 90% of labeled pixels were set aside for testing purposes. The corresponding classification maps of the 10% labeled pixels are shown in Figures5(a)–5(d). The ssMedLDA achieves a more compact and smoother classification result when compared with the SVM and ssLDA. The two visual results from SVM + MV and ssMedLDA look similar, but the latter does not require the segmentation map. Therefore, the advantages of integrating the max-margin principle, topic modeling, and pixel supervised information influence on its neighborhood based on distance consistency are confirmed.Figure 5
Classification results in the 10% labeled pixels condition for training.
(a)
SVM 10%
(b)
SVM + MV 10%
(c)
ssLDA 10%
(d)
ssMedLDA 10%Table1 displays the class-specific (producer accuracies), overall accuracies, and kappa coefficients for all methods and shows that the ssMedLDA method yields the best overall accuracy and kappa coefficients. The producer accuracies of asphalt, meadows, gravel, trees, metal sheets, bare soil, bitumen, and bricks are better than other methods. The overall accuracies and kappa coefficients from SVM + MV are better than SVM, but the producer accuracies of the shadows are not better because of the segmentation map. Therefore the influence of the segmentation on the classification is not always good.Table 1
Comparison of the each class accuracy of four classifying results.
Class
SVM
SVM + MV
ssLDA
ssMedLDA
Asphalt
92.504901%
95.370231%
97.602172%
98.386367%
Meadows
98.219744%
99.892756%
98.847123%
99.941016%
Gravel
66.22201%
72.748928%
64.173416%
72.987137%
Trees
90.013055%
81.070496%
85.411227%
96.083551%
Metal sheets
99.107807%
97.32342%
98.66171%
100%
Bare soil
62.278783%
64.028634%
78.146749%
94.690793%
Bitumen
83.308271%
96.842105%
99.097744%
99.172932%
Bricks
89.788159%
97.311244%
98.777838%
98.859316%
Shadows
100%
98.944034%
96.937698%
99.788807%
OA
88.6972%
90.9729%
92.7887
97.0723%
Kappa
0.8625
0.8896
0.9130
0.9649


### 3.3. Influence of Different Sizes of Labeled Pixels

To explore how the performance of the ssMedLDA and other object-based models behaves with different settings of the number of labeled pixels, different proportions of the ground truth pixels for training, that is, approximately 1.0%, 2.0%, 4.0%, 5.9%, 7.7%, 10.0%, 12.5%, 14.3%, 16.7%, 20.0%, and 25.0%, are conducted. Labeled training pixels for each class were acquired at equal interval from the ground truth, and the remaining labeled pixels of the ground truth were used for testing. The identical labeled pixels were used to train the SVM + MV, ssLDA, and ssMedLDA. All of the unlabeled pixels are used for training in the ssLDA and ssMedLDA. Figure6 shows the overall accuracies of the SVM + MV, ssLDA, and ssMedLDA against the proportions of labeled training pixels. Two obvious results are noted: (1) regardless of the size of labeled pixels, the ssMedLDA method achieves a better performance than the ssLDA and SVM + MV, because the ssMedLDA integrates the mechanisms of probabilistic topic models, maximum entropy discrimination, semisupervised learning, and spatial coherence; and (2) the performance of the ssMedLDA, SVM + MV, and ssLDA is not always enhanced by increasing the portion of labeled pixels, because the samples are chosen at equal intervals, which means samples of the 4% condition do not completely contain the 2% samples.Figure 6
Overall accuracy versus the proportion of labeled pixels.


## 3.1. Experimental Data

The VHR data was collected by the ROSIS optical sensor over the urban area of the University of Pavia, Italy. The image data with a size of 610 × 340 pixels and a high spatial resolution of 1.3 m/pixel was used in the experiment. Figure4(a) shows a color composite of the image, whereas Figure 4(b) presents the ground truth map, which includes nine geoobject classes of interest, that is, asphalt, meadows, gravel, trees, metal sheets, bare soil, bitumen, bricks, and shadows, for which each geoclass is represented by a color.Figure 4
The initial VHR image and its ground truth.
(a)
Remote sensing image
(b)
Ground truth


## 3.2. Comparison with Existing Methods and Parameters Setting

To evaluate the effectiveness of the proposed method, the performance of the ssMedLDA was compared to three classification methods: (1) the pixel-based SVM; (2) the spectral-spatial SVM: pixel-based SVM classification was generated and a majority voting within the neighborhoods was defined by image segmentation (termed as SVM + MV) [22]; and (3) the ssLDA which is the supervised probabilistic topic models that used the pixel-level supervised information. The segmentation map for the three existing classification methods used the entropy rate superpixel segmentation algorithm, which is effective and efficient [23]. The optimal number of segments D
=
4000 was experimentally derived.In the two SVMs, a multiclass one-versus-one SVM classification with a RBF kernel from a LIBSVM was used [24] and the other parameters were determined by parameter optimization. In the ssLDA, the number of latent topics K was 80 and the Dirichlet hyperparameter α was 1 + 50/K. In the ssMedLDA, the parameters K and α were identical to the ssLDA. The positive regularization parameter c and the cost of making a wrong prediction l were 1, the number of classes of interests L is 9 according to the ground truth, the neighborhood size h was 11, the space decay parameter σ
s
p
a was 5, and spectral decay parameter σ
s
p
e was the variance of the VHR image.In total, 10% of the labeled pixels were selected for the four methods at equal interval for training purposes and the remaining 90% of labeled pixels were set aside for testing purposes. The corresponding classification maps of the 10% labeled pixels are shown in Figures5(a)–5(d). The ssMedLDA achieves a more compact and smoother classification result when compared with the SVM and ssLDA. The two visual results from SVM + MV and ssMedLDA look similar, but the latter does not require the segmentation map. Therefore, the advantages of integrating the max-margin principle, topic modeling, and pixel supervised information influence on its neighborhood based on distance consistency are confirmed.Figure 5
Classification results in the 10% labeled pixels condition for training.
(a)
SVM 10%
(b)
SVM + MV 10%
(c)
ssLDA 10%
(d)
ssMedLDA 10%Table1 displays the class-specific (producer accuracies), overall accuracies, and kappa coefficients for all methods and shows that the ssMedLDA method yields the best overall accuracy and kappa coefficients. The producer accuracies of asphalt, meadows, gravel, trees, metal sheets, bare soil, bitumen, and bricks are better than other methods. The overall accuracies and kappa coefficients from SVM + MV are better than SVM, but the producer accuracies of the shadows are not better because of the segmentation map. Therefore the influence of the segmentation on the classification is not always good.Table 1
Comparison of the each class accuracy of four classifying results.
Class
SVM
SVM + MV
ssLDA
ssMedLDA
Asphalt
92.504901%
95.370231%
97.602172%
98.386367%
Meadows
98.219744%
99.892756%
98.847123%
99.941016%
Gravel
66.22201%
72.748928%
64.173416%
72.987137%
Trees
90.013055%
81.070496%
85.411227%
96.083551%
Metal sheets
99.107807%
97.32342%
98.66171%
100%
Bare soil
62.278783%
64.028634%
78.146749%
94.690793%
Bitumen
83.308271%
96.842105%
99.097744%
99.172932%
Bricks
89.788159%
97.311244%
98.777838%
98.859316%
Shadows
100%
98.944034%
96.937698%
99.788807%
OA
88.6972%
90.9729%
92.7887
97.0723%
Kappa
0.8625
0.8896
0.9130
0.9649


## 3.3. Influence of Different Sizes of Labeled Pixels

To explore how the performance of the ssMedLDA and other object-based models behaves with different settings of the number of labeled pixels, different proportions of the ground truth pixels for training, that is, approximately 1.0%, 2.0%, 4.0%, 5.9%, 7.7%, 10.0%, 12.5%, 14.3%, 16.7%, 20.0%, and 25.0%, are conducted. Labeled training pixels for each class were acquired at equal interval from the ground truth, and the remaining labeled pixels of the ground truth were used for testing. The identical labeled pixels were used to train the SVM + MV, ssLDA, and ssMedLDA. All of the unlabeled pixels are used for training in the ssLDA and ssMedLDA. Figure6 shows the overall accuracies of the SVM + MV, ssLDA, and ssMedLDA against the proportions of labeled training pixels. Two obvious results are noted: (1) regardless of the size of labeled pixels, the ssMedLDA method achieves a better performance than the ssLDA and SVM + MV, because the ssMedLDA integrates the mechanisms of probabilistic topic models, maximum entropy discrimination, semisupervised learning, and spatial coherence; and (2) the performance of the ssMedLDA, SVM + MV, and ssLDA is not always enhanced by increasing the portion of labeled pixels, because the samples are chosen at equal intervals, which means samples of the 4% condition do not completely contain the 2% samples.Figure 6
Overall accuracy versus the proportion of labeled pixels.


## 4. Conclusions

In this paper, a semisupervised method has been proposed to address the problem of VHR remote sensing image classification. The method combines the MedLDA model and bilateral filter through conditional topic random fields for training. The proposed method takes advantage of spatial and spectral relationships in VHR images. Additionally, the ssMedLDA does not require a segmentation map, and the pixels with their neighborhoods are used as objects to enforce spatial regularization over the classification results. The experimental results show that the proposed approach is superior to the SVM + MV and ssLDA methods.



---

*Source: 182439-2015-11-15.xml*