# A Novel Approach to Developing a Supervised Spatial Decision Support System for Image Classification: A Study of Paddy Rice Investigation

Authors: Shih-Hsun Chang
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

Paddy rice area estimation via remote sensing techniques has been well established in recent years. Texture information and vegetation indicators are widely used to improve the classification accuracy of satellite images. Accordingly, this study employs texture information and vegetation indicators as ancillary information for classifying paddy rice through remote sensing images. In the first stage, the images are attained using a remote sensing technique and ancillary information is employed to increase the accuracy of classification. In the second stage, we decide to construct an efficient supervised classifier, which is used to evaluate the ancillary information. In the third stage, linear discriminant analysis (LDA) is introduced. LDA is a well-known method for classifying images to various categories. Also, the particle swarm optimization (PSO) algorithm is employed to optimize the LDA classification outcomes and increase classification performance. In the fourth stage, we discuss the strategy of selecting different window sizes and analyze particle numbers and iteration numbers with corresponding accuracy. Accordingly, a rational strategy for the combination of ancillary information is introduced. Afterwards, the PSO algorithm improves the accuracy rate from 82.26% to 89.31%. The improved accuracy results in a much lower salt-and-pepper effect in the thematic map.


---

## Body

## 1. Introduction

Paddy rice is the major food crop in Taiwan. The main contributions of this crop in Taiwan include regional ecofriendly environment, flood control, and improvement of air quality. Food shortage has become a serious issue for many countries. The estimation of crop area is important because this information is related to national food policy. Therefore, developing a fast and accurate method for estimating crop area is desirable. With the progress of spatial data survey techniques in the geosciences, massive data or information can be easily collected and monitored. Thus, the collection of influencing variables of investigated target category becomes complicated. Advancements in applying spatial data technology have led to the effective approaches in the measurement of given categories from predictive models to the actual or practical remote sensing data. For instance, searching a target category in image classification must rely on governing ancillary attributes with specified rules.On the other hand, paddy cultivation draws the attention of governments around the world to problems caused by food shortages. The evaluation of paddy cultivation area may become a crucial problem in the near future. Rice is one of the major crops cultivated in Taiwan, and the Agriculture and Food Agency of Taiwan government has dedicated substantial efforts towards the estimation of cultivated areas and corresponding harvests. One of the best possible solutions is to use satellite image data to precisely handle the management of paddy rice area [1]. Moreover, numerous studies have endeavored to construct target GIS maps by means of remote sensing image classification [2, 3].Geographic Information System (GIS) is an extensively used tool for processing spatial data and displaying the results. GIS can be used to handle a variety of datasets, provides remedial measures, and aids in decision analysis. Gupta and Joshi [4] used GIS in assessing landslide hazard zones. Recently, several GIS-based approaches to assessing landslides have been reported [5, 6]. One significant advantage of GIS over traditional field examination and mapping methods is the ability to process various layers of data and comprehensively display the results of spatial assessments. In our study, GIS is used as a data-processing tool for assessing land cover categories and displaying the results.Data mining is one of the fastest growing fields in the computer industry. Specifically, classification is a task of grouping data with multiple attributes into relevant categories. Data mining has become the most valuable process for learning the implicit knowledge among datasets. The objective of classification herein is to classify the categories of paddy rice/non-paddy rice in consideration of multiple attributes of ancillary information and site conditions.Ancillary information is one of the most popular materials for assisting scientists to categorize green plants; thus, it is also suitable for our paddy rice study. This information is summarized as the following.(1)
Vegetation Indicator. This is a simple numerical indicator that is widely used to analyze remote sensing measurements, typically but not necessarily from a space platform, and to assess whether the target being observed contains live green vegetation. The NDVI, ABI, MSAVI, and RVI indicators are applied in this study. Thus, the goal of collecting these attributes is to extract the most influential attributes which lead to the best discernibility.(2)
Geostatistical Indices. One of the key factors in geostatistical modeling is the semivariogram, a function describing the spatial dependence of the spatial variable. The semivariogram has been widely used in remote sensing to determine spatial structures [7, 8]. In general, a semivariogram is employed as a tool to model the spatially varying phenomenon of natural land covers. We incorporate two semivariograms: (1) direct semivariogram and (2) semimadogram. A direct semivariogram model constructed from known physical properties is commonly used to measure general texture material in remote sensing. The semimadogram is a texture measure commonly defined as half of the average absolute difference between pairs of points separated by a given vector. Please refer to the work of Chica-Olmo and Abarca-Hernández [9] for details. In addition, this study employs gray-level cooccurrence matrices (GLCM), proposed by Haralick et al. [10]. GLCM is defined over an image as the distribution of cooccurring values of a given offset. In this study, four different GLCM values are employed as additional texture information.In general, the use of a supervised classifier should consider two crucial points: (a) proper training samples and (b) an effective learning process. As a matter of fact, engineers and scientists encounter obstacles to attain the aforementioned samples and rules. Thus, an effective knowledge classifier can lead to an interesting solution for image classification. This is the goal of this study. When applying particle swarm optimization (PSO) on LDA, three questions arise.(1)
Iteration Number. How many iteration numbers are required to obtain acceptable accuracy?(2)
Number of Particles. In PSO, the number of particles needs to be initialized before starting calculation. Hence, different numbers of particles should be tested to approach the best performance.(3)
Attribute Extraction. Different combinations of variables are used based upon PSO in which the classification error rate must be examined in each epoch. Accordingly, the error matrices are calculated with regard to a combination of variables. The fitness number of each variable should be computed through each epoch.The uncertainty of an image classification problem may be produced by deficiencies in the description of various categories and feature spaces [11, 12]. To resolve this problem, extensive studies have carried out the augmentation of ancillary information to improve classification accuracy. To improve the quality of classification results, many scientists have used supervised classifiers (MLH, ANN, and Fuzzy Classifier) to tackle image-processing problems [9, 10, 13–18]. Specifically, some justification of the use of a variogram and GLCM as texture measures for the optimization of LDA approaches would be useful as they play a key part in the procedures outlined [19]. Our solution is to develop an enhanced supervised classifier in our rice decision support system.The study is divided into four parts. The first part discusses the development of vegetation indicators and geostatistical indices for the study area. In the second part, the traditional LDA method is introduced. The third part briefly introduces the use of PSO to reduce the dimensions of the attributes in LDA (so called PSOLDA). The fourth part shows the results of a parallel analysis through the (a) LDA method and (b) PSOLDA method.


## 2. Materials and Methods

### 2.1. Study Area and Materials

An area of paddy rice (located in Taichung, Taiwan) is selected as a case study to demonstrate the plan of this research. The study area is located at Tanzi County, Taichung, Taiwan. The study region has complex categories of paddy rice, grass, bare land, buildings, asphalt roads, water bodies, and others, as shown in Figure1. Figure 2 shows a Quickbird image with 2096 × 2096 pixels representing an area of 215 ha. Figure 3 expresses the distribution of samples. It includes training samples and learning samples. A 7 by 7 moving window is used to calculate the texture values of the subsamples of the image data. The size of the moving window and the spectral bands selected to calculate the texture measures, GLCM and variogram, are determined by statistical methods. More detailed information and results are presented in Section 3. The goal of our study is to develop an effective supervised classifier that employs the above spatial variables in the decision support system.Figure 1
The study area (Tanzi County, Taichung City, Taiwan).Figure 2
Quickbird RS image of the study area.Figure 3
Sample distribution on the studied map (blue circle is paddy rice and red x is non-paddy rice).


### 2.2. Material Preprocessing and Study Steps

The inputs for our decision support classifier include five major steps:(1) executing image fusion, (2) employing ancillary information, (3) selection of windows size, (4) selecting proper training and testing datasets, and (5) developing a PSO + LDA model for comparison. These steps are described as follows.Step 1 (image fusion—combine spectrum image and panchromatic image).
The Quickbird image resolutions of the spectral bands are 2.88 m. The drawback of this resolution is that it cannot provide any adequate information for distinguishing vegetation categories such as grass and paddy rice. To attain a higher resolution of image data on the previous study material, we combine the image data with some ancillary information. In this study, we integrate a multispectral image (with a resolution of 2.88 m) with a higher spatial resolution panchromatic image (with a resolution of 0.69 m) from Quickbird by using ERDAS image software with the use of the PCA (principal component analysis) method.Step 2 (ancillary information—reinforce better classification performance).
In addition to spectral information, a series of vegetation indices are included in the building of our classifier. Furthermore, to improve the classification accuracy of land covers with close spectral measures, such as grass and paddy rice, the spatial structures measures are included. In this study, they are GLCM contrast, GLCM homogeneity, GLCM energy, GLCM entropy, a direct semivariogram, and a semimadogram. Please refer to Table2 for all the conditional attributes used in this study.Step 3 (selection of window size).
We propose an approach which resolves the problem of varying window size selection for a wide class of classifiers. Window size is considered as a variable estimation and testing a series of different window sizes can lead to a better understanding of window size selection. The texture measures were calculated for different window sizes, land covers, and spectral bands. All samples, which include various land covers, were used in the calculation to attain the mean texture values, and then they were depicted in figures for the sake of comparison. We present a number of results which demonstrate how the window size rules were selected in our study cases.Step 4 (preparing training and testing datasets).
The training dataset consists of 455 sample points, which are comprised of 135 paddy rice samples and 320 non-paddy rice samples. Please refer to Table1 for the distribution of samples with various land cover types. These data are input into our enhanced decision support system in the training process. Following this process, all of the image data are classified into two categories (paddy and non-paddy rice). The K-fold cross-validation method was applied. We used k=5 in our study, which means 80% of the sample dataset was randomly selected for training and the remaining 20% was used for validation. The value of each cell on the error matrix (Tables 3, 4, 5, 6, 7, 8, and 9) was obtained by averaging the 20 times of the aforementioned k-fold cross-validation calculation.Table 1
Number of samples distribution over various land covers.
Land cover
Paddy rice
Levee
Grass
Woods
Dryfarmland
Road
Building
Shadow
Number of samples
135
50
46
45
40
42
56
41Table 2
All conditional attributes used in this study.
Numbering
1
2
3
4
5
6
7
8
9
Attribute
R
G
B
IR
NDVI
CFMI
BR
SQBR
VI
Numbering
10
11
12
13
14
15
16
17
18
Attribute
SAVI
MSAVI
ABI
GLCM contrast
GLCM energy
GLCM homogeneity
GLCM entropy
Direct semivariogram
SemimadogramTable 3
Error matrix for all land cover types and 4 spectral bands (data is obtained by averaging 20 trials of calculation).
All land cover typesRGB + IR Kappa = 0.5950
Classification result
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
22.75
4.25
0.8426
Non-paddy rice
12.49
51.51
0.8048
User accuracy
0.6456
0.9238
Overall accuracy0.8160, std. = 0.0038Table 4
Error matrix for all land cover types and 4 spectral bands + NDVI (data is obtained by averaging 20 trials of calculation).
All land cover typesRGB + IR + NDVIKappa = 0.6238
Classification result
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
22.05
4.95
0.8167
Non-paddy rice
10.13
53.87
0.8417
User accuracy
0.6852
0.9158
Overall accuracy0.8343, std. = 0.0052Table 5
Error matrix for all land cover types and 4 spectral bands + NDVI + texture (data is obtained by averaging 20 trials of calculation).
All land cover typesRGB + IR + NDVI + textureKappa = 0.6640
Classification result
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
23.15
3.85
0.8574
Non-paddy rice
9.71
54.29
0.8483
User accuracy
0.7045
0.9338
Overall accuracy0.8501, std. = 0.0048Table 6
Non-PSO versus PSO error matrix for all land cover types.
All land cover typesNon-PSO Kappa = 0.5982PSO Kappa = 0.7510
Classification result (non-PSO/PSO)
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
21.62/23.48
5.38/3.52
0.8007/0.8696
Non-paddy rice
10.76/6.21
53.24/57.79
0.8319/0.9030
User accuracy
0.6677/0.7908
0.9082/0.9426
Overall accuracy0.8226, std. = 0.0380/overall accuracy 0.8931, std. = 0.0048Table 7
Non-PSO versus PSO error matrix for paddy rice versus grass.
Land cover: paddy rice versus grassNon-PSO Kappa = 0.4528PSO Kappa = 0.7396
Classification result (non-PSO/PSO)
Paddy rice
Grass
Producer accuracy
Ground truth class
Paddy rice
23.08/26.20
3.92/0.8
0.8548/0.9704
Grass
3.66/2.55
5.54/6.65
0.6022/0.7228
User accuracy
0.8631/0.9113
0.5856/0.8926
Overall accuracy0.7906, std. = 0.0598/overall accuracy 0.9075, std. = 0.0073Table 8
Non-PSO versus PSO error matrix for paddy rice versus levee.
Land cover: paddy rice versus leveeNon-PSO Kappa = 0.8502PSO Kappa = 0.8845
Classification result (non-PSO/PSO)
Paddy rice
Levee
Producer accuracy
Ground truth class
Paddy rice
26.11/26.11
0.89/0.89
0.9670/0.9670
Levee
1.27/0.80
8.73/9.20
0.8730/0.9200
User accuracy
0.9536/0.9703
0.9075/0.9118
Overall accuracy0.9416, std. = 0.0228/overall accuracy 0.9543, std. = 0.0320Table 9
Non-PSO versus PSO error matrix for paddy rice versus woods.
Land cover: paddy rice versus woodsNon-PSO Kappa = 0.7716PSO Kappa = 0.7716
Classification result (non-PSO/PSO)
Paddy rice
Woods
Producer accuracy
Ground truth class
Paddy rice
25.67/25.67
0.89/0.89
0.9507/0.9507
Woods
1.71/1.71
7.29/7.29
0.8100/0.8100
User accuracy
0.9375/0.9375
0.8457/0.8457
Overall accuracy0.9156, std.=0.0443/overall accuracy 0.9156, std. = 0.0443Step 5 (develop a PSO + LDA computer program).
In the present study, the weight coefficients of the LDA equation were obtained through the training dataset by using the MATLAB code we developed. This equation serves as a classification rule. This rule can be used to determine the class of land cover of each pixel in the RS image. The PSO algorithm was incorporated into the LDA code to optimize the classification outcome by selecting different attribute combinations.


### 2.3. Research Method

#### 2.3.1. Particle Swarm Optimization

Particle swarm optimization is a group intelligence optimization method proposed by Kennedy and Eberhart in 1995 [20]. This method has been successfully applied in many areas. It is inspired by bird flocking behaviors, in which a temporary destination is determined by the cognition and global direction of the entire group. In PSO, a population of particles is created and each particle is assigned with an initial position and velocity. Each particle moves to a new position in each calculation iteration with regard to the value of fitness function. The particle movement is based on individual best fitness and the group’s best fitness. Assume in a D-dimensional space that there are n particles described by X=(X1,X2,…,Xn), where Xi=(xi1,xi2,…,xiD)T denotes the position of the ith particle. The position of the particles is the potential solution in question. The velocity of the ith particle is Vi=(Vi1,Vi2,…,ViD)T. The best individual position and best global position, with regard to optimizing fitness, are Pi=(Pi1,Pi2,…,PiD)T and Pg=(Pg1,Pg2,…,PgD)T, respectively. The velocity and position of each particle are updated in each iteration with the following equations:
(1)Vidk+1=ωVidk+c1r1(Pidk-Xidk)+c2r2(Pgdk-Xidk)(2)Xidk+1=Xidk+Vidk+1,
where d=1,2,…,D; i=1,2,…,n; k is the current iteration step; ω is the inertial weight, c1 represents the cognition learning factor, c2 denotes the social learning factor, and r1 and r2 are random numbers.The basic steps of the PSO algorithm can be described as follows.Step 1: create a number of particles assigned with initial positions and velocities.Step 2: calculate the fitness of each particle.Step 3: calculate the velocity of each particle using (1).Step 4: update the position of each particle using (2).Step 5: stop the iteration process if termination criterion is met; otherwise return to Step 2 and continue the process.In this study, the PSO algorithm is used to accomplish feature selection. The fitness function is the function that returns classification accuracy through the LDA algorithm. The fitness functionJ is defined as the summation of Euclidean distance between the data points to its associated group center. Consider
(3)J=∑i=1M∑X∈ωi∥X-Xωl-∥2,
where M is the number of classes, ωi is a specific class, X is the vector of data points, and Xωl- is the center of class.The feature of each training sample acts as a position variablexid  and its value is normalized and is bound to be [0,1]. The result of the particle position after PSO process is examined and those features with xid<0.5 are discarded. Detailed illustrated examples can be found in work of Lin and Chen [21].


#### 2.3.2. Linear Discriminant Analysis (LDA)

Linear discriminant analysis (LDA) is a popular statistical method used for classification. In general, it is composed of linear discriminant equations which are obtained by definite and simple procedures. Due to the contribution of recent advances in satellite photography technology, high resolution images are now well accepted for analysis. However, uncertainty information may exist in such images, leading to the decrease of classification accuracy. It is thus expected that, with the efforts of attribute reduction and the data preprocessing of raw data, the classification accuracy of satellite images can be profoundly improved.


## 2.1. Study Area and Materials

An area of paddy rice (located in Taichung, Taiwan) is selected as a case study to demonstrate the plan of this research. The study area is located at Tanzi County, Taichung, Taiwan. The study region has complex categories of paddy rice, grass, bare land, buildings, asphalt roads, water bodies, and others, as shown in Figure1. Figure 2 shows a Quickbird image with 2096 × 2096 pixels representing an area of 215 ha. Figure 3 expresses the distribution of samples. It includes training samples and learning samples. A 7 by 7 moving window is used to calculate the texture values of the subsamples of the image data. The size of the moving window and the spectral bands selected to calculate the texture measures, GLCM and variogram, are determined by statistical methods. More detailed information and results are presented in Section 3. The goal of our study is to develop an effective supervised classifier that employs the above spatial variables in the decision support system.Figure 1
The study area (Tanzi County, Taichung City, Taiwan).Figure 2
Quickbird RS image of the study area.Figure 3
Sample distribution on the studied map (blue circle is paddy rice and red x is non-paddy rice).


## 2.2. Material Preprocessing and Study Steps

The inputs for our decision support classifier include five major steps:(1) executing image fusion, (2) employing ancillary information, (3) selection of windows size, (4) selecting proper training and testing datasets, and (5) developing a PSO + LDA model for comparison. These steps are described as follows.Step 1 (image fusion—combine spectrum image and panchromatic image).
The Quickbird image resolutions of the spectral bands are 2.88 m. The drawback of this resolution is that it cannot provide any adequate information for distinguishing vegetation categories such as grass and paddy rice. To attain a higher resolution of image data on the previous study material, we combine the image data with some ancillary information. In this study, we integrate a multispectral image (with a resolution of 2.88 m) with a higher spatial resolution panchromatic image (with a resolution of 0.69 m) from Quickbird by using ERDAS image software with the use of the PCA (principal component analysis) method.Step 2 (ancillary information—reinforce better classification performance).
In addition to spectral information, a series of vegetation indices are included in the building of our classifier. Furthermore, to improve the classification accuracy of land covers with close spectral measures, such as grass and paddy rice, the spatial structures measures are included. In this study, they are GLCM contrast, GLCM homogeneity, GLCM energy, GLCM entropy, a direct semivariogram, and a semimadogram. Please refer to Table2 for all the conditional attributes used in this study.Step 3 (selection of window size).
We propose an approach which resolves the problem of varying window size selection for a wide class of classifiers. Window size is considered as a variable estimation and testing a series of different window sizes can lead to a better understanding of window size selection. The texture measures were calculated for different window sizes, land covers, and spectral bands. All samples, which include various land covers, were used in the calculation to attain the mean texture values, and then they were depicted in figures for the sake of comparison. We present a number of results which demonstrate how the window size rules were selected in our study cases.Step 4 (preparing training and testing datasets).
The training dataset consists of 455 sample points, which are comprised of 135 paddy rice samples and 320 non-paddy rice samples. Please refer to Table1 for the distribution of samples with various land cover types. These data are input into our enhanced decision support system in the training process. Following this process, all of the image data are classified into two categories (paddy and non-paddy rice). The K-fold cross-validation method was applied. We used k=5 in our study, which means 80% of the sample dataset was randomly selected for training and the remaining 20% was used for validation. The value of each cell on the error matrix (Tables 3, 4, 5, 6, 7, 8, and 9) was obtained by averaging the 20 times of the aforementioned k-fold cross-validation calculation.Table 1
Number of samples distribution over various land covers.
Land cover
Paddy rice
Levee
Grass
Woods
Dryfarmland
Road
Building
Shadow
Number of samples
135
50
46
45
40
42
56
41Table 2
All conditional attributes used in this study.
Numbering
1
2
3
4
5
6
7
8
9
Attribute
R
G
B
IR
NDVI
CFMI
BR
SQBR
VI
Numbering
10
11
12
13
14
15
16
17
18
Attribute
SAVI
MSAVI
ABI
GLCM contrast
GLCM energy
GLCM homogeneity
GLCM entropy
Direct semivariogram
SemimadogramTable 3
Error matrix for all land cover types and 4 spectral bands (data is obtained by averaging 20 trials of calculation).
All land cover typesRGB + IR Kappa = 0.5950
Classification result
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
22.75
4.25
0.8426
Non-paddy rice
12.49
51.51
0.8048
User accuracy
0.6456
0.9238
Overall accuracy0.8160, std. = 0.0038Table 4
Error matrix for all land cover types and 4 spectral bands + NDVI (data is obtained by averaging 20 trials of calculation).
All land cover typesRGB + IR + NDVIKappa = 0.6238
Classification result
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
22.05
4.95
0.8167
Non-paddy rice
10.13
53.87
0.8417
User accuracy
0.6852
0.9158
Overall accuracy0.8343, std. = 0.0052Table 5
Error matrix for all land cover types and 4 spectral bands + NDVI + texture (data is obtained by averaging 20 trials of calculation).
All land cover typesRGB + IR + NDVI + textureKappa = 0.6640
Classification result
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
23.15
3.85
0.8574
Non-paddy rice
9.71
54.29
0.8483
User accuracy
0.7045
0.9338
Overall accuracy0.8501, std. = 0.0048Table 6
Non-PSO versus PSO error matrix for all land cover types.
All land cover typesNon-PSO Kappa = 0.5982PSO Kappa = 0.7510
Classification result (non-PSO/PSO)
Paddy rice
Non-paddy rice
Producer accuracy
Ground truth class
Paddy rice
21.62/23.48
5.38/3.52
0.8007/0.8696
Non-paddy rice
10.76/6.21
53.24/57.79
0.8319/0.9030
User accuracy
0.6677/0.7908
0.9082/0.9426
Overall accuracy0.8226, std. = 0.0380/overall accuracy 0.8931, std. = 0.0048Table 7
Non-PSO versus PSO error matrix for paddy rice versus grass.
Land cover: paddy rice versus grassNon-PSO Kappa = 0.4528PSO Kappa = 0.7396
Classification result (non-PSO/PSO)
Paddy rice
Grass
Producer accuracy
Ground truth class
Paddy rice
23.08/26.20
3.92/0.8
0.8548/0.9704
Grass
3.66/2.55
5.54/6.65
0.6022/0.7228
User accuracy
0.8631/0.9113
0.5856/0.8926
Overall accuracy0.7906, std. = 0.0598/overall accuracy 0.9075, std. = 0.0073Table 8
Non-PSO versus PSO error matrix for paddy rice versus levee.
Land cover: paddy rice versus leveeNon-PSO Kappa = 0.8502PSO Kappa = 0.8845
Classification result (non-PSO/PSO)
Paddy rice
Levee
Producer accuracy
Ground truth class
Paddy rice
26.11/26.11
0.89/0.89
0.9670/0.9670
Levee
1.27/0.80
8.73/9.20
0.8730/0.9200
User accuracy
0.9536/0.9703
0.9075/0.9118
Overall accuracy0.9416, std. = 0.0228/overall accuracy 0.9543, std. = 0.0320Table 9
Non-PSO versus PSO error matrix for paddy rice versus woods.
Land cover: paddy rice versus woodsNon-PSO Kappa = 0.7716PSO Kappa = 0.7716
Classification result (non-PSO/PSO)
Paddy rice
Woods
Producer accuracy
Ground truth class
Paddy rice
25.67/25.67
0.89/0.89
0.9507/0.9507
Woods
1.71/1.71
7.29/7.29
0.8100/0.8100
User accuracy
0.9375/0.9375
0.8457/0.8457
Overall accuracy0.9156, std.=0.0443/overall accuracy 0.9156, std. = 0.0443Step 5 (develop a PSO + LDA computer program).
In the present study, the weight coefficients of the LDA equation were obtained through the training dataset by using the MATLAB code we developed. This equation serves as a classification rule. This rule can be used to determine the class of land cover of each pixel in the RS image. The PSO algorithm was incorporated into the LDA code to optimize the classification outcome by selecting different attribute combinations.


## 2.3. Research Method

### 2.3.1. Particle Swarm Optimization

Particle swarm optimization is a group intelligence optimization method proposed by Kennedy and Eberhart in 1995 [20]. This method has been successfully applied in many areas. It is inspired by bird flocking behaviors, in which a temporary destination is determined by the cognition and global direction of the entire group. In PSO, a population of particles is created and each particle is assigned with an initial position and velocity. Each particle moves to a new position in each calculation iteration with regard to the value of fitness function. The particle movement is based on individual best fitness and the group’s best fitness. Assume in a D-dimensional space that there are n particles described by X=(X1,X2,…,Xn), where Xi=(xi1,xi2,…,xiD)T denotes the position of the ith particle. The position of the particles is the potential solution in question. The velocity of the ith particle is Vi=(Vi1,Vi2,…,ViD)T. The best individual position and best global position, with regard to optimizing fitness, are Pi=(Pi1,Pi2,…,PiD)T and Pg=(Pg1,Pg2,…,PgD)T, respectively. The velocity and position of each particle are updated in each iteration with the following equations:
(1)Vidk+1=ωVidk+c1r1(Pidk-Xidk)+c2r2(Pgdk-Xidk)(2)Xidk+1=Xidk+Vidk+1,
where d=1,2,…,D; i=1,2,…,n; k is the current iteration step; ω is the inertial weight, c1 represents the cognition learning factor, c2 denotes the social learning factor, and r1 and r2 are random numbers.The basic steps of the PSO algorithm can be described as follows.Step 1: create a number of particles assigned with initial positions and velocities.Step 2: calculate the fitness of each particle.Step 3: calculate the velocity of each particle using (1).Step 4: update the position of each particle using (2).Step 5: stop the iteration process if termination criterion is met; otherwise return to Step 2 and continue the process.In this study, the PSO algorithm is used to accomplish feature selection. The fitness function is the function that returns classification accuracy through the LDA algorithm. The fitness functionJ is defined as the summation of Euclidean distance between the data points to its associated group center. Consider
(3)J=∑i=1M∑X∈ωi∥X-Xωl-∥2,
where M is the number of classes, ωi is a specific class, X is the vector of data points, and Xωl- is the center of class.The feature of each training sample acts as a position variablexid  and its value is normalized and is bound to be [0,1]. The result of the particle position after PSO process is examined and those features with xid<0.5 are discarded. Detailed illustrated examples can be found in work of Lin and Chen [21].


### 2.3.2. Linear Discriminant Analysis (LDA)

Linear discriminant analysis (LDA) is a popular statistical method used for classification. In general, it is composed of linear discriminant equations which are obtained by definite and simple procedures. Due to the contribution of recent advances in satellite photography technology, high resolution images are now well accepted for analysis. However, uncertainty information may exist in such images, leading to the decrease of classification accuracy. It is thus expected that, with the efforts of attribute reduction and the data preprocessing of raw data, the classification accuracy of satellite images can be profoundly improved.


## 2.3.1. Particle Swarm Optimization

Particle swarm optimization is a group intelligence optimization method proposed by Kennedy and Eberhart in 1995 [20]. This method has been successfully applied in many areas. It is inspired by bird flocking behaviors, in which a temporary destination is determined by the cognition and global direction of the entire group. In PSO, a population of particles is created and each particle is assigned with an initial position and velocity. Each particle moves to a new position in each calculation iteration with regard to the value of fitness function. The particle movement is based on individual best fitness and the group’s best fitness. Assume in a D-dimensional space that there are n particles described by X=(X1,X2,…,Xn), where Xi=(xi1,xi2,…,xiD)T denotes the position of the ith particle. The position of the particles is the potential solution in question. The velocity of the ith particle is Vi=(Vi1,Vi2,…,ViD)T. The best individual position and best global position, with regard to optimizing fitness, are Pi=(Pi1,Pi2,…,PiD)T and Pg=(Pg1,Pg2,…,PgD)T, respectively. The velocity and position of each particle are updated in each iteration with the following equations:
(1)Vidk+1=ωVidk+c1r1(Pidk-Xidk)+c2r2(Pgdk-Xidk)(2)Xidk+1=Xidk+Vidk+1,
where d=1,2,…,D; i=1,2,…,n; k is the current iteration step; ω is the inertial weight, c1 represents the cognition learning factor, c2 denotes the social learning factor, and r1 and r2 are random numbers.The basic steps of the PSO algorithm can be described as follows.Step 1: create a number of particles assigned with initial positions and velocities.Step 2: calculate the fitness of each particle.Step 3: calculate the velocity of each particle using (1).Step 4: update the position of each particle using (2).Step 5: stop the iteration process if termination criterion is met; otherwise return to Step 2 and continue the process.In this study, the PSO algorithm is used to accomplish feature selection. The fitness function is the function that returns classification accuracy through the LDA algorithm. The fitness functionJ is defined as the summation of Euclidean distance between the data points to its associated group center. Consider
(3)J=∑i=1M∑X∈ωi∥X-Xωl-∥2,
where M is the number of classes, ωi is a specific class, X is the vector of data points, and Xωl- is the center of class.The feature of each training sample acts as a position variablexid  and its value is normalized and is bound to be [0,1]. The result of the particle position after PSO process is examined and those features with xid<0.5 are discarded. Detailed illustrated examples can be found in work of Lin and Chen [21].


## 2.3.2. Linear Discriminant Analysis (LDA)

Linear discriminant analysis (LDA) is a popular statistical method used for classification. In general, it is composed of linear discriminant equations which are obtained by definite and simple procedures. Due to the contribution of recent advances in satellite photography technology, high resolution images are now well accepted for analysis. However, uncertainty information may exist in such images, leading to the decrease of classification accuracy. It is thus expected that, with the efforts of attribute reduction and the data preprocessing of raw data, the classification accuracy of satellite images can be profoundly improved.


## 3. Results and Discussions

### 3.1. Selecting Window Sizes and Spectral Bands When Calculating Texture Information

In this study, GLCM and semivariogram texture information are a part of the condition attributes. To attain these data, it is required to determine the size of the moving window and which spectral band should be used for calculation. Four GLCM attributes, including contrast, homogeneity, energy, and entropy, are calculated for different window sizes at each sample pixel. Their mean values are obtained by averaging the samples of their corresponding land covers. The GLCM versus window size distributions (spectral band R and IR) are shown in Figures4 and 5, respectively. The x-axis is the window size and the y-axis is the value of GLCM texture information. Under different curves (land covers), the larger the separation between curves, the better the discernibility rate. It is seen from the figures that the IR band has better discernibility than does the R band. Paddy rice and grass are types of land cover that are close in spectral distributions and thus are difficult to classify. For the paddy rice field and grass, the IR band obviously depicts higher discernibility. The distributions for the B and G bands were also obtained. However, they do not present better discernibility than the IR band does and are, thus, not shown in the figures. Accordingly, the IR band is selected for calculating GLCM texture information in our study. Similar distributions for semivariogram texture cases are shown in Figures 6 and 7. In this case, it is depicted that the R band depicts better discernibility. Therefore, the R band is selected for calculating semivariogram texture information in our study. As seen in Figures 4–7, although the larger window size cases tend to have better discernibility, they may include more uncertainties and probably enclose pixels of other land covers. Accordingly, we decide to select window size 7 in our study. In summary, the IR band is used for calculating GLCM, the R band is used for calculating semivariogram, and the window size is 7.GLCM versus window size for different land covers (band: R).
(a)
(b)
(c)
(d)GLCM versus window size for different land covers (band: IR).
(a)
(b)
(c)
(d)Semivariogram versus window size for different land covers (band: R).
(a)
(b)Semivariogram versus window size for different land covers (band: IR).
(a)
(b)


### 3.2. Effects of Ancillary Attributes

To study the effectiveness of ancillary attributes, NDVI and texture information (GLCM and semivariogram) and 3 different conditional attribute combinations are used to obtain the classification outcomes. Table3 presents the error matrix with only 4 spectral bands as conditional attributes; Table 4 adds NDVI as an additional conditional attribute; Table 5 further adds texture information (4 GLCM data and 2 semivariogram data) as additional attributes. All land cover types are included in the above calculation. Comparing Tables 3, 4, and 5, it is seen that, with the inclusion of ancillary attributes, the classification outcome improves. The overall accuracy rates are increased from 81.60% to 83.43% with NDVI included and to 85.01% with texture information included.


### 3.3. PSO Attribute Reduction with LDA

#### 3.3.1. Determining the Number of Particles and Maximal Epochs in PSOLDA

This study incorporates PSO with LDA as an optimization tool to find the best combination of conditional attributes. This kind of approach is generally referred to as an attribute extraction process in data mining. PSO is an iterative calculation process in which it is necessary to set up initial conditions. The inertial weightω is set to 1.0 and the cognition learning factor and social learning factor, c1 and c2, are both set to 0.8. However, two other initial conditions, maximal epoch number and number of particles, must also be determined. Figure 8 depicts the iteration evolutions for various maximal epoch numbers (the particle number is fixed at 30). Figure 8 shows that a higher maximal epoch does not always lead to a lower classification error rate. Figure 9 depicts the iteration evolutions for different numbers of particles. Similarly, a higher number of particles set up in initial condition do not always lead to a lower classification error rate. The classification error rate always varies at different discrete values. This is due to the fact that the nature of the problem in study is an optimal combination of attributes, which is inherently discrete. Considering the balance between classifier optimization and additional required computing time, the maximal epoch is set to 400 and the number of particles is set to 40 in the rest of the study.Figure 8
Effect of maximal epoch in PSOLDA iteration evolution.Figure 9
Effect of particle number in PSOLDA iteration evolution.


#### 3.3.2. Classification Outcome with and without PSO Attribute Reduction

PSO is used as a dimension reduction technique on LDA equations in this study. To compare the results of the PSOLDA classifier with those obtained with the LDA classifier only, 4 different initial conditions are studied, including all land cover types, paddy rice field versus grass, paddy rice field versus levee, and paddy rice versus woods. The number of conditional attributes, listed in Table2, before applying PSO is 18 and is reduced to a smaller number, 15, after applying PSO. Tables 6–9 present the error matrices of non-PSO and PSO cases with different land covers. The bold face numbers are obtained through PSO. It is clear from Tables 6–8 that, with attribute reduction incorporating PSO, the classification outcomes are improved. However, as seen in Table 9, in the paddy rice versus woods case, no attributes are eliminated after incorporating PSO. The overall land cover case shows an 8.57% accuracy improvement, and the case of paddy rice versus grass reveals 14.78% improvement. The eliminated attributes are summarized in Table 10. It is noted that some of the attributes used in this study are correlated; therefore, the dropped attributes could still be influential in classifying land covers. PSO here serves to optimize the classification outcome by employing different combinations of attributes. For various correlated attributes, such as NDVI versus R and IR, it is possible that either one of them could be extracted or eliminated under the PSO process.Table 10
Dropped attributes after PSO process.
Land cover
All land cover types
Paddy rice versus  grass
Paddy rice versus  levee
Paddy rice versus  woods
Dropped attributes
IR, VI, ABI
R, G, NDVI, CMFI, SQBR,  SAVI, MSAVI, ABI,  GLCM energy,  GLCM homogeneity,  direct semivariogram
BR, VI, MSAVI, GLCM contrast,  GLCM energy
None


### 3.4. Thematic Map Comparison of Non-PSO versus PSO

A thematic map is useful for visually examining the performance of the developed classifier and estimating the area of paddy rice field. The classification outcome discussed in previous section presents better results by using PSOLDA as compared with using LDA alone. To further examine the benefit of incorporating PSOLDA, two thematic maps, Figures10 and 11, are generated for non-PSO and PSO, for the sake of comparison. These two figures are created by using the classifiers (LDA versus PSOLDA) through the same training datasets. Salt-and-pepper effect can be easily observed in Figure 10. Much lower salt-and-pepper effect is depicted in the PSO case, as shown in Figure 11. This is because the PSO case has a higher overall classification accuracy, which results in fewer misclassified points on the thematic map.Figure 10
Paddy rice thematic map using merely LDA.Figure 11
Paddy rice thematic map applying PSO on LDA.


## 3.1. Selecting Window Sizes and Spectral Bands When Calculating Texture Information

In this study, GLCM and semivariogram texture information are a part of the condition attributes. To attain these data, it is required to determine the size of the moving window and which spectral band should be used for calculation. Four GLCM attributes, including contrast, homogeneity, energy, and entropy, are calculated for different window sizes at each sample pixel. Their mean values are obtained by averaging the samples of their corresponding land covers. The GLCM versus window size distributions (spectral band R and IR) are shown in Figures4 and 5, respectively. The x-axis is the window size and the y-axis is the value of GLCM texture information. Under different curves (land covers), the larger the separation between curves, the better the discernibility rate. It is seen from the figures that the IR band has better discernibility than does the R band. Paddy rice and grass are types of land cover that are close in spectral distributions and thus are difficult to classify. For the paddy rice field and grass, the IR band obviously depicts higher discernibility. The distributions for the B and G bands were also obtained. However, they do not present better discernibility than the IR band does and are, thus, not shown in the figures. Accordingly, the IR band is selected for calculating GLCM texture information in our study. Similar distributions for semivariogram texture cases are shown in Figures 6 and 7. In this case, it is depicted that the R band depicts better discernibility. Therefore, the R band is selected for calculating semivariogram texture information in our study. As seen in Figures 4–7, although the larger window size cases tend to have better discernibility, they may include more uncertainties and probably enclose pixels of other land covers. Accordingly, we decide to select window size 7 in our study. In summary, the IR band is used for calculating GLCM, the R band is used for calculating semivariogram, and the window size is 7.GLCM versus window size for different land covers (band: R).
(a)
(b)
(c)
(d)GLCM versus window size for different land covers (band: IR).
(a)
(b)
(c)
(d)Semivariogram versus window size for different land covers (band: R).
(a)
(b)Semivariogram versus window size for different land covers (band: IR).
(a)
(b)


## 3.2. Effects of Ancillary Attributes

To study the effectiveness of ancillary attributes, NDVI and texture information (GLCM and semivariogram) and 3 different conditional attribute combinations are used to obtain the classification outcomes. Table3 presents the error matrix with only 4 spectral bands as conditional attributes; Table 4 adds NDVI as an additional conditional attribute; Table 5 further adds texture information (4 GLCM data and 2 semivariogram data) as additional attributes. All land cover types are included in the above calculation. Comparing Tables 3, 4, and 5, it is seen that, with the inclusion of ancillary attributes, the classification outcome improves. The overall accuracy rates are increased from 81.60% to 83.43% with NDVI included and to 85.01% with texture information included.


## 3.3. PSO Attribute Reduction with LDA

### 3.3.1. Determining the Number of Particles and Maximal Epochs in PSOLDA

This study incorporates PSO with LDA as an optimization tool to find the best combination of conditional attributes. This kind of approach is generally referred to as an attribute extraction process in data mining. PSO is an iterative calculation process in which it is necessary to set up initial conditions. The inertial weightω is set to 1.0 and the cognition learning factor and social learning factor, c1 and c2, are both set to 0.8. However, two other initial conditions, maximal epoch number and number of particles, must also be determined. Figure 8 depicts the iteration evolutions for various maximal epoch numbers (the particle number is fixed at 30). Figure 8 shows that a higher maximal epoch does not always lead to a lower classification error rate. Figure 9 depicts the iteration evolutions for different numbers of particles. Similarly, a higher number of particles set up in initial condition do not always lead to a lower classification error rate. The classification error rate always varies at different discrete values. This is due to the fact that the nature of the problem in study is an optimal combination of attributes, which is inherently discrete. Considering the balance between classifier optimization and additional required computing time, the maximal epoch is set to 400 and the number of particles is set to 40 in the rest of the study.Figure 8
Effect of maximal epoch in PSOLDA iteration evolution.Figure 9
Effect of particle number in PSOLDA iteration evolution.


### 3.3.2. Classification Outcome with and without PSO Attribute Reduction

PSO is used as a dimension reduction technique on LDA equations in this study. To compare the results of the PSOLDA classifier with those obtained with the LDA classifier only, 4 different initial conditions are studied, including all land cover types, paddy rice field versus grass, paddy rice field versus levee, and paddy rice versus woods. The number of conditional attributes, listed in Table2, before applying PSO is 18 and is reduced to a smaller number, 15, after applying PSO. Tables 6–9 present the error matrices of non-PSO and PSO cases with different land covers. The bold face numbers are obtained through PSO. It is clear from Tables 6–8 that, with attribute reduction incorporating PSO, the classification outcomes are improved. However, as seen in Table 9, in the paddy rice versus woods case, no attributes are eliminated after incorporating PSO. The overall land cover case shows an 8.57% accuracy improvement, and the case of paddy rice versus grass reveals 14.78% improvement. The eliminated attributes are summarized in Table 10. It is noted that some of the attributes used in this study are correlated; therefore, the dropped attributes could still be influential in classifying land covers. PSO here serves to optimize the classification outcome by employing different combinations of attributes. For various correlated attributes, such as NDVI versus R and IR, it is possible that either one of them could be extracted or eliminated under the PSO process.Table 10
Dropped attributes after PSO process.
Land cover
All land cover types
Paddy rice versus  grass
Paddy rice versus  levee
Paddy rice versus  woods
Dropped attributes
IR, VI, ABI
R, G, NDVI, CMFI, SQBR,  SAVI, MSAVI, ABI,  GLCM energy,  GLCM homogeneity,  direct semivariogram
BR, VI, MSAVI, GLCM contrast,  GLCM energy
None


## 3.3.1. Determining the Number of Particles and Maximal Epochs in PSOLDA

This study incorporates PSO with LDA as an optimization tool to find the best combination of conditional attributes. This kind of approach is generally referred to as an attribute extraction process in data mining. PSO is an iterative calculation process in which it is necessary to set up initial conditions. The inertial weightω is set to 1.0 and the cognition learning factor and social learning factor, c1 and c2, are both set to 0.8. However, two other initial conditions, maximal epoch number and number of particles, must also be determined. Figure 8 depicts the iteration evolutions for various maximal epoch numbers (the particle number is fixed at 30). Figure 8 shows that a higher maximal epoch does not always lead to a lower classification error rate. Figure 9 depicts the iteration evolutions for different numbers of particles. Similarly, a higher number of particles set up in initial condition do not always lead to a lower classification error rate. The classification error rate always varies at different discrete values. This is due to the fact that the nature of the problem in study is an optimal combination of attributes, which is inherently discrete. Considering the balance between classifier optimization and additional required computing time, the maximal epoch is set to 400 and the number of particles is set to 40 in the rest of the study.Figure 8
Effect of maximal epoch in PSOLDA iteration evolution.Figure 9
Effect of particle number in PSOLDA iteration evolution.


## 3.3.2. Classification Outcome with and without PSO Attribute Reduction

PSO is used as a dimension reduction technique on LDA equations in this study. To compare the results of the PSOLDA classifier with those obtained with the LDA classifier only, 4 different initial conditions are studied, including all land cover types, paddy rice field versus grass, paddy rice field versus levee, and paddy rice versus woods. The number of conditional attributes, listed in Table2, before applying PSO is 18 and is reduced to a smaller number, 15, after applying PSO. Tables 6–9 present the error matrices of non-PSO and PSO cases with different land covers. The bold face numbers are obtained through PSO. It is clear from Tables 6–8 that, with attribute reduction incorporating PSO, the classification outcomes are improved. However, as seen in Table 9, in the paddy rice versus woods case, no attributes are eliminated after incorporating PSO. The overall land cover case shows an 8.57% accuracy improvement, and the case of paddy rice versus grass reveals 14.78% improvement. The eliminated attributes are summarized in Table 10. It is noted that some of the attributes used in this study are correlated; therefore, the dropped attributes could still be influential in classifying land covers. PSO here serves to optimize the classification outcome by employing different combinations of attributes. For various correlated attributes, such as NDVI versus R and IR, it is possible that either one of them could be extracted or eliminated under the PSO process.Table 10
Dropped attributes after PSO process.
Land cover
All land cover types
Paddy rice versus  grass
Paddy rice versus  levee
Paddy rice versus  woods
Dropped attributes
IR, VI, ABI
R, G, NDVI, CMFI, SQBR,  SAVI, MSAVI, ABI,  GLCM energy,  GLCM homogeneity,  direct semivariogram
BR, VI, MSAVI, GLCM contrast,  GLCM energy
None


## 3.4. Thematic Map Comparison of Non-PSO versus PSO

A thematic map is useful for visually examining the performance of the developed classifier and estimating the area of paddy rice field. The classification outcome discussed in previous section presents better results by using PSOLDA as compared with using LDA alone. To further examine the benefit of incorporating PSOLDA, two thematic maps, Figures10 and 11, are generated for non-PSO and PSO, for the sake of comparison. These two figures are created by using the classifiers (LDA versus PSOLDA) through the same training datasets. Salt-and-pepper effect can be easily observed in Figure 10. Much lower salt-and-pepper effect is depicted in the PSO case, as shown in Figure 11. This is because the PSO case has a higher overall classification accuracy, which results in fewer misclassified points on the thematic map.Figure 10
Paddy rice thematic map using merely LDA.Figure 11
Paddy rice thematic map applying PSO on LDA.


## 4. Conclusion

Rice is a crop of global importance. Thus, remote sensing techniques have been applied for evaluating its production. This study combines PSO with LDA as an optimization tool for finding the best combination of conditional attributes. Five conclusions are made.(1)
This study proposes a method, PSOLDA, to improve classification accuracy in remote-sensing image classification tasks.(2)
This study presents a process to select window sizes and spectral bands for calculating texture variables. In this study, for GLCM texture, the IR band has better discernibility than does the R band; however, for semivariogram, the R band has better discernibility than the IR band does. Larger window size cases tend to have better discernibility but may include more uncertainties and probably enclose pixels of other land covers.(3)
Incorporating ancillary attributes, such as vegetation indices and texture measures, helps to improve classification accuracy.(4)
By incorporating PSO into LDA, the number of attributes is reduced and classification accuracy is improved. The proposed method leads to accuracy improvements of 8.57% and 14.78% in the overall land cover case and paddy rice versus grass case, respectively.(5)
Applying PSOLDA greatly reduces the salt-and-pepper effect in the thematic map when compared with merely applying LDA.



---

*Source: 804548-2014-08-20.xml*