# Subspace Learning via Local Probability Distribution for Hyperspectral Image Classification

Authors: Huiwu Luo; Yuan Yan Tang; Lina Yang
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

The computational procedure of hyperspectral image (HSI) is extremely complex, not only due to the high dimensional information, but also due to the highly correlated data structure. The need of effective processing and analyzing of HSI has met many difficulties. It has been evidenced that dimensionality reduction has been found to be a powerful tool for high dimensional data analysis. Local Fisher’s liner discriminant analysis (LFDA) is an effective method to treat HSI processing. In this paper, a novel approach, called PD-LFDA, is proposed to overcome the weakness of LFDA. PD-LFDA emphasizes the probability distribution (PD) in LFDA, where the maximum distance is replaced with local variance for the construction of weight matrix and the class prior probability is applied to compute the affinity matrix. The proposed approach increases the discriminant ability of the transformed features in low dimensional space. Experimental results on Indian Pines 1992 data indicate that the proposed approach significantly outperforms the traditional alternatives.


---

## Body

## 1. Introduction

With the rapid technological advancement of remote sensing, the technology of high dimensional data analysis has been gotten forward. With the great demand of need and appetite to automatical process, the remote sensing data in very high dimensional space, a series of analytical methods, and applicable toolkits were engendered one after another. Hyperspectral imaging (HSI) typically have hundreds even thousands of electromagnetic spectral bands for each pixel, and these bands are often highly correlated. To make full use of rich spectrum and to enable effective processing of HSI data, it is often dramatic to extract useful features, preventing negative effect and precaution caused by redundant data. Dimensionality reduction is an efficient technique to eliminate the redundance among data samples. Dimensionality reduction also eliminates the effects brought by the uncorrelated features and simultaneously “selects” or “extracts” the features that are beneficial to precious classification. To be specific, the aim of dimensionality reduction is to decrease computational complexity and ameliorate statistical ill-conditioning by discarding redundant features that potentially deteriorate classification performance [1]. Nevertheless, how to suppress the redundance and preserve the most valuable features still remains an open topic in the community of high dimensional data analysis.Dimension reduction techniques can be roughly categorized into linear approaches and nonlinear ones. The linear approaches include principal component analysis (PCA) [2], random projection (RP) [3], linear discriminant analysis (LDA) [4], and locality preserving projection (LPP), whereas the nonlinear approaches include isomap mapping (Isomaps) [5, 6], diffusion maps (DMaps) [7], and locally linear embedding (LLE) [8].The common drawback of nonlinear embedding methods is that these techniques are too expensive to compute HSI data when the size of samples becomes large. For instance, Isomaps employs geodesic distance to measure the distance of data samples rather than Euclidean distance, that is the classical straight-line distance. However, the theory of Isomaps is established on the basis of training samples, which is excessively reliant on the assumption of manifold-like distribution. Meantime, the mapping found by Isomaps is recessive and implicit. For the new data points, the geodesic distance has to be recomputed on the new training set to obtain the low dimensional embedding. There is no exact computational expression of new data points in Isomaps. It is clear that such computation is explicitly complex and unapplicable for the large capacity of HSI data. For this reason, Isomaps is impractical for the dimensionality reduction of HSI data. Similar drawback occurs in the construction of LLE [7]. Recent interests in discovering the intrinsic manifold of data structure have been a trend and the theory is still in the progress of development [9], yet some achievements have been gained and reported in many research articles [10].Nevertheless, the linear approaches are efficient to deal with this issue [11, 12]. PCA, an unsupervised approach, finds the global scatter as the best projected direction with the aim of minimizing the least square error of reconstruction data points [13]. Due to its “unsupervised” nature, the learning procedure is often blind and the projected direction found by PCA is usually not the optimal direction [14]. LDA is a supervised methodology which absorbs the advantage of purpose of learning [15]. Toward that goal, LDA seeks the direction that minimizes the classified error. However, the within-class scatter matrix of LDA is often singular when it is applied to the small size of samples [16]. Consequently, the optimal solution of LDA is unable to solve, and the projected direction is failed to achieve. These drawbacks will limit the wide promotion of LDA [4]. To cope with this issue, a derived discriminant analysis, which puts additional constraint on the objective function [17], was proposed in some research papers [18–20], for example, Join Global and Local Discriminant Analysis (JGLDA) [21]. The common scheme of these methods is that they are easy to compute and implement, and the mapping is explicit. Yet they have shown effeciency in most cases despite the simple models.The linear algorithms would have more advantages in the dimensionality reduction of HSI data in general. As a matter of fact, conventional linear approaches, such as PCA, LDA, and LPP, make the assumption that the distributions of data samples are Gaussian distribution or mixed Gaussian distribution. However, the assumption is often failed [22] since the distribution of real HSI data is preferred to be multimodal instead of a single modal. To be specific, the distribution of HSI data is usually unknown [23], and the single Gaussian model or Gaussian mixture model can not capture the distribution of all landmarks of the HSI data since the landmarks from different classes are multimodal [24]. In this case, the conventional methodologies work poorly. In view of this, some methods extend the idea of LDA and formulate extend-LDA algorithms; for example, Sugiyama [25] proposed Local Fisher Linear Discriminant Analysis (LFDA) for multimodal clusters. LFDA incorporates the supervised nature of LDA with local description of LPP and then the optimal projection is obtained under the constraint of multimodal samples. Li et al. [1] apply LFDA with maximum likelihood estimate (MLE), support vector machine (SVM), and Gaussian mixture model (GMM) to HSI data. As reported in his paper, LFDA is superior not only in the computational time, but also in the classified accuracy. In a word, LFDA is especially appropriate for the landmarks classification of HSI data. Nevertheless, the conventional LFDA ignores the distribution of data samples in the construction procedure of affinity matrix.In LFDA, the computation of affinity matrix is important. Note that there are clearly many different ways to define an affinity matrix, but the heat kernel derived from LPP has been shown to result in very effective locality preserving properties [26]. In this way, the local scaling of data samples in the K-nearest neighborhood is utilized. K is a self-tuning predefined parameter. To simplify the calculation procedure of parameters, [1, 27, 28] employing a fixed value of K=7 for experiments. Note that such calculation may ignore the distribution of data samples in the construction procedure of affinity matrix. Actually, the simplification of local distribution by the distance between the samples and the Kth nearest neighbor sample may be unreasonable, and the results by using this simplification may raise some error.Thus, in this paper, to overcome the weakness of conventional LFDA, a novel approach is proposed, where by adopting the local variance of local patch instead of farthest distance for weight matrix and the class prior probability for affinity matrix, the weight matrix of proposed algorithm takes into account both the distribution of HSI data samples and the objective function of HSI data after dimension reduction. This novel approach is called PD-LFDA because the probability distribution (PD) is used in LFDA algorithm. To be specific, PD-LFDA incorporates two key points, namely.(1)
The class prior probability is applied to compute the affinity matrix.(2)
The distribution of local patch is represented by the “local variance” instead of “farthest distance” to construct the weight matrix.The proposed approach essentially increases the discriminant ability of transformed features in low dimensional space. The pattern found by PD-LFDA is expected to be more accurate and is coincide with the character of HSI data and is conducive to classify HSI data.The rest of this paper is organized as follows. In the beginning of this paper, the most basic concepts of conventional linear approaches related to our work will be introduced in Section2. Precisely, Fisher’s linear discriminant analysis (LDA) and locality preserve projection (LPP) as well as local Fisher discriminant analysis (LFDA) will be presented. Proposed algorithm is developed and formalized in Section 3, which is the core of this paper. The experimental results with comparison on real HSI dateset are provided in Section 4. Finally, we conclude our work in Section 5.


## 2. Related Work

The purpose of linear approaches is to find an optimal projected direction where the information of embedding features is preserved as much as possible. To formulate our problem, letxi be the p-dimensional feature in the original space and let {x1,x2,…,xN} be the N samples. For the case of supervised learning, let li be label of xi, and then the label set of all samples can be represented by notation {l1,l2,…,lN}. Suppose that there are C classes in all, and the sample number of the cth is Nc that fulfils the condition N=∑c=1CNc. That is, the number of all samples is the total sum of each class. Let xic be the ith sample of the cth class. Then, the corresponding sample mean becomes mc=(1/Nc)∑i=1Ncxic, yet the data center of all samples is denoted by m=(1/N)∑i=1Nxi. Suppose that the data set X in p-dimensional hyperspace is distributed on a low q-dimensional subspace. A general problem of linear discriminant is to find a transformation T∈Rp×q that maps the p-dimensional data into a low q-dimensional subspace data by Y=TTX such that each yi represents xi without losing useful information. The transformed matrix T is pursued by different methods and different objective function, resulting in different algorithm.


### 2.1. Fisher’s Linear Discriminant Analysis (LDA)

LDA introduces the within-scatter matrixSw and between-scatter matrix Sb to describe the distribution of data samples:
(1)Sw=∑c=1C∑i=1Nc‍xic-mcxic-mcT,(2)Sb=∑c=1C‍Ncmc-mmc-mT.
Fisher criterion seeks a transformation T that maximized the between-class scatter while minimized the within-class scatter. This can be achieved by optimizing the following objective function:
(3)TLDA=argmax⁡T∈Rp×qtr⁡TTSbTtr⁡TTSwT.
It is implicitly assumed that TTSwT is full rank. Under this assumption, the problem can then be attributed to the generalized eigenvectors {φ1,φ2,…,φd} by solving
(4)Sbφ=λSwφ.
Finally, the solution of TLDA is given by TLDA={φ1,φ2,…,φq} which are associated with the first q largest eigenvalues λ1≥λ2≥⋯≥λq. Since the rank of between-class scatter matrix Sb is at most C-1, there are C-1 meaningful features in conventional LDA. To deal with this issue, a regularization procedure is essential in practice.


### 2.2. Locality Preserve Projection (LPP)

A drawback of LDA is that it does not consider the local structure among data points [29], and the distribution of real HSI data is often multimodal. Locality preserving projection meets this requirement [30]. The goal of LPP is to preserve the local structure of neighborhood points. Toward this goal, a graph is modeled explicitly to describe the relationship using k-nearest neighborhood. Let A denote the affinity matrix, where A(i,j)∈[0,1] represents the similarity between points xi and xj. The larger the value of A(i,j), the closer the relationship between xi and xj. A simple and effective way to define affinity matrix A is given by
(5)A(i,j)=exp⁡-xi-xj2α2,ifxi∈KNNxj,k,or:xj∈KNNxi,k0,otherwise,
where *2 denotes the square 2-norm Euclidean distance, α is a tuning parameter, and KNN(x,k) represents the K-nearest neighborhoods of x under parameter k.The transformed matrix of LPP is achieved in the following criterion [31]:
(6)TLPP=argmin⁡T∈Rp×q12∑i,j=1n‍Ai,jyi-yj2kkkkkkkkks.t.TTXDXTT=I,
where D=diag(Dii) is a diagonal matrix whose entries are the column sum (also can be a row sum since A is symmetric) of A; that is, Di,i=∑jAij. Arbitrary scaling invariance and degeneracy are guaranteed by the constraint of (6).The solution of LPP problem can be gained by solving the eigenvector problem of(7)XLXTφ=λXDXTφ,
where L≡D-A denotes the graph-Laplacian matrix in the community of spectral analysis and can be viewed as the discrete version of Laplace Beltrami operator on a compact Rimannian manifold [29]. And, finally, the transformation matrix T is given by TLPP={φ1,φ2,…,φq}∈Rp×q that correspond to eigenvalue 0=λ0≤λ1≤λ2≤⋯≤λq≤⋯≤λk.


### 2.3. Local Fisher Discriminant Analysis (LFDA)

Local Fisher discriminant analysis (LFDA) [32] measures the “weights” of two data points by the corresponding distance, and then the affinity matrix is calculated by these weights. Note that the “pairwise” representation of within-scatter matrix and between-scatter matrix is very important for LFDA. Following simple algebra steps, the within-scatter matrix (1) of LDA can be transformed into the following forms:
(8)Sw=∑c=1C∑i=1Nc‍xic-mcxic-mcT=∑c=1C∑i=1Nc‍xic-1Nc∑j=1Nc‍xjcxic-1Nc∑j=1Nc‍xjcT=∑i=1N‍xixiT-∑c=1C‍1Nc∑i,j=1Nc‍xicxjcT=∑i=1N‍∑j=1N‍Pwi,jxixiT-∑i,j=1N‍Pwi,jxixjT=12∑i,j=1N‍Pwi,jxixiT+xjxjT-xixjT-xjxiT=12∑i,j=1N‍Pwi,jxi-xjxi-xjT,
where
(9)Pw(i,j)=1Nc,ifli=lj=c0,ifli≠lj.
Let St be the total mixed matrix of LDA, and then we gain
(10)Sb=St-Sw=12∑i,j=1N‍Pbi,jxi-xjxi-xjT,
where
(11)Pb(i,j)=1N-1Nc,ifli=lj=c1N,ifli≠lj.
LFDA is achieved by weighting the pairwise data points
(12)S^w=12∑i,j=1N‍P^wi,jxi-xjxi-xjT,S^b=12∑i,j=1N‍P^bi,jxi-xjxi-xjT,
where P^w(i,j) and P^b(i,j) denote the weight matrix of different pairwise points for the within-class samples and between-class samples, respectively,
(13)P^w(i,j)≡Wi,jNc,ifli=lj=c,0,ifli≠lj,(14)P^b(i,j)≡Wi,j1N-1Nc,ifli=lj=c,1N,ifli≠lj,
where W indicates the affinity matrix. The construction of W is critical for the performance of classified accuracy; thereby, the investigation of construction is in great need to be further elaborated in the following section.


## 2.1. Fisher’s Linear Discriminant Analysis (LDA)

LDA introduces the within-scatter matrixSw and between-scatter matrix Sb to describe the distribution of data samples:
(1)Sw=∑c=1C∑i=1Nc‍xic-mcxic-mcT,(2)Sb=∑c=1C‍Ncmc-mmc-mT.
Fisher criterion seeks a transformation T that maximized the between-class scatter while minimized the within-class scatter. This can be achieved by optimizing the following objective function:
(3)TLDA=argmax⁡T∈Rp×qtr⁡TTSbTtr⁡TTSwT.
It is implicitly assumed that TTSwT is full rank. Under this assumption, the problem can then be attributed to the generalized eigenvectors {φ1,φ2,…,φd} by solving
(4)Sbφ=λSwφ.
Finally, the solution of TLDA is given by TLDA={φ1,φ2,…,φq} which are associated with the first q largest eigenvalues λ1≥λ2≥⋯≥λq. Since the rank of between-class scatter matrix Sb is at most C-1, there are C-1 meaningful features in conventional LDA. To deal with this issue, a regularization procedure is essential in practice.


## 2.2. Locality Preserve Projection (LPP)

A drawback of LDA is that it does not consider the local structure among data points [29], and the distribution of real HSI data is often multimodal. Locality preserving projection meets this requirement [30]. The goal of LPP is to preserve the local structure of neighborhood points. Toward this goal, a graph is modeled explicitly to describe the relationship using k-nearest neighborhood. Let A denote the affinity matrix, where A(i,j)∈[0,1] represents the similarity between points xi and xj. The larger the value of A(i,j), the closer the relationship between xi and xj. A simple and effective way to define affinity matrix A is given by
(5)A(i,j)=exp⁡-xi-xj2α2,ifxi∈KNNxj,k,or:xj∈KNNxi,k0,otherwise,
where *2 denotes the square 2-norm Euclidean distance, α is a tuning parameter, and KNN(x,k) represents the K-nearest neighborhoods of x under parameter k.The transformed matrix of LPP is achieved in the following criterion [31]:
(6)TLPP=argmin⁡T∈Rp×q12∑i,j=1n‍Ai,jyi-yj2kkkkkkkkks.t.TTXDXTT=I,
where D=diag(Dii) is a diagonal matrix whose entries are the column sum (also can be a row sum since A is symmetric) of A; that is, Di,i=∑jAij. Arbitrary scaling invariance and degeneracy are guaranteed by the constraint of (6).The solution of LPP problem can be gained by solving the eigenvector problem of(7)XLXTφ=λXDXTφ,
where L≡D-A denotes the graph-Laplacian matrix in the community of spectral analysis and can be viewed as the discrete version of Laplace Beltrami operator on a compact Rimannian manifold [29]. And, finally, the transformation matrix T is given by TLPP={φ1,φ2,…,φq}∈Rp×q that correspond to eigenvalue 0=λ0≤λ1≤λ2≤⋯≤λq≤⋯≤λk.


## 2.3. Local Fisher Discriminant Analysis (LFDA)

Local Fisher discriminant analysis (LFDA) [32] measures the “weights” of two data points by the corresponding distance, and then the affinity matrix is calculated by these weights. Note that the “pairwise” representation of within-scatter matrix and between-scatter matrix is very important for LFDA. Following simple algebra steps, the within-scatter matrix (1) of LDA can be transformed into the following forms:
(8)Sw=∑c=1C∑i=1Nc‍xic-mcxic-mcT=∑c=1C∑i=1Nc‍xic-1Nc∑j=1Nc‍xjcxic-1Nc∑j=1Nc‍xjcT=∑i=1N‍xixiT-∑c=1C‍1Nc∑i,j=1Nc‍xicxjcT=∑i=1N‍∑j=1N‍Pwi,jxixiT-∑i,j=1N‍Pwi,jxixjT=12∑i,j=1N‍Pwi,jxixiT+xjxjT-xixjT-xjxiT=12∑i,j=1N‍Pwi,jxi-xjxi-xjT,
where
(9)Pw(i,j)=1Nc,ifli=lj=c0,ifli≠lj.
Let St be the total mixed matrix of LDA, and then we gain
(10)Sb=St-Sw=12∑i,j=1N‍Pbi,jxi-xjxi-xjT,
where
(11)Pb(i,j)=1N-1Nc,ifli=lj=c1N,ifli≠lj.
LFDA is achieved by weighting the pairwise data points
(12)S^w=12∑i,j=1N‍P^wi,jxi-xjxi-xjT,S^b=12∑i,j=1N‍P^bi,jxi-xjxi-xjT,
where P^w(i,j) and P^b(i,j) denote the weight matrix of different pairwise points for the within-class samples and between-class samples, respectively,
(13)P^w(i,j)≡Wi,jNc,ifli=lj=c,0,ifli≠lj,(14)P^b(i,j)≡Wi,j1N-1Nc,ifli=lj=c,1N,ifli≠lj,
where W indicates the affinity matrix. The construction of W is critical for the performance of classified accuracy; thereby, the investigation of construction is in great need to be further elaborated in the following section.


## 3. Proposed Scheme

The calculation of (13) and (14) is very important to the performance of LFDA. There are many methods to compute the affinity matrix W. The simplest one is that W is equivalent to a constant; that is,
(15)Wi,j≡a,
where a in the above equation is a real nonnegative number. However, the equations of (13) and (14) are derived to the state-of-the-art Fisher’s linear discriminant analysis under this construction.Another construction adopts the heat kernel derived from LPP(16)Wi,j=exp⁡-xi-xj2σ2,
where σ is a tuning parameter. Yet, the affinity is valued by the distance of data points, and the computation is too simple to represent the locality of data patches. A more adaptive version [26] of (16) is proposed as follows:
(17)Wi,j=exp⁡-xi-xj2σ2,ifxi∈KNNxj,K,or:xj∈KNNxi,K,0,otherwise.Compared with the former computation, (17) is in conjunction with K-nearest data points, which is computationally fast and light. Moreover, the property of local patches can be characterized by (17). However, the affinity defined in (16) and (17) is globally computed; thus, it may be apt to overfit the training points and be sensitive to noise. Furthermore, the density of HSI data points may vary according to different patches. Hence, a local scaling technique is proposed in LFDA to cope with this issue [29], where the sophisticated computation is given by
(18)W(i,j)=exp⁡-xi-xj2ρiρj,ifxi∈KNNxj,K,or:xj∈KNNxi,K,0,otherwise,
where ρi denotes the local scaling around the corresponding sample xi with the following definition:
(19)ρi=xi-xiK,
where xiK represents the Kth nearest neighbor of xi, *2 denotes the square Euclidean distance, and K is a self-tuning predefined parameter.To simplify the calculation, many researches considered a fixed value ofK, and a recommended value of K=7 is studied in [1, 28]. Note that ρi is used to represent the distribution of local data around sample xi. However, the above work ignored the distribution around each individual sample. The diversity of adjacent HSI pixels is approximate; thus, the spectrum of the neighboring landmarks has great similarity. That is, the pixels of HSI data which have resembling spectrums tend to be of the same landmark. This phenomenon indicates that the adjacency of local patches not only lies in the spectrum space but also in the spatial space. For a local point, the calculation of making use of the diversity of its Kth nearest neighborhoods is not fully correct.An evident example is illustrated in Figure1, where, two groups of points have different distributions. In group (a), most neighbor points are closed to point x0, while, in group (b), most neighbor points are far from point x0. However, the measurement of two cases are the same according to (19). This can be found in Figure 1, where, the distances between point x0 and its Kth nearest neighborhoods (K=7) are same in both distributions, which can be shown in Figures 1(a) and 1(b), L1=L2. This example indicates that the simplification of local distribution by the distance between the sample xi and the Kth nearest neighbor sample is unreasonable. Actually, the result by using of this simplification may raise some errors.Different distributions ofx0 and the corresponding K-nearest neighborhoods (K=7). (a) Most neighbors are closed to point x0. (b) Most neighbors are far from point x0. The distances between point x0 and its Kth nearest neighbors are the same in both distributions, L1=L2.
(a)
(b)Based on the discussion above, a novel approach, which is called PD-LFDA, is proposed to overcome the weakness of LFDA. To be specific, PD-LFDA incorporates two key points, namely.(1)
The class prior probability is applied to compute the affinity matrix.(2)
The distribution of local patch is represented by the “local variance” instead of the “farthest distance” to construct the weight matrix.The proposed approach essentially increases the discriminant ability of transformed features in low dimensional space. The pattern found by PD-LFDA is expected to be more accurate and coincids with the character of HSI data, and is conducive to classify HSI data.In this way, a more sophisticated construction of affinity matrix, which is derived from [29], is proposed as follows:
(20)W(i,j)=pli2exp⁡-xi-xj2ρ^iρ^j·1+exp⁡-xi-xj2ρ^iρ^j,ifli=lj=c;0,ifli≠lj,
where p(li) stands for the class prior probability of class xi and ρ^i indicates the local variance. Note that the denominator item of (13) is 1/Nc, which will cancel out our prior effect if we use p(li) to replace p(li)2 (the construction of p(li) will be given in (21)). Different part of this derivation plays the same role as the original formulation; for example, for the last item, on one hand, it plays the role of intraclass discriminating weight and, on the other hand, the product result of W may reach zero if the Euclidean square distance · is very small for some data points. For this case, an extra item (1+exp⁡(-xi-xj2/ρ^iρ^j)) is added to the construction of intraclass discriminating weight to prevent accuracy truncation. By doing so, our derivation can be viewed as an integration of class prior probability, the local weight, and the discriminating weight. This construction is expected to preserve both the local neighborhood structure and the class information. Besides, this construction is expected to share the same advantages detailed in the original work.It is clear that (20) consists of two new factors compared with LFDA method: (1) class prior probability p(li) and (2) local variance ρ^i.Suppose classxi to be class c; that is li=c, so that the probability of class xi can be calculated by
(21)pli=pc=NcN,
where Nc is the number of the samples in class c, whole N denotes the total number of samples, and N=∑c=1CNc.Please note that the item(1+exp⁡(-xi-xj2/ρ^iρ^j)) in (20) is used to prevent the extra rounding error produced from the first two items and to keep the total value of
(22)pli2exp⁡-xi-xj2ρ^iρ^j1+exp⁡-xi-xj2ρ^iρ^j
which does not reach the minimum. Here, ρ^* denotes the local scaling around x*. In this paper, a local scaling x* is measured by the standard deviation of local square distance. Assume that x1(i),x2(i),…,xK(i) are the K-nearest samples of xi, and then the square distance between xi and xk(i) is given by
(23)dk(i)=xi-xk(i)2,k=1,2,…,K.The corresponding meanm^(i) can be defined as
(24)m^(i)=1K∑k=1K‍dk(i)=1K∑k=1K‍xi-xki2,
where *2 represents a square Euclidean distance and K is a predefined parameter whose recommended value is K=7. The standard deviation can be calculated as
(25)ρ^i=1K∑k=1K‍dki-m^i2.
Note that, in the above equation, the item 1/K becomes a constant that can be shifted outside. Thus, an equivalent formula is given by
(26)ρ^i=1K∑k=1K‍dki-m^i2.
Similar procedure can be deduced to xj. Hence, we have
(27)ρ^m=1K∑i=1K‍xi(mk)2-1K∑j=1K‍xj(mk)2.Comparing (19) with (27), it is noticeable that (28) holds
(28)ρ^m≤ρi.
Compared with the former definitions, our definition has at least the following advantages.(i)
By incorporating the prior probability of each class with local techniquep(li), the proposed scheme is expect to be a benefit for the classified accuracy.(ii)
The representation of local patches equation (26) is described by local standard deviation ρ^i rather than absolute diversity in (19), which is more accurate in measuring the local variance of data samples.(iii)
Compared with the global calculation, the proposed calculation is taken on local patches, which is efficient in getting rid of over-fitting.(iv)
The proposed local scaling technique meets the character of HSI data, which is more applicable for the processing of hyperspectral image in real applications.Based on the above affinity defined, an extended affinity matrix can also be defined in a similar way. Our definition only provides a heuristic exploration for reference. The affinity can be further sparse, for example, by introducing the idea ofε-nearest neighborhoods [31].The optimal solution of improved scheme can be achieved by maximize the following criterion:(29)TPD-LFDA≡argmax⁡T∈Rp×qtr⁡TTS^bTtr⁡TTS^wT.
It is evident that (29) has the similar form of (3). This finding enlightens us that the transformation T can be simply achieved by solving the generalized eigenvalue decomposition of S^w-1S^b. Moreover, Let G∈Rq×q be a q-dimensional invertible square matrix. It is clear that TPD-LFDAG is also an optimal solution of (29). This property indicates that the optimal solution is not uniquely determined because of arbitrary arithmetic transformation of TPD-LFDAG. Let φ^i be the eigenvector of S^w-1S^b corresponding to eigenvalue λ^i; that is, S^bφ^i=λ^iS^wφ^i. To cope with this issue, a rescaling procedure is adopted [25]. Each eigenvector {φ^i}i=1q is rescaled to satisfy the following constraint:
(30)φ^iS^wφ^j=1,ifi=j;0,ifi≠j.
Then, each eigenvector is weighted by the square root of its associated eigenvalue. The transformed matrix TPD-LFDA of the proposed scheme is finally given by
(31)TPD-LFDA=λ^1φ^1,λ^2φ^2,…,λ^q1φ^q∈Rp×q
with descending order: λ^1≥λ^2≥⋯≥λ^q.For a new testing pointsx^, the projected point in the new feature space can be captured by y^=TPFDATx^; thus, it can be further analyzed in the transformed space.According to the above analysis, we can design an algorithm, which is called PD-LFDA Algorithm, to perform our proposed method. The detailed description of this algorithm can be found in the appendix (Algorithm2). A summary of the calculation steps of PD-LFDA Algorithm is presented in Algorithm 1.Algorithm 1:PD-LFDA Algorithm.
Input: HSI training samples X∈Rp×N, dimensionality to be embedded q, the parameter K of KNN,
and a test samplext∈Rp
Step 1. For each sample xi from the same class, calculate the Wi,j by (20),
where the local scaling factorρ^i is calculated via (26) or (27);
Step 2. Equations (13) and (14) can be globally and uniformly transformed into an equivalent formula via:
P^w=W⋅W1P^b=W⋅W2-W1,                (i)
where the operatorA⋅B denotes the dot product between A and B, and
W(i,j)=p(li)2exp⁡-xi-xj2ρ^iρ^j1+exp⁡-xi-xj2ρ^iρ^j,ifli=lj=c;ifli≠lj.0, (iia)
W1=1Nc,ifli=lj=c;0,others.                 (iib)
W2=1N1N×111×N                  (iic)
By the above formulas the product of elements in different matrices can be achieved via dot product
between matrices. The equations (iia), (iib) and (iic) can be gained by integrating the number of
each classNc, the number of total training samples N, and the local scaling ρ^i, then matrices W,W1,W2
can be calculated.
Step 3. Construct within-scatter matrix P^w and between-scatter matrix P^b, according to (i).
Step 4. Define Laplacian matrices L* below:
L*=D*-P^*,                  (iii)
whereD* is the row sum or column sum of W*: Di,i*=∑j‍P^*(i,j) (or: Di,i*=∑i‍P^*(i,j)), and the
notation* denotes one letter in {w,b}.
Step 5. On the basis of (29) and (30), the transformation matrix can be achieved via
eigenvectorsT=λ^1φ^1,λ^2φ^2,…,λ^q1φ^q, T∈Rp×q that corresponding the q leading
eigenvaluesP^bφ^i=λ^iP^wφ^i in solving the general problem of P^bφ^i=λ^iP^wφ^i.
Step 6. For a testing sample xt∈Rp, the extracted feature is zt=TTxt∈Rq.
Output: Transformation matrix T, and the extracted feature zt.Algorithm 2:Proposed PD-LFDA method.
Input: HSI data samples X={x1,x2,…,xN}∈Rp×N, the objective dimension to be embedded q,
the nearest neighbor parameterK (default: K≡7), and the test sample xt∈Rp
Output: Transformation matrix T∈Rp×q
Steps are as follows:
(1) % Initialize matrices
(2)S^w←0p×p; % within-class scatter
(3)S^b←0p×p; % between-class scatter
(4)
(5) % Compute within-class affinity matrixWw
(6)for  c=1,2,…,C  do % in a classwise manner
(7){xic}i=1Nc←{xj∣lj=c}; % the cth  class data samples
(8)X←{x1c,x2c,…,xNcc} % sample matrix
(9)Wi=1Nc×111×NcNc;
(10)
(11)   % Determine the local scaling
(12)for  i=1,2,…,Nc  do
(13)xk(i)← the kth nearest neighbor of xic, k=1,2,…,K;
(14)for  k=1,2,…,K  do
(15)dk(i)=xi-xk(i)2;
(16)end for
(17)m^(i)←1K∑k=1Kxi-xk(i)2;
(18)ρ^(i)=1K∑k=1Kdk(i)-m^i2‍
(19)end for
(20)
(21)   % Define local affinity matrix
(22)for  i,j=1,2,…,Nc  do
(23)p(li)←Nc/N; % prior probability
(24)Ai,j←pliexp⁡-xi-xj2ρ^iρ^j1+exp⁡-xi-xj2ρ^iρ^j
(25)end for
(26)Ac=A
(27)end for
(28)Ww=diag⁡W1,W2,…,WC; % in block diagonal manner
(29)Aw=diag⁡A1,A2,…,AC; % also in block diagonal manner
(30)for  i,j=1,2,…,N  do
(31)P^wi,j←Awi,jWwi,j;
(32)end for
(33)
(34) % Compute between-class affinity matrixWb
(35)Wb←1N×111×NN-diag⁡W1,W2,…,WC;
(36) % LetFnz denotes the nonzero flag of elements in Ww: Fnzi,j=1 if Wwi,j≠0; Fnzi,j=0 if Wwi,j=0
(37)Fnz←Wb′≠0;
(38)Ab←0N×N;
(39)AbFnz=AwFnz;
(40)Ab¬Fnz=1;
(41)for  i,j=1,2,…,N  do
(42)P^bi,j←Abi,jWbi,j;
(43)end for
(44) % Now construct Laplacian matrix for within affinity matrixP^w and between affinity matrix P^b
(45) Let
(46)Di,iw=∑jP^wi,j, Di,ib=∑jP^bi,j;
(47) Then:
(48)Lw=P^w-Dw, Lb=P^b-Db;
(49) Construct two matrixs below:
(50)S^b=XLbXT, S^w=XLwXT;
(51) Letφ^1,φ^2,…,φ^q be the general eigenvector of:
(52)S^bφ^i=λ^iS^wφ^i, ∀i∈1,2,…,q
(53) with the corresponding eigenvalue in descending order:λ^1≥λ^2≥⋯≥λ^q
(54)
(55) Finally, the transformation matrix can be represented as:
(56)T=λ1φ^1,λ1φ^2,…,λ1φ^q∈Rp×q;
(57)
(58) For a new test samplext, the embedding zt is given by:
(59)zt=TTxt∈RqThe advantage of PD-LFDA is discussed as follows.Firstly, to investigate the rank of the between-class scatter matrixSb of LDA, Sb can be rewritten as
(32)Sb=∑l=1C‍Nlml-mml-mT=N1ml-m,N2ml-m,…,NLml-m·N1ml-m,N2ml-m,…,NLml-mT.
Thereby,
(33)rankSb⩽rankN1ml-m,N2ml-m,KiKKKKK…,NLml-m⩽C-1.It is easy to infer that the rank of the between-class scatter matrixSb is C-1 at most; thus, there are up to C-1 meaningful subfeatures that can be extracted. Thanks to the help of affinity matrix W, when compared with the conventional LDA, the reduced subspace of proposed PD-LFDA can be any subdimensional space. On the other hand, the classical local fisher’s linear discriminant only weights the value of sample pairs in the same classes, while our method also takes in account the sample pairs in different classes. Hereafter, the proposed method will be more flexible, and the results will be more adaptive. The objective function of proposed method is quite similar to the conventional LDA; hereby, the optimal solution is almost same as the conventional LDA, which indicates that it is also simple to implement and easy to revise.To further explore the relationship of LDA and PD-LFDA, we now rewrite the objective function of LDA and PD-LFDA, respectively:(34)TLDA=argmax⁡T∈Rp×qtraceTTSbTsubjectto:TTSwT=I,(35)TPD-LFDA≡argmax⁡T∈Rp×qtrace{TTS^bT}subjectto:TTS^wT=I.This implies that LDA tries to maximize the between-class scatter and simultaneously constraint the within-class scatter to a certain level. However, such restriction is hard to constraint and no relaxation is imposed. When the data is not a single modal, that is, multimodal, or unknown modal, LDA often fails. On the other hand, benefiting from the flexible designing of affinity matrixW, PD-LFDA gains more freedom in (35). That is, the separability of PD-LFDA will be more distinct, and the degree of freedom remains more than the conventional LDA; thus, our method is expected to be more robust and significantly preponderant.For large scale data sets, we discuss a scheme that can accelerate the computation procedure of the within-scatter matrixS^w. In our algorithm, owning to the fact that we have put penalty on the affinity matrix for different class samples in constructing the between-scatter matrix, the accelerated procedure will remain for further discussion.The within-class scatterS^w can be reformulated as
(36)S^w=12∑i,j=1N‍P^wi,jxi-xjxi-xjT=12∑i,j=1N‍P^wi,jxixiT+xjxjT-xixjT-xjxiT=∑i=1N‍∑j=1N‍P^wi,jxixiT-∑i,j=1N‍P^wi,jxixjT=XD^w-P^wXT=XL^wXT.
Here,
(37)D^wi,i=∑j=1N‍P^wi,j,L^w=D^w-P^w.P
^
w can be block diagonal if all samples {xi}i=1N are sorted according to their labels. This property implies that D^w and L^w can also be block diagonal matrix. Hence, if we compute S^w through (36), then the procedure will be much more efficient. Similarly, S^b can also be formulated as
(38)S^b=XL^bXT=XD^b-P^bXT.Nevertheless,P^b is dense and can not be further simplified. However, the simplified computational procedure of P^w saves for us part of time in a way. In this paper, we adopt the above procedure to accelerate S^w and pursue S^b normally. In addition to locality structure, some papers show that another property, for example, marginal information, is also important and should be preserved in the reduced space. The theory of extended LDA and LPP algorithm is developed rapidly recently. Yan et al. [33] summarized these algorithms in a graph embedding framework and also proposed a marginal fisher analysis embedding (MFA) algorithm under this framework.In MFA, the criterion is characterized by intraclass compactness and interclass marginal superability, which is replaced for the within-class scatter and between-class scatter, severally. The intraclass relationship is reflected by an intrinsic graph which is constructed byK-nearest neighborhood sample data points in the same class, while the interclass superability is mirrored by a penalty graph computed for marginal points from different classes. Following this idea, the intraclass compactness is given as follows:
(39)Si=∑i,j:i∈N(k)(j)‍or:j∈N(k)(i)TTxi-TTxj2=2TTXD-WXTT,
where
(40)Wi,j=1,ifi∈Nkjorj∈Nki;0,otherwise.
Here, N(k)(j) represents the K-nearest neighborhood index set of xj from the same class and D is the row sum (or column sum) of  W: Di,i=∑iWij. Interclass separability is indicated by a penalty graph whose term is expressed as follows:
(41)Se=∑i,j:(i,j)∈P(k)(lj)‍or:(i,j)∈P(k)(li)TTxi-TTxj2=2TTXD~-W~XTT,
of which
(42)W~(i,j)=1,ifi,j∈Pkljori,j∈Pkli;0,otherwise.
Note that Si and Se are corresponding to “within-scatter matrix” and “between-scatter matrix” of the traditional LDA, alternatively. The optimal solution of MFA can be achieved by solving the following minimization problem. That is,
(43)T^=argTmin⁡TTXD-WXTTTTXD~-W~XTT.
We know that (43) is also a general eigenvalue decomposition problem. Let TPCA indicate the transformation matrix from the original space to PCA subspace with certain energy remaining, and then the final projection of MFA is output as
(44)TMFA=TPCAT^.As can be seen, MFA constructs two weighted matricesW and W~ according to the intraclass compactness and interclass separability. In LFDA and PD-LFDA, only one affinity is constructed. The difference lies in that the “weight” in LFDA and PD-LFDA is in the range of [0,1] according to the level of difference. Yet, MFA distributes the same weight to its K-nearest neighborhoods. The optimal solution of MFA, LFDA, and PD-LFDA can be attributed to a general eigenvalue decomposition problem. Hence, the idea of MFA, LFDA, and PD-LFDA is approximately similar in a certain interpretation. Relationship with other methodologies can be analyzed in an analogous way.


## 4. Experimental Results

To illustrate the performance of PD-LFDA, experiments on a real hyperspectral remote sensing image data set—AVIRIS Indian Pine 1992, are conducted in this section. The AVIRIS Indian Pines 1992 data set was gathered by National Aeronautics and Space Administrator (NASA) with Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor over the Indian Pine test site in northwest Indians in June 1992. This data set consists of 145 × 145 pixels and 224 spectral reflectance bands ranging from 0.4μm to 2.45 μm with a spatial resolution of 20 m. The Indian Pines scene is composed of two-third agriculture and one-third forest or other natural perennial vegetation. Some other landmarks such as dual lane highways, rail line, low density housing, and smaller roads are also in this image. Since the scene was taken in June, some main crops, for example, soybeans and corn, are in their early growth stage with less than 5% coverage, while the no-till, min-till, and clean-till indicate the amount of previous crop residue remaining. The region map can be referred to Figure 5(a). The 20 water absorption bands (i.e., [108–112, 154–167] 224) were discarded.In this section, performance of different dimension reduction methods, that is, PCA, LPP, LFDA, LDA, JGLDA and RP [34], is compared with the proposed PD-LFDA. Classification accuracy is reported via the concrete classifiers. Generally, many dimension reduction research papers adopt K-nearest neighborhood classifier (KNN) and support vector machine (SVM) classifier to measure the performance of the extracting features after the dimension reduction, where the overall accuracy and kappa coefficient are detailed in the reports. Hereby, in this paper, we also adopt KNN classifier and SVM classifier for performance measurement. In KNN classifier, we select the value of K as 1, 5, and 9, so that three classifiers based on nearest neighborhoods are formed, which are called 1NN, 5NN, and 9NN. In SVM classifier, we seek a hyperplane to separate classes in kernel-induced space where the linear nonseparable classes in the original feature space can be separated via kernel trick. SVM, as a robust and successful classifier, has been widely used to evaluate the performance of multifarious methods in many areas. For simplicity and convenience, we use LIBSVM package [35] for experiments. Accuracy of dimension reduced performance will be reported by classified performance from SVM classifier. In the following schedule, the feature subspace will be calculated at the first step from training samples by different dimensional algorithms. Table 1 gives a numerical statistics of training samples corresponding to each class. Then, the new sample will be projected into a low subspace by the transformed matrix. Finally, all the new samples are classified by SVM classifier.Table 1
Training set in AVIRIS Indian Pines 1992 database.
ID
Class Name
1
Alfalfa (46/18, 39.13%)
2
Corn-notill (1428/136, 9.52%)
3
Corn-mintill (830/87, 10.48%)
4
Corn (237/34, 14.34%)
5
Grass-pasture (483/54, 11.18%)
6
Grass-trees (730/71, 9.73%)
7
Grass-pasture-mowed (28/17, 60.71%)
8
Hay-windrowed (478/50, 10.46%)
9
Oats (20/15, 75.00%)
10
Soybean-notill (972/86, 8.84%)
11
Soybean-mintill (2455/214, 8.72%)
12
Soybean-clean (593/54, 9.11%)
13
Wheat (205/28, 13.66%)
14
Woods (1265/102, 8.06%)
15
Buildings-Grass-Trees-Drives (386/39, 10.10%)
16
Stone-Steel-Towers (93/24, 25.81%)
Total
10249/1029, 10.04%
Numerical value in this table refers to number of  samples, number of training samples, and p.c., respectively.In this experiment, a total of 1,029 samples were selected for training, and the remaining samples are used for testing. Note that all the labeled samples in database are unbalanced, and the available samples of each category differ dramatically. The following strategy is imposed for sample division. A fixed number of 15 samples are randomly selected to form the training sample, yet the absent samples are randomly selected from the remaining samples. Under this strategy, the training samples and testing samples are listed in Table1.Figure2 shows the overall accuracy of different dimension reduction methods applied to AVIRIS Indian 92AV3C data set. The neighborhood of KNN classifier is selected as 1, 5, and 9, respectively, which produces three classifiers, that is, 1NN, 5NN, and 9NN. Three different kernel functions are adopted for SVM classifier. Then, derived classifiers are also used in this experiment, that is, linear SVM, polynomial SVM, and RBFSVM. It can be deduced from Figures 2(a)~2(c) that when the embedding space is greater than 5, proposed PD-LFDA performs the best, while JGLDA performs the worst. The results produced by RP is slightly better than JGLDA. PCA, LDA, LPP, and RP show the similar classified results under KNN classifier. That is, the proposed PD-LFDA outperforms the others. Meantime, compared with LFDA, the proposed PD-LFDA leads to 2% more improvements on average. Moreover, it can be observed from (d) that the classified accuracy increases steadily as the embedded space increases. However, LDA demonstrates the highest overall accuracy when the reduced features vary to 9, while LFDA has the significant improvements when the number of reduced features is greater than 9. This phenomenon of Figure 2(d) indicates the instability of linear SVM. Nevertheless, the situation reversed for polynomial SVM and RBF SVM in Figures 2(e) and 2(f), wherein the proposed PD-LFDA wins a little improvement against LFDA and has significant improvement compared with the others. Inspired effects of proposed PD-LFDA algorithm were achieved in all cases. Furthermore, Table 2 gives the detailed overall accuracy under different feature dimensions using 3NN, 7NN, and RBFSVM classifiers, which validates the feasibility of the proposed scheme in this paper.Table 2
Overall accuracy obtained using different dimensions of feature and different classifiers for Indian Pine Scene database (%).
Item
Dims
7
9
11
13
15
3NN
PCA
70.07
69.96
70.26
70.35
70.87
LPP
65.75
68.71
68.35
66.95
65.46
LFDA
75.13
78.95
79.63
79.92
80.09
LDA
64.15
66.06
66.16
65.96
65.36
PD-LFDA
77.13
80.94
81.6
82.18
82.63
JGLDA
54.05
56.02
57.04
56.42
57.51
RP
61.73
64.44
67.45
66.32
66.52
7NN
PCA
69.02
69.02
69.1
69.33
69.72
LPP
67.82
70.57
70.27
68.92
66.81
LFDA
74.19
77.77
78.27
78.63
78.62
LDA
66.15
69.09
68.99
69.2
68.82
PD-LFDA
77.36
80.9
81.55
81.72
82.35
JGLDA
56.65
58.11
58.09
58.12
59.14
RP
61.94
64.81
66.96
66.13
66.03
RBF-SVM
PCA
80.51
77.94
77.18
79.35
81.76
LPP
71.01
75.14
76.14
75.01
73.14
LFDA
78.26
81.87
82
81.13
79.15
LDA
67.7
68.34
68.77
66.42
67.92
PD-LFDA
78.58
82.67
83.95
84.23
81.66
JGLDA
56.86
56.06
56.63
56.89
58.79
RP
69.57
72.82
74.41
74.16
75.43Overall accuracy by different dimension reduction methods and different classifiers applied to AVIRIS Indian Pines database.
(a)
1NN
(b)
5NN
(c)
9NN
(d)
Liner SVM
(e)
Polynomial SVM
(f)
RBF-SVMFigure3 displays the results of kappa coefficient obtained using the different dimension reduction algorithms under KNN classifier and SVM classifiers. The experimental circumstance of Figure 3 is same as that of Figure 2. We can find that, from these results, JGLDA performs the worst in most cases except in Figure 3(e). The proposed PD-LFDA method outperforms the other methods and achieves the highest kappa numerical value in most cases except using the linear SVM as the classifier. In fact, none of them work steady in the case of linear SVM (Figure 3(d)). Note that the situation is improved in polynomial SVM, where the kappa numerical value of the proposed PD-LFDA is significantly better than the others. All the achievements demonstrate the robustness of our contribution in PD-LFDA. Simultaneously, it is noticeable that LPP exhibits an average kappa level. The kappa value gained by LPP is not seriously bad and is not dramatically good. The kappa results produced by RP are approximately the same as LPP. A significant advantage of RP is the simple construction and computation, where the accuracy is closed to LPP. More details are summarized in Table 3. It can be concluded that the kappa coefficient of proposed algorithm is higher than the other approaches, which is more appropriate for the classification of HSI data.Table 3
Kappa coefficient by different dimension reduction methods and different classifiers applied to Indian Pine Scene database (%).
Item
Dims
7
9
11
13
15
3NN
PCA
65.92
65.8
66.12
66.24
66.81
LPP
60.84
64.22
63.82
62.19
60.5
LFDA
71.45
75.86
76.64
76.97
77.18
LDA
59.13
61.26
61.37
61.18
60.48
PD-LFDA
73.92
78.24
79.01
79.66
80.16
JGLDA
47.78
49.92
50.82
50.06
51.19
RP
56.4
59.63
63.03
61.72
61.92
7NN
PCA
64.66
64.66
64.75
65.01
65.45
LPP
63.04
66.29
65.85
64.29
61.85
LFDA
70.47
74.57
75.15
75.57
75.57
LDA
61.32
64.6
64.53
64.78
64.31
PD-LFDA
74.17
78.2
78.93
79.13
79.85
JGLDA
50.29
51.9
51.61
51.59
52.63
RP
56.38
59.92
62.37
61.35
61.24
RBF-SVM
PCA
77.67
74.7
73.93
76.36
79.00
LPP
66.77
71.5
72.64
71.15
69.21
LFDA
74.86
79.15
79.24
78.22
75.84
LDA
62.9
63.62
64.27
61.53
63.23
PD-LFDA
75.51
80.11
81.59
81.87
78.99
JGLDA
51.09
50.66
51.33
51.47
53.69
RP
65.3
69.08
70.85
70.6
72.03Kappa coefficient by different dimension reduction methods and different classifiers applied to AVIRIS Indian Pines database.
(a)
1NN
(b)
5NN
(c)
9NN
(d)
Linear SVM
(e)
Polynomial SVM
(f)
RBF SVMThe visual results of all methods are presented in Figures5~6, where the class labels are converted to pseudocolor image. The pseudocolor image of the hyperspectral image from Indian 92AV3C database is shown in Figure 4(a). The available labeled image which represents the truth ground is illustrated in Figure 4(b), where the labels are made by human. The training samples are selected from the labeled image, represented as points in the image as shown in Figure 4(c). Each label number (ID) corresponds to each class name, which is indexed in Table 1. In this experiment, all the available labeled samples are used for testing, while approximate 10% of samples are used for training. The subspace is fixed to 13 (the number here is only used for reference; it can be changed). For each experiment, the dimension from original feature space is reduced to the objective dimensionality; thereafter, the classified maps are induced by 7NN classifier and RBF-SVM classifier. The overall accuracy, kappa coefficient, and average accuracy are also included at the top of each map, respectively. Figure 5 displays the classified maps of classic methods in pseudocolor images, and the classified maps of proposed PD-LFDA are presented in Figure 6. It can be observed from these maps that the best performance is achieved by PD-LFDA when 7NN classifier is used; in this case, the overall accuracy is 83.79%, the kappa coefficient is 81.69%, and the average accuracy is 89.91%. Moreover, the worst algorithm is JGLDA whose overall accuracy is 61.95%, the kappa coefficient becomes 56.51%, and the average accuracy is only 62.09%. Other methods such as PCA, LPP, and RP can produce the comparable results, and no one can outperform the other. However, LDA outperforms PCA, LPP, and RP, yielding better result. Similar conclusions can be achieved in the group of RBF-SVM. Generally, proposed PD-LFDA significantly outperforms the rest in this experiment, which indicates the correctness of improvements in proposed PD-LFDA.Illustration of sample partition.
(a)
Pseudo 3-channel color image, band =[12, 79, 140]
(b)
Ground truth
(c)
Distribution of training sampleClassified map generated by different dimension reduction methods, where the overall accuracy, kappa coefficient, and average classification accuracy are listed at the top of each map, respectively.
(a)
PCA-7NN
(b)
LPP-7NN
(c)
LFDA-7NN
(d)
LDA-7NN
(e)
JGLDA-7NN
(f)
RP-7NN
(g)
PCA-RBFSVM
(h)
LPP-RBFSVM
(i)
LFDA-RBFSVM
(j)
LDA-RBFSVM
(k)
JGLDA-RBFSVM
(l)
RP-RBFSVMClassified map of the proposed method.
(a)
PD-LFDA-7NN
(b)
D-LFDA-RBFSVMFinally, details of assessment deduced by 7NN and RBF-SVM is summarized in Table4, where the corresponding overall accuracy, kappa coefficient, and average accuracy of different methods can be located collectively.Table 4
Performance of dimension reduction on the whole labeled samples (%).
Evaluated item
Methodologies
PCA
LDA
LPP
LFDA
JGLDA
RP
PD-LFDA
7NN
Overall accuracy
68.68
73.9
68.75
79.23
61.95
67.09
83.79
Kappa coefficient
64.71
70.41
64.7
76.53
56.51
62.94
81.69
Average accuracy
73.7
81.3
76.07
86.09
62.09
71.64
89.91
RBF-SVM
Overall accuracy
79.92
72.87
76.74
83.75
58.24
76.31
84.86
Kappa coefficient
77.28
69.28
73.55
81.51
53.77
73.27
82.79
Average accuracy
85.7
80.71
82.88
88.22
71.21
83.65
89.68


## 5. Conclusions

In this paper, we have analyzed local Fisher discriminant analysis (LFDA) and found its weakness. By replacing the maximum distance with local variance for the construction of weight matrix, introducing class prior probability into the computation of affinity matrix, an improved LFDA algorithm has been proposed. This novel approach is called PD-LFDA, because the probability distribution (PD) is applied in LFDA algorithm. The proposed approach essentially can increase the discriminant ability of transformed features in low dimensional space. The pattern found by the new approach is expected to be more accurate and coincides with the character of HSI data and is conducive to classify HSI data. PD-LFDA has been evaluated on a real removing sensing AVIRIS Indian Pine 92AV3C data set. We have compared the performance of the proposed PD-LFDA with that of PCA, LPP, LFDA, LDA, JGLDA, and RP. Both the numerical results and visual inspection of class maps have been obtained. In the experiments, KNN classifier and SVM classifier have been used. We have argued that the proposed PD-LFDA exhibits the best performance and serves as a very effective dimensionality reduction tool for high dimensional data, such as hyperspectral image (HSI) data.



---

*Source: 145136-2015-02-12.xml*