HYPERSPECTRAL IMAGE CLASSIFICATION WITH PARTIAL LEAST SQUARE FOREST
Junshi Xia, Naoto Yokoya, Akira Iwasaki
Research Center for Advanced Science and Technology, The University of Tokyo,
153-0041 Tokyo, Japan
ABSTRACTIn the hyperspectral remote sensing community, decision
forests combine the predictions of multiple decision trees(DTs) to achieve better prediction performance. Two wellknown and powerful decision forests are Random Forest (RF)and Rotation Forest (RoF). In this work, a novel decision forest, called Partial Least Square Forest (PLSF), is proposed.
In the PLSF, we adapt PLS to obtain the components for thehyperplane splitting. Moreover, the projection bootstrap technique is used to retain the full spectral bands for the selectionof split in the projected space. Experimental results on three
hyperspectral datasets indicated the effectiveness of the proposed PLSF because it enhances the diversity and accuracy
within the ensemble when compared to RF and RoF.
Index Terms— Composite kernel learning, ensemble
learning, classification, hyperspectral image.
1. INTRODUCTION
Hyperspectral sensors capture the images over narrows contiguous hundreds or thousands of bands of the electromagnetic spectrum. The detailed spectral information, as well as
high spatial resolution, provides enhanced capability for thematic applications, i.e., land cover classification [1]. Highdimensionality should be addressed to exploit the hyperspectral datasets. Several approaches have been proposed to tackle
this issue [2, 3].
Multiple classifier systems (MCSs) or ensemble learning
has proven to be effective in the classification of hyperspectral images among the currently available supervised learning
methods. It is formed by combing the predictions of multiple base learners to achieve better performance than the single learner. In order to construct a strong MCS, the individualclassifiers should be with high diversity and high accuracy [4].
Two decision tree (DT) ensemble methods, random forest(RF) [5] and rotation forest (RoF) [6], have attracted increasing attention in hyperspectral remote sensing community due
to their great classification capability, fast out of sample prediction, and with slight parameter tuning [7, 8]. Two bagging
strategies, i.e., each tree is trained on bootstrapped samples,and a subset of features is considered for the split of leaf, areused to inject the randomness into the construction of RF [5].

In the RoF, the accuracy and diversity are promoted by using principal component analysis (PCA) to extract features inseveral disjoint subsets for learning individual base DT classifiers [6]. The aforementioned techniques make RF and RoF
widely used not only in hyperspectral data analysis but also
in synthetic aperture radar (SAR) [9] and very high spatial
resolution image analysis [10].
In order to further improve the accuracy and diversity
within the ensemble, we propose Partial Least Square Forest (PLSF), which is constructed by several individual DTs
that utilizes partial least square for the hyperplane splitting.
Moreover, the projection bootstrap technique, which retains
all spectral bands for split selection in the projected space,results in a significant increase in accuracy of diversity.
2. PARTIAL LEAST SQUARE FOREST
Let S ≡ {1, ..., N } denote a set of integers indexing theN pixels of a hyperspectral image; let K ≡ {1, ..., K}
be a set of K labels; let x ≡ {x1 ; ...; xN } ∈ RN ×D
denote an image of D−dimensional feature vectors; lety ≡ {y1 , ..., yN } ∈ K be a set of labels for the N pixels;
and let D ≡ {X, Y} ≡ {(x1 , y1 ) , ..., (xn , yn )} be the training set, where n is the number of training samples. Theobjective of classification is to assign a label yi ∈ K to eachpixel i ∈ S, based on the vector xi , resulting in an image ofclass label yi .
As a new DT ensemble, PLSF combines individual
oblique DTs, in which PLS is used to project the original dataand select the best split in the projected space. In this work,
we pay attention to Orthonormalized PLS (OPLS) [11, 12].
OPLS exploits the correlation between the features andthe target data by combining the merits of canonical variate
analysis and PLS [11,12]. Let {X, Y} ≡ {(x1 , y1 ) , ..., (xn , yn )}be the training set, where n is the number of training samples. Let Y is converted to 1-of-K labels Y ∈ In×K ,where Yik = 1 implies the pixel i belongs to class k andCxy = n1 X> Y represent the covariance between X and Y,whereas the covariance matrix of X is given by Cxx = n1 X> X.
W ∈ RD×d is referred as the projection matrix, thus the extracted features is achieved by X0 = XW. It should be notedthat OPLS only extracts the projection matrix from the input
data X.

DT 1

DT 2

DT T

Bagging

Bagging

Bagging

OPLS

OPLS

OPLS……

Fig. 1. PLSF
The objective of OPLS is formulated byonW
OPLS:
maximize: Tr W> Cxy C>
xysubject to:

W> Cxx W = I(1)Let T = {t1 , ..., tT } denote a PLSF composed of TDTs ti . Each DT is composed of a set of split nodes
S = {sj }j∈J /∂J and a set of leaf nodes L = {lj }j∈∂J ,where J is a set of node indices and ∂J ⊆ Jis the subset ofleaf node indices. Each split node is defined s1j , s2j , φj , zj ,where s1j , s2j ⊆ J /j are the ids of two child nodes, φj is aweight vector used to project the input features and zj is thepoint at which the splitting occurs in the projected space Xφj .
As shown in Fig. 1 and Algorithm 1, the detailed training
steps are summarized as follows:• X is centered with zero mean and unit variance.
• the new training set X0 , Y 0 is randomly selected from{X, Y} using Bagging technique.
• the new training set is used as the input of the GROWTREE algorithm (seen in Algorithm 2).
 0 0
In the GROWTREEn algorithm,o the new training set X , Yis selected from Xj(:,λ) , Y by using Bagging technique, andthen is input to the OPLS. The projection matrix W obtainedfrom OPLS is used to produce the new features. At node j,Algorithm 1 PLSF
Training phase
Input: {X, Y}: training samples.D: number of spectral
bands. T : ensemble size. M : number of sampling features.
Output: CCF T = {ti }i=1,...,T
1: Centering
X with zero mean and unit variance 0
2: X , Y 0 ←− bootstrap sampling from {X, Y}0
3: [., S, L] = GROWTREE X , Y 0 , D, M
4: ti = {S, L}
5: return T = {ti }i=1,...,TPrediction phase
Input: The ensemble T = {ti }i=1,...,T . A new sample x∗ .
Output: class label y ∗1: get the output ensemble using T = {ti }i=1,...,T .
PT∗ ∗2: p(y ∗ |x∗ ) = T1
j=1 p(y |x : tj )TP
3: y ∗ = argmax
1k∈{1,2,...,K} j:tj (x∗ )=kwe randomly select λ = min(M, |Fj |) features without replacement from the available feature set Fj . The split projection vector Wj is set as the column of W in which the best
split existed in the training phrase at node j. zj is the corresponding split point at XWj . Furthermore, the index of theprojection giving the best split ξ is obtained from the function
FINDBESTSPLIT that is calculated by maximizing the information gain [13]. The samples are then split into left and rightnodes according to the rule ∃X(i,:) φj ≤ zj . The tree is grown
until some stopping criterion, such as a maximum tree depth,
is reached. It should be noticed that OPLS is only performedat the training stage while the split rule is directly used in thetest stage.
In the prediction phase, for a new sample x∗ , the final
result is generated by combining the results from individual
DT in the ensemble using a majority voting rule.
3. EXPERIMENTAL RESULTS AND ANALYSIS
3.1. Hyperspectral datasets
In this work, three hyperspectral datasets are used to assess
the performance of the proposed PLSF and compared withRF and RoF classifiers.
• Kennedy space center (KSC): This dataset was acquired by the AVIRIS sensor with a ground sampling
distance (GSD) of 18 m. We keep 176 bands after
removing water absorption and low SNR bands. 13
classes are used in this scene.
• Salinas: This scene was also collected by the AVIRIS
sensor over Salinas Valley, California with a GSD of
3.7 m. The area covered comprises 512 lines by 217
samples with 16 classes and 224 bands.
• Botswana. This dataset was obtained by the Hyperion
sensor on the NASA EO-1 satellite over the Okavango

9095

95

90

90

OA(%)

85

OA(%)

85

OA(%)

Algorithm 2 GROWTREE
Input: Xj , Y j : training samples at node j, M : number ofsampling features. Fj : available features.
Output: sub-tree root node identifier j, sub-tree discriminantnodes Ψ, sub-tree leaf nodes Θ1: set current node index j to an unique node identifier
2: randomly selecting λ features by taking min(M, |Fj |)samples without replacement from Fj no 0 0
3: X , Y←− bootstrap sampling from Xj(:,λ) , Y j0
4: [W, ·] = OPLS X , Y 0
5: W(λ:end,λ:end) ← 0
6: U = XW
7: [ξ, zj , gain] = FINDBESTSPLIT(U)8: φj = W(:,ξ)
9: If ∃X(i,:) φj ≤ zj then τl else τr EndIf
jj
10: s1j , Sl , Ll = GROWTREE X(τ ,:) , Y(τ ,:) , |Fj |, Ml l 2jj
11: sj , Sr , Lr = GROWTREE X(τ ,:) , Y(τ ,:) , |Fj |, Mrr
12: obtain s1j , s2j , φj , zj
13: return [j, {Sl ∪ Sr } , {Ll ∪ Lr }]

85

80
80

70
10RFRoF
PLSF

RFRoF
PLSF

75

203040

Number of samples per class

(a)

50

75
10203040

RFRoF
PLSF

80

50

Number of samples per class(b)75
1020304050

Number of samples per class(c)Fig. 2. Overall accuracies with different sizes of training set.
(a) KSC. (b) Salinas. (c) Botswana.
Delta, Botswana. The scene consists of 14 identifiedclasses and 145 bands.
3.2. Results
The ensemble size (T ) is set to be 20. The number of featuresin a subset is set to be the square root of the number of theused features and 10 for the RF and RoF, respectively. For
the PLSF, M = 9. We randomly select the training set fromthe ground truth for ten Monte Carlo runs and report the meanvalues.
Fig. 2 shows the overall accuracies with different sizes oftraining set. Here, 10, 20, 30, 40 and 50 samples per classare chosen to form the training set. It can be seen that PLSF
shows the best classification results in all the cases. Table. 1
presents the global and class-individual accuracies producedby PLSF as well as RF and RoF. From this table, PLSF not
only provides the best global accuracies but also yields thebest performance for individual classes (11 out of 13, 14 outof 16, 13 out of 14 for KSC, Salinas and Botswana).
Table 2 presents the measures such as the “OA (%)”, thepercentage average overall accuracies of the individual DTclassifier, “AOA (%)”, and the Coincident Failure Diversity(CFD) [14]. A stronger diversity is represented by a higher
value of CFD. It is apparent that the best performance of
PLSF is attributed to the strongest diversities for all threedatasets although RoF gives the best results of AOA. It provesthat the diversity is of critical importance within the ensemble
in a practical viewpoint.
Fig. 3 compares the performances of the three decision
forests (i.e., RF, RoF, and PLSF) by using different values of
the number of features in a subset (M ). As in the detailed test,
PLSF tends to have better performance in small values of M .
In contrast, this value is data dependent for RoF.
4. CONCLUSION
In this paper, we proposed a decision tree ensemble classifier(PLSF) for hyperspectral image classification. PLSF provides
a natural way to enhance the diversity of an ensemble by using
OPLS and the projection bagging technique. Experimental
results support the theoretical basis as mentioned above.

Table 1. Overall, average and class-specific accuracies obtained for the Kennedy Space Center, Salinas and Botswana images
when ten samples per class are used to form the training set.
Kennedy Space Center
RFOA
76.85
AA
72.71
κ
74.28
Scrub
80.78
Willow swamp
73.87
Cabbage palm hammock84.57
Cabbage palm/oak hammock
46.11
59.50
Slash pineoak/broadleaf hammock
46.90
Hardwood swamp
89.05
48.84
Graminoid marsh
Spartina marsh
77.92
Cattail marsh
71.44
92.96
Salt marsh
Mud flats
75.19
Water
98.05
-

Salinas
RoF
83.26
79.87
81.41
80.85
85.43
83.75
48.29
63.17
62.84
92.57
72.62
83.37
94.18
91.98
79.64
99.67
-

PLSF
86.44
83.22
84.95
79.84
86.13
89.49
50.40
71.61
61.2292.67
84.52
94.27
93.22
92.20
86.34
100.00-OA
AAκBrocoli green weeds 1
Brocoli green weeds 2
Fallow
Fallow rough plow
Fallow smoothStubbleCeleryGrapes untrained
Soil vinyard developCorn senesced green weeds
Lettuce romaine 4wk
Lettuce romaine 5wk
Lettuce romaine 6wk
Lettuce romaine 7wk
Vinyard untrained
Vinyard vertical trellis

Table 2. Comparison among RF, RoF and PLSF by using“OA (%)”, “AOA (%)”, and diversities.
DatasetsRF
RoF PLSF
SalinasOA (%)AOA (%)
Diversity

79.84
70.83
67.3783.91
77.12
69.93

85.32
69.97
71.85

Botswana

OA (%)AOA (%)
Diversity

80.45
66.83
64.36

84.04
73.38
68.12

89.44
72.92
69.77

KSC

OA (%)AOA (%)
Diversity

76.85
64.32
59.13

83.26
73.26
66.19

86.44
68.33
69.02

86

90

90

85
84
83
85

85

RF
RoF
PLSF

80

OA(%)

OA(%)

OA(%)

82
81
80
7980

78

RF
RoF
PLSF

7776
75

75
10203040506070

Number of features in a subset

(a)

80

RF
RoF
PLSF
75

20406080

100

10Number of features in a subset(b)203040506070

Number of features in a subset(c)Fig. 3. Sensitivity to the change of the number of features ina subset. (a) KSC. (b) Salinas. (c) Botswana.
5. ACKNOWLEDGMENT
The authors would like to thank Prof D. Landgrebe from Purdue University for providing the image. The authors would
also like to that KAKENHI Grant Number 24360347 and16H04587, and JSPS KAKENHI Grant Number 16F16053
for supporting this work.

RF79.84
87.78
77.72
98.18
97.60
84.27
99.11
94.75
96.06
97.54
51.90
93.22
70.88
87.43
94.81
96.42
91.97
62.82
87.43

RoF
83.91
90.44
82.15
96.49
94.65
92.67
99.17
96.01
96.65
98.80
63.97
97.18
80.44
91.40
98.69
97.18
93.17
63.10
87.50

PLSF
85.31
92.47
83.70
98.00
99.46
92.42
99.45
97.56
99.59
99.69
61.52
99.25
84.30
94.70
99.56
98.25
93.74
65.04
97.03

Botswana
RFOA
80.45
AA
82.34
κ
78.85
Water
97.52
Hippo grass
90.89
Floodplain grass1
86.65
Floodplain grass2
83.72
Reeds 1
74.72
Riparian
50.33
Firescar2
91.27
Island interior
87.44
Acacia woodlands
63.89
Acacia shrublands
75.16
Acacia grasslands
86.55
Short mopane
89.67
Mixed mopane
75.49Exposed soils
99.47
-RoF
84.04
85.62
82.73
98.44
91.39
88.29
91.95
79.18
55.92
92.93
89.80
74.01
79.31
86.89
93.48
77.57
99.58
-PLSF
89.44
90.60
88.57
100.00
98.61
96.81
94.47
79.63
71.56
97.26
93.69
84.52
83.91
90.33
95.03
84.48
98.11

6. REFERENCES[1] C. I. Chang, Hyperspectral Data Exploitation: Theory and Applications, Wiley-Interscience, Hoboken, NJ, 2007.
[2] P. Du, J. Xia, W. Zhang, K. Tan, Y. Liu, and S. Liu, “Multiple classifiersystem for remote sensing image classification: A review,” Sensors,
vol. 12, no. 4, pp. 4764–4792, 2012.
[3] A. Plaza, J. A. Benediktsson, J. W. Boardman, J. Brazile, L. Bruzzone, G. Camps-Valls, J. Chanussot, M. Fauvel, P. Gamba, A. Gualtieri,
M. Marconcini, J. C. Tilton, and G. Trianni, “Recent advances in techniques for hyperspectral image processing,” Remote Sensing of Environment, vol. 113, no. S1, pp. 110 – 122, 2009.
[4] L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, Wiley-Interscience, 2004.
[5] L. Breiman, “Random forest,” Machine Learn., vol. 45, no. 1, pp. 5–32,2001.
[6] J. J. Rodriguez, L. I. Kuncheva, and C. J. Alonso, “Rotation forest: A
new classifier ensemble method.,” IEEE Trans. Pattern Anal. Mach.
Intell, vol. 28, no. 10, pp. 1619–1630, 2006.
[7] J. Xia, P. Du, X. He, and J. Chanussot, “Hyperspectral remote sensing
image classification based on rotation forest,” IEEE Geosci. RemoteSens. Lett., vol. 11, no. 1, pp. 239 – 243, 2014.
[8] J. Xia, J. Chanussot, P. Du, and X. He, “Spectral-spatial classificationfor hyperspectral data using rotation forests with local feature extraction and markov random fields,” IEEE Trans. Geosci. Remote Sens.,
vol. 53, no. 5, pp. 2532–2546, 2015.
[9] P. Du, A. Samat, B. Waske, S. Liu, and Z. Li, “Random forest androtation forest for fully polarized sar image classification using polarimetric and spatial features,” ISPRS J. Photogramm Remote. Sens., vol.
105, pp. 38 – 53, 2015.
[10] T. Kavzoglu, I. Colkesen, and T. Yomralioglu, “Object-based classification with rotation forest ensemble learning algorithm using veryhigh-resolution worldview-2 image,” Remote Sensing Letters, vol. 6,no. 11, pp. 834–843, 2015.
[11] J. Arenas-Garcı́a, K. Petersen, G. Camps-Valls, and L. K. Hansen,“Kernel multivariate analysis framework for supervised subspace learning: A tutorial on linear and kernel multivariate methods,” IEEE Signal
Process. Mag., vol. 30, no. 4, pp. 16–29, 2013.
[12] J. Arenas-Garcı́a and G. Camps-Valls, “Efficient kernel orthonormalized pls for remote sensing applications,” IEEE Trans. Geosci. RemoteSens., vol. 46, no. 10, pp. 2872–2881, 2008.
[13] J.R. Quinlan, “Induction of decision trees,” Machine Learning, vol. 1,no. 1, pp. 81–106, 1986.
[14] L. I. Kuncheva and C. J. Whitaker, “Measures of diversity in classifier
ensembles and their relationship with the ensemble accuracy,” MachLearn., vol. 51, no. 2, pp. 181–207, 2003.