# Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification

Danfeng Hong, Xin Wu, Pedram Ghamisi, Jocelyn Chanussot, Naoto Yokoya, Xiao Xiang Zhu

► To cite this version:

Danfeng Hong, Xin Wu, Pedram Ghamisi, Jocelyn Chanussot, Naoto Yokoya, et al. Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58 (6), pp.3791-3808. 10.1109/TGRS.2019.2957251. hal-03142171

HAL Id: hal-03142171

https://hal.science/hal-03142171v1

Submitted on 28 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification

Danfeng Hong<sup>1,†</sup>, Member, IEEE, Xin Wu, Student Member, IEEE, Pedram Ghamisi<sup>1,†</sup>, Senior Member, IEEE, Jocelyn Chanussot<sup>1,†</sup>, Fellow, IEEE, Naoto Yokoya<sup>1,†</sup>, Member, IEEE, and Xiao Xiang Zhu<sup>1,†</sup>, Senior Member, IEEE

**Abstract**—So far, a large number of advanced techniques have been developed to enhance and extract the spatially semantic information in hyperspectral image processing and analysis. However, locally semantic change, such as scene composition, relative position between objects, spectral variability caused by illumination, atmospheric effects, and material mixture, has been less frequently investigated in modeling spatial information. Consequently, identifying the same materials from spatially different scenes or positions can be difficult. In this article, we propose a solution to address this issue by locally extracting invariant features from hyperspectral imagery (HSI) in both spatial and frequency domains, using a method called invariant attribute profiles (IAPs). IAPs extract the spatial invariant features by exploiting isotropic filter banks or convolutional kernels on HSI and spatial aggregation techniques (e.g., superpixel segmentation) in the Cartesian coordinate system. Furthermore, they model invariant behaviors (e.g., shift, rotation) by the means of a continuous histogram of oriented gradients constructed in a Fourier polar coordinate. This yields a combinatorial representation of

spatial-frequency invariant features with application to HSI classification. Extensive experiments conducted on three promising hyperspectral data sets (Houston2013 and Houston2018) to demonstrate the superiority and effectiveness of the proposed IAP method in comparison with several state-of-the-art profile-related techniques. The codes will be available from the website: https://sites.google.com/view/danfeng-hong/data-code.

**Index Terms**—Attribute profile (AP), feature extraction, Fourier, frequency, hyperspectral image, invariant, remote sensing, spatial information modeling, spatial-spectral classification.

## I. INTRODUCTION

LAND use and land cover (LULC) classification has been playing an increasingly vital role in the high-level image interpretation and analysis of remote sensing [1], [2]. Owing to the rich spectral information, hyperspectral imagery (HSI) enables the identification and detection of the materials at a more accurate level, which has been proven to be effective for LULC-related tasks, such as HSI classification [3]–[6], multimodality data analysis [7]–[10], and anomaly detection [11]–[13]. In spite of the fine spectral discrepancies in HSI, the noisy pixels, manual labeling uncertainty, and the intrinsic or extrinsic spectral variability inevitably degrade the classification performance when only the spectral profile is considered as the feature input. Fortunately, except for the spectral dimension, the two-dimensional image space can provide the extra spatial information to correct the errors in a local region by linking with different objects, and robustly eliminating the effects of spectral variability [14]–[17] between the same materials.

In [19], mathematical morphology has shown its superiority in modeling and extracting the spatial information of an image related to the geometric shape and scale of different objects. Based on this concept, Pesaresi and Benediktsson [20] developed morphological profiles (MPs) to segment high-resolution satellite imagery by applying a sequence of opening and closing operators to reconstruct or connect the targeted objects with a size-increasing structurized element (SE). The resulting morphological operator has been successfully extended and applied in the HSI classification [21], where the extended MPs (EMPs) are built on the first principal component (PC) of HSI obtained using principal component analysis (PCA) [22]. Fauvel et al. [23] designed a novel strategy of jointly

Manuscript received July 10, 2019; revised September 17, 2019 and October 28, 2019; accepted November 18, 2019. Date of publication January 16, 2020; date of current version May 21, 2020. This work was supported in part by the German Research Foundation (DFG) under Grant ZH 498/7-2, in part by the Helmholtz Association through the Framework of the Young Investigators Group SiPEO under Grant VH-NG-1018, and in part by the European Research Council (ERC) through the European Union's Horizon 2020 Research and Innovation Programme (Acronym: So2Sat) under Grant ERC-2016-StG-714087. The work of Naoto Yokoya was supported by the Japan Society for the Promotion of Science under Grant KAKENHI 18K18067. (Corresponding author: Xiao Xiang Zhu.)

Danfeng Hong and Xiao Xiang Zhu are with the Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234 Weßling, Germany, and also with the Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM), 80333 Munich, Germany (e-mail: danfeng.hong@dlr.de; xiaoxiang.zhu@dlr.de).

Xin Wu is with the School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China, and also with the Beijing Key Laboratory of Fractional Signals and Systems, Beijing Institute of Technology, Beijing 100081, China (e-mail: 040251522wuxin@163.com).

Pedram Ghamisi is with the Exploration Division, Machine Learning Group, Helmholtz Institute Freiberg for Resource Technology, Helmholtz-Zentrum Dresden-Rossendorf, 09599 Freiberg, Germany (e-mail: p.ghamisi@gmail.com).

Jocelyn Chanussot is with the Université Grenoble Alpes, INRIA, CNRS, Grenoble INP, LJK, F-38000 Grenoble, France, and also with the Faculty of Electrical and Computer Engineering, University of Iceland, 101 Reykjavik, Iceland (e-mail: jocelyn@hi.is).

Naoto Yokoya is with the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo 103-0027, Japan (e-mail: naoto.yokoya@riken.jp).

Color versions of one or more of the figures in this article are available online at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TGRS.2019.2957251
using SVMs and MPs for spatial-spectral hyperspectral classification. With the growing attention to MPs, a considerable volume of work related to HSI classification has frequently been reported in the literature [9], [24], [25]. Nevertheless, the MPs' ability to extract the diverse geometric features (e.g., textural, semantic) from hyperspectral images still remains limited, since its concept has a few limitations: for example, the shape of SEs is fixed and SEs are not able to characterize information related to the gray-level or higher-level (e.g., HSI) characteristics of the regions. More specifically, rich spectral information in HSIs makes the geometric structure of hyperspectral regions or scenes more complex, leading to difficulties in representing and extracting MPs with the SEs in an appropriate way.

To reduce the limitations of MPs, morphological attribute profiles (APs) were developed in [26] by applying a set of attribute filters (AFs) [27], connected operators utilized to process an image by considering only its connected components, to integrate the attribute components of the given image at different levels. Moreover, APs are flexible tools since they can be of different types (e.g., they can be purely geometric or related to the spectral values of the pixels or different characteristics such as spatial relations to other connected components). APs can be viewed as a generalized extension of MPs, yet APs are advantageous over MPs due to their flexibility in capturing a variety of region-based attributes (e.g., scale, geometry, and size). For example, APs allow geometrical characterization to be extracted hierarchically [28], thereby yielding a more effective analysis in remote-sensing images [29]–[31].

It is well known, however, that the connectivity between these defined AFs relies heavily on the geodesic reconstruction. This might lead to a problem of the so-called leakage, also known as over-reconstruction [18], where multiple regions corresponding to different objects could be merged into a single region due to improper linking. To alleviate the problem, many advanced models were proposed by using tree-based image representations to construct the AFs or APs by the means of nonredundant representations, automatic strategies, and optimization techniques [32]–[36]. Recently, a feasible solution to address the leakage problem was proposed in [37] in which the AFs were partially reconstructed in order to model and extract the spatially geometrical information to a particular specification. Another representative AFs-based method to automatically and precisely extract the spatial and contextual information is extinction profiles (EPs) [38] built based on extinction filters (EFs). EPs are less sensitive to changeable image resolution, as the used EFs are determined by extrema rather than a threshold given manually in the AP. Thus, EPs have shown their effectiveness in reducing the redundant details and preserving the discriminative geometrical features, making them more useful for the classification of remote-sensing data [39]–[41]. Similarly, the applications of APs and EPs in HSI classification are called as extended attribute profiles (EAPs), and extended EPs (EEPs), respectively. Yet the APs and its variants hardly consider and investigate semantic variations. For example, two visually similar patches from a particular scene,

e.g., Patch (red) and Patch (blue), Patch (green) and Patch (orange) in Fig. 1, should share basically identical feature representation. However, in reality, the following holds.

1. On one hand, due to the shift and rotation of pixels (objects), the extracted features inevitably suffer from substantial differences between the two similar batches: green and orange (see the histograms of Fig. 1 in the bottom left-hand corner). More specifically, the centered pixels for the two patches stand for the same material: trees, but the locally semantic change (e.g., rotation) makes the extracted APs largely different between two patches. This might be explained by the fact that these previous APs-based methods fail to robustly merge the structure information of surrounding pixels, tending to absorb “negative” or “easy-changing” characteristics that are sensitive to a semantic change.

2. On the other hand, despite presenting similar semantic characteristics, the slightly different components (e.g., some objects added or missed), the changed arrangement in the location and order of the objects, or spectral variability caused by illumination, topology change, atmospheric effects, and intrinsic mixing of the materials) would lead to substantial differences in the APs of the two similar patches: red and blue (see the histograms of Fig. 1 in the top left-hand corner). Moreover, for Patch (red) and Patch (blue) that are expected to be identified as the same material from the centered pixel perspective, there is a big difference between their APs, due to the moderately semantic change (other materials involved, e.g., trees) in Patch (blue) compared to Patch (red). This indicates that the APs are relatively sensitive to semantic change.

Limited by the two factors described above, the resulting APs might be vulnerable and sensitive to semantic change in a local region, tending to further enlarge the negative effects on follow-up feature matching or classifier learning. Although some researchers from the remote-sensing community have attempted to investigate the invariant feature representation [42] by integrating the APs and geometric invariant moments (GIMs) [43], the poor discriminative ability of GIM limits the classification performance of the HSI to a great extent.

To overcome the challenges and pitfalls of those previously proposed AP approaches, we propose to extract the invariant attributes (IAs) that can be robust against the semantic change in a hyperspectral scene by empowering the invariance to the APs, yielding the proposed IA profiles (IAPs). Direct evidence is shown on the right-hand side of Fig. 1, where there are only slight perturbations between the extracted features of the two-paired patches. The IAPs consist of two parts: spatial invariant features (SIFs) and frequency invariant features (FIFs). The former are constructed in the Cartesian coordinates, where the isotropic filter banks or convolutional kernels are first exploited for HSI, and the over-segmentation techniques are further used to spatially aggregate the filtered features. The latter convert discrete APs to continuous profiles by modeling the variant behaviors of pixels or objects (e.g., shift and rotation)
<figure><img src="image_2.png" /><figcaption>Fig. 1. Illustration clarifying the motivation of the proposed IAPs. Due to the difference in composition between the Red and Bule patches—that is, the Blue patch holds more materials, there is a big difference in their APs, while the proposed IAPs are only slightly different for the two patches. Similarly, for the shift or rotation of the local scene, the IAPs are obviously more robust than the conventional APs. Given an HSI’s patch, PCA is first performed on the patch to reduce its dimension to be 3. Then, the APs can be extracted from the three PCs, respectively, and the final APs can be obtained by stacking all APs. The AF used in the APs is the region attribute and the code can be found in [18]. Note that the APs or IAPs are extracted based on the whole image and the cropped patches are only visual examples and the corresponding histograms are only the profile representation of the centered pixels.</figcaption></figure>

in a Fourier polar coordinate system. More specifically, our
contributions can be highlighted as follows.

1) We propose a novel feature extractor for HSI, called IAPs. As the name suggests, these aim at extracting invariant feature representation by applying a sequence of well-designed AFs that are insensitive and even invariant to the change of pixels or materials in local regions.

2) The proposed approach is capable of modeling SIFs by isotropically filtering HSI in the first step. The filtered features can then be grouped into the semantically meaningful object-based representation.

3) To further improve the completeness and discrimination of the invariant features, the FIFs in this article are also designed by extracting the continuous Fourier convolutional features in polar coordinates.

4) Two relatively new and challenging hyperspectral data sets are used to assess and compare the classification accuracies of state-of-the-art Aps, and our IAPs. Experimental results indicate that the classification performance of IAPs is superior to that of using the APs-based approaches, demonstrating the necessity and progressiveness of investigating and handling the issue of local semantic change in HSI classification.

The rest of this article is organized as follows. Section II presents the methodology for extracting IAPs and the workflow for HSI classification, focusing on the design of SIFs, and FIFs. In Section III, we provide the experimental results and analysis, as well as brief discussion of two hyperspectral data sets from both a qualitative and quantitative perspective. Finally, our main conclusions are summarized in Section IV.

## II. METHODOLOGY

In this section, the proposed IAPs are first introduced from two different aspects: SIFs and FIFs. The holistic workflow with the designed IAPs as the input is then developed for the HSI classification.

### A. Invariant Feature Extraction in the Spatial Domain

As shown in Fig. 1, the APs in spatial domain are sensitive to various factors that can give rise to semantic change, further leading to unexpected degradation in modeling spatial information. One feasible solution for this problem is to find and extract the invariant feature representation from the image space. It is well known that isotropic filtering (or convolution) [44] is a good tool that performs robustly against shift or rotation behavior of image patches and can simultaneously eliminate various other variabilities (e.g., salt-pepper noise, missing local information) effectively. Therefore, the robust convolutional features (RCFs) can be extracted from the HSI in the form of feature set $\Omega(\cdot)$

$$ \mathcal{F}_{RCF} = \Omega_1(\mathbf{I}) = [\mathcal{F}_1, \dots, \mathcal{F}_k, \dots, \mathcal{F}_D] \quad (1) $$

where the features of the kth band ($\mathcal{F}_k$) can be computed by

$$ \mathcal{F}_k = \mathbf{I}_k \otimes K_{\text{conv}}. \quad (2) $$

$\mathbf{I} \in \mathbb{R}^{W \times H \times D}$ is the HSI with D bands by $W \times H$ pixels, where $\mathbf{I}_k$ denotes the kth band of $\mathbf{I}$; $K_{\text{conv}}$ is defined as the convolutional kernel for isotropically aggregating the local spatial information, and the operator $\otimes$ denotes the convolutional operation.

Furthermore, we combine the filtered or convolutional features ($\mathcal{F}_{RCF}$) with spatial aggregation (SA) techniques,
<figure><img src="image_3.png" /><figcaption>Fig. 2. Example illustrating the two-step extraction process of SIFs, where SLIC means simple linear iterative clustering.</figcaption></figure>

such as super-pixel segmentation, to enhance the object-based semantic features, i.e., edges, shape, and the invariance of these features. In our case, we select a popular super-pixel segmentation approach: simple linear iterative clustering (SLIC) [45]. More specifically, the SA step is implemented on the basis of RCF; thus the pixel-wise (e.g., ith pixel) SIFs can be represented as

$$ \mathcal{F}_{\text{SIFs}}^{i} = \frac{1}{N_q} \sum_{j=1}^{N_q} \mathcal{F}_{\text{RCF}}^{j}, \quad j \in \phi_{i,q} \qquad (3) $$

where $N_q$ is the number of pixels in the qth super-pixel, while $\phi_{i,q}$ is defined as a pixel set (qth super-pixel) including the ith targeted pixel. The final SIFs are simply stacked as

$$ \mathcal{F}_{\text{SIFs}} = [\mathcal{F}_{\text{SIFs}}^{1}, \dots, \mathcal{F}_{\text{SIFs}}^{i}, \dots, \mathcal{F}_{\text{SIFs}}^{N}] \qquad (4) $$

where $N = W \times H$ denotes the number of pixels in a given hyperspectral scene. It should be noted that in the original SLIC algorithm, the superpixels are segmented on the CIELAB color space rather than RGB space, since the differences or changes between pixels (or materials) in the CIELAB space can be perceived or captured more easily. From the feature-level perspective, CIELAB should hold a more discriminative feature representation compared to the RGB space. Naturally, the HSI is capable of better identifying the pixels due to the richer and more discriminative spectral information. However, considering the spectral redundancy in HSI, we first applied the PCA on the whole HSI to reduce the spectral dimension and simultaneously preserve the spectral information as much as possible, then performed the SLIC on the first three PCs.

The aforementioned two steps yield the SIFs of the IAPs, as illustrated in Fig. 2.

## B. Invariant Feature Extraction in the Frequency Domain

Although SIFs are insensitive to the semantic changes in a local neighborhood, it still fails to accurately describe the shift or rotation behavior of an image (or an object) due to the quantization artifacts in a discrete coordinate system [46]. Instead of locally estimating the discrete coordinates with the pose normalization, i.e., histogram of oriented gradients (HOG) [47], a Fourier-based analysis technique has guaranteed the invariance of feature extraction in a polar coordinate system [46], [48], [49]. More significantly, the features only extracted from the spatial domain are relatively limited in representation ability and diversity, particularly for the complex hyperspectral scene, e.g., including various spectral variabilities. Therefore, the benefits of feature extraction in the frequency domain are, on the one hand, to enrich the diversity of the features, thereby further improving the performance of HSI classification;

and, on the other hand, to be capable of effectively modeling the invariant behaviors by the means of continuous signal representation, yielding robust feature extraction against a variety of semantic change in HSI.

For example, the rotation behavior of an object or local image patch cannot be well modeled by a discrete histogram, but can be explained by a circular shift by continuous and smooth signal representation. For this reason, the continuous Fourier transformation has been proven to be an effective tool to model the rotation behavior with any angles, e.g., either integer or noninteger, more accurately [46]. Fig. 3 illustrates the two strategies for extracting feature descriptors for the traditional discrete HOG, and continuous Fourier-based HOG. In detail, the differences from the pixel-wise feature design to the spatially regional feature aggregation are clarified by the two approaches, respectively. Another can be seen in Fig. 3, where there is a relatively obvious change between two image regions or patches in the form of a discrete histogram (e.g., HOG), due to the 10° rotation behavior. Conversely, the Fourier-based HOG descriptor is robust to the big semantic gap when the image scene changes significantly, which only yields a slight shift along the horizontal coordinate, and basically keeps the feature shape unchanged. In the following, we will detail the procedure for extracting the Fourier-based continuous HOG from the pixel-wise features to the region-based representation.

1) Analysis of Rotation-Invariance: As the name suggests, rotation-invariance refers to the extracted features $f(x, y)$ for a given pixel located in the $(x, y)$ of the image remaining the same or unchanged when the image rotates with a $g^\circ$ angle. Similar to [49], the rotation behavior can be formulated as

$$ gf(x, y) = f(x, y), \text{ or } h(\mathbf{I}(x, y) \circ \mathbf{T}_g) = h(\mathbf{I}(x, y)) \qquad (5) $$

where $h(\mathbf{I})$ is abstracted as a feature extractor from the input image $\mathbf{I}$, and $\mathbf{T}_g$ represents an operation of coordinate transformation. For those pixels that do not change the locations, the invariant features can be simply deduced by (5) as $gf(x, y) = f(x, y)$. For other pixels whose locations are changed, the coordinate transformation $\mathbf{T}_g$ has to be considered to formulate the rotation-invariance [49] as

$$ gf(x, y) = f(\mathbf{T}_g(x, y)) = f(x, y) \circ \mathbf{T}_g. \qquad (6) $$

We then have the equivalent condition $gf = f \circ \mathbf{T}_g$, which has been proven and widely used in many works [50]–[52].

2) Pixel-Wise Fourier Features: Let a 2-D location of any one pixel in a given image $\mathbf{I}$ be $(x, y)$ in Cartesian coordinates or $(\|\mathbf{D}(x, y)\|, \theta(\mathbf{D}(x, y)))$ in polar coordinates, where $\|\mathbf{D}(x, y)\|$ and $\theta(\mathbf{D}(x, y))$ are defined as the magnitude and the phase information of a complex number: $\mathbf{D}(x, y) = dx + dyi$, and $dx$ and $dy$ are denoted as the horizontal and
<figure><img src="image_4.png" /><figcaption>Fig. 3. Illustration clarifying the advantage of continuous HOG, represented by the polarized Fourier analysis from pixel-wise features to regional descriptors over the discrete HOG encoded by gradient binning (cells) and regional aggregation (blocks) in the real image regions before and after 10° rotation.</figcaption></figure>

vertical gradients, respectively, in the location of (x, y) of **I** in Cartesian coordinates. Thus, given an input pixel *p*, its *m*-th order Fourier representation, denoted as **F**<sub>*m*</sub>(x, y), can be obtained by

$$ 
\begin{aligned}
\mathbf{F}_m(x, y) &= \frac{1}{2\pi} \int_0^{2\pi} h(\varphi)e^{-im\varphi} \\
&= \|\mathbf{D}(x, y)\| e^{-im\theta(\mathbf{D}(x, y))} \quad (7)
\end{aligned}
$$

where $h(\varphi)$ is an orientation distribution function with respect to the angle $\varphi$ in the pixel *p*. This function can be given by an impulse response function $\delta$ with integral $\|\mathbf{D}(x, y)\|$

$$ h(\varphi) := \|\mathbf{D}(x, y)\| \delta(\varphi - \theta(\mathbf{D}(x, y))). \quad (8) $$

Unlike the Fourier transformation in Cartesian coordinates, the polarized Fourier transformation has been proven to be effective for separating the angular information ($e^{-im\theta(\mathbf{D}(x, y))}$) and radial basis from the Fourier representations [46]. As shown in (7), the rotated angle in Cartesian coordinates can be equivalently explained by a shift behavior in the polar representation.

As shown in (7), one natural and intuitive way to eliminate the effects of the image rotation in feature extraction is to fit the angular information. It is, however, hardly possible to directly estimate the phase information [$\theta(\mathbf{D}(x, y))$], due to its complexity, and uncertainty. Alternatively, we may enforce the

Fourier order *m* to be zero, thereby achieving the same goal for removing the phase information. When a $g^\circ$ rotation is applied on **F**<sub>*m*</sub>(x, y), accordingly to the mathematical property of Fourier transformation in Polar coordinates [46], we then deduce the rotated Fourier representation $g\mathbf{F}^\circ_m(x, y)$ as

$$ 
\begin{aligned}
g\mathbf{F}_m(x, y) &= [\|\mathbf{D}(x, y)\| e^{-im(\theta(\mathbf{D}(x, y)) + g^\circ)}] \circ \mathbf{T}_g \\
&= e^{-img^\circ} [\mathbf{F}_m(x, y) \circ \mathbf{T}_g]. \quad (9)
\end{aligned}
$$

Owing to the self-steerability of the Fourier basis under polar coordinates [53]—that is, such a basis can be self-steered to any orientation—we can construct a sequence of Fourier base with self-adaptive rotation angles. According to the special property, the rotated Fourier representation in (9) can be multiplied or convoluted by another Fourier basis with the same rotation behavior (e.g., a $g^\circ$ angle)

$$ 
\begin{aligned}
g(\mathbf{F}_m(x, y) * \mathbf{F}_{m'}(x, y)) \\
&= e^{-i(m+m')g^\circ} [\mathbf{F}_m(x, y) * \mathbf{F}_{m'}(x, y)] \circ \mathbf{T}_g \\
g(\mathbf{F}_m(x, y) \cdot \mathbf{F}_{m'}(x, y)) \\
&= e^{-i(m+m')g^\circ} [\mathbf{F}_m(x, y) \cdot \mathbf{F}_{m'}(x, y)] \circ \mathbf{T}_g \quad (10)
\end{aligned}
$$

where * and · operators denote the convolution and multiplication behaviors, respectively. In order to make the extracted features rotation-invariant, we have to meet the condition of (6): that is, $gf(x, y) = f(x, y) \circ \mathbf{T}_g$. It is, therefore, natural to have a solution of rotation-invariance as long as $m+m' = 0$
is satisfied. Note that the convolutional representation of (10) is regarded as the final rotation-invariant output.

Supported by the above analysis and derivation of rotation-invariance in theory, we further detail the procedures to extract the rotation-invariant features from the images in practice. This process consists of three parts, as follows.

1) Part 1 (Magnitude): By applying the polarized Fourier transformation on each pixel of the input image, we can obtain the m magnitude features corresponding to m different Fourier orders, which can be formulated for each pixel as $\mathcal{F}_m^1(x, y) = \|\mathbf{D}_m(x, y)\|$, $m = 0, 1, \dots, m$.

2) Part 2 (Absolute Rotation-Invariant Features:) The rotation behavior can be compensated by completely removing the phase information by using (10), and arranging $m + m' = 0$. Then, the features of part 2 can be represented as $\mathcal{F}_m^2(x, y) = \mathbf{F}_m(x, y) * \mathbf{F}_{m'(x, y)}$, and $m = -m'$.

3) Part 3 (Relative Rotation-Invariant Features:) To reduce the loss of rich phase information, the relative rotation-invariant features are developed by coupling the adjunct convolutional Fourier representations obtained with two neighboring convolutional kernel-radii [54]. This process can be performed by

$$ \mathcal{F}_m^3(x, y) = \frac{(\mathbf{F}_m(x, y) * \mathbf{F}_{m',r_1}(x, y))(\overline{\mathbf{F}_m(x, y) * \mathbf{F}_{m',r_2}(x, y)})}{\| (\mathbf{F}_m(x, y) * \mathbf{F}_{m',r_1}(x, y))(\overline{\mathbf{F}_m(x, y) * \mathbf{F}_{m',r_2}(x, y)}) \|} \quad (11) $$

which is subject to $m \neq m'$. The terms $r_1$ and $r_2$ are the radii of two different convolutional kernels, and the symbol $\overline{\mathbf{F}}$ is defined as the complex conjugate.

Combined with the three parts, the final pixel-wise Fourier features (PWFFs) can be written as

$$ \mathcal{F}_{\text{PWFF}}(x, y) = [\mathcal{F}_0^1(x, y), \dots, \mathcal{F}_m^1(x, y), \dots, \mathcal{F}_0^2(x, y), \dots, \mathcal{F}_m^2(x, y), \dots, \mathcal{F}_0^3(x, y), \dots, \mathcal{F}_m^3(x, y)]. \quad (12) $$

By collecting all PWFF, we then have

$$ \mathcal{F}_{\text{PWFF}} = \{ \mathcal{F}_{\text{PWFF}}(x, y) \}_{x=1}^{W} \}_{y=1}^{H}. \quad (13) $$

3) Regional Descriptors: Analogous to the HOG blocks, which aim to describe object-based contextual information, we aggregate the PWFF into region-based descriptors with the use of isotropically triangular convolutional kernels. In our case, multiscaled convolutional kernels are used to capture the semantic information of different receptive fields. The scaling setting will be discussed in the experimental section. Finally, the resulting FIFs are

$$ \mathcal{F}_{\text{FIFs}} = [\mathcal{F}_{\text{PWFF}}^{C_1}, \dots, \mathcal{F}_{\text{PWFF}}^{C_j}, \dots] \quad (14) $$

where $\mathcal{F}_{\text{PWFF}}^{C_j}$ denotes the regional descriptors with the jth convolutional kernel.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Step-wise workflow to extract the proposed IAPs that consist of SIFs and FIFs of order fitting and magnitude maps.</figcaption></figure>

### C. IAPs

For our proposed IAPs, the fusion strategy of SIFs and FIFs is nothing but a stacking operation. Despite its simplicity, this fusion strategy has been widely and successfully applied in various feature extraction tasks. Fig. 4 illustrates the step-wise workflow of extracting SIFs and FIFs, which can be delineated more specifically as follows.

1) Step 1 Group the HSI and Compute Its Gradients: Due to the redundancy of HSI, some adjunct bands might share similar invariance. To address that circumstance, we first group the HSI, e.g., using clustering techniques (k-means algorithm in our case), then compute the horizontal and vertical gradients for each group by means of the maximum magnitude response (see the first step in Fig. 4). Note that the final classification performance is sensitive to the number of grouped HSIs, that is, excessively large or small ones would make the information redundant or coupled across bands. Therefore, a proper parameter setting is needed. In our case, the number of grouped HSIs can be effectively
<figure><img src="image_6.png" /><figcaption>Fig. 5. Proposed IAPs-based classification framework.</figcaption></figure>

determined by cross-validation on the labeled training set.

2) *Step 2 Extract Polarized Fourier Features by Gradients:* The Fourier complex form can be generated by gradients (see Section II-B2). Thus, the polarized Fourier representation given in (7) can be obtained by applying Fourier transform to the complex number.

3) *Step 3 Construct the Regional Descriptors on Both Spatial and Frequency Domains:* We model the region-based representation by using isotropic spatial filters, and Fourier convolutional kernels.

4) *Step 4 Generate the Proposed IAPs:* The SA technique, i.e., superpixel segmentation, is used to generate the SIFs. In the frequency domain, there are two parts in the FIFs: magnitude, and order fitting. The latter consists of absolute rotation-invariant features and relative rotation-invariant features. In that way, the IAPs can be abstracted as $F_{\text{IAPs}} = [F_{\text{SIFs}}, F_{\text{FIFs}}]$.

## D. IAPs-Based HSI Classification Framework

With the proposed IAPs, we intend to develop an automatic HSI classification system. For this purpose, an IAPs-based HSI classification framework is designed. As shown in Fig. 5, the framework is mainly composed of HSI grouping, IAPs extraction, feature stacking, and feature learning (FL) [or dimensionality reduction (DR)] [55]. For more details, the HSI is grouped in a band-wise way, while the FL step is simply and effectively conducted by PCA, in our case. In the experiments below, we found that the FL or refinement step plays a significant role in improving the classification performance. This could be reasonably explained by the redundant and noisy concentrated features.

## E. Feasibility and Effectiveness Analysis of the Proposed IAPs for HSI Classification

Different from pixel-wise semantic labeling of natural images, an effective and accurate HSI classification algorithm usually depends on jointly modeling spatial and spectral information. Owing to the effective spatial structure

(or pattern) modeling, the MPs (or APs) and their variants have been widely and successfully applied for HSI classification. With this motivation, the proposed IAPs similarly consider the spatially structural information in the form of semantic patches or objects with SA strategy (e.g., superpixel), yielding SIFs. In [56], the SIFs can be regarded as the low-level shift-invariant feature representation, which has been theoretically proven to be effective for robustly addressing variability within the same class. On the other hand, the IAPs model the irregular textural information from the frequency domain, yielding FIFs, in order to further improve the feature discrimination. Similarly, there is also a good theoretical support for the extracted FIFs in [46] and [48], demonstrating its effectiveness in extracting the invariant features from optical remote-sensing images for classification, and detection tasks. The compact combination of the two features with their IAs makes it possible for the proposed IAPs to classify the HSI more robustly, and accurately. More notably, unlike those previously proposed MPs and APs that are only performed on first few components obtained by PCA, our IAPs are not only capable of extracting spatial semantic features with SA techniques and convolutional (or filtering) operators, but also fully considering the spectral information by the means of grouping strategy, maximum magnitude response, and FL or DR after feature extraction instead of DR before feature extraction (e.g., using PCA).

# III. EXPERIMENTS

## A. Data Description

We quantitatively and qualitatively evaluate the algorithm performance on three representative and promising HSI data sets: Pavia University, Houston2013, and Houston2018, in the form of image classification. These two data sets are briefly introduced as follows.

1) *Pavia University Data Set:* The hyperspectral scene was acquired by the ROSIS sensor over the campus of Pavia University, Paiva, Italy. The image consists of 610 × 340 pixels at a ground sampling distance (GSD) of
<figure><img src="image_7.png" /><figcaption>Fig. 6. Scene categories and visualization of false-color image, the distribution of training and test samples, and classification maps of different compared algorithms using two classifiers (top: NN; bottom: RF) on the Pavia University data set.</figcaption></figure>

<table><caption>TABLE I
SCENE CATEGORIES OF THE PAVIA UNIVERSITY DATA SET WITH THE NUMBER OF TRAINING AND TEST SAMPLES SHOWN FOR EACH CLASS</caption><thead><tr><th>Class No.</th><th>Class Name</th><th>Training</th><th>Test</th></tr></thead><tbody><tr><td>1</td><td>Asphalt</td><td>548</td><td>6631</td></tr><tr><td>2</td><td>Meadows</td><td>540</td><td>18649</td></tr><tr><td>3</td><td>Gravel</td><td>392</td><td>2099</td></tr><tr><td>4</td><td>Trees</td><td>524</td><td>3064</td></tr><tr><td>5</td><td>Metal Sheets</td><td>265</td><td>1345</td></tr><tr><td>6</td><td>Bare Soil</td><td>532</td><td>5029</td></tr><tr><td>7</td><td>Bitumen</td><td>375</td><td>1330</td></tr><tr><td>8</td><td>Bricks</td><td>514</td><td>3682</td></tr><tr><td>9</td><td>Shadows</td><td>231</td><td>947</td></tr></tbody><tfoot><tr><td colspan="2">Total</td><td>3921</td><td>42776</td></tr></tfoot></table>

1.3 m with 103 spectral bands in the range of 430–860 nm. In this data set, nine main categories are investigated for land cover classification task. The number of training and test samples are specifically listed in Table I, while the corresponding sample distribution is given in Fig. 6.

2) Houston2013 Data Set: The ITRES CASI-1500 sensor was used to acquire the data over the campus of University of Houston and its surrounding areas, in Houston, TX, USA. This data set was provided for the 2013 IEEE GRSS data fusion contest, and is composed of 349 × 1905 pixels with 144 spectral channels ranging from 364 to 1046 nm at a spectral sampling of 10 nm. There are 15 challenging classes in the form of LULC in the scene. Table II lists the scene categories and the number of training and test samples used in the classification task, while Fig. 7 visualizes the false-color image of the hyperspectral scene and the sample distribution of the training and test set.

3) Houston2018 Data Set: The data set is an airborne multimodal data product, where the hyperspectral data was acquired by the same IRTES CASI-1500 sensor at a GSD of 1 m. This data has become available from the 2018 IEEE GRSS data fusion contest and its size is 601 × 2384 with 50 spectral bands sampling the wavelength of between 380 and 1050 nm at intervals of 10 nm. The specific training and test

<table><caption>TABLE II
SCENE CATEGORIES OF THE HOUSTON2013 DATA SET WITH THE NUMBER OF TRAINING AND TEST SAMPLES SHOWN FOR EACH CLASS</caption><thead><tr><th>Class No.</th><th>Class Name</th><th>Training</th><th>Test</th></tr></thead><tbody><tr><td>1</td><td>Healthy Grass</td><td>198</td><td>1053</td></tr><tr><td>2</td><td>Stressed Grass</td><td>190</td><td>1064</td></tr><tr><td>3</td><td>Synthetic Grass</td><td>192</td><td>505</td></tr><tr><td>4</td><td>Tree</td><td>188</td><td>1056</td></tr><tr><td>5</td><td>Soil</td><td>186</td><td>1056</td></tr><tr><td>6</td><td>Water</td><td>182</td><td>143</td></tr><tr><td>7</td><td>Residential</td><td>196</td><td>1072</td></tr><tr><td>8</td><td>Commercial</td><td>191</td><td>1053</td></tr><tr><td>9</td><td>Road</td><td>193</td><td>1059</td></tr><tr><td>10</td><td>Highway</td><td>191</td><td>1036</td></tr><tr><td>11</td><td>Railway</td><td>181</td><td>1054</td></tr><tr><td>12</td><td>Parking Lot1</td><td>192</td><td>1041</td></tr><tr><td>13</td><td>Parking Lot2</td><td>184</td><td>285</td></tr><tr><td>14</td><td>Tennis Court</td><td>181</td><td>247</td></tr><tr><td>15</td><td>Running Track</td><td>187</td><td>473</td></tr></tbody><tfoot><tr><td colspan="2">Total</td><td>2832</td><td>12197</td></tr></tfoot></table>

information for the data is detailed in Table III, and a false-color image and the locations of labeled samples of training and test set are also given in Fig. 8.

In the second data set, we artificially chose a number of challenging pixels to act as the training samples with multiple different experiments on the given ground truth (see [57] for more details). The benefits of the strategy to separate the training and test sets are twofold. On one hand, unlike random selection for training and test set that usually yields a very high performance, such a challenging setting may help us assess the potential and performance of extracted features more effectively. On the other hand, using the fixed training and test samples also contributes to making a consistent fair comparison by reproducing the experimental results using different methods.

## B. Experimental Setup

1) Evaluation Metrics: Pixel-wise classification is explored as a potential application for quantitatively evaluating the performance of these feature extraction algorithms.
<figure><img src="image_8.png" /><figcaption>Fig. 7. Scene categories and visualization of false-color image, the distribution of training and test samples, and classification maps of different compared algorithms using two classifiers (top: NN; bottom: RF) on the Houston2013 data set.</figcaption></figure>

Three commonly used criteria, overall accuracy (OA), average accuracy (AA), and Kappa coefficient (κ), are adopted to intuitively quantify the experimental results using two simple classifiers: nearest neighbor (NN) and random forest (RF). The two classifiers have been widely used in many works related to HSI classification [58], [59]. Please note that the reason for selecting the two classifiers in our case is to emphasize the performance gain from the features rather than from the complex and advanced classifiers.

2) *State-of-the-Art Comparison in Related Works:* The MP-based or AP-based methods we investigate in this article are obviously categorized into unsupervised feature extraction. Therefore, the spatial-spectral features are extracted from the whole HSI. To demonstrate the effectiveness and superiority of IAPs, the baseline [original spectral features (OSFs)] and other six advanced MPs or APs are compared with the proposed method. Specifically, they are EMPs, EAPs<sub>a</sub>, EAPs<sub>s</sub>, EAPs<sub>i</sub>, EAPs<sub>all</sub>, and EEPs, where *a*, *s*, *i*, and all denote the attributes for the area of the regions, the standard deviation in the regions, moment of inertia, and the stack of the previous three attributes, respectively.

3) *Algorithm Configuration:* As a matter of routine, PCA was performed on the original HSI to obtain several PCs (the first three PCs in our case) before applying opening and closing operations in MPs or thinning and thickening ones in APs. Moreover, the parameters for MPs and each attribute of APs are set by following the studies in [18], specifically defined as follows.

1) *MPs:* The scales are 10, that is, there were ten openings and closings.

2) $AP_a: \lambda_a = [100, 500, 1000, 2000, 3000, \dots, 7000, 8000]$.

3) $AP_s: \lambda_s = [0.1, 0.5, 1, 2, 3, 4, 5, 6, 7, 8]$.

4) $AP_i: \lambda_i = [0.1, 0.15, 0.2, 0.25, 0.3, \dots, 0.5, 0.55]$.

For the EP, only one parameter, called desired levels (*dl*), needs to be given, due to its automatic process. To extract the EPs in our case, we fully follow the same setup steps suggested in [38]; the *dl* is assigned to be 7. More details can be found in [38] and [60].

Furthermore, the parameters in our IAPs consist of the number of scaled convolution kernels (*n*<sub>s</sub>) and their radii
<table><caption>TABLE III
SCENE CATEGORIES OF THE HOUSTON2018 DATA SET, WITH THE NUMBER OF TRAINING AND TEST SAMPLES SHOWN FOR EACH CLASS</caption><thead><tr><th>Class No.</th><th>Class Name</th><th>Training</th><th>Test</th></tr></thead><tbody><tr><td>1</td><td>Healthy Grass</td><td>500</td><td>9299</td></tr><tr><td>2</td><td>Stressed Grass</td><td>500</td><td>32002</td></tr><tr><td>3</td><td>Artificial Turf</td><td>68</td><td>616</td></tr><tr><td>4</td><td>Evergreen Trees</td><td>500</td><td>13088</td></tr><tr><td>5</td><td>Deciduous Trees</td><td>500</td><td>4548</td></tr><tr><td>6</td><td>Bare Earth</td><td>451</td><td>4065</td></tr><tr><td>7</td><td>Water</td><td>26</td><td>240</td></tr><tr><td>8</td><td>Residential Buildings</td><td>500</td><td>39262</td></tr><tr><td>9</td><td>Non-Residential Buildings</td><td>500</td><td>223184</td></tr><tr><td>10</td><td>Roads</td><td>500</td><td>45310</td></tr><tr><td>11</td><td>Sidewalks</td><td>500</td><td>33502</td></tr><tr><td>12</td><td>Crosswalks</td><td>151</td><td>1365</td></tr><tr><td>13</td><td>Major Thoroughfares</td><td>500</td><td>45858</td></tr><tr><td>14</td><td>Highways</td><td>500</td><td>9349</td></tr><tr><td>15</td><td>Railways</td><td>500</td><td>6437</td></tr><tr><td>16</td><td>Paved Parking Lots</td><td>500</td><td>10975</td></tr><tr><td>17</td><td>Unpaved Parking Lots</td><td>14</td><td>135</td></tr><tr><td>18</td><td>Cars</td><td>500</td><td>6078</td></tr><tr><td>19</td><td>Trains</td><td>500</td><td>4865</td></tr><tr><td>20</td><td>Stadium Seats</td><td>500</td><td>6324</td></tr><tr><td></td><td>Total</td><td>8210</td><td>496502</td></tr></tbody></table>

of convolution kernels in spatially isotropic filtering (r), the number of Fourier orders (m), and the dimension in FL or DR (d) as well as the number of the grouped HSIs ($n_g$). In practical applications, these parameters can be determined by cross-validation on the available training set in order to achieve an automatic system for HSI feature extraction, and classification. More specifically, the parameter combination is set to $n_s = 3$, $r = [2, 4, 6]$, $m = [0, 1, 2, 3]$, $d = 30$, and $n_g = 5$ for the Pavia University data set, $n_s = 3$, $r = [2, 4, 6]$, $m = [0, 1, 2, 3]$, $d = 30$, and $n_g = 4$ for the Houston2013 data set, and $n_s = 3$, $r = [2, 4, 6]$, $m = [0, 1, 2]$, $d = 40$, and $n_g = 4$ for the Houston2018 data set. Note that the dimension of the original IAPs without FL or DR is 341 for the first data set, 396 for the second data set, and 208 for the latter data set in our case, which includes the original spectral signatures (103, 144, and 50), SIFs (103, 144, and 50), and FIFs (135, 108, and 84). In Section III-C, we experimentally discuss and analyze the various parameters used in the process of extracting the IAPs.

### C. Results and Discussion

1) *Pavia University Data Set:* The classification maps and corresponding quantitative results in terms of OA, AA, and $\kappa$ for different compared methods using NN and RF classifiers are given in Fig. 6 and Table IV, respectively.

In detail, the classification accuracies using extracted features are much higher than that of using only OSF (at least 7% increase) for both classifiers. For example, the EMPs can effectively represent the structure information of objects in the HSI, yielding competitive classification performance even better than many single-attribute EAPs, such as EAPs$_s$, EAPs$_i$. By focusing on HSI's spatial information, the EAP$_a$ method is able to excavate the regional features, leading to a more smooth classification result (see Fig. 6). As expected, collecting multiple attributes into a stacked vector representation (EAPs$_{all}$) achieves a great improvement in classification

performance (around 5% in terms of OAs) compared to the single-attribute one. However, the use of EFs makes another breakthrough on the basis of AF-based profiles, with an additional 2% increase in terms of OAs using the two classifiers (EEPs versus EAPs$_{all}$).

Although the above-mentioned methods have shown superiority in spatial information modeling and further obtained excellent performance on the pixel-wise classification task, yet their ability in addressing various spectral variabilities and semantic changes of local scene or objects is relatively weak. In contrast, the proposed IAP method is capable of extracting the intrinsic invariant feature representation from the HSI, achieving a more effective feature extraction. As shown in Fig. 6, there are less noisy points in the classification maps obtained by our method. In particular, the Meadows class located in the bottom of the studied scene shows a more reasonable result, which is approaching to the manually labeled ground truth. Similarly, the IAP approach dramatically outperforms the others with a quantitative comparison in Table IV.

2) *Houston2013 Data Set:* Fig. 7 visualizes the classification maps of different feature extraction methods using two classifiers on the Houston2013 data set, while Table V quantifies the corresponding experimental results in terms of OA, AA, and $\kappa$ as well as the classification accuracy for each class.

Generally speaking, the classification performance with the feature extraction step is obviously superior to that with the OSF (Baseline), which is consistent for both NN and RF classifiers. Although the EMPs yield a similar performance compared to the EAPs with a single attribute (e.g., EAPs$_a$, EAPs$_s$, and EAPs$_i$) using the two classifiers, it is worth noting that those single-attribute EAPs usually hold a lower feature dimension. This indicates, to some extent, that EAPs would have the advantages over EMPs in extracting the spatial information of the image, since EAPs based methods could make it possible to model more geometrical features (e.g., size, shape, and texture). As expected, stacking all single EAPs is of great benefit in finding a more discriminative feature representation; thus the resulting EAPs$_{all}$ performs better at classifying each material than any single EAP or EMP. With the use of EFs, the developed EEPs outperform the aforementioned methods. Particularly when using the RF classifier, there is an obvious improvement in classification accuracy, improving the OAs by approximately 2% and 5%, over the EAPs$_{all}$ and EMPs, respectively.

Remarkably, our proposed IAP method not only outperforms other feature extraction operators overall in terms of OA, AA, and $\kappa$, but also it achieves the desirable results for each class, especially for challenging classes like Commercial, Highway, and Parking Lot1. For these classes, the IAPs make a significant performance improvement, at an increase of at least 10% accuracy. We can also observe from Fig. 7 that the classification maps obtained by the proposed feature extractor are smoother in the regions sharing the same materials, and sharper on the edges between different materials. Furthermore, there are some object deformations (e.g., shift and rotation) in the stadium and its surroundings located in the middle of
<figure><img src="image_9.png" /><figcaption>Fig. 8. Scene categories and visualization of false-color image, the distribution of training and test samples, and classification maps of different compared algorithms using two classifiers (NN and RF) on the Houston2018 data set.</figcaption></figure>

the studied HSI. Owing to the robustness in local semantic
changes of the scene, the IAPs obtain more accurate classifi-
cation maps in the area in and around the stadium.

3) *Houston2018 Data Set:* For the Houston2018 data set, we make a similar visual comparison for eight compared approaches, as given in Fig. 8; the specific numerical results are detailed in Table VI.

As observed in Table VI, the same basic trend appears when
using NN and RF for all candidates: that is, the classifica-
tion accuracies using RF are higher than those using NN.
More specifically, with the OSFs as the input, the baseline

only holds 60.99% and 65.85% results using NN and RF
classifiers, respectively, due to the lack of spatial information
modeling. In MPs, the spatial information is considered in
the form of openings and closings, yet the EMPs yield a
relatively poor classification performance. This might result
from some intrinsic limitations of MPs. For example, the MPs
are not capable of adaptively making an effective connec-
tion between different scaled objects, further limiting their
ability to characterize the semantic structural information.
Despite the use of mathematical morphological attribute filers
that are able to generate a richer geometrical description,
<table><caption>TABLE IV
QUANTITATIVE CLASSIFICATION PERFORMANCE FOR EIGHT DIFFERENT ALGORITHMS ON THE PAVIA UNIVERSITY DATA SET</caption><thead><tr><th>Methods</th><th colspan="2">Baseline (103)</th><th colspan="2">EMP (63)</th><th colspan="2">EAP<sub>a</sub> (63)</th><th colspan="2">EAP<sub>s</sub> (63)</th><th colspan="2">EAP<sub>i</sub> (63)</th><th colspan="2">EAP<sub>all</sub> (189)</th><th colspan="2">EEP (213)</th><th colspan="2">IAP (30)</th></tr><tr><th>Classifier</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th></tr></thead><tbody><tr><td>OA</td><td>71.85</td><td>71.50</td><td>78.22</td><td>81.19</td><td>79.87</td><td>84.35</td><td>78.24</td><td>78.64</td><td>69.24</td><td>75.30</td><td>83.68</td><td>89.65</td><td>85.84</td><td>91.96</td><td>92.43</td><td><b>95.78</b></td></tr><tr><td>AA</td><td>81.15</td><td>82.13</td><td>88.05</td><td>89.40</td><td>83.68</td><td>86.96</td><td>84.37</td><td>83.61</td><td>82.47</td><td>84.03</td><td>89.45</td><td>91.47</td><td>87.38</td><td>89.73</td><td>91.94</td><td><b>93.15</b></td></tr><tr><td>κ</td><td>0.6501</td><td>0.6499</td><td>0.7279</td><td>0.7602</td><td>0.7414</td><td>0.7956</td><td>0.7259</td><td>0.7286</td><td>0.6275</td><td>0.6894</td><td>0.7914</td><td>0.8629</td><td>0.8156</td><td>0.8920</td><td>0.9005</td><td><b>0.9441</b></td></tr><tr><td>Class1</td><td>73.17</td><td>80.86</td><td>90.86</td><td>93.55</td><td>74.97</td><td>82.51</td><td>82.34</td><td>87.45</td><td>88.19</td><td>91.66</td><td>91.98</td><td>96.15</td><td>77.03</td><td>88.99</td><td>87.65</td><td><b>96.40</b></td></tr><tr><td>Class2</td><td>61.32</td><td>55.64</td><td>63.69</td><td>71.23</td><td>78.64</td><td>85.06</td><td>71.22</td><td>73.29</td><td>48.64</td><td>64.13</td><td>75.97</td><td><b>90.91</b></td><td>87.31</td><td><b>99.08</b></td><td>94.88</td><td>98.01</td></tr><tr><td>Class3</td><td>60.17</td><td>51.74</td><td>72.84</td><td>77.13</td><td>70.65</td><td>76.46</td><td>60.03</td><td>52.36</td><td>58.55</td><td>57.60</td><td>72.56</td><td><b>78.99</b></td><td>70.08</td><td>59.31</td><td>75.46</td><td>69.56</td></tr><tr><td>Class4</td><td>97.39</td><td>98.73</td><td>98.20</td><td>98.83</td><td>99.35</td><td>98.43</td><td>96.34</td><td>99.48</td><td>96.83</td><td>98.20</td><td>99.15</td><td>97.68</td><td><b>99.51</b></td><td>99.18</td><td>91.19</td><td>97.45</td></tr><tr><td>Class5</td><td>99.26</td><td>99.11</td><td>99.48</td><td>99.70</td><td>98.07</td><td><b>100.00</b></td><td>99.41</td><td>98.96</td><td>99.03</td><td>99.26</td><td>99.48</td><td>99.70</td><td>98.59</td><td><b>100.00</b></td><td>99.48</td><td>96.65</td></tr><tr><td>Class6</td><td>73.35</td><td>79.16</td><td>76.38</td><td>66.16</td><td>62.99</td><td>63.07</td><td>78.86</td><td>65.22</td><td>76.42</td><td>60.49</td><td>75.46</td><td>61.64</td><td>76.44</td><td>69.80</td><td>95.01</td><td><b>95.82</b></td></tr><tr><td>Class7</td><td>84.96</td><td>84.44</td><td>97.74</td><td>99.62</td><td>79.10</td><td>82.78</td><td>87.67</td><td>89.85</td><td>95.64</td><td>95.04</td><td><b>99.92</b></td><td><b>99.92</b></td><td>87.59</td><td>95.56</td><td>94.66</td><td>90.08</td></tr><tr><td>Class8</td><td>85.36</td><td>91.53</td><td>99.02</td><td><b>99.62</b></td><td>97.66</td><td>97.64</td><td>84.44</td><td>89.98</td><td>86.45</td><td>91.93</td><td>98.70</td><td>99.40</td><td>97.91</td><td>98.51</td><td>91.01</td><td>98.67</td></tr><tr><td>Class9</td><td>95.35</td><td>97.99</td><td>94.19</td><td>98.73</td><td>91.66</td><td>96.73</td><td><b>99.05</b></td><td>95.88</td><td>92.50</td><td>97.99</td><td>91.87</td><td>98.84</td><td>91.97</td><td>97.15</td><td>98.1</td><td>95.67</td></tr></tbody></table>

<table><caption>TABLE V
QUANTITATIVE CLASSIFICATION PERFORMANCE FOR EIGHT DIFFERENT ALGORITHMS ON THE HOUSTON2013 DATA SET</caption><thead><tr><th>Methods</th><th colspan="2">Baseline (144)</th><th colspan="2">EMP (63)</th><th colspan="2">EAP<sub>a</sub> (63)</th><th colspan="2">EAP<sub>s</sub> (63)</th><th colspan="2">EAP<sub>i</sub> (63)</th><th colspan="2">EAP<sub>all</sub> (189)</th><th colspan="2">EEP (213)</th><th colspan="2">IAP (30)</th></tr><tr><th>Classifier</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th></tr></thead><tbody><tr><td>OA</td><td>72.83</td><td>73.11</td><td>77.45</td><td>77.90</td><td>77.79</td><td>78.17</td><td>76.91</td><td>77.78</td><td>75.25</td><td>77.08</td><td>79.34</td><td>79.01</td><td>78.74</td><td>81.80</td><td>83.86</td><td><b>88.79</b></td></tr><tr><td>AA</td><td>76.16</td><td>77.01</td><td>80.80</td><td>81.07</td><td>81.62</td><td>81.47</td><td>79.91</td><td>80.80</td><td>79.09</td><td>80.30</td><td>82.36</td><td>82.43</td><td>82.17</td><td>84.87</td><td>86.01</td><td><b>90.09</b></td></tr><tr><td>κ</td><td>0.7079</td><td>0.7110</td><td>0.7565</td><td>0.7606</td><td>0.7591</td><td>0.7633</td><td>0.7518</td><td>0.7608</td><td>0.7336</td><td>0.7519</td><td>0.7769</td><td>0.7724</td><td>0.7704</td><td>0.8035</td><td>0.8248</td><td><b>0.8783</b></td></tr><tr><td>Class1</td><td>82.15</td><td>82.62</td><td>82.34</td><td>81.67</td><td>80.34</td><td>82.05</td><td>81.77</td><td>82.05</td><td>80.06</td><td>82.05</td><td>81.86</td><td>82.05</td><td>80.63</td><td>81.67</td><td>81.29</td><td><b>82.81</b></td></tr><tr><td>Class2</td><td>81.86</td><td>83.36</td><td>82.52</td><td>82.80</td><td>83.27</td><td>82.61</td><td>83.93</td><td>80.55</td><td>83.83</td><td>78.48</td><td>84.12</td><td>80.45</td><td>83.93</td><td>84.02</td><td>83.27</td><td><b>85.06</b></td></tr><tr><td>Class3</td><td>99.60</td><td>97.82</td><td>99.01</td><td>99.60</td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td>99.21</td><td>99.80</td><td><b>100.00</b></td><td>99.60</td></tr><tr><td>Class4</td><td><b>91.76</b></td><td>91.67</td><td>88.64</td><td>83.90</td><td>77.27</td><td>85.80</td><td>88.64</td><td>89.11</td><td>81.44</td><td>90.91</td><td>82.10</td><td>90.53</td><td>86.74</td><td>91.29</td><td>90.53</td><td>91.38</td></tr><tr><td>Class5</td><td>97.06</td><td>96.50</td><td>99.62</td><td>98.48</td><td>99.24</td><td>98.39</td><td>97.73</td><td>96.59</td><td>96.88</td><td>94.60</td><td>99.34</td><td>95.93</td><td>99.62</td><td>97.63</td><td>99.62</td><td><b>100.00</b></td></tr><tr><td>Class6</td><td>95.10</td><td><b>99.30</b></td><td>95.10</td><td>95.10</td><td><b>99.30</b></td><td>99.30</td><td>95.10</td><td>94.41</td><td>96.50</td><td>95.10</td><td>92.31</td><td>98.60</td><td>95.10</td><td>94.41</td><td>97.20</td><td>98.60</td></tr><tr><td>Class7</td><td>73.60</td><td>74.53</td><td>86.75</td><td>81.90</td><td>81.34</td><td>83.86</td><td>80.41</td><td>85.07</td><td>72.57</td><td>88.06</td><td>83.86</td><td>86.47</td><td>88.34</td><td>84.14</td><td>81.72</td><td><b>85.35</b></td></tr><tr><td>Class8</td><td>36.37</td><td>33.43</td><td>48.72</td><td>40.84</td><td>47.58</td><td>44.35</td><td>37.70</td><td>42.07</td><td>36.47</td><td>40.17</td><td>37.70</td><td>45.39</td><td>39.98</td><td>65.91</td><td>64.58</td><td><b>74.45</b></td></tr><tr><td>Class9</td><td>66.19</td><td>68.84</td><td>78.28</td><td>81.96</td><td>80.83</td><td>76.39</td><td>74.03</td><td>79.04</td><td>75.17</td><td>73.94</td><td>79.51</td><td>76.02</td><td>75.64</td><td>83.10</td><td>82.91</td><td><b>83.76</b></td></tr><tr><td>Class10</td><td>49.23</td><td>44.11</td><td>53.38</td><td>47.97</td><td>48.07</td><td>47.39</td><td>48.07</td><td>48.17</td><td>65.73</td><td>61.68</td><td>64.48</td><td>54.05</td><td>53.76</td><td>67.28</td><td>58.59</td><td><b>94.02</b></td></tr><tr><td>Class11</td><td>67.74</td><td>69.83</td><td>76.09</td><td>77.13</td><td>73.72</td><td>93.26</td><td>74.95</td><td>79.98</td><td>71.63</td><td>76.57</td><td>76.19</td><td>83.49</td><td>69.92</td><td>69.26</td><td>90.32</td><td><b>90.39</b></td></tr><tr><td>Class12</td><td>54.27</td><td>56.29</td><td>58.89</td><td>69.26</td><td>77.23</td><td>61.48</td><td>76.66</td><td>69.93</td><td>58.79</td><td>58.98</td><td>79.25</td><td>69.84</td><td>81.84</td><td>70.70</td><td>84.53</td><td><b>90.39</b></td></tr><tr><td>Class13</td><td>51.93</td><td>60.00</td><td>65.61</td><td>75.79</td><td>76.14</td><td>68.07</td><td>61.75</td><td>67.37</td><td>67.02</td><td>71.23</td><td>74.74</td><td>76.14</td><td>82.46</td><td><b>89.12</b></td><td>73.33</td><td>76.14</td></tr><tr><td>Class14</td><td>97.57</td><td>99.19</td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td>98.79</td><td>99.19</td><td><b>100.00</b></td><td>97.98</td><td><b>100.00</b></td><td><b>100.00</b></td><td>95.95</td><td>99.60</td><td>99.60</td><td>99.60</td></tr><tr><td>Class15</td><td>97.89</td><td>97.67</td><td>97.04</td><td>99.58</td><td><b>100.00</b></td><td>99.15</td><td>99.15</td><td>98.52</td><td><b>100.00</b></td><td>94.71</td><td><b>100.00</b></td><td>97.46</td><td>99.37</td><td>95.14</td><td>98.94</td><td>99.79</td></tr></tbody></table>

EAP-related approaches fail to improve classification performance dramatically, since there are more challenging categories, and more complex semantic variations in the studied scene. Nevertheless, the ability of EAPs<sub>all</sub> to fuse the different attributes still contributes to its power in spatial information extraction, yielding an increase in accuracy of around 5% using the RF classifier, compared to the others previously mentioned. Unlike EMPs and EAPs, EEPs utilize a sequence of extrema-oriented connected filters to extract the extinction values as the features. Due to these constructed features, the classification accuracy of EEPs increases by approximately 5% using NN, in spite of a slight decrease in using RF (about 2 percentage points less than EAPs<sub>all</sub>).

Beyond those algorithms developed based on MPs and APs, we propose to model the invariant features from both the spatial and frequency domains, which has been theoretically proven to be robust against the semantic change caused by various image deformations (e.g., shift, rotation, sensor noises, and distortions). Therefore, the resulting IAPs outperform the previously proposed MPs or APs methods, showing the best overall performance and the highly competitive classification results for each class, as shown in Table VI.

Fig. 8 also highlights the superiority of the proposed IAP method by means of classification maps. Generally speaking, our method tends to lead to more smooth classification maps: that is, the IAPs aggregate the same materials more easily while separating the different materials. Our IAPs also remove the effects of hot pixels like salt-and-pepper noise from classification maps effectively and simultaneously preserve the semantically meaningful structure or objects, leading to an increase of about 20% in classification accuracy. This is particularly the case for the class of nonresidential buildings that constitute large-scale coverage of the whole scene.

From the experimental results on the different data sets, we can observe an interesting and meaningful phenomenon that the proposed IAPs are more apt to recognize objects or materials with a regular structure (or shape), such as road, residential, and commercial, while for those irregular classes (e.g., grass and tree) that seem to hold more disorderly textual features, the extracted IAPs might not be so discriminative to identify these materials, limiting the gain in the final classification performance to some extent.

<footnote>* The best is shown in bold. The feature dimensions are given in the parentheses after the method's name.</footnote>

<footnote>* The best is shown in bold. The feature dimensions are given in the parentheses after the method's name.</footnote>
<table><caption>TABLE VI
QUANTITATIVE CLASSIFICATION PERFORMANCE FOR EIGHT DIFFERENT ALGORITHMS ON THE HOUSTON2018 DATA SET</caption><thead><tr><th>Methods</th><th colspan="2">Baseline (50)</th><th colspan="2">EMP (63)</th><th colspan="2">EAP<sub>a</sub> (63)</th><th colspan="2">EAP<sub>s</sub> (63)</th><th colspan="2">EAP<sub>i</sub> (63)</th><th colspan="2">EAP<sub>all</sub> (189)</th><th colspan="2">EEP (213)</th><th colspan="2">IAP (40)</th></tr><tr><th>Classifier</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th></tr></thead><tbody><tr><td>OA</td><td>60.99</td><td>65.85</td><td>63.87</td><td>71.88</td><td>65.98</td><td>73.48</td><td>61.41</td><td>73.67</td><td>61.42</td><td>68.66</td><td>65.33</td><td>77.36</td><td>70.69</td><td>75.09</td><td>79.11</td><td><b>83.68</b></td></tr><tr><td>AA</td><td>77.07</td><td>81.09</td><td>81.16</td><td>85.69</td><td>82.21</td><td>85.99</td><td>80.10</td><td>85.55</td><td>78.58</td><td>82.64</td><td>82.38</td><td>87.78</td><td>86.63</td><td>88.55</td><td><b>90.27</b></td><td>88.84</td></tr><tr><td>κ</td><td>0.5358</td><td>0.5922</td><td>0.5689</td><td>0.6570</td><td>0.5912</td><td>0.6742</td><td>0.5421</td><td>0.6751</td><td>0.5426</td><td>0.6209</td><td>0.5851</td><td>0.7187</td><td>0.6471</td><td>0.6958</td><td>0.7408</td><td><b>0.7931</b></td></tr><tr><td>Class1</td><td>93.15</td><td><b>95.03</b></td><td>91.95</td><td>93.29</td><td>91.69</td><td>94.16</td><td>91.49</td><td>93.72</td><td>90.84</td><td>93.15</td><td>91.26</td><td>93.99</td><td>91.33</td><td>93.66</td><td>90.38</td><td>93.73</td></tr><tr><td>Class2</td><td>87.48</td><td><b>91.35</b></td><td>87.71</td><td>90.39</td><td>87.60</td><td>90.02</td><td>84.46</td><td>89.21</td><td>83.48</td><td>90.46</td><td>86.60</td><td>90.98</td><td>89.96</td><td>90.70</td><td>86.27</td><td>89.20</td></tr><tr><td>Class3</td><td><b>100.00</b></td><td><b>100.00</b></td><td>98.54</td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td>96.10</td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td>99.19</td><td><b>100.00</b></td><td>99.84</td><td></td></tr><tr><td>Class4</td><td>91.31</td><td>94.64</td><td>93.92</td><td>94.84</td><td>92.85</td><td>95.17</td><td>90.46</td><td>93.64</td><td>92.28</td><td>94.28</td><td>93.96</td><td>95.31</td><td>95.07</td><td>95.31</td><td>97.65</td><td><b>98.39</b></td></tr><tr><td>Class5</td><td>87.36</td><td>90.22</td><td>90.11</td><td>92.52</td><td>94.33</td><td>94.85</td><td>91.82</td><td>93.54</td><td>91.16</td><td>92.85</td><td>93.65</td><td>95.05</td><td>96.26</td><td>95.40</td><td><b>97.98</b></td><td>96.17</td></tr><tr><td>Class6</td><td>98.28</td><td>98.35</td><td>97.81</td><td>98.97</td><td>99.68</td><td>99.93</td><td>99.58</td><td>99.80</td><td>99.19</td><td>99.85</td><td>99.80</td><td><b>100.00</b></td><td>99.02</td><td>99.68</td><td>99.90</td><td>99.98</td></tr><tr><td>Class7</td><td>92.92</td><td>90.83</td><td>98.33</td><td>98.33</td><td>98.75</td><td>98.75</td><td>95.42</td><td>98.33</td><td>90.83</td><td>96.67</td><td>98.33</td><td>98.33</td><td>97.92</td><td>97.92</td><td><b>99.58</b></td><td>98.33</td></tr><tr><td>Class8</td><td>67.03</td><td>82.24</td><td>83.52</td><td>89.78</td><td>86.12</td><td>89.72</td><td>82.63</td><td>89.64</td><td>80.55</td><td>86.42</td><td>85.93</td><td>90.80</td><td>85.74</td><td><b>91.75</b></td><td>90.69</td><td>90.17</td></tr><tr><td>Class9</td><td>58.46</td><td>60.59</td><td>57.63</td><td>66.79</td><td>61.30</td><td>70.42</td><td>55.22</td><td>71.72</td><td>54.96</td><td>64.14</td><td>59.39</td><td>75.02</td><td>62.97</td><td>68.69</td><td>77.01</td><td><b>85.57</b></td></tr><tr><td>Class10</td><td>41.30</td><td>46.42</td><td>44.61</td><td>55.44</td><td>48.32</td><td>58.41</td><td>44.36</td><td>57.23</td><td>45.78</td><td>54.57</td><td>47.72</td><td>62.59</td><td>59.61</td><td>62.81</td><td>65.48</td><td><b>65.51</b></td></tr><tr><td>Class11</td><td>33.74</td><td>44.47</td><td>41.23</td><td>50.07</td><td>44.25</td><td>52.36</td><td>37.40</td><td>45.93</td><td>43.29</td><td>46.07</td><td>47.03</td><td>54.73</td><td>49.00</td><td>56.32</td><td>55.52</td><td><b>62.59</b></td></tr><tr><td>Class12</td><td>37.51</td><td>31.79</td><td>45.13</td><td>47.84</td><td>56.41</td><td>51.36</td><td>50.62</td><td>46.23</td><td>45.35</td><td>37.58</td><td>52.53</td><td>50.40</td><td>67.77</td><td>62.42</td><td><b>81.39</b></td><td>48.79</td></tr><tr><td>Class13</td><td>43.89</td><td>50.80</td><td>51.48</td><td>63.28</td><td>49.43</td><td>59.19</td><td>44.13</td><td>61.10</td><td>45.60</td><td>52.21</td><td>49.47</td><td>68.91</td><td>68.22</td><td>71.68</td><td>75.04</td><td><b>76.84</b></td></tr><tr><td>Class14</td><td>84.32</td><td>86.20</td><td>86.62</td><td>91.77</td><td>82.98</td><td>91.40</td><td>85.36</td><td>92.41</td><td>80.30</td><td>86.20</td><td>86.46</td><td>95.46</td><td>96.66</td><td>95.87</td><td><b>98.49</b></td><td>97.85</td></tr><tr><td>Class15</td><td>98.12</td><td>98.07</td><td>98.66</td><td>98.90</td><td>96.36</td><td>98.18</td><td>98.79</td><td>99.75</td><td>93.24</td><td>98.14</td><td>96.16</td><td><b>99.78</b></td><td>99.60</td><td>99.63</td><td>99.52</td><td>99.33</td></tr><tr><td>Class16</td><td>86.82</td><td>89.09</td><td>86.62</td><td>93.43</td><td>84.78</td><td>90.21</td><td>86.76</td><td>92.65</td><td>84.56</td><td>92.00</td><td>86.76</td><td>93.94</td><td>95.21</td><td>97.51</td><td><b>97.87</b></td><td>95.24</td></tr><tr><td>Class17</td><td>95.56</td><td>91.85</td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td><b>100.00</b></td><td>99.26</td><td>99.26</td><td><b>100.00</b></td><td><b>100.00</b></td><td>98.52</td><td><b>100.00</b></td><td>99.26</td><td>83.70</td><td></td></tr><tr><td>Class18</td><td>69.69</td><td>87.87</td><td>79.85</td><td>89.98</td><td>79.25</td><td>88.10</td><td>77.41</td><td>88.98</td><td>75.60</td><td>82.79</td><td>81.29</td><td>90.61</td><td>86.9</td><td>93.73</td><td>94.92</td><td><b>96.08</b></td></tr><tr><td>Class19</td><td>80.10</td><td>94.12</td><td>91.12</td><td>99.03</td><td>90.61</td><td>97.97</td><td>86.87</td><td>98.34</td><td>81.21</td><td>86.70</td><td>91.68</td><td><b>99.88</b></td><td>95.38</td><td>98.71</td><td>99.22</td><td>99.61</td></tr><tr><td>Class20</td><td>94.43</td><td>97.93</td><td>98.39</td><td>99.21</td><td>99.49</td><td>99.67</td><td>99.16</td><td>99.60</td><td>98.09</td><td>98.80</td><td>99.54</td><td>99.86</td><td>98.24</td><td>99.24</td><td>99.30</td><td><b>99.95</b></td></tr></tbody></table>

## D. Parameter Sensitivity Analysis

The effectiveness of a feature extractor largely depends on parameter selection. It is, therefore, indispensable to discuss the sensitivity of parameters involved in the proposed IAP. These parameters include the number of scaled convolution kernels ($n_s$) and the corresponding radii of these convolution kernels ($r$) in spatially isotropic filtering, the number of Fourier orders ($m$) in the process of extracting frequency features, and the reduced dimension ($d$) in FL or DR. Among them, some of them are listed in the following.

1. The search range of $n_s$ is constrained from 1 to 5.

2. $r$ can be investigated through the combination of [1, 2, 3], [2, 4, 6], [4, 6, 8], and [6, 8, 10].<fnref n="1" />

3. The order $m$ can be selected from the different set, i.e., [0, 1], [0, 1, 2], [0, 1, 2, 3], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4, 5], ..., [0, 1, 2, ..., 9, 10].

4. The feature dimension after FL or DR is determined ranging from 10 to the dimension of the original IAPs (e.g., 300 for the Pavia University, 390 for the Houston2013 and 180 for the Houston2018) at a 10 interval.

We experimentally analyze the effects of different parameters for the overall classification accuracy (OA in our case) with two classifiers. The resulting quantitative results are shown in Fig. 9 for the three data sets.

It is evident from Fig. 9 that the optimal parameter combinations are $n_s = 3$, $r = [2, 4, 6]$, $m = [0, 1, 2, 3]$, and $d = 30$ for the Pavia University data set, $n_s = 3$, $r = [2, 4, 6]$, $m = [0, 1, 2, 3]$, and $d = 30$ for the Houston2013 data set, and $n_s = 3$, $r = [2, 4, 6]$, $m = [0, 1, 2]$, and $d = 40$ for the Houston2018. Moreover, we also discovered an

interesting phenomenon: classification performance is insensitive to the parameters of $n_s$ and $r$ on the Houston2018 data sets, while for the Pavia University and Houston2013 they have a moderate effect on OAs. Especially, the Fourier orders ($m$) are associated with classification results, but a progressively increased $m$ starts with a sharp performance decrease and then becomes stable with a relatively poor result. This reveals that $m$ is a noteworthy and important parameter during the process of extracting IAPs. The graph showing results for the parameter $d$ in Fig. 9 demonstrates that the extracted IAPs are, to some extent, redundant and can be further improved in feature discriminative ability by means of FL or DR, as the OAs start to quickly reach a certain optimal value (e.g., 30 for the Pavia University data set, 30 for the Houston2013 data set, and 40 for the Houston2018 data set), then hold basically unchanged over a period of the time, and finally gradually diminish for the robust RF classifier. For the NN classifier, the OAs dramatically decrease when inputting the IAPs with a higher dimension. This could be result of the curse of dimensionality.

1) *Effects of DR or FL in Feature Extraction:* For a fair comparison, we also investigate the effects of a DR technique on all studied methods, as listed in Table VII.

There is an interesting regular pattern in Table VII. Generally, the classification performance using the dimension-reduced features is obviously superior to that using original features (without DR), e.g., Baseline and our IAP method. It should be noted that, however, those EsP-based approaches with a DR technique usually perform worse than their previous versions without DR. A possible reason is the use of a DR technique (e.g., PCA) before feature extraction. If the DR technique is applied again after feature extraction, then this might suffer from the effects of reuse, leading to the limitation and even degradation in the final classification performance.

<footnote>* The best is shown in bold. The feature dimensions are given in the parentheses after the method's name.</footnote>

<footnote n="1">¹In the original literature [46], $n_s = 3$ is suggested. In addition, we experimentally found in our case that the $n_s$ is equal to 3, yielding the best performance on both data sets.</footnote>
<figure><img src="image_10.png" /><figcaption>Fig. 9. Parameter sensitivity analysis of the proposed IAPs with respect to the number of scaled convolution kernels ($n_s$) and their radii of these convolution kernels ($r$) in spatially isotropic filtering, the number of Fourier orders ($m$), and feature dimension ($d$) after FL or DR, respectively. (a) Analysis results of two classifiers (NN and RF) in terms of OAs on the Pavia University data set. (b) Analysis results of two classifiers (NN and RF) in terms of OAs on the Houston2013 data set. (c) Analysis results of two classifiers in terms of OAs on the Houston2018 data set.</figcaption></figure>

## E. Ablation Studies

In addition, we investigate and analyze the performance improvement of our IAP method by step-wise adding the different components, since the IAPs are involved with multiple components, i.e., SIFs, FIFs, and FL or DR. Table VIII shows an increasing performance gain as the different techniques are gradually fused. We found that successively adding each component into the proposed IAPs led to a progressive enhancement in feature representation ability, yielding a higher classification result for the HSI. This also indicates the reasonableness and advancement of the proposed IAPs-based HSI classification framework we designed (see Section II-D and Fig. 5).

The key points from the ablation analysis can be summarized as follows.

1. There is a great improvement in classification performance after applying the FL or DR on the original IAPs, from which we conclude that the FL or DR step plays a significant role in our proposed IAPs based HSI classification framework. This might be explained by the fact that directly stacking the OSF, SIFs, and FIFs as a final input would hurt classification performance,

since they come from different feature spaces. Moreover, this is also due to the reason that FL or DR can address the curse of dimensionality (the lack of balance between the number of features and training samples), as well as the existence of high redundancy among IAPs.

2. Although the results with SIFs are superior to those with FIFs alone, performance is still limited without FIFs, showing the power of FIFs in enriching the diversity and robustness of the extracted features, particularly when they are jointly used, i.e., IAPs (OSF+SIFs+FIFs).

3. Using FIFs individually would yield poor results, which indicates that frequency information alone is not sufficiently discriminative to identify a variety of materials.

## F. Computational Cost in the Proposed IAP

All experiments in this article were implemented with Matlab 2016a on a Windows 10 operation system and conducted on Intel Core i7-6700HQ 2.60 GHZ laptop with 16 GB memory. With the setting, the running time of the proposed IAP is given in Table IX. Overall, the running time is acceptable in practice and it shows an approximately linear increase as the size of the image increases.
<table><caption>TABLE VII
CLASSIFICATION PERFORMANCE IN TERMS OF OA FOR DIFFERENT STUDIED METHODS WITH DR TECHNIQUES ON THE THREE DATA SETS.
NOTE THAT THE OPTIMAL DIMENSION FOR EACH METHOD IS SELECTED BY THE MEANS OF A CROSS-VALIDATION STRATEGY</caption><thead><tr><th>Methods</th><th colspan="2">Baseline</th><th colspan="2">EMP</th><th colspan="2">EAP<sub>a</sub></th><th colspan="2">EAP<sub>s</sub></th><th colspan="2">EAP<sub>i</sub></th><th colspan="2">EAP<sub>all</sub></th><th colspan="2">EEP</th><th colspan="2">IAP</th></tr><tr><th>Classifier</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th><th>NN</th><th>RF</th></tr><tr><th>Dimension</th><td>(30, 20, 30)</td><td></td><td>(40, 30, 30)</td><td></td><td>(30, 20, 20)</td><td></td><td>(20, 20, 30)</td><td></td><td>(40, 30, 20)</td><td></td><td>(20, 40, 30)</td><td></td><td>(30, 30, 20)</td><td></td><td>(30, 30, 40)</td><td></td></tr></thead><tbody><tr><th>Pavia University</th><td>71.75</td><td>77.50</td><td>78.04</td><td>80.69</td><td>79.75</td><td>82.23</td><td>78.50</td><td>84.47</td><td>67.15</td><td>74.26</td><td>82.67</td><td>87.35</td><td>85.64</td><td>91.06</td><td>92.43</td><td><strong>95.78</strong></td></tr><tr><th>Houston2013</th><td>72.78</td><td>83.75</td><td>77.31</td><td>80.31</td><td>78.00</td><td>81.32</td><td>76.99</td><td>77.91</td><td>74.87</td><td>77.07</td><td>79.23</td><td>80.06</td><td>78.56</td><td>81.50</td><td>83.86</td><td><strong>88.79</strong></td></tr><tr><th>Houston2018</th><td>60.98</td><td>72.75</td><td>63.83</td><td>66.47</td><td>65.27</td><td>69.95</td><td>63.12</td><td>72.35</td><td>60.99</td><td>65.33</td><td>65.16</td><td>73.39</td><td>70.62</td><td>69.46</td><td>79.11</td><td><strong>83.68</strong></td></tr></tbody></table>

<table><caption>TABLE VIII
ABLATION ANALYSIS OF THE PROPOSED IAPS IN TERMS OF OA ON THE THREE DATA SETS. DIMENSION WITH DIFFERENT COMPONENT COMBINATION IS ALSO SHOWN</caption><thead><tr><th rowspan="2">Methods</th><th rowspan="2">OSF</th><th rowspan="2">SIFs</th><th rowspan="2">FIFs</th><th rowspan="2">FL or DR</th><th rowspan="2">Dimension</th><th colspan="2">Pavia University</th><th colspan="2">Houston2013</th><th colspan="2">Houston2018</th></tr><tr><th>NN</th><th>RF</th><th>NN</th><th>RF</th></tr></thead><tbody><tr><td>IAPs</td><td>✓</td><td>×</td><td>×</td><td>×</td><td>(103, 144, 50)</td><td>71.85</td><td>71.50</td><td>72.83</td><td>73.11</td><td>60.99</td><td>65.85</td></tr><tr><td>IAPs</td><td>×</td><td>✓</td><td>×</td><td>×</td><td>(103, 144, 50)</td><td>70.77</td><td>82.48</td><td>71.21</td><td>82.89</td><td>67.94</td><td>74.23</td></tr><tr><td>IAPs</td><td>×</td><td>×</td><td>✓</td><td>×</td><td>(135, 108, 84)</td><td>70.46</td><td>76.37</td><td>40.44</td><td>63.79</td><td>35.57</td><td>52.27</td></tr><tr><td>IAPs</td><td>✓</td><td>✓</td><td>×</td><td>×</td><td>(206, 288, 100)</td><td>72.48</td><td>86.63</td><td>72.35</td><td>83.67</td><td>64.95</td><td>79.29</td></tr><tr><td>IAPs</td><td>✓</td><td>✓</td><td>✓</td><td>×</td><td>(341, 396, 184)</td><td>81.16</td><td>90.33</td><td>74.46</td><td>87.05</td><td>69.04</td><td>81.74</td></tr><tr><td>IAPs</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>(30, 30, 40)</td><td>92.43</td><td>95.78</td><td>83.86</td><td>88.79</td><td>79.11</td><td>83.68</td></tr></tbody></table>

✓ and × mean the current component with and without being involved or considered in the HSI classification framework, respectively.

<table><caption>TABLE IX
PROCESSING TIME FOR THE PROPOSED IAP ON THE THREE DATA SETS</caption><thead><tr><th>Datasets</th><th>Pavia University</th><th>Houston2013</th><th>Houston2018</th></tr></thead><tbody><tr><th>Running Time (s)</th><td>11.2</td><td>29.0</td><td>62.7</td></tr></tbody></table>

## IV. CONCLUSION

To effectively improve the robustness of a feature extractor in modeling spatial information of HSI, we propose a novel AP-like feature descriptor called IAP that designs the invariant APs in both spatial and frequency domains. The proposed IAP method aims at overcoming the shortcomings of previous MPs or APs, in which extracted spatial features are apt to be degraded, leading to a large difference between the same materials, or confusion with other different materials, due to the local semantic change in a hyperspectral scene. Combined with the sound theory in modeling our proposed SIFs and FIFs, the resulting IAPs have demonstrated their potential and superiority in the HSI classification task. In the future, we will further extend our model to a supervised or semi-supervised version in an end-to-end fashion (e.g., deep learning) to conceive invariant feature extraction and classification as a whole.

## REFERENCES

[1] J. R. Anderson, *A Land Use Land Cover Classification System for Use With Remote Sensor Data*, vol. 964. Washington, DC, USA: GPO, 1976.

[2] J. Kang, M. Körner, Y. Wang, H. Taubenböck, and X. X. Zhu, “Building instance classification using street view images,” *ISPRS J. Photogramm. Remote Sens.*, vol. 145, pp. 44–59, Nov. 2018.

[3] H. Li, Y. Song, and C. L. P. Chen, “Hyperspectral image classification based on multiscale spatial information fusion,” *IEEE Trans. Geosci. Remote Sens.*, vol. 55, no. 9, pp. 5302–5312, Sep. 2017.

[4] R. Hang, Q. Liu, D. Hong, and P. Ghamisi, “Cascaded recurrent neural networks for hyperspectral image classification,” *IEEE Trans. Geosci. Remote Sens.*, vol. 57, no. 8, pp. 5384–5394, Aug. 2019.

[5] Q. Gao, S. Lim, and X. Jia, “Spectral-spatial hyperspectral image classification using a multiscale conservative smoothing scheme and adaptive sparse representation,” *IEEE Trans. Geosci. Remote Sens.*, vol. 57, no. 10, pp. 7718–7730, Oct. 2019.

[6] Z. Ye, J. Chen, H. Li, Y. Wei, G. Xiao, and J. A. Benediktsson, “Supervised functional data discriminant analysis for hyperspectral image classification,” *IEEE Trans. Geosci. Remote Sens.*, to be published, doi: 10.1109/tgrs.2019.2940991.

[7] R. Hang, Q. Liu, H. Song, and Y. Sun, “Matrix-based discriminant subspace ensemble for hyperspectral image spatial-spectral feature fusion,” *IEEE Trans. Geosci. Remote Sens.*, vol. 54, no. 2, pp. 783–794, Feb. 2016.

[8] D. Hong, N. Yokoya, J. Chanussot, and X. X. Zhu, “CoSpace: Common subspace learning from hyperspectral–multispectral correspondences,” *IEEE Trans. Geosci. Remote Sens.*, vol. 57, no. 7, pp. 4349–4359, Jul. 2019.

[9] J. Hu, D. Hong, Y. Wang, and X. Zhu, “A comparative review of manifold learning techniques for hyperspectral and polarimetric SAR image fusion,” *Remote Sens.*, vol. 11, no. 6, p. 681, Mar. 2019.

[10] D. Hong, N. Yokoya, N. Ge, J. Chanussot, and X. X. Zhu, “Learnable manifold alignment (LeMA): A semi-supervised cross-modality learning framework for land cover and land use classification,” *ISPRS J. Photogram. Remote Sens.*, vol. 147, pp. 193–205, Jan. 2019.

[11] C. Li, L. Gao, Y. Wu, B. Zhang, J. Plaza, and A. Plaza, “A real-time unsupervised background extraction-based target detection method for hyperspectral imagery,” *J. Real-Time Image Process.*, vol. 15, no. 3, pp. 597–615, Oct. 2018.

[12] Y. Xu, Z. Wu, J. Chanussot, and Z. Wei, “Joint reconstruction and anomaly detection from compressive hyperspectral images using Mahalanobis distance-regularized tensor RPCA,” *IEEE Trans. Geosci. Remote Sens.*, vol. 56, no. 5, pp. 2919–2930, May 2018.

[13] Y. Wu, S. López, B. Zhang, F. Qiao, and L. Gao, “Approximate computing for onboard anomaly detection from hyperspectral images,” *J. Real-Time Image Process.*, vol. 16, no. 1, pp. 99–114, Feb. 2019.

[14] D. Hong, N. Yokoya, J. Chanussot, and X. X. Zhu, “Learning a low-coherence dictionary to address spectral variability for hyperspectral unmixing,” in *Proc. IEEE Int. Conf. Image Process. (ICIP)*, Sep. 2017, pp. 1–5.

[15] D. Hong and X. X. Zhu, “SULORA: Subspace unmixing with low-rank attribute embedding for hyperspectral data analysis,” *IEEE J. Sel. Topics Signal Process.*, vol. 12, no. 6, pp. 1351–1363, Dec. 2018.

[16] M. Tang, B. Zhang, A. Marinoni, L. Gao, and P. Gamba, “Multiharmonic postnonlinear mixing model for hyperspectral nonlinear unmixing,” *IEEE Geosci. Remote Sens. Lett.*, vol. 15, no. 11, pp. 1765–1769, Nov. 2018.
[17] D. Hong, N. Yokoya, J. Chanussot, and X. X. Zhu, "An augmented linear mixing model to address spectral variability for hyperspectral unmixing," *IEEE Trans. Image Process.*, vol. 28, no. 4, pp. 1923-1938, Apr. 2019.

[18] W. Liao et al., "Taking optimal advantage of fine spatial resolution: Promoting partial image reconstruction for the morphological analysis of very-high-resolution images," *IEEE Geosci. Remote Sens. Mag.*, vol. 5, no. 2, pp. 8-28, Jun. 2017.

[19] P. Soille, *Morphological Image Analysis: Principles and Applications*. Berlin, Germany: Springer-Verlag, 2003.

[20] M. Pesaresi and J. A. Benediktsson, "A new approach for the morphological segmentation of high-resolution satellite imagery," *IEEE Trans. Geosci. Remote Sens.*, vol. 39, no. 2, pp. 309-320, Feb. 2001.

[21] J. Benediktsson, J. Palmason, and J. Sveinsson, "Classification of hyperspectral data from urban areas based on extended morphological profiles," *IEEE Trans. Geosci. Remote Sens.*, vol. 43, no. 3, pp. 480-491, Mar. 2005.

[22] S. Wold, K. Esbensen, and P. Geladi, "Principal component analysis," *Chemometrics Intell. Lab. Syst.*, vol. 2, nos. 1-3, pp. 37-52, 1987.

[23] M. Fauvel, J. Chanussot, J. A. Benediktsson, and J. R. Sveinsson, "Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles," in *Proc. IGARSS*, Jul. 2007, pp. 4834-4837.

[24] G. Licciardi, P. R. Marpu, J. Chanussot, and J. A. Benediktsson, "Linear versus nonlinear PCA for the classification of hyperspectral data based on the extended morphological profiles," *IEEE Geosci. Remote Sens. Lett.*, vol. 9, no. 3, pp. 447-451, May 2012.

[25] M. Fauvel, Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, "Advances in spectral-spatial classification of hyperspectral images," *Proc. IEEE*, vol. 101, no. 3, pp. 652-675, Mar. 2013.

[26] M. D. Mura, J. A. Benediktsson, B. Waske, and L. Bruzzone, "Morphological attribute profiles for the analysis of very high resolution images," *IEEE Trans. Geosci. Remote Sens.*, vol. 48, no. 10, pp. 3747-3762, Oct. 2010.

[27] E. J. Breen and R. Jones, "Attribute openings, thinnings, and granulometries," *Comput. Vis. Image Understand.*, vol. 64, no. 3, pp. 377-389, Nov. 1996.

[28] D. Hong, W. Liu, J. Su, Z. Pan, and G. Wang, "A novel hierarchical approach for multispectral palmprint recognition," *Neurocomputing*, vol. 151, pp. 511-521, Mar. 2015.

[29] M. D. Mura, A. Villa, J. A. Benediktsson, J. Chanussot, and L. Bruzzone, "Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis," *IEEE Geosci. Remote Sens. Lett.*, vol. 8, no. 3, pp. 542-546, May 2011.

[30] B. Song et al., "Remotely sensed image classification using sparse representations of morphological attribute profiles," *IEEE Trans. Geosci. Remote Sens.*, vol. 52, no. 8, pp. 5122-5136, Aug. 2014.

[31] P. Ghamisi, M. D. Mura, and J. A. Benediktsson, "A survey on spectral-spatial classification techniques based on attribute profiles," *IEEE Trans. Geosci. Remote Sens.*, vol. 53, no. 5, pp. 2335-2353, May 2015.

[32] Y. Xu, T. Géraud, and L. Najman, "Morphological filtering in shape spaces: Applications using tree-based image representations," in *Proc. ICPR*, Nov. 2012, pp. 485-488.

[33] N. Falco, J. A. Benediktsson, and L. Bruzzone, "Spectral and spatial classification of hyperspectral images based on ICA and reduced morphological attribute profiles," *IEEE Trans. Geosci. Remote Sens.*, vol. 53, no. 11, pp. 6223-6240, Nov. 2015.

[34] B. Demir and L. Bruzzone, "Histogram-based attribute profiles for classification of very high resolution remote sensing images," *IEEE Trans. Geosci. Remote Sens.*, vol. 54, no. 4, pp. 2096-2107, Apr. 2016.

[35] G. Cavallaro, M. D. Mura, J. A. Benediktsson, and A. Plaza, "Remote sensing image classification using attribute filters defined over the tree of shapes," *IEEE Trans. Geosci. Remote Sens.*, vol. 54, no. 7, pp. 3899-3911, Jul. 2016.

[36] G. Cavallaro, N. Falco, M. D. Mura, and J. A. Benediktsson, "Automatic attribute profiles," *IEEE Trans. Image Process.*, vol. 26, no. 4, pp. 1859-1872, Apr. 2017.

[37] W. Liao, M. D. Mura, J. Chanussot, R. Bellens, and W. Philips, "Morphological attribute profiles with partial reconstruction," *IEEE Trans. Geosci. Remote Sens.*, vol. 54, no. 3, pp. 1738-1756, Mar. 2016.

[38] P. Ghamisi, R. Souza, J. A. Benediktsson, X. X. Zhu, L. Rittner, and R. A. Lotufo, "Extinction profiles for the classification of remote sensing data," *IEEE Trans. Geosci. Remote Sens.*, vol. 54, no. 10, pp. 5631-5645, Oct. 2016.

[39] P. Ghamisi and B. Hofle, "LiDAR data classification using extinction profiles and a composite kernel support vector machine," *IEEE Geosci. Remote Sens. Lett.*, vol. 14, no. 5, pp. 659-663, May 2017.

[40] L. Fang, N. He, S. Li, P. Ghamisi, and J. A. Benediktsson, "Extinction profiles fusion for hyperspectral images classification," *IEEE Trans. Geosci. Remote Sens.*, vol. 56, no. 3, pp. 1803-1815, Mar. 2018.

[41] J. Xia, P. Ghamisi, N. Yokoya, and A. Iwasaki, "Random forest ensembles and extended multiextinction profiles for hyperspectral image classification," *IEEE Trans. Geosci. Remote Sens.*, vol. 56, no. 1, pp. 202-216, Jan. 2018.

[42] G. A. Licciardi, A. Villa, M. D. Mura, L. Bruzzone, J. Chanussot, and J. A. Benediktsson, "Retrieval of the height of buildings from worldview-2 multi-angular imagery using attribute filters and geometric invariant moments," *IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens.*, vol. 5, no. 1, pp. 71-79, Feb. 2012.

[43] M.-K. Hu, "Visual pattern recognition by moment invariants," *IRE Trans. Inf. Theory*, vol. 8, no. 2, pp. 179-187, Feb. 1962.

[44] X. Wu, D. Hong, P. Ghamisi, W. Li, and R. Tao, "MsRi-CCF: Multi-scale and rotation-insensitive convolutional channel features for geospatial object detection," *Remote Sens.*, vol. 10, no. 12, p. 1990, Dec. 2018.

[45] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, "SLIC superpixels compared to state-of-the-art superpixel methods," *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 34, no. 11, pp. 2274-2282, Nov. 2012.

[46] K. Liu et al., "Rotation-invariant HOG descriptors using Fourier analysis in polar and spherical coordinates," *Int. J. Comput. Vis.*, vol. 106, no. 3, pp. 342-364, Feb. 2014.

[47] D. Hong, W. Liu, X. Wu, Z. Pan, and J. Su, "Robust palmprint recognition based on the fast variation Vese-Osher model," *Neurocomputing*, vol. 174, pp. 999-1012, Jan. 2016.

[48] X. Wu, D. Hong, J. Tian, J. Chanussot, W. Li, and R. Tao, "ORSIm detector: A novel object detection framework in optical remote sensing imagery using spatial-frequency channel features," *IEEE Trans. Geosci. Remote Sens.*, vol. 57, no. 7, pp. 5146-5158, Jul. 2019.

[49] X. Wu, D. Hong, J. Chanussot, Y. Xu, R. Tao, and Y. Wang, "Fourier-based rotation-invariant feature boosting: An efficient framework for geospatial object detection," *IEEE Geosci. Remote Sens. Lett.*, to be published, doi: 10.1109/lgrs.2019.2919755.

[50] M. Reisert and H. Burkhardt, "Equivariant holomorphic filters for contour denoising and rapid object detection," *IEEE Trans. Image Process.*, vol. 17, no. 2, pp. 190-203, Feb. 2008.

[51] Q. Wang, O. Ronneberger, and H. Burkhardt, "Rotational invariance based on Fourier analysis in polar and spherical coordinates," *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 31, no. 9, pp. 1715-1722, Sep. 2009.

[52] A. Vedaldi, M. Blaschko, and A. Zisserman, "Learning equivariant structured output SVM regressors," in *Proc. Int. Conf. Comput. Vis. (ICCV)*, Nov. 2011, pp. 959-966.

[53] G. Jacovitti and A. Neri, "Multiresolution circular harmonic decomposition," *IEEE Trans. Signal Process.*, vol. 48, no. 11, pp. 3242-3247, Nov. 2000.

[54] G. B. Giannakis, "Signal reconstruction from multiple correlations: Frequency- and time-domain approaches," *J. Opt. Soc. Amer. A, Opt. Image Sci.*, vol. 6, no. 5, pp. 682-697, Feb. 1989.

[55] D. Hong, N. Yokoya, and X. X. Zhu, "Learning a robust local manifold representation for hyperspectral dimensionality reduction," *IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens.*, vol. 10, no. 6, pp. 2960-2975, Jun. 2017.

[56] P. Dollar, R. Appel, S. Belongie, and P. Perona, "Fast feature pyramids for object detection," *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 36, no. 8, pp. 1532-1545, Aug. 2014.

[57] P. R. Marpu, M. Pedergnana, M. D. Mura, S. Peeters, J. A. Benediktsson, and L. Bruzzone, "Classification of hyperspectral data using extended attribute profiles based on supervised and unsupervised feature extraction techniques," *Int. J. Image Data Fusion*, vol. 3, no. 3, pp. 269-298, Sep. 2012.

[58] D. Hong, N. Yokoya, J. Xu, and X. Zhu, "Joint and progressive learning from high-dimensional data for multi-label classification," in *Proc. ECCV*, 2018, pp. 469-484.

[59] D. Hong, N. Yokoya, J. Chanussot, J. Xu, and X. X. Zhu, "Learning to propagate labels on graphs: An iterative multitask regression framework for semi-supervised hyperspectral dimensionality reduction," *ISPRS J. Photogramm. Remote Sens.*, vol. 158, pp. 35-49, Dec. 2019.

[60] B. Rasti, P. Ghamisi, and R. Gloaguen, "Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis," *IEEE Trans. Geosci. Remote Sens.*, vol. 55, no. 7, pp. 3997-4007, Jul. 2017.
<figure><img src="image_11.png" /></figure>

**Danfeng Hong** (Member, IEEE) was born in Shandong, China, in 1989. He received the B.Sc. degree in computer science and technology from the Neusoft College of Information, Northeastern University, Shenyang, China, in 2012, the M.Sc. degree (summa cum laude) in computer vision from the College of Information Engineering, Qingdao University, Qingdao, China, in 2015, and the Dr. Ing. degree (summa cum laude) from the Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM), Munich, Germany, in 2019.

Since 2015, he has been a Research Associate with the Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Weßling, Germany. He is currently a Research Scientist with EO Data Science, IMF, DLR, where he leads the Spectral Vision Group. In 2018 and 2019, he was a Visiting Scholar with the GIPSA-lab, Grenoble Institute of Technology (Grenoble INP), CNRS, University of Grenoble Alpes, Grenoble, France, and the RIKEN Artificial Intelligent Project (AIP), RIKEN, Tokyo, Japan. His research interests include signal/image processing and analysis, pattern recognition, machine/deep learning, and their applications in Earth vision.

Dr. Ghamisi was a recipient of the Best Researcher Award for M.Sc. students at the K. N. Toosi University of Technology in the academic year of 2010–2011, the IEEE Mikio Takagi Prize for winning the Student Paper Competition at the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) in 2013, the Talented International Researcher by Iran's National Elites Foundation in 2016, the First Prize of the Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee (IADF), IEEE Geoscience and Remote Sensing Society, in 2017, the Best Reviewer Prize of the IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (GRSL) in 2017, the Alexander von Humboldt Fellowship from the Technical University of Munich, and the High Potential Program Award from HZDR. He is also the Vice-Chair of the IEEE Image Analysis and Data Fusion Committee. He serves as an Associate Editor for *Remote Sensing* (MDPI) and the IEEE GRSL.

<figure><img src="image_12.png" /></figure>

**Xin Wu** (Student Member, IEEE) received the B.S. degree in science and technology from the Electronic Information Department, Mudanjiang Normal University, Mudanjiang, China, in 2011, and the M.Sc. degree in computer science and technology from Qingdao University, Qingdao, China, in 2014. She is currently pursuing the Ph.D. degree in information and communication engineering with the Beijing Institute of Technology (BIT), Beijing, China.

In 2018, she was a Visiting Student with the Photogrammetry and Image Analysis Department, Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Weßling, Germany. Her research interests include signal/image processing, fractional Fourier transform, deep learning, and their applications in biometrics and geospatial object detection.

<figure><img src="image_13.png" /></figure>

**Jocelyn Chanussot** (Fellow, IEEE) received the M.Sc. degree in electrical engineering from the Grenoble Institute of Technology (Grenoble INP), Grenoble, France, in 1995, and the Ph.D. degree from the Université de Savoie, Annecy, France, in 1998.

In 1999, he was with the Geography Imagery Perception Laboratory, Delegation Generale de l'Armement (DGA), French National Defense Department, Arcueil, France. Since 1999, he has been with Grenoble INP, where he is currently a Professor of signal and image processing. He is conducting his research at GIPSA-Lab. He has been a Visiting Scholar with Stanford University, Stanford, CA, USA, the KTH Royal Institute of Technology, Stockholm, Sweden, and the National University of Singapore, Singapore. Since 2013, he has been an Adjunct Professor with the University of Iceland, Reykjavik, Iceland. From 2015 to 2017, he was a Visiting Professor with the University of California at Los Angeles (UCLA), Los Angeles, CA, USA. His research interests include image analysis, multicomponent image processing, nonlinear filtering, and data fusion in remote sensing.

Dr. Chanussot was a member of the IEEE Geoscience and Remote Sensing Society AdCom from 2009 to 2010, in charge of membership development. He is also a member of the Institut Universitaire de France from 2012 to 2017. He was the Founding President of the IEEE Geoscience and Remote Sensing French Chapter from 2007 to 2010, which received the 2010 IEEE GRSS Chapter Excellence Award. He was a co-recipient of the NORSIG 2006 Best Student Paper Award, the IEEE GRSS 2011 and 2015 Symposium Best Paper Award, the IEEE GRSS 2012 Transactions Prize Paper Award and the IEEE GRSS 2013 Highest Impact Paper Award. He was the General Chair of the first IEEE GRSS Workshop on Hyperspectral Image and Signal Processing, Evolution in Remote Sensing (WHISPERS). He was the Chair and the Co-Chair of the GRS Data Fusion Technical Committee from 2009 to 2011 and from 2005 to 2008, respectively. He was a member of the Machine Learning for Signal Processing Technical Committee of the IEEE Signal Processing Society from 2006 to 2008 and the Program Chair of the IEEE International Workshop on Machine Learning for Signal Processing in 2009. He was an Associate Editor of the IEEE GEOSCIENCE AND REMOTE SENSING LETTERS from 2005 to 2007 and *Pattern Recognition* from 2006 to 2008. He was the Editor-in-Chief of the IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING from 2011 to 2015. Since 2007, he has been an Associate Editor of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING. Since 2018, he has also been an Associate Editor of the IEEE TRANSACTIONS ON IMAGE PROCESSING. He was a Guest Editor of the PROCEEDINGS OF THE IEEE and the *IEEE Signal Processing Magazine* in 2013 and 2014, respectively.

<figure><img src="image_14.png" /></figure>

**Pedram Ghamisi** (Senior Member, IEEE) received the B.Sc. degree in civil (survey) engineering from Islamic Azad University at South Tehran, Tehran, Iran, in 2008, the M.Sc. degree (Hons.) in remote sensing with the K. N. Toosi University of Technology, Tehran, in 2012, and the Ph.D. degree in electrical and computer engineering with the University of Iceland, Reykjavik, Iceland, in 2015.

From 2013 to 2014, he was with the School of Geography, Planning and Environmental Management, University of Queensland, Brisbane, QLD, Australia. In 2015, he was an Alexander von Humboldt Research Fellow with the Technical University of Munich (TUM), Munich, Germany, and Heidelberg University, Heidelberg, Germany. From 2015 to 2018, he was a Research Scientist with the German Aerospace Center (DLR), Weßling, Germany. Since 2018, he has been working as the Head of the Machine Learning Group, Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Freiberg, Germany. He is currently the CTO and the Co-Founder of VasoGnosis Inc., Milwaukee, WI, USA, where he is involved in the development of advanced diagnostic and analysis tools for brain diseases using cloud computing and deep learning algorithms. His research interests include interdisciplinary research on remote sensing and machine (deep) learning, image and signal processing, and multisensor data fusion.
<figure><img src="image_15.png" /></figure>

**Naoto Yokoya** (Member, IEEE) received the M.Eng. and Ph.D. degrees in aerospace engineering from The University of Tokyo, Tokyo, Japan, in 2010 and 2013, respectively.

He was an Assistant Professor with The University of Tokyo from 2013 to 2017. From 2015 to 2017, he was an Alexander von Humboldt Fellow with the German Aerospace Center (DLR), Weßling, Germany, and the Technical University of Munich (TUM), Munich, Germany. He is currently a Unit Leader with the RIKEN Center for Advanced Intel-

ligence Project, Tokyo, where he has been leading the Geoinformatics Unit since 2018. He has been a Visiting Associate Professor with the Tokyo University of Agriculture and Technology, Fuchū, Japan, since 2019. His research is focused on the development of image processing, data fusion, and machine learning algorithms for understanding remote sensing images, with applications to disaster management.

Dr. Yokoya received the First Place in the 2017 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee (IADF TC). He was the Co-Chair of the IEEE GRSS IADF TC from 2017 to 2019, where he is the Chair for the period 2019–2021. He has been the Secretary of the IEEE GRSS All Japan Joint Chapter since 2018. He has been an Associate Editor of the IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING (JSTARS) since 2018. He is/was a Guest Editor of the IEEE JSTARS from 2015 to 2016, *Remote Sensing* from 2016 to 2019, and the IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (GRSL) from 2018 to 2019.

<figure><img src="image_16.png" /></figure>

**Xiao Xiang Zhu** (Senior Member, IEEE) received the M.Sc. degree, the Dr. Ing. degree, and the "Habilitation" in the field of signal processing from the Technical University of Munich (TUM), Munich, Germany, in 2008, 2011, and 2013, respectively.

She was a Guest Scientist or Visiting Professor at the Italian National Research Council (CNR-IREA), Naples, Italy; Fudan University, Shanghai, China; The University of Tokyo, Tokyo, Japan; and the University of California, Los Angeles, CA, USA, in 2009, 2014, 2015, and 2016, respectively. She

is currently a Professor for signal processing in earth observation with the Technical University of Munich (TUM) and German Aerospace Center (DLR); the Head of EO Data Science, DLR Earth Observation Center; and the Head of the Helmholtz Young Investigator Group, SIPEO, at DLR and TUM. Since 2019, she has been co-coordinating the Munich Data Science Research School. She is also leading the Helmholtz Artificial Intelligence Cooperation Unit (HAICU)—Research Field, Aeronautics, Space and Transport. Her main research interests are remote sensing and earth observation, signal processing, machine learning, and data science, with a special application focus on global urban mapping.

Dr. Zhu is a member of Young Academy (Junge Akademie/Jungen Kolleg) at the Berlin-Brandenburg Academy of Sciences and Humanities as well as the German National Academy of Sciences Leopoldina and the Bavarian Academy of Sciences and Humanities. She is an Associate Editor of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING.