# Hilbert–Schmidt Independence Criterion Subspace Learning on Hybrid Region Covariance Descriptor for Image Classification

Authors: Xi Liu; Peng Yang; Zengrong Zhan; Zhengming Ma
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

The region covariance descriptor (RCD), which is known as a symmetric positive definite (SPD) matrix, is commonly used in image representation. As SPD manifolds have a non-Euclidean geometry, Euclidean machine learning methods are not directly applicable to them. In this work, an improved covariance descriptor called the hybrid region covariance descriptor (HRCD) is proposed. The HRCD incorporates the mean feature information into the RCD to improve the latter’s discriminative performance. To address the non-Euclidean properties of SPD manifolds, this study also proposes an algorithm called the Hilbert-Schmidt independence criterion subspace learning (HSIC-SL) for SPD manifolds. The HSIC-SL algorithm is aimed at improving classification accuracy. This algorithm is a kernel function that embeds SPD matrices into the reproducing kernel Hilbert space and further maps them to a linear space. To make the mapping consider the correlation between SPD matrices and linear projection, this method introduces global HSIC maximization to the model. The proposed method is compared with existing methods and is proved to be highly accurate and valid by classification experiments on the HRCD and HSIC-SL using the COIL-20, ETH-80, QMUL, face data FERET, and Brodatz datasets.


---

## Body

## 1. Introduction

A growing number of non-Euclidean data, such as symmetric positive definite (SPD) manifolds [1] and Grassmann manifolds [2], are often encountered in vision recognition tasks. In particular, SPD manifolds have attracted increased attention in the form of the region covariance descriptor (RCD) [3, 4], Gaussian mixed model (GMM) [5], tensors [6–9], etc. In this work, we mainly discuss the image classification on SPD manifolds.The RCD has been proved to be an effective descriptor in a variety of applications [10–12]. It captures the correlation between different features of an image and represents the image with a covariance matrix. However, the mean vector of features has been proved to be significant in image recognition tasks [13, 14]. In this work, we construct a new image descriptor by directly incorporating the mean feature information into the RCD. The new image descriptor is called the hybrid region covariance descriptor (HRCD). The HRCD inherits the advantages of the RCD, and it is more discriminable than the RCD. The images represented by the HRCD are also SPD matrices that lie on SPD manifolds. Most classical machine learning algorithms are constructed on linear spaces. Given the non-Euclidean geometry of Riemannian manifolds, directly using most of the conventional machine learning methods on Riemannian manifolds is inadequate [15, 16]. Therefore, the classification of the points on Riemannian manifolds has become a hot research topic.Two main approaches are generally adopted to cope with the nonlinearity of Riemannian manifolds. The first approach is to construct learning methods by directly considering the Riemannian geometry; one such method is the widely used tangent approximation [17, 18]. Most existing SPD classification methods have been proposed by making use of Riemannian metrics [15, 16] or matrix divergences [19, 20] as the distance measure for SPD matrices [21–23].The other approach is to project the SPD matrices to another space, such as a high-dimensional reproducing kernel Hilbert space (RKHS) [24] and another low-dimensional SPD manifold [25]. Classification algorithms can be constructed on the projection space. Benefiting from the success of kernel methods in Euclidean spaces, the kernel-based classification scheme is a good choice for the analysis of SPD manifolds and has shown promising performance [26, 27]. Kernel-based methods embed manifolds into RKHSs and further project these manifolds to Euclidean spaces via an explicit mapping. Hence, algorithms designed for linear spaces can be extended to Riemannian manifolds. However, the mapping from RKHSs to Euclidean spaces using existing methods is based on a linear assumption. Moreover, the intrinsic connections of SPD matrices and low-dimensional projections are ignored.To circumvent this limitation of kernel-based methods, we propose introducing the Hilbert–Schmidt independence criterion (HSIC) to the kernel trick and refer to the resulting method as the HSIC subspace learning (HSIC-SL) algorithm. Specifically, we derive the log-linear and log-Gaussian kernels to embed SPD matrices into a high-dimensional RKHS and then project these points into a low-dimensional vector space of the RKHS. To align the low-dimensional representation with the intrinsic features of the input data, we introduce statistical dependence between the SPD matrices and the low-dimensional representation. In this work, explicit mapping is obtained on the basis of subspace learning and HSIC maximization. Here, HSIC can be used to characterize the statistical correlation between two datasets.The main contributions of this study are as follows:(1)
We propose a novel covariance descriptor called the HRCD. The proposed descriptor explores discriminative information effectively.(2)
The HSIC is first applied to the kernel framework on SPD Riemannian manifolds, and a novel subspace learning algorithm called the HSIC-SL is proposed. The proposed method achieves effective classification on the basis of global HSIC maximization.(3)
We identify two simple kernel functions involved in the HSIC-SL algorithm. The diversity of kernels improves the flexibility of the HSIC-SL.The rest of the paper is organized as follows. We provide a review of previous work in Section2. A brief description about RCD, RKHS, and HSIC is presented in Section 3. We derive the proposed descriptor and algorithm in detail in Section 4. The experimental results are presented in Section 5 to demonstrate the effectiveness of the HRCD and HSIC-SL. Conclusions and future research directions are established in Section 6.


## 2. Literature Review

This section presents a brief review of RCDs, as well as recent manifold classification methods constructed on SPD manifolds.The RCD was first introduced by Tuzel et al. [28]. It represents an image region with a nonsingular SPD matrix by extracting the covariance matrix of multiple features. The covariance matrix does not have any information about size and ordering, which implies certain scale and rotation independence. The RCD is used not only in image recognition but also in image set recognition tasks, in which an image set is modeled with its natural second-order statistic [4, 29]. The GMM could also serve as the SPD descriptor of an image set. Under the assumption of the multi-Gaussian distribution of an image set [30], hundreds of images in the image set are assigned to a small number of Gaussian components. Each Gaussian component is represented as an SPD matrix [31]. Thus, the image set is described by multiple SPD matrices. As mentioned previously, mean vectors have also been proved to be important in recognition tasks. In [32], the mean information was utilized in an improved log-Euclidean Gaussian kernel. However, this approach is limited to a specific algorithm and lacks generality. In the current work, we propose to incorporate the feature mean information and covariance matrix into a new SPD matrix and introduce first-order statistic information into the image RCD to improve the discriminant ability of the descriptor.When the manifold under consideration is an SPD manifold, the tangent space of a particular point is a linear space. Most works map SPD matrices onto the tangent space of a particular point; thus, traditional linear classifiers can be applied. Under this framework, dimensionality reduction and clustering methods, such as Laplacian eigenmaps, local linear embedding (LLE), and Hessian LLE, have been extended to Riemannian manifolds [17]. Tuzel et al. introduced LogitBoost for classification on Riemannian manifolds [18]. The classifier has been generalized to multiclass classification [33]. Sparse coding by embedding manifolds into identity tangent spaces to identify the Lie algebra of SPD manifolds was considered in [34]. Such tangent space approximations could preserve manifold value data and eliminate the swelling effect. However, flattening a manifold through tangent spaces may generate inaccurate modeling, especially for regions far away from the tangent pole.Except for tangent approximation, many efforts have been devoted to the distance measure on SPD manifolds to measure the true SPD manifold geometry; examples include the log-Euclidean Riemannian metric (LERM) [15] and the affine invariance Riemannian metric (AIRM) [16]. Although matrix divergences are not real Riemannian metric, they provide fast and approximate distance computation. Sivalingam et al. proposed tensor sparse coding (TSC) for positive definite matrices [35] that utilizes the Burg divergence to perform sparse coding and dictionary learning on SPD manifolds. Riemannian dictionary learning and sparse coding (DLSC) [36] represents data as sparse combinations of SPD dictionary atoms via a Riemannian geometric approach and characterizes the loss of optimization for DLSC via the affine invariant Riemannian metric. However, these methods cannot be applied to other Riemannian manifolds because of the specificity of the specific metrics used. Embedding discriminant analysis (EDA) [37] identifies a bilinear isometric mapping such that the resulting representation maximizes the preservation of Riemannian geodesic distance.As for the proposed kernel methods for SPD manifolds, Riemannian locality preserving projections (RLPPs) [38] embed Riemannian manifolds into low-dimensional vector spaces by defining Riemannian kernels; moreover, their computational complexity is heavy, and the kernel is not always positive definite. Jayasumana et al. [39] presented a framework on Riemannian manifolds to identify the positive definiteness of Gaussian RBF kernels and utilized the log-Euclidean Gaussian kernel in kernel principal component analysis (KPCA) for a recognition task. Caseiro et al. proposed a heat kernel mean shift on Riemannian manifolds [40]. In [41], kernel DLSC based on LERM was introduced. Harandi et al. proposed to seek sparse coding by embedding the space of SPD matrices into Hilbert spaces through two types of Bregman matrix divergences [42]. Covariance discriminative learning (CDL) [4] utilizes a matrix logarithm operator to define kernel functions and then explicitly maps the covariance matrices from a Riemannian manifold to a Euclidean space. Zhuang et al. proposed a data-dependent kernel learning framework on the basis of the kernel learning and Riemannian metric (KLRM) [43]. In [44], multi-kernel SSCR (MKSSCR) created a linear combination of a set of Riemannian kernels. Considerable results are achieved in the kernel framework. To improve the performance of the kernel trick, we introduce a statistical dependence constraint between SPD matrices and projections and measure the statistical dependence with the HSIC.


## 3. Related Work

In this section, we briefly review the RCD and the properties of the RKHS and HSIC.


### 3.1. Region Covariance Descriptor

Region covariance descriptor (RCD), as a special case of SPD matrices, proposes a natural way of fusing multiple features. SupposeR is an image region of size h×w, and we can extract multiple features of every pixel in R. The features could be location, grey values, and gradients. We denote the feature vector of the k-th pixel as(1)zk=x,y,I,Ix,Iy,where x and y denote the location, I is the grey value, and Ix and Iy are the gradients with respect to x and y. The RCD of R is defined as(2)Σ=1n−1∑k=1nzk−μzk−μT,where n=h×w and μ=1/n∑k=1nzk∈Rd denotes the mean of the points. Then, the image region can be presented by a d×d SPD matrix, where d depends on the number of features.


### 3.2. RKHS

Reproducing kernel Hilbert space (RKHS) is the theoretical basis of kernel methods. After projecting the data into a RKHS, various machine learning methods will be implemented in the RKHS.LetSΩ be a function space, and ⋅,⋅ is an inner product defined on SΩ. The complete inner product space H=SΩ,⋅,⋅ induced by ⋅,⋅ is a Hilbert space. For all x∈Ω and f∈SΩ, if the function k satisfies fx=f,k⋅,x, then k is the reproducing kernel of the RKHS H. We denote the mapping defined by the reproducing kernel as ϕx=k⋅,x=kx∈H. We can induce that(3)ϕx,ϕy=kx,k⋅,y=kxy=ky,x=kx,y.The functionk could be a kernel function only if the kernel matrix K is symmetric positive definite, where K=kx1,x1⋯kx1,xn⋮⋱⋮kxn,x1⋯kxn,xn. According to Mercer’s theorem [45], once a valid reproducing kernel is defined, we can generate a unique Hilbert space.


### 3.3. Hilbert–Schmidt Independence Criterion (HSIC)

The HSIC [46] is usually used to characterize the statistical correlation of two datasets. The mathematical theory of HSIC has been studied for a long time and there are many achievements [47–51]. In the computation of HSIC, the two datasets are firstly embedded onto two RKHSs, and then the HSIC of the two set of data is measured by the Hilbert–Schmidt (HS) operator of these two RKHSs.LetX be a random variable/vector defined on ΩX and Y be a random variable/vector defined on ΩY, HX and HY be two separate Hilbert spaces, and ϕX:ΩX⟶HX and ϕY:ΩY⟶HY be the kernel mappings defined by the reproducing kernels, respectively.


#### 3.3.1. Hilbert–Schmidt (HS) Operators

LetT:HX⟶HY be a compact operator and eiX|i∈I be the orthonormal basis of HX; if ∑i∈ITeiXY2<+∞, then T is called a Hilbert–Schmidt (HS) operator [52]. If for all T,S∈HSHX⟶HY, ∑i∈ITeiX,SeiXY<+∞, then HSHX⟶HY,⋅,⋅HS is a Hilbert space. The inner product ⋅,⋅HS is defined as T,SHS=∑i∈ITeiX,SeiXY. For f0∈HX and g0∈HY, the tensor product of f0 and g0 is denoted as f0⊗g0. Since f0⊗g0f=f0,fXg0∈HY, then f0⊗g0∈HSHX⟶HY [53].


#### 3.3.2. Mean Functions and Cross Covariance Operators

LetΦX:HX⟶R be a continuous linear functional over HX; for all T∈HSHX⟶HY,(4)ΦXf=EXφXX,fX.According to Riesz theorem, there must a unique HS operatorμX∈HX such that for all f∈HX, ΦXf=f,μXX; then, μX is called the mean function of φXX. Similarly, the mean function μY of φYY is defined in the same way.LetΦ be a continuous linear functional over HSHX⟶HY; for all T∈HSHX⟶HY,(5)ΦT=EXYφXX⊗φYY,THS.Then, according to Riesz theorem, there must a unique HS operatorCXY∈HSHX,HY, such that for all T∈HSHX,HY,(6)ΦT=T,CXYHS,where CXY is called the cross covariance operator between φXX and φYY.The relationship betweenCXY, μX, and μY is illustrated in Figure 1. The two datasets ΩX and ΩY are embedded into HX and HY by the kernel functions ϕX:ΩX⟶HX and ϕY:ΩY⟶HY, respectively. μX and μY are the mean functions. The HSIC of ΩX and ΩY is given by the Hilbert–Schmidt (HS) operator CXY of HX and HY.Figure 1
The sketch mapping of HSIC.


#### 3.3.3. HSIC

HSIC of two random variables/vectors is defined as follows:(7)HSICX,Y=EXYφXX−μX⊗φYY−μYHS2.It can be seen from the definition ofHSICX,Y that instead of directly calculating the covariance of X and Y, i.e., EXYX−EXXY−EYY, HSIC first transforms X and Y into HX and HY, respectively, and then calculates the covariance of φXX and φYY by using HS operators between HX and HY. In practice, HX and HY are generated from kernel functions kX and kY.If the joint probability distribution ofX and Y is given or known, HSICX,Y can be calculated as follows:(8)HSICX,Y=EXYφXX−μX⊗φYY−μYHS2=CXY−μX⊗μYHS2=CXY,CXYHS−2CXY,μX⊗μYHS+μX⊗μY,μX⊗μYHS,where(9)CXY,CXYHS=EXYEX′Y′kXX,X′kYY,Y′,(10)CXY,μX⊗μYHS=EXYEX′kXX,X′EY′kYY,Y′,(11)μX⊗μY,μX⊗μYHS=μX,μXXμY,μYY.Generally speaking, the joint probability distribution ofX and Y is unknown, and only some samples of X and Y are given: X≜x1,…,xN⊆ΩX,Y≜y1,…,yN⊆ΩY. In this case, the statistical average can be approximated by the sample average. Moreover, it is assumed that when i≠j, the probability of the random event X=xi;Y=yj is 0; then, the cross covariance operator CXY and the mean functions μX and μY can be approximated as follows:(12)CXY≈1N∑i=1NφXxi⊗φYyi,μX≈1N∑i=1NφXxi,μY≈1N∑i=1NφYyi.Substituting equation (12) into equations (9) and (10) gives(13)CXY,CXYHS≈1N2trKXKY,CXY,μX⊗μYHS≈1N3ΓNTKXKYΓN,μX⊗μY,μX⊗μYHS≈1N4ΓNTKXΓNΓNTKYΓN,where ΓN=1,…,1T∈RN is the N-dimensional vector with all elements being 1 and KX and KY are the kernel matrices of X and Y, respectively.At last, the calculation formula ofHSICX,Y can be obtained by(14)HSICX,Y=CXY−μX⊗μYHS2=CXY,CXYHS+μX⊗μY,μX⊗μYHS−2CXY,μX⊗μYHS≈1N2trKYCNKXCN,where CN=IN−1/NΓNΓNT is the centralizing matrix.


## 3.1. Region Covariance Descriptor

Region covariance descriptor (RCD), as a special case of SPD matrices, proposes a natural way of fusing multiple features. SupposeR is an image region of size h×w, and we can extract multiple features of every pixel in R. The features could be location, grey values, and gradients. We denote the feature vector of the k-th pixel as(1)zk=x,y,I,Ix,Iy,where x and y denote the location, I is the grey value, and Ix and Iy are the gradients with respect to x and y. The RCD of R is defined as(2)Σ=1n−1∑k=1nzk−μzk−μT,where n=h×w and μ=1/n∑k=1nzk∈Rd denotes the mean of the points. Then, the image region can be presented by a d×d SPD matrix, where d depends on the number of features.


## 3.2. RKHS

Reproducing kernel Hilbert space (RKHS) is the theoretical basis of kernel methods. After projecting the data into a RKHS, various machine learning methods will be implemented in the RKHS.LetSΩ be a function space, and ⋅,⋅ is an inner product defined on SΩ. The complete inner product space H=SΩ,⋅,⋅ induced by ⋅,⋅ is a Hilbert space. For all x∈Ω and f∈SΩ, if the function k satisfies fx=f,k⋅,x, then k is the reproducing kernel of the RKHS H. We denote the mapping defined by the reproducing kernel as ϕx=k⋅,x=kx∈H. We can induce that(3)ϕx,ϕy=kx,k⋅,y=kxy=ky,x=kx,y.The functionk could be a kernel function only if the kernel matrix K is symmetric positive definite, where K=kx1,x1⋯kx1,xn⋮⋱⋮kxn,x1⋯kxn,xn. According to Mercer’s theorem [45], once a valid reproducing kernel is defined, we can generate a unique Hilbert space.


## 3.3. Hilbert–Schmidt Independence Criterion (HSIC)

The HSIC [46] is usually used to characterize the statistical correlation of two datasets. The mathematical theory of HSIC has been studied for a long time and there are many achievements [47–51]. In the computation of HSIC, the two datasets are firstly embedded onto two RKHSs, and then the HSIC of the two set of data is measured by the Hilbert–Schmidt (HS) operator of these two RKHSs.LetX be a random variable/vector defined on ΩX and Y be a random variable/vector defined on ΩY, HX and HY be two separate Hilbert spaces, and ϕX:ΩX⟶HX and ϕY:ΩY⟶HY be the kernel mappings defined by the reproducing kernels, respectively.


### 3.3.1. Hilbert–Schmidt (HS) Operators

LetT:HX⟶HY be a compact operator and eiX|i∈I be the orthonormal basis of HX; if ∑i∈ITeiXY2<+∞, then T is called a Hilbert–Schmidt (HS) operator [52]. If for all T,S∈HSHX⟶HY, ∑i∈ITeiX,SeiXY<+∞, then HSHX⟶HY,⋅,⋅HS is a Hilbert space. The inner product ⋅,⋅HS is defined as T,SHS=∑i∈ITeiX,SeiXY. For f0∈HX and g0∈HY, the tensor product of f0 and g0 is denoted as f0⊗g0. Since f0⊗g0f=f0,fXg0∈HY, then f0⊗g0∈HSHX⟶HY [53].


### 3.3.2. Mean Functions and Cross Covariance Operators

LetΦX:HX⟶R be a continuous linear functional over HX; for all T∈HSHX⟶HY,(4)ΦXf=EXφXX,fX.According to Riesz theorem, there must a unique HS operatorμX∈HX such that for all f∈HX, ΦXf=f,μXX; then, μX is called the mean function of φXX. Similarly, the mean function μY of φYY is defined in the same way.LetΦ be a continuous linear functional over HSHX⟶HY; for all T∈HSHX⟶HY,(5)ΦT=EXYφXX⊗φYY,THS.Then, according to Riesz theorem, there must a unique HS operatorCXY∈HSHX,HY, such that for all T∈HSHX,HY,(6)ΦT=T,CXYHS,where CXY is called the cross covariance operator between φXX and φYY.The relationship betweenCXY, μX, and μY is illustrated in Figure 1. The two datasets ΩX and ΩY are embedded into HX and HY by the kernel functions ϕX:ΩX⟶HX and ϕY:ΩY⟶HY, respectively. μX and μY are the mean functions. The HSIC of ΩX and ΩY is given by the Hilbert–Schmidt (HS) operator CXY of HX and HY.Figure 1
The sketch mapping of HSIC.


### 3.3.3. HSIC

HSIC of two random variables/vectors is defined as follows:(7)HSICX,Y=EXYφXX−μX⊗φYY−μYHS2.It can be seen from the definition ofHSICX,Y that instead of directly calculating the covariance of X and Y, i.e., EXYX−EXXY−EYY, HSIC first transforms X and Y into HX and HY, respectively, and then calculates the covariance of φXX and φYY by using HS operators between HX and HY. In practice, HX and HY are generated from kernel functions kX and kY.If the joint probability distribution ofX and Y is given or known, HSICX,Y can be calculated as follows:(8)HSICX,Y=EXYφXX−μX⊗φYY−μYHS2=CXY−μX⊗μYHS2=CXY,CXYHS−2CXY,μX⊗μYHS+μX⊗μY,μX⊗μYHS,where(9)CXY,CXYHS=EXYEX′Y′kXX,X′kYY,Y′,(10)CXY,μX⊗μYHS=EXYEX′kXX,X′EY′kYY,Y′,(11)μX⊗μY,μX⊗μYHS=μX,μXXμY,μYY.Generally speaking, the joint probability distribution ofX and Y is unknown, and only some samples of X and Y are given: X≜x1,…,xN⊆ΩX,Y≜y1,…,yN⊆ΩY. In this case, the statistical average can be approximated by the sample average. Moreover, it is assumed that when i≠j, the probability of the random event X=xi;Y=yj is 0; then, the cross covariance operator CXY and the mean functions μX and μY can be approximated as follows:(12)CXY≈1N∑i=1NφXxi⊗φYyi,μX≈1N∑i=1NφXxi,μY≈1N∑i=1NφYyi.Substituting equation (12) into equations (9) and (10) gives(13)CXY,CXYHS≈1N2trKXKY,CXY,μX⊗μYHS≈1N3ΓNTKXKYΓN,μX⊗μY,μX⊗μYHS≈1N4ΓNTKXΓNΓNTKYΓN,where ΓN=1,…,1T∈RN is the N-dimensional vector with all elements being 1 and KX and KY are the kernel matrices of X and Y, respectively.At last, the calculation formula ofHSICX,Y can be obtained by(14)HSICX,Y=CXY−μX⊗μYHS2=CXY,CXYHS+μX⊗μY,μX⊗μYHS−2CXY,μX⊗μYHS≈1N2trKYCNKXCN,where CN=IN−1/NΓNΓNT is the centralizing matrix.


## 3.3.1. Hilbert–Schmidt (HS) Operators

LetT:HX⟶HY be a compact operator and eiX|i∈I be the orthonormal basis of HX; if ∑i∈ITeiXY2<+∞, then T is called a Hilbert–Schmidt (HS) operator [52]. If for all T,S∈HSHX⟶HY, ∑i∈ITeiX,SeiXY<+∞, then HSHX⟶HY,⋅,⋅HS is a Hilbert space. The inner product ⋅,⋅HS is defined as T,SHS=∑i∈ITeiX,SeiXY. For f0∈HX and g0∈HY, the tensor product of f0 and g0 is denoted as f0⊗g0. Since f0⊗g0f=f0,fXg0∈HY, then f0⊗g0∈HSHX⟶HY [53].


## 3.3.2. Mean Functions and Cross Covariance Operators

LetΦX:HX⟶R be a continuous linear functional over HX; for all T∈HSHX⟶HY,(4)ΦXf=EXφXX,fX.According to Riesz theorem, there must a unique HS operatorμX∈HX such that for all f∈HX, ΦXf=f,μXX; then, μX is called the mean function of φXX. Similarly, the mean function μY of φYY is defined in the same way.LetΦ be a continuous linear functional over HSHX⟶HY; for all T∈HSHX⟶HY,(5)ΦT=EXYφXX⊗φYY,THS.Then, according to Riesz theorem, there must a unique HS operatorCXY∈HSHX,HY, such that for all T∈HSHX,HY,(6)ΦT=T,CXYHS,where CXY is called the cross covariance operator between φXX and φYY.The relationship betweenCXY, μX, and μY is illustrated in Figure 1. The two datasets ΩX and ΩY are embedded into HX and HY by the kernel functions ϕX:ΩX⟶HX and ϕY:ΩY⟶HY, respectively. μX and μY are the mean functions. The HSIC of ΩX and ΩY is given by the Hilbert–Schmidt (HS) operator CXY of HX and HY.Figure 1
The sketch mapping of HSIC.


## 3.3.3. HSIC

HSIC of two random variables/vectors is defined as follows:(7)HSICX,Y=EXYφXX−μX⊗φYY−μYHS2.It can be seen from the definition ofHSICX,Y that instead of directly calculating the covariance of X and Y, i.e., EXYX−EXXY−EYY, HSIC first transforms X and Y into HX and HY, respectively, and then calculates the covariance of φXX and φYY by using HS operators between HX and HY. In practice, HX and HY are generated from kernel functions kX and kY.If the joint probability distribution ofX and Y is given or known, HSICX,Y can be calculated as follows:(8)HSICX,Y=EXYφXX−μX⊗φYY−μYHS2=CXY−μX⊗μYHS2=CXY,CXYHS−2CXY,μX⊗μYHS+μX⊗μY,μX⊗μYHS,where(9)CXY,CXYHS=EXYEX′Y′kXX,X′kYY,Y′,(10)CXY,μX⊗μYHS=EXYEX′kXX,X′EY′kYY,Y′,(11)μX⊗μY,μX⊗μYHS=μX,μXXμY,μYY.Generally speaking, the joint probability distribution ofX and Y is unknown, and only some samples of X and Y are given: X≜x1,…,xN⊆ΩX,Y≜y1,…,yN⊆ΩY. In this case, the statistical average can be approximated by the sample average. Moreover, it is assumed that when i≠j, the probability of the random event X=xi;Y=yj is 0; then, the cross covariance operator CXY and the mean functions μX and μY can be approximated as follows:(12)CXY≈1N∑i=1NφXxi⊗φYyi,μX≈1N∑i=1NφXxi,μY≈1N∑i=1NφYyi.Substituting equation (12) into equations (9) and (10) gives(13)CXY,CXYHS≈1N2trKXKY,CXY,μX⊗μYHS≈1N3ΓNTKXKYΓN,μX⊗μY,μX⊗μYHS≈1N4ΓNTKXΓNΓNTKYΓN,where ΓN=1,…,1T∈RN is the N-dimensional vector with all elements being 1 and KX and KY are the kernel matrices of X and Y, respectively.At last, the calculation formula ofHSICX,Y can be obtained by(14)HSICX,Y=CXY−μX⊗μYHS2=CXY,CXYHS+μX⊗μY,μX⊗μYHS−2CXY,μX⊗μYHS≈1N2trKYCNKXCN,where CN=IN−1/NΓNΓNT is the centralizing matrix.


## 4. HRCD and HSIC Subspace Learning

The algorithm can be divided into four steps. First, we model the labeled training samples through the proposed HRCD. Each training sample is described by an SPD matrix. Second, we embed the SPD matrices into a high-dimensional RKHSH with a defined kernel function and further project the elements in the RKHS into a vector space H′. The mapping f:H⟶H′ is explored by solving the optimization problem. Third, we use the explicit map in mapping the training and test samples onto the low-dimensional and relatively discriminative space. Finally, the classification task can be realized by executing a classifier on H′. An overall illustration of the algorithm is shown in Figure 2.Figure 2
Framework of the proposed method. Each image is represented by an SPD matrix on the manifoldM. The points on M are embedded into the RKHS H and are further projected to a low-dimensional and discriminative subspace H′. The map is optimized via the cost function f.Given a set of training samples belonging toc classes χ=X1,X2,…,XN⊆M, Xi∈Rd×d is an SPD matrix, and l=l1,l2,…,lN denote the corresponding labels. The representation of χ on the low-dimensional vector space H′ is denoted as Y=y1,y2,…,yN, yi∈Rm. In the framework of the kernel analysis, the low-dimensional representation yi of Xi is obtained by the mapping yi=WKiRow, where KiRow=ki1,ki2,…,kiNT.


### 4.1. Hybrid Region Covariance Descriptor

As mentioned previously, we propose incorporating the feature mean information into the RCD to improve the discrimination of the descriptor. We refer to the resulting descriptor as the HRCD. Given an image regionR, we extract multiple features of each point in R and then compute the mean vector and covariance matrix of the features. Suppose that the feature vector of the k-th pixel is zk; the mean vector μ∈Rd and covariance matrix Σ∈Rd×d can then be computed as(15)μi=1n∑k=1nzk∈Rd,Σ=1n−1∑k=1nzk−μzk−μT.Following the information geometry theory [54], we combine the mean and the covariance matrix into a new matrix without additional computational complexity. The new matrix is constructed as(16)X=Σ−1/d+1Σ+μμTμμT1.Here,d is the dimensionality of the feature vector, and ⋅ is the determinant operator. The d+1×d+1 SPD matrix X is the HRCD of the image. As a result of the inheritance from the covariance matrix, the HRCD is not only effective, robust, and low-dimensional but also more discriminable than the RCD.


### 4.2. Kernel Function in HSIC-SL

In defining a valid RKHS, the kernel must be symmetric positive definite. Many discussions on the symmetric positive definiteness of kernel functions are based on vector spaces. In this section, we introduce two typical kernel functions on SPD Riemannian manifolds.


#### 4.2.1. Log-Linear Kernel

The polynomial kernel is one of the commonly used kernel functions in Euclidean spaces. The polynomial kernel function in a vector space is defined as(17)kxi,xj=αxiTxj+βγ,where xi,xj∈Rn. If α=γ=1 and β=0, then equation (17) is a linear kernel.(18)kxi,xj=xiTxj.When the linear kernel is developed into SPD Riemannian manifolds, it should be redefined in a sophisticated form. The linear kernel on SPD Riemannian manifolds can be defined as(19)kXi,Xj=trlogXiTlogXj=trlogXilogXj.We denote the kernel as the log-linear kernel.


#### 4.2.2. Log-Gaussian Kernel

The Gaussian kernel is another popular kernel function in Euclidean spaces. The definition of a Gaussian kernel function is(20)kxi,xj=exp−d2xi,xj2σ2,where dxi,xj=xi−xjF. A good effect can be achieved by replacing the Euclidean distance with a log-Euclidean distance. The log-Gaussian kernel is defined by(21)kLEXi,Xj=exp−dLE2Xi,Xj2σ2,where dLEXi,Xj=logXi−logXjF is the log-Euclidean distance between Xi and Xj. The positive definiteness of kLE was proved in [39].The parameterσ is an important parameter in the Gaussian kernel. To make the log-Gaussian kernel sensitive to distances, we suggest setting σ to the average value of distances between the training samples.


### 4.3. HSIC Subspace Learning

After embedding the matrices to the RKHS, we further project the points into a vector space through explicit mapping. We aim to find the explicit mapping from the RKHS to the vector space by maximizing the HSIC between the SPD matrices and the low-dimensional representation, as well as preserving the local information. The proposed HSIC-SL includes global HSIC maximization and within-class information preservation.We denote the HSIC ofχ and the low-dimensional representation Y as HSICχ,Y. According to equation (14), HSICχ,Y can be computed as(22)HSICχ,Y=1N2trKYCNKχCN.The input dataχ and projection Y are represented by Kχ and KY, respectively. To explicitly realize the low-dimensional representation, we define the kernel function of Y in HSICχ,Y as kY:Rd×Rd⟶R, ∀y′,y″∈Rd; that is,(23)kYy′,y″=y′Ty″.We denote the kernel matrix ofkY as KY. It can be computed by(24)KY=y1Ty1⋯y1TyN⋮⋱⋮yNTy1⋯yNTyN=YTY.Substituting equation (24) to equation (22) yields(25)HSICχ,Y=1N2trKYCNKχCN=1N2trYTYCNKχCN=1N2trYCNKχCNYT.AsN is not related to Y, the coefficient 1/N2 in equation (25) can be omitted. Then, we have(26)HSICχ,Y=trWKχCNKχCNKχTWT=trWLHWT,where LH=KχCNKχCNKχT.The within-class information is represented by the within-class scatterSW, which is defined as(27)SW=tr∑i=1c∑j=1Niyk−miyk−miT,where Ni is the number of training samples of the i-th class, ∑i=1cNi=N, and mi is the mean vector corresponding to the i-th class. According to the relationship between yi and Xi, equation (27) can be further transformed into(28)SW=trW∑i=1c∑j=1NiKjRow−KmiKjRow−KmiTWT=trWLWWT,where Kmi=1/Ni∑j=1Nikji, LW=∑i=1c∑j=1NiKjRow−miKjRow−miT.In sum, the objective function is formulated as(29)JW=argmaxWHSICχ,Y=argmaxWWLHWTs.t.trWLWW=trLW.The Rayleigh quotient maximum problem is commonly used in optimization problems because of the fast and simple calculation. The problem shown in equation (29) can be solved by calculating the Rayleigh quotient maximum. To tackle the singularity, we add a small perturbation ε to the diagonal elements of LW. The optimal projection matrix W is composed of the eigenvectors corresponding to the m biggest eigenvalues of LW+εIN−1LH, where IN is the identity matrix.Hence, for the given test image, we first compute its HRCD and denote the result asXt. The projection can be obtained by yt=WKtRow. Then, the class of the test image can be predicted through the nearest neighbor classifier.


## 4.1. Hybrid Region Covariance Descriptor

As mentioned previously, we propose incorporating the feature mean information into the RCD to improve the discrimination of the descriptor. We refer to the resulting descriptor as the HRCD. Given an image regionR, we extract multiple features of each point in R and then compute the mean vector and covariance matrix of the features. Suppose that the feature vector of the k-th pixel is zk; the mean vector μ∈Rd and covariance matrix Σ∈Rd×d can then be computed as(15)μi=1n∑k=1nzk∈Rd,Σ=1n−1∑k=1nzk−μzk−μT.Following the information geometry theory [54], we combine the mean and the covariance matrix into a new matrix without additional computational complexity. The new matrix is constructed as(16)X=Σ−1/d+1Σ+μμTμμT1.Here,d is the dimensionality of the feature vector, and ⋅ is the determinant operator. The d+1×d+1 SPD matrix X is the HRCD of the image. As a result of the inheritance from the covariance matrix, the HRCD is not only effective, robust, and low-dimensional but also more discriminable than the RCD.


## 4.2. Kernel Function in HSIC-SL

In defining a valid RKHS, the kernel must be symmetric positive definite. Many discussions on the symmetric positive definiteness of kernel functions are based on vector spaces. In this section, we introduce two typical kernel functions on SPD Riemannian manifolds.


### 4.2.1. Log-Linear Kernel

The polynomial kernel is one of the commonly used kernel functions in Euclidean spaces. The polynomial kernel function in a vector space is defined as(17)kxi,xj=αxiTxj+βγ,where xi,xj∈Rn. If α=γ=1 and β=0, then equation (17) is a linear kernel.(18)kxi,xj=xiTxj.When the linear kernel is developed into SPD Riemannian manifolds, it should be redefined in a sophisticated form. The linear kernel on SPD Riemannian manifolds can be defined as(19)kXi,Xj=trlogXiTlogXj=trlogXilogXj.We denote the kernel as the log-linear kernel.


### 4.2.2. Log-Gaussian Kernel

The Gaussian kernel is another popular kernel function in Euclidean spaces. The definition of a Gaussian kernel function is(20)kxi,xj=exp−d2xi,xj2σ2,where dxi,xj=xi−xjF. A good effect can be achieved by replacing the Euclidean distance with a log-Euclidean distance. The log-Gaussian kernel is defined by(21)kLEXi,Xj=exp−dLE2Xi,Xj2σ2,where dLEXi,Xj=logXi−logXjF is the log-Euclidean distance between Xi and Xj. The positive definiteness of kLE was proved in [39].The parameterσ is an important parameter in the Gaussian kernel. To make the log-Gaussian kernel sensitive to distances, we suggest setting σ to the average value of distances between the training samples.


## 4.2.1. Log-Linear Kernel

The polynomial kernel is one of the commonly used kernel functions in Euclidean spaces. The polynomial kernel function in a vector space is defined as(17)kxi,xj=αxiTxj+βγ,where xi,xj∈Rn. If α=γ=1 and β=0, then equation (17) is a linear kernel.(18)kxi,xj=xiTxj.When the linear kernel is developed into SPD Riemannian manifolds, it should be redefined in a sophisticated form. The linear kernel on SPD Riemannian manifolds can be defined as(19)kXi,Xj=trlogXiTlogXj=trlogXilogXj.We denote the kernel as the log-linear kernel.


## 4.2.2. Log-Gaussian Kernel

The Gaussian kernel is another popular kernel function in Euclidean spaces. The definition of a Gaussian kernel function is(20)kxi,xj=exp−d2xi,xj2σ2,where dxi,xj=xi−xjF. A good effect can be achieved by replacing the Euclidean distance with a log-Euclidean distance. The log-Gaussian kernel is defined by(21)kLEXi,Xj=exp−dLE2Xi,Xj2σ2,where dLEXi,Xj=logXi−logXjF is the log-Euclidean distance between Xi and Xj. The positive definiteness of kLE was proved in [39].The parameterσ is an important parameter in the Gaussian kernel. To make the log-Gaussian kernel sensitive to distances, we suggest setting σ to the average value of distances between the training samples.


## 4.3. HSIC Subspace Learning

After embedding the matrices to the RKHS, we further project the points into a vector space through explicit mapping. We aim to find the explicit mapping from the RKHS to the vector space by maximizing the HSIC between the SPD matrices and the low-dimensional representation, as well as preserving the local information. The proposed HSIC-SL includes global HSIC maximization and within-class information preservation.We denote the HSIC ofχ and the low-dimensional representation Y as HSICχ,Y. According to equation (14), HSICχ,Y can be computed as(22)HSICχ,Y=1N2trKYCNKχCN.The input dataχ and projection Y are represented by Kχ and KY, respectively. To explicitly realize the low-dimensional representation, we define the kernel function of Y in HSICχ,Y as kY:Rd×Rd⟶R, ∀y′,y″∈Rd; that is,(23)kYy′,y″=y′Ty″.We denote the kernel matrix ofkY as KY. It can be computed by(24)KY=y1Ty1⋯y1TyN⋮⋱⋮yNTy1⋯yNTyN=YTY.Substituting equation (24) to equation (22) yields(25)HSICχ,Y=1N2trKYCNKχCN=1N2trYTYCNKχCN=1N2trYCNKχCNYT.AsN is not related to Y, the coefficient 1/N2 in equation (25) can be omitted. Then, we have(26)HSICχ,Y=trWKχCNKχCNKχTWT=trWLHWT,where LH=KχCNKχCNKχT.The within-class information is represented by the within-class scatterSW, which is defined as(27)SW=tr∑i=1c∑j=1Niyk−miyk−miT,where Ni is the number of training samples of the i-th class, ∑i=1cNi=N, and mi is the mean vector corresponding to the i-th class. According to the relationship between yi and Xi, equation (27) can be further transformed into(28)SW=trW∑i=1c∑j=1NiKjRow−KmiKjRow−KmiTWT=trWLWWT,where Kmi=1/Ni∑j=1Nikji, LW=∑i=1c∑j=1NiKjRow−miKjRow−miT.In sum, the objective function is formulated as(29)JW=argmaxWHSICχ,Y=argmaxWWLHWTs.t.trWLWW=trLW.The Rayleigh quotient maximum problem is commonly used in optimization problems because of the fast and simple calculation. The problem shown in equation (29) can be solved by calculating the Rayleigh quotient maximum. To tackle the singularity, we add a small perturbation ε to the diagonal elements of LW. The optimal projection matrix W is composed of the eigenvectors corresponding to the m biggest eigenvalues of LW+εIN−1LH, where IN is the identity matrix.Hence, for the given test image, we first compute its HRCD and denote the result asXt. The projection can be obtained by yt=WKtRow. Then, the class of the test image can be predicted through the nearest neighbor classifier.


## 5. Experiment

The performance of the HRCD and the proposed algorithm is verified in this section. We considered five widely studied image datasets: COIL-20 (Columbia Object Image Library) dataset [55], ETH-80 dataset [56], Queen Mary University of London (QMUL) dataset [57], face data FERET dataset [58], and Brodatz dataset [59]. All of the compared methods were implemented in MATLAB R2014 and tested on an Intel(R) Core(TM) i5-4670K (3.40 GHz) machine.


### 5.1. Performance of HRCD

To verify that the HRCD is an effective image descriptor, we directly used the KNN classifier on the image feature space represented by the HRCD and RCD without feature extraction. By adopting the Euclidean metric, LERM, AIRM, and Burg divergence as the measurements, the classification experiments were performed on COIL-20 and ETH-80. The COIL-20 dataset contains 20 objects, each of which contains 72 images measuring128×128 at different directions. Figure 3 shows the sample pictures. Features including grey values and first- and second-order gradients were extracted to calculate the RCD and HRCD of an image. Hence, the RCD and HRCD of an image were a 5×5 SPD matrix and a 6×6 SPD matrix, respectively. The images were randomly split into the training set and test set, with 10 pictures assigned to the training set and the remaining images assigned to the test set.Figure 3
Sample images of COIL-20.ETH-80 is an image set containing eight types of objects, such as apple, pears, cars, and dogs. Each object has 10 instances, and each instance contains images from 41 different viewpoints. The images in ETH-80 were resized to128×128 (Figure 4). For the RCD and HRCD representations, we extracted the following features:(30)Fx,y=x,y,Rx,y,Gx,y,Bx,y,Ix,y,Ix,Iy,Ixx,Iyy,where Rx,y, Gx,y, Bx,y are the RGB color values of a pixel at the position of x and y, Ix,y is the greyscale value, and Ix, Iy, Ixx, Iyy are the first-and second-order gradients of intensities. The RCD and HRCD of the image were a 10×10 SPD matrix and a 11×11 SPD matrix, respectively. Half of the instances in every object were used for training, and the remaining instances were used for the test. Each instance in the training and test sets comprised 100 random samples. Therefore, the training and test sets each contained 800 images.Figure 4
Sample images of ETH-80.Table1 lists the classification accuracies and runtimes under different metrics. To eliminate the randomness of the experiment, we obtained the average accuracy and runtime for 20 tests.Table 1
Comparison of RCD and HRCD in terms of classification accuracy (%) and runtime (seconds).
MetricDescriptorCOIL-20ETH-80Classification accuracy (%)Runtime (s)Classification accuracy (%)Runtime (s)EuclideanRCD74.982.94662.635.234HRCD59.882.90866.345.299LERMRCD84.814.14971.037.1HRCD88.994.26272.047.759AIRMRCD87.10118.3371.64482.47HRCD91.06125.7973.35509.15Burg divergenceRCD89.2310.10272.0726.368HRCD91.7110.49273.6326.66


### 5.2. Performance of HSIC-SL

The proposed HSIC-SL was compared with several recognition methods on SPD manifolds. The compared methods included RLPP [38], KSLR [32], CDL [4], KPCA using the log-Gaussian kernel [39], RSR [42], TSC [35], Riem-DLSC [36], logEuc-SC [34], Geometry-DR [25], KLRM-DL [43], EDA [37], and MKSSCR [44]. For brevity, we denote the HSIC-SL with the log-linear kernel as HSIC-SL (log-linear) and that with the log-Gaussian kernel as HSIC-SL (log-Gaussian). HSIC-SL (log-linear) and HSIC-SL (log-Gaussian) were combined with the RCD and HRCD. Thus, for the proposed HSIC-SL, four different combinations were tested. For equality, the important parameters of the comparison methods were set according to the suggestion of the original paper.


#### 5.2.1. Experiments on QMUL Dataset

The QMUL dataset [44] is a set of images of human heads collected from airport terminal cameras. The dataset is composed of 20,005 images. It is divided into five classes according to the direction of the head images: back, front, left, right, and background. The samples from QMUL are shown in Figure 5. The dataset was divided into the training and test sets in advance. Table 2 shows the number of training and test sets in every class. The extracted feature of any pixel is(31)Fx,y=ILx,y,Iax,y,Ibx,y,Ix2+Iy2,arctanIx2Iy2,G1x,y,…,G8x,y,where ILx,y, Iax,y, and Ibx,y are the three channel values of the CIELAB color space, Ix and Iy are the first-order gradients in the x- and y-directions of ILx,y, respectively, and Gix,y;i=1,…,8 is the response of eight difference-of-Gaussians filters. We obtained a 13×13 SPD matrix for the RCD and a 14×14 SPD matrix for the HRCD. The training data consisted of 200 randomly selected samples for each category, and the test set consisted of 100 randomly selected samples. The KNN (k=12) search was used to construct the neighborhood graphs in the RLPP and Geometry-DR. The parameters (σ) in the kernels of the KPCA, RLPP, KSLR, and HSIC-SL were set to the average distances. The parameter γ in the KSLR was set to 0.3. The parameter ε in the proposed method was set to 0.001. We evaluated the performance of the CDL, RLPP, KSLR, Geometry-DR, and HSIC-SL for various dimensions and reported the maximum performance. In logEuc-SC, RSR, TSC, Riem-DLSC, and KLRM-DL, 50 dictionaries and kernel parameters were learned from the training set. The kernel function in the RSR and the basic kernel in the KLRM-DL was the Stein kernel. The parameter alpha was set to 0.1, and the number of data samples was set to 30. The 1NN classifier was adopted in all the algorithms.Figure 5
Sample images of QMUL dataset.Table 2
Distribution of QMUL dataset.
LabelBackBackgroundFrontLeftRightTraining22562256225622562256Test20961107177215022248In Table3, we show the recognition accuracy of the HSIC-SL and the other existing algorithms. To eliminate the randomness of the experiment, we used the average recognition rate for 20 tests. HSIC-SL (log-Gaussian) + HRCD and HSIC-SL (log-linear) + HRCD achieved impressive performance while HSIC-SL (log-Gaussian) + HRCD obtained the highest classification accuracy. Moreover, the accuracy of the HRCD was greater than that of the RCD in the experiment. These results indicated that the HRCD was better than the RCD. Furthermore, HSIC-SL + HRCD was better than the other algorithms.Table 3
Comparison of classification accuracies on QMUL dataset.
MethodsAccuracy (%)KPCA42.5CDL76RLPP58.4KSLR77.4logEuc-SC66.3RSR73.2TSC61.7Riem-DLSC36.6Geometry-DR69.2KLRM_DL70.74EDA75.85MKSSCR78.14HSIC-SL (log-linear) + RCD76.22HSIC-SL (log-linear) + HRCD78.29HSIC-SL (log-Gaussian) + RCD76.16HSIC-SL (log-Gaussian) + HRCD79.65


#### 5.2.2. Experiments on FERET Dataset

To conduct the face recognition experiment, we used the “b” subset of the FERET dataset [56], which consists of 2,000 face images of 200 people. The images are those of 71 females and 129 males of diverse ethnicities, genders, and ages. The images were cropped and downsampled to 64×64. The training set was composed of images with “ba,” “bc,” “bh,” and “bk” labels. Images marked as “bd,” “be,” “bf,” and “bg” constituted the test set. The feature vector for computing the RCD and HRCD is described by(32)Fx,y=x,y,Ix,y,G00x,y,…,G47x,y,where x and y denote the position, Ix,y is the intensity, and Guvx,y is the response value of the Gabor filter. The direction u of the Gabor filter was from 0 to 4, and the scale v was from 0 to 7. Thus, the RCD and HRCD of each image were a 43×43 SPD matrix and a 44×44 SPD matrix, respectively. The neighborhood graphs constructed in the RLPP and Geometry-DR were KNN (k=3). The kernel functions with Jeffrey and Stein divergences were adopted in RSR and, respectively, denoted as RSR-J and RSR-S for brevity. In RSR, TSC, Riem-DLSC, KLRM-DL, and logEuc-SC, all training samples were regarded as dictionary atoms. The settings of the other parameters were the same as those for the QMUL dataset.Table4 shows the recognition rates of the compared algorithms. The proposed method was not the best algorithm for the FERET dataset. It only achieved the highest recognition accuracy in the “bd” test scenario. Nevertheless, the average recognition accuracies of HSIC-SL were still better than those of the other algorithms and were only slightly worse than those of KLRM-DL. Hence, HSIC-SL was still a feasible algorithm for the FERET dataset. We also noticed that HSIC-SL (log-Gaussian) performed better than HSIC-SL (log-linear). Therefore, the log-Gaussian kernel was more suitable than the log-linear kernel for this dataset.Table 4
Comparison of classification accuracies on FERET dataset.
MethodsbdbebfbgAverageCDL76.5075.0088.5084.5081.13RLPP58.4060.0067.0060.5061.48KSLR83.0090.0096.0091.0090.00logEuc-SC74.0094.0097.5080.5086.50RSR-S82.5094.5098.0083.5089.63RSR-J79.5096.5097.5086.0089.88TSC36.0073.0073.5044.5056.75Riem-DLSC88.2593.5096.5091.7592.50Geometry-DR80.5078.0086.5083.0082.00KLRM-DL89.5096.0097.0094.0094.13EDA86.0090.0095.5092.0090.88MKSSCR88.5092.0096.0094.5092.75HSIC-SL (log-linear) + RCD83.5087.5094.0091.0089.00HSIC-SL (log-linear) + HRCD83.5087.0093.5091.5088.88HSIC-SL (log-Gaussian) + RCD88.0088.5095.0093.0091.13HSIC-SL (log-Gaussian) + HRCD90.0090.5096.5093.5092.63


#### 5.2.3. Experiments on Brodatz Dataset

We performed two texture classification experiments on the Brodatz dataset [57]. Examples from the Brodatz dataset are shown in Figure 6. The first experiment was a grouping experiment with selected textures, and the other was a classification experiment for all texture images.Figure 6
Sample images of Brodatz dataset.In the first experiment, we followed the test setup designed in [35] and selected three of the test schemes. The schemes included one of the 5-texture groups, one of the 10-texture groups, and one of the 16-texture groups. The number of classes selected in each test scheme is shown in Table 5. Each image was resized to 256×256, and then 64 regions measuring 32×32 were extracted. The covariance matrices were computed from a five-dimensional feature vector, including intensity and the first- and second-order gradients. In each test scheme, eight samples in one image were randomly selected as the training data, and the remaining samples were used for the test. Geometry-DR was not suitable for the dataset because of the low dimension of the SPD matrices in this experiment. The results shown in Figure 7 are the average results for 20 tests.Table 5
Images selected for each group.
Experiment numberImages selected5-textureD77, D84, D55, D53, D2410-textureD4, D9, D19, D21, D24, D28, D29, D36, D37, D3816-textureD3, D4, D5, D6, D9, D21, D24, D29, D32, D33, D54, D55, D57, D68, D77, D84Figure 7
Comparison of algorithms for 1NN classification accuracy in Brodatz dataset.HSIC-SL achieved the highest classification result on all test schemes, except for the 5-texture test, in which the recognition rates of most of the algorithms were relatively close. In this dataset, HSIC-SL (log-linear) performed better than HSIC-SL (log-Gaussian).In the second experiment, 20 random samples were chosen as the training set, and 10 random samples were chosen as the test set from all the texture images. The average results for 20 tests are presented in Table6. HSIC-SL (log-linear) and HSIC-SL (log-Gaussian) outperformed the other methods, with the latter being marginally better than the former when all texture pictures were classified. In addition, HSIC-SL modeled in the HRCD was much higher than HSIC-SL modeled in the RCD. The discriminative ability of the HRCD was verified again.Table 6
Accuracy of all texture classifications of Brodatz.
MethodsAccuracy (%)CDL85.25RLPP75.8KSLR85.55logEuc-SC63.24RSR-S78.32RSR-J76.7Riem-DLSC37.1Geometry-DR74.9KLRM-DL79.23HSIC-SL (log-linear) + RCD84.87HSIC-SL (log-linear) + HRCD88.87HSIC-SL (log-Gaussian) + RCD85.98HSIC-SL (log-Gaussian) + HRCD89.66


#### 5.2.4. Experiments on COIL-20 and ETH-80 Datasets

In this experiment, we used the COIL-20 dataset [55] and ETH-80 dataset in the object categorization task. The experimental procedure was the same as that described in Section 5.1. We compared the proposed method with KPCA [39], RLPP [38], KSLR [32], and CDL [4]. In addition, KPCA and RLPP were conducted on the HRCD and, respectively, denoted as KPCA + HRCD and RLPP + HRCD. The classifier adopted in all of the algorithms was the 1NN classifier.Table7 shows the classification accuracies of the methods on COIL-20 and ETH-80. First, HSIC-SL obtained the best accuracy in all of the datasets. This result indicated that the introduction of the HSIC improved the effectiveness of the recognition algorithm. Second, the classification accuracies of RLPP, KPCA, and HSIC-SL in the RCD were lower than those in the HRCD (i.e., RLPP + HRCD, KPCA + HRCD, and HSIC-SL + HRCD). This result proved once again that the HRCD had advantages over the RCD. Finally, the effectiveness of the log-linear kernel and log-Gaussian kernel in HSIC-SL was demonstrated in the experiments.Table 7
Comparison of recognition rates (%) of different methods.
MethodsCOIL-20ETH-80KPCA81.0572.61KPCA + HRCD83.7973.7RLPP85.8974.08RLPP + HRCD88.7975.63CDL94.5479.92KSLR96.2481.66HSIC-SL (log-linear) + RCD96.7282.80HSIC-SL (log-linear) + HRCD97.7584.60HSIC-SL (log-Gaussian) + RCD96.8782.40HSIC-SL (log-Gaussian) + HRCD97.9285.28


### 5.3. Analysis of Dimensionality

The parameterm was regarded as the dimensionality of the vector space after feature extraction. The curves of the classification accuracies of the compared algorithms on COIL-20 [55], ETH-80 [37], and Brodatz versus m are shown in Figures 8 and 9. The experimental setups were the same as those described in the previous section.Figure 8
Recognition rates versus different dimensionalities on COIL-20 database.Figure 9
Recognition rates versus different dimensionalities on ETH-80 database.With the increase of the dimensionality, the recognition accuracy curves showed an upward trend. When the recognition accuracy reached a certain value, the recognition rate remained basically stable within a certain range of the subspace dimension.


### 5.4. Discussion

In the above experiments, the performance of the RCD and HRCD and the effectiveness of HSIC-SL and the other algorithms were compared. The following observations were made:(1)
The classification accuracy in the image feature space represented by the HRCD was better than that by the RCD regardless of which classifier was used (i.e., KNN classifier without feature extraction or the proposed HSIC-SL). The result showed that the proposed image descriptor HRCD outperformed the RCD.(2)
When the RCD was used as the image descriptor, the HSIC-SL method was superior to most of the methods, except for the FERET and Brodatz datasets. In FERET, the performance of Riem-DLSC, MKSSCR, and KLRM-DL was slightly better than that of HSIC-SL + RCD (log-Gaussian kernel). In Brodatz, the performance of HSIC-SL + RCD was slightly worse than that of the other methods in the 5-texture group, 10-texture group, and 16-texture group. Nevertheless, the recognition accuracy of HSIC-SL + RCD in the experiment on all texture images was higher than those of the other methods. The results showed that HSIC-SL was indeed an excellent algorithm on SPD manifolds, but it was inferior in the classification of datasets with subtle features, such as face recognition and texture recognition. At the same time, the HRCD makes up for this defect to a certain extent. The performance of HSIC-SL + HRCD was almost superior to that of all methods. However, in the FERET dataset, the average recognition accuracy of HSIC-SL was lower than that of KLRM-DL.(3)
In the experiments, we also compared the performance of the log-Gaussian kernel and log-linear kernel. In general, the log-Gaussian kernel was better than the log-linear kernel. However, in the experiments on QMUL and Brodatz, the log-linear kernel obtained better results than the log-Gaussian kernel. The difference in performance indicated that the choice of kernel affected the performance of HSIC-SL. We can improve the performance of HSIC-SL by selecting a suitable kernel function.


## 5.1. Performance of HRCD

To verify that the HRCD is an effective image descriptor, we directly used the KNN classifier on the image feature space represented by the HRCD and RCD without feature extraction. By adopting the Euclidean metric, LERM, AIRM, and Burg divergence as the measurements, the classification experiments were performed on COIL-20 and ETH-80. The COIL-20 dataset contains 20 objects, each of which contains 72 images measuring128×128 at different directions. Figure 3 shows the sample pictures. Features including grey values and first- and second-order gradients were extracted to calculate the RCD and HRCD of an image. Hence, the RCD and HRCD of an image were a 5×5 SPD matrix and a 6×6 SPD matrix, respectively. The images were randomly split into the training set and test set, with 10 pictures assigned to the training set and the remaining images assigned to the test set.Figure 3
Sample images of COIL-20.ETH-80 is an image set containing eight types of objects, such as apple, pears, cars, and dogs. Each object has 10 instances, and each instance contains images from 41 different viewpoints. The images in ETH-80 were resized to128×128 (Figure 4). For the RCD and HRCD representations, we extracted the following features:(30)Fx,y=x,y,Rx,y,Gx,y,Bx,y,Ix,y,Ix,Iy,Ixx,Iyy,where Rx,y, Gx,y, Bx,y are the RGB color values of a pixel at the position of x and y, Ix,y is the greyscale value, and Ix, Iy, Ixx, Iyy are the first-and second-order gradients of intensities. The RCD and HRCD of the image were a 10×10 SPD matrix and a 11×11 SPD matrix, respectively. Half of the instances in every object were used for training, and the remaining instances were used for the test. Each instance in the training and test sets comprised 100 random samples. Therefore, the training and test sets each contained 800 images.Figure 4
Sample images of ETH-80.Table1 lists the classification accuracies and runtimes under different metrics. To eliminate the randomness of the experiment, we obtained the average accuracy and runtime for 20 tests.Table 1
Comparison of RCD and HRCD in terms of classification accuracy (%) and runtime (seconds).
MetricDescriptorCOIL-20ETH-80Classification accuracy (%)Runtime (s)Classification accuracy (%)Runtime (s)EuclideanRCD74.982.94662.635.234HRCD59.882.90866.345.299LERMRCD84.814.14971.037.1HRCD88.994.26272.047.759AIRMRCD87.10118.3371.64482.47HRCD91.06125.7973.35509.15Burg divergenceRCD89.2310.10272.0726.368HRCD91.7110.49273.6326.66


## 5.2. Performance of HSIC-SL

The proposed HSIC-SL was compared with several recognition methods on SPD manifolds. The compared methods included RLPP [38], KSLR [32], CDL [4], KPCA using the log-Gaussian kernel [39], RSR [42], TSC [35], Riem-DLSC [36], logEuc-SC [34], Geometry-DR [25], KLRM-DL [43], EDA [37], and MKSSCR [44]. For brevity, we denote the HSIC-SL with the log-linear kernel as HSIC-SL (log-linear) and that with the log-Gaussian kernel as HSIC-SL (log-Gaussian). HSIC-SL (log-linear) and HSIC-SL (log-Gaussian) were combined with the RCD and HRCD. Thus, for the proposed HSIC-SL, four different combinations were tested. For equality, the important parameters of the comparison methods were set according to the suggestion of the original paper.


### 5.2.1. Experiments on QMUL Dataset

The QMUL dataset [44] is a set of images of human heads collected from airport terminal cameras. The dataset is composed of 20,005 images. It is divided into five classes according to the direction of the head images: back, front, left, right, and background. The samples from QMUL are shown in Figure 5. The dataset was divided into the training and test sets in advance. Table 2 shows the number of training and test sets in every class. The extracted feature of any pixel is(31)Fx,y=ILx,y,Iax,y,Ibx,y,Ix2+Iy2,arctanIx2Iy2,G1x,y,…,G8x,y,where ILx,y, Iax,y, and Ibx,y are the three channel values of the CIELAB color space, Ix and Iy are the first-order gradients in the x- and y-directions of ILx,y, respectively, and Gix,y;i=1,…,8 is the response of eight difference-of-Gaussians filters. We obtained a 13×13 SPD matrix for the RCD and a 14×14 SPD matrix for the HRCD. The training data consisted of 200 randomly selected samples for each category, and the test set consisted of 100 randomly selected samples. The KNN (k=12) search was used to construct the neighborhood graphs in the RLPP and Geometry-DR. The parameters (σ) in the kernels of the KPCA, RLPP, KSLR, and HSIC-SL were set to the average distances. The parameter γ in the KSLR was set to 0.3. The parameter ε in the proposed method was set to 0.001. We evaluated the performance of the CDL, RLPP, KSLR, Geometry-DR, and HSIC-SL for various dimensions and reported the maximum performance. In logEuc-SC, RSR, TSC, Riem-DLSC, and KLRM-DL, 50 dictionaries and kernel parameters were learned from the training set. The kernel function in the RSR and the basic kernel in the KLRM-DL was the Stein kernel. The parameter alpha was set to 0.1, and the number of data samples was set to 30. The 1NN classifier was adopted in all the algorithms.Figure 5
Sample images of QMUL dataset.Table 2
Distribution of QMUL dataset.
LabelBackBackgroundFrontLeftRightTraining22562256225622562256Test20961107177215022248In Table3, we show the recognition accuracy of the HSIC-SL and the other existing algorithms. To eliminate the randomness of the experiment, we used the average recognition rate for 20 tests. HSIC-SL (log-Gaussian) + HRCD and HSIC-SL (log-linear) + HRCD achieved impressive performance while HSIC-SL (log-Gaussian) + HRCD obtained the highest classification accuracy. Moreover, the accuracy of the HRCD was greater than that of the RCD in the experiment. These results indicated that the HRCD was better than the RCD. Furthermore, HSIC-SL + HRCD was better than the other algorithms.Table 3
Comparison of classification accuracies on QMUL dataset.
MethodsAccuracy (%)KPCA42.5CDL76RLPP58.4KSLR77.4logEuc-SC66.3RSR73.2TSC61.7Riem-DLSC36.6Geometry-DR69.2KLRM_DL70.74EDA75.85MKSSCR78.14HSIC-SL (log-linear) + RCD76.22HSIC-SL (log-linear) + HRCD78.29HSIC-SL (log-Gaussian) + RCD76.16HSIC-SL (log-Gaussian) + HRCD79.65


### 5.2.2. Experiments on FERET Dataset

To conduct the face recognition experiment, we used the “b” subset of the FERET dataset [56], which consists of 2,000 face images of 200 people. The images are those of 71 females and 129 males of diverse ethnicities, genders, and ages. The images were cropped and downsampled to 64×64. The training set was composed of images with “ba,” “bc,” “bh,” and “bk” labels. Images marked as “bd,” “be,” “bf,” and “bg” constituted the test set. The feature vector for computing the RCD and HRCD is described by(32)Fx,y=x,y,Ix,y,G00x,y,…,G47x,y,where x and y denote the position, Ix,y is the intensity, and Guvx,y is the response value of the Gabor filter. The direction u of the Gabor filter was from 0 to 4, and the scale v was from 0 to 7. Thus, the RCD and HRCD of each image were a 43×43 SPD matrix and a 44×44 SPD matrix, respectively. The neighborhood graphs constructed in the RLPP and Geometry-DR were KNN (k=3). The kernel functions with Jeffrey and Stein divergences were adopted in RSR and, respectively, denoted as RSR-J and RSR-S for brevity. In RSR, TSC, Riem-DLSC, KLRM-DL, and logEuc-SC, all training samples were regarded as dictionary atoms. The settings of the other parameters were the same as those for the QMUL dataset.Table4 shows the recognition rates of the compared algorithms. The proposed method was not the best algorithm for the FERET dataset. It only achieved the highest recognition accuracy in the “bd” test scenario. Nevertheless, the average recognition accuracies of HSIC-SL were still better than those of the other algorithms and were only slightly worse than those of KLRM-DL. Hence, HSIC-SL was still a feasible algorithm for the FERET dataset. We also noticed that HSIC-SL (log-Gaussian) performed better than HSIC-SL (log-linear). Therefore, the log-Gaussian kernel was more suitable than the log-linear kernel for this dataset.Table 4
Comparison of classification accuracies on FERET dataset.
MethodsbdbebfbgAverageCDL76.5075.0088.5084.5081.13RLPP58.4060.0067.0060.5061.48KSLR83.0090.0096.0091.0090.00logEuc-SC74.0094.0097.5080.5086.50RSR-S82.5094.5098.0083.5089.63RSR-J79.5096.5097.5086.0089.88TSC36.0073.0073.5044.5056.75Riem-DLSC88.2593.5096.5091.7592.50Geometry-DR80.5078.0086.5083.0082.00KLRM-DL89.5096.0097.0094.0094.13EDA86.0090.0095.5092.0090.88MKSSCR88.5092.0096.0094.5092.75HSIC-SL (log-linear) + RCD83.5087.5094.0091.0089.00HSIC-SL (log-linear) + HRCD83.5087.0093.5091.5088.88HSIC-SL (log-Gaussian) + RCD88.0088.5095.0093.0091.13HSIC-SL (log-Gaussian) + HRCD90.0090.5096.5093.5092.63


### 5.2.3. Experiments on Brodatz Dataset

We performed two texture classification experiments on the Brodatz dataset [57]. Examples from the Brodatz dataset are shown in Figure 6. The first experiment was a grouping experiment with selected textures, and the other was a classification experiment for all texture images.Figure 6
Sample images of Brodatz dataset.In the first experiment, we followed the test setup designed in [35] and selected three of the test schemes. The schemes included one of the 5-texture groups, one of the 10-texture groups, and one of the 16-texture groups. The number of classes selected in each test scheme is shown in Table 5. Each image was resized to 256×256, and then 64 regions measuring 32×32 were extracted. The covariance matrices were computed from a five-dimensional feature vector, including intensity and the first- and second-order gradients. In each test scheme, eight samples in one image were randomly selected as the training data, and the remaining samples were used for the test. Geometry-DR was not suitable for the dataset because of the low dimension of the SPD matrices in this experiment. The results shown in Figure 7 are the average results for 20 tests.Table 5
Images selected for each group.
Experiment numberImages selected5-textureD77, D84, D55, D53, D2410-textureD4, D9, D19, D21, D24, D28, D29, D36, D37, D3816-textureD3, D4, D5, D6, D9, D21, D24, D29, D32, D33, D54, D55, D57, D68, D77, D84Figure 7
Comparison of algorithms for 1NN classification accuracy in Brodatz dataset.HSIC-SL achieved the highest classification result on all test schemes, except for the 5-texture test, in which the recognition rates of most of the algorithms were relatively close. In this dataset, HSIC-SL (log-linear) performed better than HSIC-SL (log-Gaussian).In the second experiment, 20 random samples were chosen as the training set, and 10 random samples were chosen as the test set from all the texture images. The average results for 20 tests are presented in Table6. HSIC-SL (log-linear) and HSIC-SL (log-Gaussian) outperformed the other methods, with the latter being marginally better than the former when all texture pictures were classified. In addition, HSIC-SL modeled in the HRCD was much higher than HSIC-SL modeled in the RCD. The discriminative ability of the HRCD was verified again.Table 6
Accuracy of all texture classifications of Brodatz.
MethodsAccuracy (%)CDL85.25RLPP75.8KSLR85.55logEuc-SC63.24RSR-S78.32RSR-J76.7Riem-DLSC37.1Geometry-DR74.9KLRM-DL79.23HSIC-SL (log-linear) + RCD84.87HSIC-SL (log-linear) + HRCD88.87HSIC-SL (log-Gaussian) + RCD85.98HSIC-SL (log-Gaussian) + HRCD89.66


### 5.2.4. Experiments on COIL-20 and ETH-80 Datasets

In this experiment, we used the COIL-20 dataset [55] and ETH-80 dataset in the object categorization task. The experimental procedure was the same as that described in Section 5.1. We compared the proposed method with KPCA [39], RLPP [38], KSLR [32], and CDL [4]. In addition, KPCA and RLPP were conducted on the HRCD and, respectively, denoted as KPCA + HRCD and RLPP + HRCD. The classifier adopted in all of the algorithms was the 1NN classifier.Table7 shows the classification accuracies of the methods on COIL-20 and ETH-80. First, HSIC-SL obtained the best accuracy in all of the datasets. This result indicated that the introduction of the HSIC improved the effectiveness of the recognition algorithm. Second, the classification accuracies of RLPP, KPCA, and HSIC-SL in the RCD were lower than those in the HRCD (i.e., RLPP + HRCD, KPCA + HRCD, and HSIC-SL + HRCD). This result proved once again that the HRCD had advantages over the RCD. Finally, the effectiveness of the log-linear kernel and log-Gaussian kernel in HSIC-SL was demonstrated in the experiments.Table 7
Comparison of recognition rates (%) of different methods.
MethodsCOIL-20ETH-80KPCA81.0572.61KPCA + HRCD83.7973.7RLPP85.8974.08RLPP + HRCD88.7975.63CDL94.5479.92KSLR96.2481.66HSIC-SL (log-linear) + RCD96.7282.80HSIC-SL (log-linear) + HRCD97.7584.60HSIC-SL (log-Gaussian) + RCD96.8782.40HSIC-SL (log-Gaussian) + HRCD97.9285.28


## 5.2.1. Experiments on QMUL Dataset

The QMUL dataset [44] is a set of images of human heads collected from airport terminal cameras. The dataset is composed of 20,005 images. It is divided into five classes according to the direction of the head images: back, front, left, right, and background. The samples from QMUL are shown in Figure 5. The dataset was divided into the training and test sets in advance. Table 2 shows the number of training and test sets in every class. The extracted feature of any pixel is(31)Fx,y=ILx,y,Iax,y,Ibx,y,Ix2+Iy2,arctanIx2Iy2,G1x,y,…,G8x,y,where ILx,y, Iax,y, and Ibx,y are the three channel values of the CIELAB color space, Ix and Iy are the first-order gradients in the x- and y-directions of ILx,y, respectively, and Gix,y;i=1,…,8 is the response of eight difference-of-Gaussians filters. We obtained a 13×13 SPD matrix for the RCD and a 14×14 SPD matrix for the HRCD. The training data consisted of 200 randomly selected samples for each category, and the test set consisted of 100 randomly selected samples. The KNN (k=12) search was used to construct the neighborhood graphs in the RLPP and Geometry-DR. The parameters (σ) in the kernels of the KPCA, RLPP, KSLR, and HSIC-SL were set to the average distances. The parameter γ in the KSLR was set to 0.3. The parameter ε in the proposed method was set to 0.001. We evaluated the performance of the CDL, RLPP, KSLR, Geometry-DR, and HSIC-SL for various dimensions and reported the maximum performance. In logEuc-SC, RSR, TSC, Riem-DLSC, and KLRM-DL, 50 dictionaries and kernel parameters were learned from the training set. The kernel function in the RSR and the basic kernel in the KLRM-DL was the Stein kernel. The parameter alpha was set to 0.1, and the number of data samples was set to 30. The 1NN classifier was adopted in all the algorithms.Figure 5
Sample images of QMUL dataset.Table 2
Distribution of QMUL dataset.
LabelBackBackgroundFrontLeftRightTraining22562256225622562256Test20961107177215022248In Table3, we show the recognition accuracy of the HSIC-SL and the other existing algorithms. To eliminate the randomness of the experiment, we used the average recognition rate for 20 tests. HSIC-SL (log-Gaussian) + HRCD and HSIC-SL (log-linear) + HRCD achieved impressive performance while HSIC-SL (log-Gaussian) + HRCD obtained the highest classification accuracy. Moreover, the accuracy of the HRCD was greater than that of the RCD in the experiment. These results indicated that the HRCD was better than the RCD. Furthermore, HSIC-SL + HRCD was better than the other algorithms.Table 3
Comparison of classification accuracies on QMUL dataset.
MethodsAccuracy (%)KPCA42.5CDL76RLPP58.4KSLR77.4logEuc-SC66.3RSR73.2TSC61.7Riem-DLSC36.6Geometry-DR69.2KLRM_DL70.74EDA75.85MKSSCR78.14HSIC-SL (log-linear) + RCD76.22HSIC-SL (log-linear) + HRCD78.29HSIC-SL (log-Gaussian) + RCD76.16HSIC-SL (log-Gaussian) + HRCD79.65


## 5.2.2. Experiments on FERET Dataset

To conduct the face recognition experiment, we used the “b” subset of the FERET dataset [56], which consists of 2,000 face images of 200 people. The images are those of 71 females and 129 males of diverse ethnicities, genders, and ages. The images were cropped and downsampled to 64×64. The training set was composed of images with “ba,” “bc,” “bh,” and “bk” labels. Images marked as “bd,” “be,” “bf,” and “bg” constituted the test set. The feature vector for computing the RCD and HRCD is described by(32)Fx,y=x,y,Ix,y,G00x,y,…,G47x,y,where x and y denote the position, Ix,y is the intensity, and Guvx,y is the response value of the Gabor filter. The direction u of the Gabor filter was from 0 to 4, and the scale v was from 0 to 7. Thus, the RCD and HRCD of each image were a 43×43 SPD matrix and a 44×44 SPD matrix, respectively. The neighborhood graphs constructed in the RLPP and Geometry-DR were KNN (k=3). The kernel functions with Jeffrey and Stein divergences were adopted in RSR and, respectively, denoted as RSR-J and RSR-S for brevity. In RSR, TSC, Riem-DLSC, KLRM-DL, and logEuc-SC, all training samples were regarded as dictionary atoms. The settings of the other parameters were the same as those for the QMUL dataset.Table4 shows the recognition rates of the compared algorithms. The proposed method was not the best algorithm for the FERET dataset. It only achieved the highest recognition accuracy in the “bd” test scenario. Nevertheless, the average recognition accuracies of HSIC-SL were still better than those of the other algorithms and were only slightly worse than those of KLRM-DL. Hence, HSIC-SL was still a feasible algorithm for the FERET dataset. We also noticed that HSIC-SL (log-Gaussian) performed better than HSIC-SL (log-linear). Therefore, the log-Gaussian kernel was more suitable than the log-linear kernel for this dataset.Table 4
Comparison of classification accuracies on FERET dataset.
MethodsbdbebfbgAverageCDL76.5075.0088.5084.5081.13RLPP58.4060.0067.0060.5061.48KSLR83.0090.0096.0091.0090.00logEuc-SC74.0094.0097.5080.5086.50RSR-S82.5094.5098.0083.5089.63RSR-J79.5096.5097.5086.0089.88TSC36.0073.0073.5044.5056.75Riem-DLSC88.2593.5096.5091.7592.50Geometry-DR80.5078.0086.5083.0082.00KLRM-DL89.5096.0097.0094.0094.13EDA86.0090.0095.5092.0090.88MKSSCR88.5092.0096.0094.5092.75HSIC-SL (log-linear) + RCD83.5087.5094.0091.0089.00HSIC-SL (log-linear) + HRCD83.5087.0093.5091.5088.88HSIC-SL (log-Gaussian) + RCD88.0088.5095.0093.0091.13HSIC-SL (log-Gaussian) + HRCD90.0090.5096.5093.5092.63


## 5.2.3. Experiments on Brodatz Dataset

We performed two texture classification experiments on the Brodatz dataset [57]. Examples from the Brodatz dataset are shown in Figure 6. The first experiment was a grouping experiment with selected textures, and the other was a classification experiment for all texture images.Figure 6
Sample images of Brodatz dataset.In the first experiment, we followed the test setup designed in [35] and selected three of the test schemes. The schemes included one of the 5-texture groups, one of the 10-texture groups, and one of the 16-texture groups. The number of classes selected in each test scheme is shown in Table 5. Each image was resized to 256×256, and then 64 regions measuring 32×32 were extracted. The covariance matrices were computed from a five-dimensional feature vector, including intensity and the first- and second-order gradients. In each test scheme, eight samples in one image were randomly selected as the training data, and the remaining samples were used for the test. Geometry-DR was not suitable for the dataset because of the low dimension of the SPD matrices in this experiment. The results shown in Figure 7 are the average results for 20 tests.Table 5
Images selected for each group.
Experiment numberImages selected5-textureD77, D84, D55, D53, D2410-textureD4, D9, D19, D21, D24, D28, D29, D36, D37, D3816-textureD3, D4, D5, D6, D9, D21, D24, D29, D32, D33, D54, D55, D57, D68, D77, D84Figure 7
Comparison of algorithms for 1NN classification accuracy in Brodatz dataset.HSIC-SL achieved the highest classification result on all test schemes, except for the 5-texture test, in which the recognition rates of most of the algorithms were relatively close. In this dataset, HSIC-SL (log-linear) performed better than HSIC-SL (log-Gaussian).In the second experiment, 20 random samples were chosen as the training set, and 10 random samples were chosen as the test set from all the texture images. The average results for 20 tests are presented in Table6. HSIC-SL (log-linear) and HSIC-SL (log-Gaussian) outperformed the other methods, with the latter being marginally better than the former when all texture pictures were classified. In addition, HSIC-SL modeled in the HRCD was much higher than HSIC-SL modeled in the RCD. The discriminative ability of the HRCD was verified again.Table 6
Accuracy of all texture classifications of Brodatz.
MethodsAccuracy (%)CDL85.25RLPP75.8KSLR85.55logEuc-SC63.24RSR-S78.32RSR-J76.7Riem-DLSC37.1Geometry-DR74.9KLRM-DL79.23HSIC-SL (log-linear) + RCD84.87HSIC-SL (log-linear) + HRCD88.87HSIC-SL (log-Gaussian) + RCD85.98HSIC-SL (log-Gaussian) + HRCD89.66


## 5.2.4. Experiments on COIL-20 and ETH-80 Datasets

In this experiment, we used the COIL-20 dataset [55] and ETH-80 dataset in the object categorization task. The experimental procedure was the same as that described in Section 5.1. We compared the proposed method with KPCA [39], RLPP [38], KSLR [32], and CDL [4]. In addition, KPCA and RLPP were conducted on the HRCD and, respectively, denoted as KPCA + HRCD and RLPP + HRCD. The classifier adopted in all of the algorithms was the 1NN classifier.Table7 shows the classification accuracies of the methods on COIL-20 and ETH-80. First, HSIC-SL obtained the best accuracy in all of the datasets. This result indicated that the introduction of the HSIC improved the effectiveness of the recognition algorithm. Second, the classification accuracies of RLPP, KPCA, and HSIC-SL in the RCD were lower than those in the HRCD (i.e., RLPP + HRCD, KPCA + HRCD, and HSIC-SL + HRCD). This result proved once again that the HRCD had advantages over the RCD. Finally, the effectiveness of the log-linear kernel and log-Gaussian kernel in HSIC-SL was demonstrated in the experiments.Table 7
Comparison of recognition rates (%) of different methods.
MethodsCOIL-20ETH-80KPCA81.0572.61KPCA + HRCD83.7973.7RLPP85.8974.08RLPP + HRCD88.7975.63CDL94.5479.92KSLR96.2481.66HSIC-SL (log-linear) + RCD96.7282.80HSIC-SL (log-linear) + HRCD97.7584.60HSIC-SL (log-Gaussian) + RCD96.8782.40HSIC-SL (log-Gaussian) + HRCD97.9285.28


## 5.3. Analysis of Dimensionality

The parameterm was regarded as the dimensionality of the vector space after feature extraction. The curves of the classification accuracies of the compared algorithms on COIL-20 [55], ETH-80 [37], and Brodatz versus m are shown in Figures 8 and 9. The experimental setups were the same as those described in the previous section.Figure 8
Recognition rates versus different dimensionalities on COIL-20 database.Figure 9
Recognition rates versus different dimensionalities on ETH-80 database.With the increase of the dimensionality, the recognition accuracy curves showed an upward trend. When the recognition accuracy reached a certain value, the recognition rate remained basically stable within a certain range of the subspace dimension.


## 5.4. Discussion

In the above experiments, the performance of the RCD and HRCD and the effectiveness of HSIC-SL and the other algorithms were compared. The following observations were made:(1)
The classification accuracy in the image feature space represented by the HRCD was better than that by the RCD regardless of which classifier was used (i.e., KNN classifier without feature extraction or the proposed HSIC-SL). The result showed that the proposed image descriptor HRCD outperformed the RCD.(2)
When the RCD was used as the image descriptor, the HSIC-SL method was superior to most of the methods, except for the FERET and Brodatz datasets. In FERET, the performance of Riem-DLSC, MKSSCR, and KLRM-DL was slightly better than that of HSIC-SL + RCD (log-Gaussian kernel). In Brodatz, the performance of HSIC-SL + RCD was slightly worse than that of the other methods in the 5-texture group, 10-texture group, and 16-texture group. Nevertheless, the recognition accuracy of HSIC-SL + RCD in the experiment on all texture images was higher than those of the other methods. The results showed that HSIC-SL was indeed an excellent algorithm on SPD manifolds, but it was inferior in the classification of datasets with subtle features, such as face recognition and texture recognition. At the same time, the HRCD makes up for this defect to a certain extent. The performance of HSIC-SL + HRCD was almost superior to that of all methods. However, in the FERET dataset, the average recognition accuracy of HSIC-SL was lower than that of KLRM-DL.(3)
In the experiments, we also compared the performance of the log-Gaussian kernel and log-linear kernel. In general, the log-Gaussian kernel was better than the log-linear kernel. However, in the experiments on QMUL and Brodatz, the log-linear kernel obtained better results than the log-Gaussian kernel. The difference in performance indicated that the choice of kernel affected the performance of HSIC-SL. We can improve the performance of HSIC-SL by selecting a suitable kernel function.


## 6. Conclusions

In this work, we propose an improved covariance descriptor called the HRCD, which represents images with SPD matrices. The HRCD inherits the advantages of the RCD and is more effective.To address the classification problem on SPD Riemannian manifolds, we propose an efficient image classification method that is based on a kernel framework. We refer to it as HSIC-SL. Through the definition of the log-linear kernel and log-Gaussian kernel, the input images represented by SPD matrices can be embedded into the RKHS. To seek explicit mapping from the RKHS to the vector space, HSIC-SL constructs the objective function on the basis of the framework of subspace learning and HSIC maximization. HSIC-SL always outperforms other representative methods without increasing computational complexity.The proposed algorithm also has certain limitations. The average classification accuracy is slightly worse than that of KLRM-DL on the FERET dataset. Hence, the covariance descriptor is not strong enough to handle the classification of small details, such as face recognition. For our future work, we will employ other effective features to form the covariance matrices. We will also explore other useful kernel functions to suit different types of datasets.



---

*Source: 6663710-2021-07-21.xml*