# Bio-Inspired Structure Representation Based Cross-View Discriminative Subspace Learning via Simultaneous Local and Global Alignment

Authors: Ao Li; Yu Ding; Xunjiang Zheng; Deyun Chen; Guanglu Sun; Kezheng Lin
Journal: Complexity
 (2020)

---

## Abstract

Recently, cross-view feature learning has been a hot topic in machine learning due to the wide applications of multiview data. Nevertheless, the distribution discrepancy between cross-views leads to the fact that instances of the different views from same class are farther than those within the same view but from different classes. To address this problem, in this paper, we develop a novel cross-view discriminative feature subspace learning method inspired by layered visual perception from human. Firstly, the proposed method utilizes a separable low-rank self-representation model to disentangle the class and view structure layers, respectively. Secondly, a local alignment is constructed with two designed graphs to guide the subspace decomposition in a pairwise way. Finally, the global discriminative constraint on distribution center in each view is designed for further alignment improvement. Extensive cross-view classification experiments on several public datasets prove that our proposed method is more effective than other existing feature learning methods.


---

## Body

## 1. Introduction

Under the modern technique background, there are many artificial intelligence methods inspired by nature, such as machine learning [1–4], reinforcement learning [5], and artificial immune recognition [6]. Among them, machine learning can effectively deal with image recognition problems. However, some researches have indicated that the adaptive ability of traditional machine learning drops sharply, when the learned images have large distribution discrepancy, such as cross-view data [2]. This discrepancy means that data variance in the view space is larger than data variance in the class space. It generates that the different views become the major factor affecting recognition. Therefore, we mainly focus on cross-view subspace learning to deal with the distribution discrepancy problems in this paper.In recent years, subspace learning (SL) has made great contributions in the field of machine learning and has a wide application in computer vision, data mining, and so on [7–21]. One of the most typical methods is principal component analysis (PCA) [22], which uses an orthogonal transformation to reduce the dimensionality while preserving unique information (principal component) of the data. However, PCA is an unsupervised dimensionality reduction method, which disregards the discriminative information attached to the semantic component. So, a supervised dimensionality reduction method, linear discriminant analysis (LDA), using semantic component was proposed in [23]. LDA learns a supervised linear combination to adjust the spatial dispersion. However, LDA generates the overfitting phenomenon when processing noisy data. To overcome corruption, rank minimization technique has been spotlighted in recent years. Candès et al. enforced low-rank and sparse constraints to eliminate corrupt information in the data [24]. After that, low-rank representation (LRR) was proposed to restore clean data through dictionary representation in multiple spaces [7]. In the last decade, LRR model has achieved satisfactory results in various fields [8–16, 25–35], such as domain adaptation [8], clustering [9], transfer learning [25], and low-rank texture structure [26]. However, Liu et al. pointed out that the dictionary of LRR may fail when the data is insufficient. To solve this problem, the latent LRR (LatLRR) was proposed to enhance subspace learning by latent information [10]. However, LatLRR is still unsupervised. Inspired by LRR models and LDA, Li et al. unified linear discriminant constraint and low-rank representation into subspace learning to enhance the learned low-dimensional feature [11]. Afterwards, the low-rank embedding (LRE) proposed in [12] provides a robust embedding subspace learning framework that eliminates reconstruction errors by adopting l2,1-norm constraint on the projection residual. The latent low-rank and sparse embedding (LLRSE) was developed in [13] based on LRE. LLRSE additionally introduces a reconstructed orthogonal matrix, which makes the projection space contain more unique feature.Recently, a lot of algorithms dedicated to cross-view feature learning based on above methods have been developed [27–31]. The low-rank common subspace (LRCS) was proposed in [27], which finds a view common subspace by LRR. Nevertheless, LRCS only considers the view label and ignores the discrimination of the class label. From the perspective of multiview data structure, a supervised subspace learning method, namely, multiview discriminant analysis (MVDA), was proposed in [28] by using the discriminative information in the different views. After that, a multiview manifold learning with locality alignment (MVML-LA) framework proposed in [29] provides us with a discriminative low-dimensional latent space. Most recently, the robust cross-view learning (RCVL) was designed to learn a common view-invariance discriminative subspace by adopting a novel rank minimization technique [30]. However, RCVL ignores the global discriminative information.Human visual perception works through visual circuit, whose function is to understand the visual signal in a layer-wise way. In detail, our brain uses a small feature extractor (each layer in the visual circuit) to obtain some simple features from the real complex signal. Drawing inspiration from layered processing in human visual system, we represent the cross-view data in two different structure layers, class structure layer and view structure layer, respectively, in order to construct the view-consistency feature learning model. Hence, we design two novel discriminative alignment constraints from simultaneous local and global viewpoints, which can not only disentangle the class and view layers but also bridge the gap existent in cross-view data. Our contributions are as follows:(1)
The dual low-rank representation model is set up to discover the two latent structures existent in cross-view data, which are view and class structures, respectively. These two distinguished structures are conducive to discovering the potential feature for cross-view classification task.(2)
A local alignment constraint based on two designed local graphs is utilized to transform the neighbouring relationships between each pair of samples in the learned subspace. This constraint can make the view and class structures be separated effectively.(3)
A global alignment constraint designed in our framework ulteriorly cut down view discrepancy. The projected samples from view and class subspaces are used to compose the discriminative constraint in global alignment by enforcing the mean distance between classes in different views.Figure1 illustrates how to learn an aligned subspace in which samples have a large distance between classes and a close distance within class from both of class perspective and view perspective. The structure of this paper is organized as follows. Related works simply review the baseline of our work. Part three presents the proposed model and its solution process. Experiments show the results of the comparison experiments and parameter experiments. At last, conclusions summarize this paper.Figure 1
Illustration of the discriminative alignment constraint framework.


## 2. Related Works

Our method has an intimate connection with the two following methods: (1) low-rank representation and (2) linear discriminant analysis.


### 2.1. Low-Rank Representation

LRR is unaffected by errors and can explore the underlying structure of data. Suppose thatX=X1,X2,…,Xk is a matrix of natural data from k classes. The model of LRR can be expressed by(1)minZ,ErankZ+λE1s.t.X=XZ+E,where Z is the low-rank linear combination coefficient matrix of data X. Matrix E with l1-norm can fit the corrupted information in real life. λ>0 is used to balance the level of corruption. Therefore, LRR that can simulate the corrupted data with representation framework possesses a feasible skill, which can handle the cross-view data well.


### 2.2. Linear Discriminant Analysis

LDA is a familiar supervised method proposed for dimensional reduction, which is constrained by discriminative semantic information. The main principle of LDA is to find a discriminative subspace with the largest interclass variance and the smallest intraclass variance. Assume thatn samples from m classes are X,y=x1,y1,…,xn,yn, where X represents samples and y represents labels of different samples. In addition, x^ and x^i denote the center of all samples and samples belonging to the ith class, respectively. Hence, between-class scatter and within-class scatter are as follows:(2)Sb=∑i=1mnix^i−x^x^i−x^T,Sw=∑i=1m∑x∈Xix−x^ix−x^iT,where ni is the number of samples from the ith class. Therefore, the generalized Rayleigh quotient can be described as follows by Fisher discriminant criterion:(3)maxwTrwTSbwTrwTSww,where w is a projection matrix and Tr⋅ denotes the trace operator. Furthermore, the solution of equation (3) is relatively complicated, so we transform it into trace-difference problem as follows [36]:(4)maxwTrwTSbw−TrwTSww.LDA not only can retain as much information for recognition as possible while reducing the dimensionality but also can remove superfluous and dependent variable feature to the disadvantage of classification task. Nevertheless, due to the distribution of cross-view data, the performance of LDA only considered class semantic information is not outstanding.


## 2.1. Low-Rank Representation

LRR is unaffected by errors and can explore the underlying structure of data. Suppose thatX=X1,X2,…,Xk is a matrix of natural data from k classes. The model of LRR can be expressed by(1)minZ,ErankZ+λE1s.t.X=XZ+E,where Z is the low-rank linear combination coefficient matrix of data X. Matrix E with l1-norm can fit the corrupted information in real life. λ>0 is used to balance the level of corruption. Therefore, LRR that can simulate the corrupted data with representation framework possesses a feasible skill, which can handle the cross-view data well.


## 2.2. Linear Discriminant Analysis

LDA is a familiar supervised method proposed for dimensional reduction, which is constrained by discriminative semantic information. The main principle of LDA is to find a discriminative subspace with the largest interclass variance and the smallest intraclass variance. Assume thatn samples from m classes are X,y=x1,y1,…,xn,yn, where X represents samples and y represents labels of different samples. In addition, x^ and x^i denote the center of all samples and samples belonging to the ith class, respectively. Hence, between-class scatter and within-class scatter are as follows:(2)Sb=∑i=1mnix^i−x^x^i−x^T,Sw=∑i=1m∑x∈Xix−x^ix−x^iT,where ni is the number of samples from the ith class. Therefore, the generalized Rayleigh quotient can be described as follows by Fisher discriminant criterion:(3)maxwTrwTSbwTrwTSww,where w is a projection matrix and Tr⋅ denotes the trace operator. Furthermore, the solution of equation (3) is relatively complicated, so we transform it into trace-difference problem as follows [36]:(4)maxwTrwTSbw−TrwTSww.LDA not only can retain as much information for recognition as possible while reducing the dimensionality but also can remove superfluous and dependent variable feature to the disadvantage of classification task. Nevertheless, due to the distribution of cross-view data, the performance of LDA only considered class semantic information is not outstanding.


## 3. Our Proposed Method

This section contains four parts. The first part of this chapter is to specify the symbols in our algorithm. Part two is a detailed introduction and direction about our framework. The third one develops a numerical scheme to obtain the approximate solutions iteratively. The last part discusses the computational complexity of our proposed algorithm.


### 3.1. Notations

Assume thatX1 and X2X=X1,X2∈Rd×n are two matrices of different views from same c classes, where n and d denote the number and dimensionality of all training samples, respectively. Class structure Zc∈Rn×n and view structure Zv∈Rn×n are two linear combination matrices, which are included in local graph framework to discover view-invariant structure of cross-view data. P∈Rd×p is a basis transformation projection matrix, where p is dimensionality of projected data. E∈Rd×n is a matrix of error data designed to obtain a robust subspace from noise. In addition, V1,V2,V^1,V^2∈Rn×c are four constant coefficient matrices utilized for aligning the global information of cross-view data.


### 3.2. Objective Function

To address the cross-view discriminative analysis, we formulate our subspace learning model with simultaneous local and global alignments as follows:(5)minZc,Zv,E,PZc∗+Zv∗+λ1E2,1︷DZc,Zv,E+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT︷ℒP,Zc,Zv+λ2TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv︷GP,Zc,Zvs.t.X=XZc+Zv+E,PTP=I,where DZc,Zv,E is a low-rank framework of two potential manifolds in cross-view data. ℒP,Zc,Zv enforces the view-specific discriminative local neighbor relationship among instances. GP,Zc,Zv performs the discriminative clustering and separation of the global structure in the class manifold and the view structure, respectively. In the following, the above terms are illustrated in detail.


#### 3.2.1. Dual Low-Rank Representations

Only one linear combination matrix, which is constrained by the rank minimum, is used by methods based on low-rank model. Nevertheless, unitary low-rank structure gives rise to the failure of linear representation due to differences between distributions of views, which endow cross-view data from a same class with a large divergence. Therefore, the two structure matricesZc and Zv are adopted to solve this specific problem, where the between-view samples from the same class are far away and the within-view samples from the different classes are closer. The first term is defined with dual low-rank representations to strip down the class and view structures as follows:(6)DZc,Zv,E=Zc∗+Zv∗+λ1E2,1s.t.X=XZc+Zv+E,where ⋅∗ is a sign of the nuclear norm, which is close to a representation of the rank minimum problem, and its solution is relatively convenient. Assuming that part of the data from real world contains corruption, we adopt the l2,1-norm to make matrix E have the structured sparsity as the noisy data. l2,1-norm can effectively remove the corruption of specific sample while holding the other clean samples. λ1>0 is used to balance the corruption.


#### 3.2.2. Graph-Based Discriminative Local Alignment

To introduce the local discriminative constraint, two graph-based constraints are constructed on each pair of synthetic samples withZc and Zv from class and view subspaces, respectively, as follows, which can better cluster intraclass samples and decentralize interclass ones.(7)ℒc=∑i,jYc,i−Yc,j2Wi,jc,ℒv=∑i,jYv,i−Yv,j2Wi,jv,where Yc,i and Yv,i denote the ith projected samples of cross-view data from the class space Yc=PTXZc and view space Yv=PTXZv, respectively. Correspondingly, Yc,j and Yv,j denote the jth projected samples. Wi,jc and Wi,jv denote graph weight matrices that are defined as follows:(8)Wi,jc=1,ifxi∈Nk1cxj,andli=lj,0,othervise,Wi,jv=1,ifxi∈Nk2vxj,butli≠lj,0,othervise,where li and lj are the labels of samples xi and xj, respectively. xi∈Nk1cxj denotes that xi belongs to the k1 adjacent datasets of the same sample xj. xi∈Nk2vxj means that xi belongs to the k2 adjacent datasets of the same view sample xj. Hence, ℒc can calculate the distance between two similar samples from same class. Similarly, ℒv denotes the distance between two similar samples from different classes. With the help of Fisher criterion, the pairwise local discriminative constraint ℒZc,Zv,P can be rewritten as follows:(9)ℒZc,Zv,P=αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT,where Lc and Lv denote the Laplacian operators of Wc and Wv. α is a balance parameter. Minimizing the subtraction of trace in equation (9) can weaken the impact of view information and separate the class structure and the view structure.


#### 3.2.3. Discriminative Global Alignment

The discriminative projection of all pairwise samples throughℒZc,Zv,P can reduce the impact between views, but the differences between learned features from different classes are not significant enough. So, to further enhance the separation of two manifolds, we design a global discriminative constraint for cross-view analysis as the third term GP,Zc,Zv:(10)GP,Zc,Zv=TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv,where SWPTXZc is within-class and between-view scatter matrix in class manifold and SBiPTXZvi=1,2 is within-view and between-class scatter matrix in view manifold. These scatter matrices are formulated as(11)SWPTXZc=∑j=1cμj1−μj2μj1−μj2T,SBiPTXZv=∑j=1cμji−μiμji−μiT,where μii=1,2 denotes the overall mean feature from the ith view and μji denotes the mean feature of the jth class from the ith view. In this way, the within-class view margin in class structure can be reduced, and the margin of between-class data from same view in view structure can be magnified. The third term can be framed as(12)GP,Zc,Zv=λ2TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv=λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2,where Vi is the coefficient matrix of the within-class mean feature of the ith view and V^ii=1,2 is the coefficient matrix of the overall mean feature of the ith view. In detail, Vik,m=1/nim only if xk belongs to the mth class from the ith view, where nim denotes the number of samples of the mth class from the ith view; in other cases, Vik,m=0. V^ik,m=1/ni only if xk belongs to the ith view, where ni denotes the number of samples from the ith view; in other cases, V^ik,m=0. λ2 is a trade-off parameter. Equation (12) achieves global alignment by the mean vectors of joint synthetic samples from global representation and further enforces the view-invariant constraint on the same class.In addition, we add an orthogonal constraintPTP=I to neglect trivial solutions. In the end, we rewrite equation (5) with all the terms as(13)minZc,Zv,E,PZc∗+Zv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2s.t.X=XZc+Zv+E,PTP=I.


### 3.3. Optimization Scheme

To obtain the feasible solution ofZc and Zv, we adopt two auxiliary variables Jc and Jv. Then, equation (13) can be transformed into the following term:(14)minZc,Zv,E,P,Jc,JvJc∗+Jv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2s.t.X=XZc+Zv+E,PTP=I,Jc=Zc,Jv=Zv.For optimization problems with equality constrains, the Augmented Lagrangian method is an effective solution. The Augmented Lagrangian form of equation (14) is as follows:(15)minZc,Zv,E,P,Jc,JvJc∗+Jv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2+TrY1TX−XZc+Zv−E+TrY2TJc−Zc+TrY3TJv−Zv+η2X−XZc+Zv−EF2+Jc−ZcF2+Jv−ZvF2s.t.PTP=I,where Y1,Y2, and Y3 are the Lagrange multipliers and η>0 is the penalty parameter. We use an alternating solution to optimize iteratively all variables. We define the left bottom of the variable plus t as the t-th solution.First, by ignoring the other variables exceptP, equation (15) becomes(16)minPtαTrPtTXZc,tLcPtTXZc,tT−TrPtTXZv,tLvPtTXZv,tT+λ2PtTXZc,tV1−V2F2−PtTXZv,tV1−V^1F2−PtTXZv,tV2−V^2F2s.t.PTP=I.We obtain the projection matrixPt one by one, because Pt is an orthogonal matrix. For the ith column of Pt, the objective function is rewritten as(17)minPi,tαTrPi,tTXZc,tLcPi,tTXZc,tT−TrPi,tTXZv,tLvPi,tTXZv,tT+εi,tPi,tTPi,t−I+λ2Pi,tTXZc,tV1−V2F2−Pi,tTXZv,tV1−V^1F2−Pi,tTXZv,tV2−V^2F2.We enforce the derivative of function (17) to be zero.(18)XZv,tV1−V^1V1−V^1T+V2−V^2V2−V^2T+αLvZv,tT−Zc,tV1−V2V1−V2T+αLcZc,tTXTPi,t=εi,tPi,t.Therefore,Pi,t is the ith eigenvector of the matrix in equation (18) and P can be simply solved.UpdateJc:(19)Jc,t+1=minJc,tJc,t∗+TrY2TJc,t−Zc,t+ηt2Jc,t−Zc,tF2=1ηtJc,t∗+12Jc,t−Zc,t+Y2,tηtF2.The singular value thresholding is an approximate method to solve the above kernel norm minimization equations [37].UpdateJv:(20)Jv,t+1=minJv,tJv,t∗+TrY3TJv,t−Zv,t+ηt2Jv,t−Zv,tF2=1ηtJv,t∗+12Jv,t−Zv,t+Y3,tηtF2.Equation (20) can be addressed in the same way as equation (19).We enforce the derivative of equation (15) with respect to Zc to be zero.(21)XTPtPtTX−1I+XTXZc,t+1+Zc,t+1λ2V1−V2V1−V2T+αLcηt=XTPtPtTX−1XTX−XZv,t−Et+Jc,t+XTY1,t−Y2,tηt.It is obvious that equation (21) is a Sylvester equation. We can easily solve it by [38]. Similarly, we enforce the derivative of equation (15) with respect to Zv to be zero.(22)XTPtPtTX−1I+XTXZv,t+1+Zv,t+1λ2V1−V^1V1−V^1T+λ2V2−V^2V2−V^2T+αLvηt=XTPtPtTX−1XTX−XZc,t−Et+Jv,t+XTY1,t−Y3,tηt.Equation (22) can be addressed in the same way as equation (21).UpdateE:(23)Et+1=minEtλ1ηtEt2,1+12Et−X−XZc,t+Zv,t+Y1ηtF2.The above equation is al2,1-norm minimization problem whose solution is shown in [39].The entire numerical iterative scheme for equation (14) is shown in Algorithm 1, where the parameters ρ,θ,tmax,η, and ηmax are set empirically. Moreover, the matrices Zc,Zv,E,Y1,Y2, and Y3 are initialized as 0 and the parameters α,λ1, and λ2 are tuned by the experiments.Algorithm 1:Algorithm for our proposed framework.
Input: data matrix X, parameters α,λ1 and λ2Initialize:ρ=1.3, θ=10−9, t=0, tmax=200, η0=0.1 and ηmax=1010;while not converged ort≤tmaxdo(1)SolvingPt+1 by equation (18);(2)SolvingJc,t+1 by equation (19);(3)SolvingJv,t+1 by equation (20);(4)SolvingZc,t+1 by equation (21);(5)SolvingZv,t+1 by equation (22);(6)SolvingEt by equation (23);(7)UpdatingY1,t+1 by Y1,t+1=Y1,t+ηtX−XZc,t+1+Zv,t+1−E;(8)UpdatingY2,t+1 by Y2,t+1=Y2,t+1+ηtJc,t+1−Zc,t+1;(9)UpdatingY3,t+1 by Y3,t+1=Y3,t+1+ηtJv,t+1−Zv,t+1;(10)Updating the parameterηt+1 by ηt+1=minηmax,ρηt;(11)Checking convergence bymaxX−XZc,t+1+Zv,t+1−Et+1∞,Jc,t+1−Zc,t+1∞,Jv,t+1−Zv,t+1∞<θ;(12)t=t+1.end whileOutput:Jc,Jv,Zc,Zv,E,P


### 3.4. Complexity Analysis

According to the above computational process and Algorithm1, we discuss the computational complexity of the proposed algorithm in detail. In Algorithm 1, the main factor of algorithm complexity depends on Steps 1–5. Equation (18) is a typical characteristic equation, which costs On3, where n is the number of the training samples. The SVD decomposition in Steps 2 and 3 takes about On3, but it can be reduced Orn2 due to the low-rank matrices Jc and Jv, where n is the rank of low-rank matrix. Equations (21) and (22) are two Sylvester equations whose computational complexity is On3. In summary, the computational complexity of our proposed method is On3.


## 3.1. Notations

Assume thatX1 and X2X=X1,X2∈Rd×n are two matrices of different views from same c classes, where n and d denote the number and dimensionality of all training samples, respectively. Class structure Zc∈Rn×n and view structure Zv∈Rn×n are two linear combination matrices, which are included in local graph framework to discover view-invariant structure of cross-view data. P∈Rd×p is a basis transformation projection matrix, where p is dimensionality of projected data. E∈Rd×n is a matrix of error data designed to obtain a robust subspace from noise. In addition, V1,V2,V^1,V^2∈Rn×c are four constant coefficient matrices utilized for aligning the global information of cross-view data.


## 3.2. Objective Function

To address the cross-view discriminative analysis, we formulate our subspace learning model with simultaneous local and global alignments as follows:(5)minZc,Zv,E,PZc∗+Zv∗+λ1E2,1︷DZc,Zv,E+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT︷ℒP,Zc,Zv+λ2TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv︷GP,Zc,Zvs.t.X=XZc+Zv+E,PTP=I,where DZc,Zv,E is a low-rank framework of two potential manifolds in cross-view data. ℒP,Zc,Zv enforces the view-specific discriminative local neighbor relationship among instances. GP,Zc,Zv performs the discriminative clustering and separation of the global structure in the class manifold and the view structure, respectively. In the following, the above terms are illustrated in detail.


### 3.2.1. Dual Low-Rank Representations

Only one linear combination matrix, which is constrained by the rank minimum, is used by methods based on low-rank model. Nevertheless, unitary low-rank structure gives rise to the failure of linear representation due to differences between distributions of views, which endow cross-view data from a same class with a large divergence. Therefore, the two structure matricesZc and Zv are adopted to solve this specific problem, where the between-view samples from the same class are far away and the within-view samples from the different classes are closer. The first term is defined with dual low-rank representations to strip down the class and view structures as follows:(6)DZc,Zv,E=Zc∗+Zv∗+λ1E2,1s.t.X=XZc+Zv+E,where ⋅∗ is a sign of the nuclear norm, which is close to a representation of the rank minimum problem, and its solution is relatively convenient. Assuming that part of the data from real world contains corruption, we adopt the l2,1-norm to make matrix E have the structured sparsity as the noisy data. l2,1-norm can effectively remove the corruption of specific sample while holding the other clean samples. λ1>0 is used to balance the corruption.


### 3.2.2. Graph-Based Discriminative Local Alignment

To introduce the local discriminative constraint, two graph-based constraints are constructed on each pair of synthetic samples withZc and Zv from class and view subspaces, respectively, as follows, which can better cluster intraclass samples and decentralize interclass ones.(7)ℒc=∑i,jYc,i−Yc,j2Wi,jc,ℒv=∑i,jYv,i−Yv,j2Wi,jv,where Yc,i and Yv,i denote the ith projected samples of cross-view data from the class space Yc=PTXZc and view space Yv=PTXZv, respectively. Correspondingly, Yc,j and Yv,j denote the jth projected samples. Wi,jc and Wi,jv denote graph weight matrices that are defined as follows:(8)Wi,jc=1,ifxi∈Nk1cxj,andli=lj,0,othervise,Wi,jv=1,ifxi∈Nk2vxj,butli≠lj,0,othervise,where li and lj are the labels of samples xi and xj, respectively. xi∈Nk1cxj denotes that xi belongs to the k1 adjacent datasets of the same sample xj. xi∈Nk2vxj means that xi belongs to the k2 adjacent datasets of the same view sample xj. Hence, ℒc can calculate the distance between two similar samples from same class. Similarly, ℒv denotes the distance between two similar samples from different classes. With the help of Fisher criterion, the pairwise local discriminative constraint ℒZc,Zv,P can be rewritten as follows:(9)ℒZc,Zv,P=αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT,where Lc and Lv denote the Laplacian operators of Wc and Wv. α is a balance parameter. Minimizing the subtraction of trace in equation (9) can weaken the impact of view information and separate the class structure and the view structure.


### 3.2.3. Discriminative Global Alignment

The discriminative projection of all pairwise samples throughℒZc,Zv,P can reduce the impact between views, but the differences between learned features from different classes are not significant enough. So, to further enhance the separation of two manifolds, we design a global discriminative constraint for cross-view analysis as the third term GP,Zc,Zv:(10)GP,Zc,Zv=TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv,where SWPTXZc is within-class and between-view scatter matrix in class manifold and SBiPTXZvi=1,2 is within-view and between-class scatter matrix in view manifold. These scatter matrices are formulated as(11)SWPTXZc=∑j=1cμj1−μj2μj1−μj2T,SBiPTXZv=∑j=1cμji−μiμji−μiT,where μii=1,2 denotes the overall mean feature from the ith view and μji denotes the mean feature of the jth class from the ith view. In this way, the within-class view margin in class structure can be reduced, and the margin of between-class data from same view in view structure can be magnified. The third term can be framed as(12)GP,Zc,Zv=λ2TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv=λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2,where Vi is the coefficient matrix of the within-class mean feature of the ith view and V^ii=1,2 is the coefficient matrix of the overall mean feature of the ith view. In detail, Vik,m=1/nim only if xk belongs to the mth class from the ith view, where nim denotes the number of samples of the mth class from the ith view; in other cases, Vik,m=0. V^ik,m=1/ni only if xk belongs to the ith view, where ni denotes the number of samples from the ith view; in other cases, V^ik,m=0. λ2 is a trade-off parameter. Equation (12) achieves global alignment by the mean vectors of joint synthetic samples from global representation and further enforces the view-invariant constraint on the same class.In addition, we add an orthogonal constraintPTP=I to neglect trivial solutions. In the end, we rewrite equation (5) with all the terms as(13)minZc,Zv,E,PZc∗+Zv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2s.t.X=XZc+Zv+E,PTP=I.


## 3.2.1. Dual Low-Rank Representations

Only one linear combination matrix, which is constrained by the rank minimum, is used by methods based on low-rank model. Nevertheless, unitary low-rank structure gives rise to the failure of linear representation due to differences between distributions of views, which endow cross-view data from a same class with a large divergence. Therefore, the two structure matricesZc and Zv are adopted to solve this specific problem, where the between-view samples from the same class are far away and the within-view samples from the different classes are closer. The first term is defined with dual low-rank representations to strip down the class and view structures as follows:(6)DZc,Zv,E=Zc∗+Zv∗+λ1E2,1s.t.X=XZc+Zv+E,where ⋅∗ is a sign of the nuclear norm, which is close to a representation of the rank minimum problem, and its solution is relatively convenient. Assuming that part of the data from real world contains corruption, we adopt the l2,1-norm to make matrix E have the structured sparsity as the noisy data. l2,1-norm can effectively remove the corruption of specific sample while holding the other clean samples. λ1>0 is used to balance the corruption.


## 3.2.2. Graph-Based Discriminative Local Alignment

To introduce the local discriminative constraint, two graph-based constraints are constructed on each pair of synthetic samples withZc and Zv from class and view subspaces, respectively, as follows, which can better cluster intraclass samples and decentralize interclass ones.(7)ℒc=∑i,jYc,i−Yc,j2Wi,jc,ℒv=∑i,jYv,i−Yv,j2Wi,jv,where Yc,i and Yv,i denote the ith projected samples of cross-view data from the class space Yc=PTXZc and view space Yv=PTXZv, respectively. Correspondingly, Yc,j and Yv,j denote the jth projected samples. Wi,jc and Wi,jv denote graph weight matrices that are defined as follows:(8)Wi,jc=1,ifxi∈Nk1cxj,andli=lj,0,othervise,Wi,jv=1,ifxi∈Nk2vxj,butli≠lj,0,othervise,where li and lj are the labels of samples xi and xj, respectively. xi∈Nk1cxj denotes that xi belongs to the k1 adjacent datasets of the same sample xj. xi∈Nk2vxj means that xi belongs to the k2 adjacent datasets of the same view sample xj. Hence, ℒc can calculate the distance between two similar samples from same class. Similarly, ℒv denotes the distance between two similar samples from different classes. With the help of Fisher criterion, the pairwise local discriminative constraint ℒZc,Zv,P can be rewritten as follows:(9)ℒZc,Zv,P=αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT,where Lc and Lv denote the Laplacian operators of Wc and Wv. α is a balance parameter. Minimizing the subtraction of trace in equation (9) can weaken the impact of view information and separate the class structure and the view structure.


## 3.2.3. Discriminative Global Alignment

The discriminative projection of all pairwise samples throughℒZc,Zv,P can reduce the impact between views, but the differences between learned features from different classes are not significant enough. So, to further enhance the separation of two manifolds, we design a global discriminative constraint for cross-view analysis as the third term GP,Zc,Zv:(10)GP,Zc,Zv=TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv,where SWPTXZc is within-class and between-view scatter matrix in class manifold and SBiPTXZvi=1,2 is within-view and between-class scatter matrix in view manifold. These scatter matrices are formulated as(11)SWPTXZc=∑j=1cμj1−μj2μj1−μj2T,SBiPTXZv=∑j=1cμji−μiμji−μiT,where μii=1,2 denotes the overall mean feature from the ith view and μji denotes the mean feature of the jth class from the ith view. In this way, the within-class view margin in class structure can be reduced, and the margin of between-class data from same view in view structure can be magnified. The third term can be framed as(12)GP,Zc,Zv=λ2TrSWPTXZc−TrSB1PTXZv−TrSB2PTXZv=λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2,where Vi is the coefficient matrix of the within-class mean feature of the ith view and V^ii=1,2 is the coefficient matrix of the overall mean feature of the ith view. In detail, Vik,m=1/nim only if xk belongs to the mth class from the ith view, where nim denotes the number of samples of the mth class from the ith view; in other cases, Vik,m=0. V^ik,m=1/ni only if xk belongs to the ith view, where ni denotes the number of samples from the ith view; in other cases, V^ik,m=0. λ2 is a trade-off parameter. Equation (12) achieves global alignment by the mean vectors of joint synthetic samples from global representation and further enforces the view-invariant constraint on the same class.In addition, we add an orthogonal constraintPTP=I to neglect trivial solutions. In the end, we rewrite equation (5) with all the terms as(13)minZc,Zv,E,PZc∗+Zv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2s.t.X=XZc+Zv+E,PTP=I.


## 3.3. Optimization Scheme

To obtain the feasible solution ofZc and Zv, we adopt two auxiliary variables Jc and Jv. Then, equation (13) can be transformed into the following term:(14)minZc,Zv,E,P,Jc,JvJc∗+Jv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2s.t.X=XZc+Zv+E,PTP=I,Jc=Zc,Jv=Zv.For optimization problems with equality constrains, the Augmented Lagrangian method is an effective solution. The Augmented Lagrangian form of equation (14) is as follows:(15)minZc,Zv,E,P,Jc,JvJc∗+Jv∗+λ1E2,1+αTrPTXZcLcPTXZcT−TrPTXZvLvPTXZvT+λ2PTXZcV1−V2F2−PTXZvV1−V^1F2−PTXZvV2−V^2F2+TrY1TX−XZc+Zv−E+TrY2TJc−Zc+TrY3TJv−Zv+η2X−XZc+Zv−EF2+Jc−ZcF2+Jv−ZvF2s.t.PTP=I,where Y1,Y2, and Y3 are the Lagrange multipliers and η>0 is the penalty parameter. We use an alternating solution to optimize iteratively all variables. We define the left bottom of the variable plus t as the t-th solution.First, by ignoring the other variables exceptP, equation (15) becomes(16)minPtαTrPtTXZc,tLcPtTXZc,tT−TrPtTXZv,tLvPtTXZv,tT+λ2PtTXZc,tV1−V2F2−PtTXZv,tV1−V^1F2−PtTXZv,tV2−V^2F2s.t.PTP=I.We obtain the projection matrixPt one by one, because Pt is an orthogonal matrix. For the ith column of Pt, the objective function is rewritten as(17)minPi,tαTrPi,tTXZc,tLcPi,tTXZc,tT−TrPi,tTXZv,tLvPi,tTXZv,tT+εi,tPi,tTPi,t−I+λ2Pi,tTXZc,tV1−V2F2−Pi,tTXZv,tV1−V^1F2−Pi,tTXZv,tV2−V^2F2.We enforce the derivative of function (17) to be zero.(18)XZv,tV1−V^1V1−V^1T+V2−V^2V2−V^2T+αLvZv,tT−Zc,tV1−V2V1−V2T+αLcZc,tTXTPi,t=εi,tPi,t.Therefore,Pi,t is the ith eigenvector of the matrix in equation (18) and P can be simply solved.UpdateJc:(19)Jc,t+1=minJc,tJc,t∗+TrY2TJc,t−Zc,t+ηt2Jc,t−Zc,tF2=1ηtJc,t∗+12Jc,t−Zc,t+Y2,tηtF2.The singular value thresholding is an approximate method to solve the above kernel norm minimization equations [37].UpdateJv:(20)Jv,t+1=minJv,tJv,t∗+TrY3TJv,t−Zv,t+ηt2Jv,t−Zv,tF2=1ηtJv,t∗+12Jv,t−Zv,t+Y3,tηtF2.Equation (20) can be addressed in the same way as equation (19).We enforce the derivative of equation (15) with respect to Zc to be zero.(21)XTPtPtTX−1I+XTXZc,t+1+Zc,t+1λ2V1−V2V1−V2T+αLcηt=XTPtPtTX−1XTX−XZv,t−Et+Jc,t+XTY1,t−Y2,tηt.It is obvious that equation (21) is a Sylvester equation. We can easily solve it by [38]. Similarly, we enforce the derivative of equation (15) with respect to Zv to be zero.(22)XTPtPtTX−1I+XTXZv,t+1+Zv,t+1λ2V1−V^1V1−V^1T+λ2V2−V^2V2−V^2T+αLvηt=XTPtPtTX−1XTX−XZc,t−Et+Jv,t+XTY1,t−Y3,tηt.Equation (22) can be addressed in the same way as equation (21).UpdateE:(23)Et+1=minEtλ1ηtEt2,1+12Et−X−XZc,t+Zv,t+Y1ηtF2.The above equation is al2,1-norm minimization problem whose solution is shown in [39].The entire numerical iterative scheme for equation (14) is shown in Algorithm 1, where the parameters ρ,θ,tmax,η, and ηmax are set empirically. Moreover, the matrices Zc,Zv,E,Y1,Y2, and Y3 are initialized as 0 and the parameters α,λ1, and λ2 are tuned by the experiments.Algorithm 1:Algorithm for our proposed framework.
Input: data matrix X, parameters α,λ1 and λ2Initialize:ρ=1.3, θ=10−9, t=0, tmax=200, η0=0.1 and ηmax=1010;while not converged ort≤tmaxdo(1)SolvingPt+1 by equation (18);(2)SolvingJc,t+1 by equation (19);(3)SolvingJv,t+1 by equation (20);(4)SolvingZc,t+1 by equation (21);(5)SolvingZv,t+1 by equation (22);(6)SolvingEt by equation (23);(7)UpdatingY1,t+1 by Y1,t+1=Y1,t+ηtX−XZc,t+1+Zv,t+1−E;(8)UpdatingY2,t+1 by Y2,t+1=Y2,t+1+ηtJc,t+1−Zc,t+1;(9)UpdatingY3,t+1 by Y3,t+1=Y3,t+1+ηtJv,t+1−Zv,t+1;(10)Updating the parameterηt+1 by ηt+1=minηmax,ρηt;(11)Checking convergence bymaxX−XZc,t+1+Zv,t+1−Et+1∞,Jc,t+1−Zc,t+1∞,Jv,t+1−Zv,t+1∞<θ;(12)t=t+1.end whileOutput:Jc,Jv,Zc,Zv,E,P


## 3.4. Complexity Analysis

According to the above computational process and Algorithm1, we discuss the computational complexity of the proposed algorithm in detail. In Algorithm 1, the main factor of algorithm complexity depends on Steps 1–5. Equation (18) is a typical characteristic equation, which costs On3, where n is the number of the training samples. The SVD decomposition in Steps 2 and 3 takes about On3, but it can be reduced Orn2 due to the low-rank matrices Jc and Jv, where n is the rank of low-rank matrix. Equations (21) and (22) are two Sylvester equations whose computational complexity is On3. In summary, the computational complexity of our proposed method is On3.


## 4. Experiments

In this section, we evaluate the performance of our proposed method with classification task. Firstly, we introduce four cross-view datasets: face database, object databases, image-text database, and experimental setup. Secondly, we adopt several excellent subspace learning algorithms for comparison with ours. The initializations of all unknown parameters are tuned to get the best experimental results. The analysis of parameters is shown in Figure2. In addition, each experiment is repeated 10 times and the average classification results are shown.Figure 2
Classification results of different experiments on 20% corrupted COIL-100.


### 4.1. Experimental Datasets

CMU-PIE Face dataset contains face images in 9 postures and 21 illumination conditions of 68 people. In our experiment, we chose 4 kinds of postures, which are P05, P09, P27, and P29. The original dimension of the cropped face images is 64×64. To enhance efficiency of our algorithm, the 300-dimensional principal feature extracted by PCA was adopted to our experiment.Wikipedia dataset, which is an image-text bimodal dataset, consists of 2866 pairwise samples from 10 classes. The dimensions of the image and text are 4096 and 100, respectively. Therefore, due to inconsistency of dimensionality of the two features, we use PCA to adjust the image dimension.COIL-20 object dataset is composed of 20 objects from a level 360-degree view. There is 5° between every two adjacent images, so each category has 72 samples. We divide the 72 images into two groups, G1 and G2. In addition, G1 is composed of samples from V1 [0°, 85°] and V2 [185°, 265°]. Similarly, G2 is composed of samples from V3 [90°, 175°] and V4 [270°, 355°].COIL-100 object dataset is an extension of the COIL-20. The only difference is that the COIL-100 is composed of 100 objects from a level 360-degree view. Therefore, the set of the COIL-100 dataset is similar to that of the COIL-20 dataset.


### 4.2. Experimental Results and Analysis

In experiments, we need not use any information about the test set, including class and view information. We select several subspace learning methods as comparison methods, that is, PCA, LDA, locality preserving projections (LPP) [40], LatLRR, SRRS, and RCVL. After extracting feature with the comparison methods, we uniformly choose KNN as classifier to evaluate their performance. In addition, we also add 10 percent and 20 percent of random noise to part of the datasets to demonstrate the adaptability of our subspace learning algorithm to different levels of corrupted data and some instances are shown in Figure 3.Figure 3
The instances of original data and corrupted data. (a) CMU-PIE face dataset. (b) COIL-20 object dataset.
(a)(b)For CMU-PIE, we randomly perform cross-view subspace learning on two poses, with a total of 6 experimental groups, which are C1{P05,P09}, C2{P05,P27}, C3{P05,P29}, C4{P09,P27}, C5{P09,P29}, and C6{P27,P29}. Tables1–3 respectively show the classification results of all experimental algorithms on the original data, 10% noisy data, and 20% noisy data. For COIL-20 and COIL-100 object datasets, we select two sets of samples from G1 and G2 as a cross-view training set, respectively, and the others as a test set. So we get 4 experimental groups from COIL-20 and COIL-100 datasets, including C1{V1,V3}, C2{V1,V4}, C3{V2,V4}, and C4{V2,V3}. Tables 4–6 display the classification results of all experimental algorithms on the original data, 10% noisy data, and 20% noisy data from COIL-20. Figures 2, 4, and 5 show the experimental results of four groups from original COIL-100 dataset, 10% corrupted COIL-100 dataset, and 20% corrupted COIL-100 dataset. For Wikipedia, we use the reduced dimensionality image feature and text feature as two views and Figure 6 displays the results of comparison experiments on the original data, 10% noisy data, and 20% noisy data.Table 1
Classification results (%) of all methods on original CMU-PIE dataset.
MethodsC1C2C3C4C5C6LDA62.96 ± 0.6666.76 ± 0.5862.16 ± 1.2061.50 ± 1.1456.54 ± 0.5561.83 ± 0.88PCA48.28 ± 0.5950.50 ± 0.7249.07 ± 0.9548.43 ± 1.1145.51 ± 0.8449.68 ± 0.43LatLRR65.10 ± 0.9766.61 ± 1.5762.47 ± 1.3463.09 ± 2.0861.04 ± 1.8760.42 ± 0.73LPP62.40 ± 0.8060.17 ± 0.2761.97 ± 0.3362.13 ± 0.4558.34 ± 0.1060.72 ± 0.28SRRS95.35 ± 1.0591.66 ± 1.8495.82 ± 1.3590.22 ± 0.2696.04 ± 1.0787.16 ± 0.55RCVL97.14 ± 0.0993.70 ± 0.5997.26 ± 0.0492.99 ± 0.1297.55 ± 0.0688.60 ± 0.03Ours98.27 ± 0.1193.47 ± 0.0698.15 ± 0.0891.69 ± 0.0898.53 ± 0.0290.27 ± 0.14Table 2
Classification results (%) of all methods on 10% corrupted CMU-PIE dataset.
MethodsC1C2C3C4C5C6LDA20.45 ± 0.0421.01 ± 0.0619.16 ± 0.1716.42 ± 0.1213.64 ± 0.1514.96 ± 0.05PCA48.09 ± 0.7446.90 ± 0.2547.74 ± 0.3147.99 ± 0.0846.90 ± 0.1046.63 ± 0.06LatLRR59.14 ± 1.8260.38 ± 1.1057.94 ± 0.8158.10 ± 1.2457.03 ± 0.5656.98 ± 0.96LPP38.47 ± 0.2734.97 ± 0.8037.85 ± 0.4738.06 ± 0.2433.16 ± 0.6335.77 ± 0.57SRRS69.54 ± 0.4166.91 ± 0.9670.02 ± 0.9668.89 ± 0.3170.28 ± 0.9765.64 ± 0.51RCVL87.14 ± 0.0478.10 ± 0.0785.52 ± 0.1277.75 ± 0.0786.87 ± 0.1076.09 ± 0.05Ours88.68 ± 0.0579.22 ± 0.0287.31 ± 0.0479.75 ± 0.1190.69 ± 0.0978.39 ± 0.04Table 3
Classification results (%) of all methods on 20% corrupted CMU-PIE dataset.
MethodsC1C2C3C4C5C6LDA9.23 ± 0.108.66 ± 0.078.57 ± 0.136.27 ± 0.116.82 ± 0.126.59 ± 0.10PCA25.98 ± 0.0625.99 ± 0.1526.58 ± 0.1421.97 ± 0.1022.01 ± 0.1920.15 ± 0.17LatLRR38.26 ± 1.0434.67 ± 1.1835.00 ± 0.8436.11 ± 1.1234.97 ± 0.7435.76 ± 0.78LPP34.25 ± 0.2230.08 ± 0.6729.89 ± 0.9333.27 ± 0.2530.95 ± 0.5131.07 ± 0.59SRRS74.30 ± 0.1463.02 ± 0.1972.79 ± 0.1559.21 ± 0.3668.73 ± 0.1754.98 ± 0.27RCVL70.63 ± 0.0861.39 ± 0.1671.44 ± 0.1057.34 ± 0.1665.02 ± 0.0953.05 ± 0.07Ours74.58 ± 0.0364.74 ± 0.1273.17 ± 0.0760.86 ± 0.0869.06 ± 0.0656.56 ± 0.10Table 4
Classification results (%) of all methods on original COIL-20 dataset.
MethodsC1C2C3C4LDA63.85 ± 1.8961.23 ± 1.0264.12 ± 2.0762.77 ± 1.73PCA65.3 ± 0.7663.42 ± 1.8866.83 ± 1.4570.31 ± 1.60LatLRR80.65 ± 1.5781.7 ± 2.1380.39 ± 1.0586.8 ± 1.29LPP76.06 ± 1.1774.6 ± 1.0477.53 ± 1.4578.99 ± 1.86SRRS85.56 ± 1.0583.89 ± 0.6383.47 ± 0.8494.58 ± 0.77RCVL83.22 ± 0.9779.04 ± 1.2581.44 ± 0.7590.23 ± 1.00Ours86.39 ± 0.5584.61 ± 0.9985.67 ± 0.2695.06 ± 0.57Table 5
Classification results (%) of all methods on 10% corrupted COIL-20 dataset.
MethodsC1C2C3C4LDA34.13 ± 1.1135.12 ± 0.8735.27 ± 1.2438.55 ± 1.47PCA49.44 ± 0.9751.69 ± 0.9448.31 ± 1.4950.81 ± 0.61LatLRR58.71 ± 1.5857.74 ± 1.4957.66 ± 1.2760.00 ± 1.51LPP57.63 ± 1.2854.84 ± 1.4258.25 ± 1.3158.62 ± 1.01SRRS80.72 ± 0.7074.36 ± 0.7975.50 ± 0.4189.06 ± 0.57RCVL78.20 ± 1.0771.92 ± 1.0273.67 ± 0.8087.13 ± 0.55Ours82.22 ± 0.9075.14 ± 0.6577.08 ± 0.3190.28 ± 0.93Table 6
Classification results (%) of all methods on 20% corrupted COIL-20 dataset.
MethodsC1C2C3C4LDA21.45 ± 1.2220.76 ± 1.5721.15 ± 1.0620.77 ± 1.39PCA43.49 ± 1.4644.16 ± 0.6443.98 ± 1.1045.03 ± 1.42LatLRR33.71 ± 1.1334.92 ± 0.8830.73 ± 1.2833.55 ± 1.36LPP35.13 ± 1.0431.26 ± 0.9732.72 ± 1.6034.16 ± 1.31SRRS75.64 ± 1.0266.72 ± 0.8470.56 ± 1.1781.14 ± 0.45RCVL73.20 ± 0.9464.96 ± 0.4866.03 ± 0.5679.02 ± 1.26Ours76.25 ± 0.5768.36 ± 0.2471.69 ± 1.1082.64 ± 1.16Figure 4
Classification results of different experiments on COIL-100.Figure 5
Classification results of different experiments on 10% corrupted COIL-100.Figure 6
Classification results of comparison experiments on original and corrupted Wikipedia.The results of experiments prove that our method achieves the persistent higher classification results compared to other methods. For noisy data, the classification results of most methods based on LRR are more robust than those of other methods. It is due to the fact that low-rank representation framework can restore raw information from corrupted data by learning latent structure. Besides, another result also can be found that the classification results of methods used for cross-view data are better than those of other comparisons. Our proposed method projects data into the discriminative view-invariant subspace via dual low-rank representations framework, so that the method can better learn from cross-view data.


### 4.3. Performance Evaluations

In this part, we test what parameters should we assign to ensure that the performance of our method can get a best grade. Then, we show the convergence of our algorithm.There are three tunable parametersα,λ1, and λ2 in our framework. We evaluate the effect of parameters on COIL-20 C1. α,λ2 are two parameters to adjust discriminative local alignment and discriminative global alignment. From Figures 7(a) and 7(b), it can be seen that our method gets the best result, when α=100 and λ2=0.01. Furthermore, λ1 is a parameter to constrain the corrupted data and the classification result is optimal around 1.Figure 7
Classification results with different parameters (a)α. (b)λ2. (c)λ1.
(a)(b)(c)In the end, we show the convergence analysis of our method on different datasets: the original, the 10% corrupted COIL-20 C1, and the CMU-PIE C1. The maximum value ofX−XZc,t+1+Zv,t+1−Et+1∞, Jc,t+1−Zc,t+1∞, and Jv,t+1−Zv,t+1∞is used as convergence criterion in each iteration. The variation of the maximum value with the increase of the number of iterations is shown in Figure 8. The curves point out that the proposed algorithm converges steadily and efficiently after 20 iterations.Figure 8
Convergence curves on different datasets.


## 4.1. Experimental Datasets

CMU-PIE Face dataset contains face images in 9 postures and 21 illumination conditions of 68 people. In our experiment, we chose 4 kinds of postures, which are P05, P09, P27, and P29. The original dimension of the cropped face images is 64×64. To enhance efficiency of our algorithm, the 300-dimensional principal feature extracted by PCA was adopted to our experiment.Wikipedia dataset, which is an image-text bimodal dataset, consists of 2866 pairwise samples from 10 classes. The dimensions of the image and text are 4096 and 100, respectively. Therefore, due to inconsistency of dimensionality of the two features, we use PCA to adjust the image dimension.COIL-20 object dataset is composed of 20 objects from a level 360-degree view. There is 5° between every two adjacent images, so each category has 72 samples. We divide the 72 images into two groups, G1 and G2. In addition, G1 is composed of samples from V1 [0°, 85°] and V2 [185°, 265°]. Similarly, G2 is composed of samples from V3 [90°, 175°] and V4 [270°, 355°].COIL-100 object dataset is an extension of the COIL-20. The only difference is that the COIL-100 is composed of 100 objects from a level 360-degree view. Therefore, the set of the COIL-100 dataset is similar to that of the COIL-20 dataset.


## 4.2. Experimental Results and Analysis

In experiments, we need not use any information about the test set, including class and view information. We select several subspace learning methods as comparison methods, that is, PCA, LDA, locality preserving projections (LPP) [40], LatLRR, SRRS, and RCVL. After extracting feature with the comparison methods, we uniformly choose KNN as classifier to evaluate their performance. In addition, we also add 10 percent and 20 percent of random noise to part of the datasets to demonstrate the adaptability of our subspace learning algorithm to different levels of corrupted data and some instances are shown in Figure 3.Figure 3
The instances of original data and corrupted data. (a) CMU-PIE face dataset. (b) COIL-20 object dataset.
(a)(b)For CMU-PIE, we randomly perform cross-view subspace learning on two poses, with a total of 6 experimental groups, which are C1{P05,P09}, C2{P05,P27}, C3{P05,P29}, C4{P09,P27}, C5{P09,P29}, and C6{P27,P29}. Tables1–3 respectively show the classification results of all experimental algorithms on the original data, 10% noisy data, and 20% noisy data. For COIL-20 and COIL-100 object datasets, we select two sets of samples from G1 and G2 as a cross-view training set, respectively, and the others as a test set. So we get 4 experimental groups from COIL-20 and COIL-100 datasets, including C1{V1,V3}, C2{V1,V4}, C3{V2,V4}, and C4{V2,V3}. Tables 4–6 display the classification results of all experimental algorithms on the original data, 10% noisy data, and 20% noisy data from COIL-20. Figures 2, 4, and 5 show the experimental results of four groups from original COIL-100 dataset, 10% corrupted COIL-100 dataset, and 20% corrupted COIL-100 dataset. For Wikipedia, we use the reduced dimensionality image feature and text feature as two views and Figure 6 displays the results of comparison experiments on the original data, 10% noisy data, and 20% noisy data.Table 1
Classification results (%) of all methods on original CMU-PIE dataset.
MethodsC1C2C3C4C5C6LDA62.96 ± 0.6666.76 ± 0.5862.16 ± 1.2061.50 ± 1.1456.54 ± 0.5561.83 ± 0.88PCA48.28 ± 0.5950.50 ± 0.7249.07 ± 0.9548.43 ± 1.1145.51 ± 0.8449.68 ± 0.43LatLRR65.10 ± 0.9766.61 ± 1.5762.47 ± 1.3463.09 ± 2.0861.04 ± 1.8760.42 ± 0.73LPP62.40 ± 0.8060.17 ± 0.2761.97 ± 0.3362.13 ± 0.4558.34 ± 0.1060.72 ± 0.28SRRS95.35 ± 1.0591.66 ± 1.8495.82 ± 1.3590.22 ± 0.2696.04 ± 1.0787.16 ± 0.55RCVL97.14 ± 0.0993.70 ± 0.5997.26 ± 0.0492.99 ± 0.1297.55 ± 0.0688.60 ± 0.03Ours98.27 ± 0.1193.47 ± 0.0698.15 ± 0.0891.69 ± 0.0898.53 ± 0.0290.27 ± 0.14Table 2
Classification results (%) of all methods on 10% corrupted CMU-PIE dataset.
MethodsC1C2C3C4C5C6LDA20.45 ± 0.0421.01 ± 0.0619.16 ± 0.1716.42 ± 0.1213.64 ± 0.1514.96 ± 0.05PCA48.09 ± 0.7446.90 ± 0.2547.74 ± 0.3147.99 ± 0.0846.90 ± 0.1046.63 ± 0.06LatLRR59.14 ± 1.8260.38 ± 1.1057.94 ± 0.8158.10 ± 1.2457.03 ± 0.5656.98 ± 0.96LPP38.47 ± 0.2734.97 ± 0.8037.85 ± 0.4738.06 ± 0.2433.16 ± 0.6335.77 ± 0.57SRRS69.54 ± 0.4166.91 ± 0.9670.02 ± 0.9668.89 ± 0.3170.28 ± 0.9765.64 ± 0.51RCVL87.14 ± 0.0478.10 ± 0.0785.52 ± 0.1277.75 ± 0.0786.87 ± 0.1076.09 ± 0.05Ours88.68 ± 0.0579.22 ± 0.0287.31 ± 0.0479.75 ± 0.1190.69 ± 0.0978.39 ± 0.04Table 3
Classification results (%) of all methods on 20% corrupted CMU-PIE dataset.
MethodsC1C2C3C4C5C6LDA9.23 ± 0.108.66 ± 0.078.57 ± 0.136.27 ± 0.116.82 ± 0.126.59 ± 0.10PCA25.98 ± 0.0625.99 ± 0.1526.58 ± 0.1421.97 ± 0.1022.01 ± 0.1920.15 ± 0.17LatLRR38.26 ± 1.0434.67 ± 1.1835.00 ± 0.8436.11 ± 1.1234.97 ± 0.7435.76 ± 0.78LPP34.25 ± 0.2230.08 ± 0.6729.89 ± 0.9333.27 ± 0.2530.95 ± 0.5131.07 ± 0.59SRRS74.30 ± 0.1463.02 ± 0.1972.79 ± 0.1559.21 ± 0.3668.73 ± 0.1754.98 ± 0.27RCVL70.63 ± 0.0861.39 ± 0.1671.44 ± 0.1057.34 ± 0.1665.02 ± 0.0953.05 ± 0.07Ours74.58 ± 0.0364.74 ± 0.1273.17 ± 0.0760.86 ± 0.0869.06 ± 0.0656.56 ± 0.10Table 4
Classification results (%) of all methods on original COIL-20 dataset.
MethodsC1C2C3C4LDA63.85 ± 1.8961.23 ± 1.0264.12 ± 2.0762.77 ± 1.73PCA65.3 ± 0.7663.42 ± 1.8866.83 ± 1.4570.31 ± 1.60LatLRR80.65 ± 1.5781.7 ± 2.1380.39 ± 1.0586.8 ± 1.29LPP76.06 ± 1.1774.6 ± 1.0477.53 ± 1.4578.99 ± 1.86SRRS85.56 ± 1.0583.89 ± 0.6383.47 ± 0.8494.58 ± 0.77RCVL83.22 ± 0.9779.04 ± 1.2581.44 ± 0.7590.23 ± 1.00Ours86.39 ± 0.5584.61 ± 0.9985.67 ± 0.2695.06 ± 0.57Table 5
Classification results (%) of all methods on 10% corrupted COIL-20 dataset.
MethodsC1C2C3C4LDA34.13 ± 1.1135.12 ± 0.8735.27 ± 1.2438.55 ± 1.47PCA49.44 ± 0.9751.69 ± 0.9448.31 ± 1.4950.81 ± 0.61LatLRR58.71 ± 1.5857.74 ± 1.4957.66 ± 1.2760.00 ± 1.51LPP57.63 ± 1.2854.84 ± 1.4258.25 ± 1.3158.62 ± 1.01SRRS80.72 ± 0.7074.36 ± 0.7975.50 ± 0.4189.06 ± 0.57RCVL78.20 ± 1.0771.92 ± 1.0273.67 ± 0.8087.13 ± 0.55Ours82.22 ± 0.9075.14 ± 0.6577.08 ± 0.3190.28 ± 0.93Table 6
Classification results (%) of all methods on 20% corrupted COIL-20 dataset.
MethodsC1C2C3C4LDA21.45 ± 1.2220.76 ± 1.5721.15 ± 1.0620.77 ± 1.39PCA43.49 ± 1.4644.16 ± 0.6443.98 ± 1.1045.03 ± 1.42LatLRR33.71 ± 1.1334.92 ± 0.8830.73 ± 1.2833.55 ± 1.36LPP35.13 ± 1.0431.26 ± 0.9732.72 ± 1.6034.16 ± 1.31SRRS75.64 ± 1.0266.72 ± 0.8470.56 ± 1.1781.14 ± 0.45RCVL73.20 ± 0.9464.96 ± 0.4866.03 ± 0.5679.02 ± 1.26Ours76.25 ± 0.5768.36 ± 0.2471.69 ± 1.1082.64 ± 1.16Figure 4
Classification results of different experiments on COIL-100.Figure 5
Classification results of different experiments on 10% corrupted COIL-100.Figure 6
Classification results of comparison experiments on original and corrupted Wikipedia.The results of experiments prove that our method achieves the persistent higher classification results compared to other methods. For noisy data, the classification results of most methods based on LRR are more robust than those of other methods. It is due to the fact that low-rank representation framework can restore raw information from corrupted data by learning latent structure. Besides, another result also can be found that the classification results of methods used for cross-view data are better than those of other comparisons. Our proposed method projects data into the discriminative view-invariant subspace via dual low-rank representations framework, so that the method can better learn from cross-view data.


## 4.3. Performance Evaluations

In this part, we test what parameters should we assign to ensure that the performance of our method can get a best grade. Then, we show the convergence of our algorithm.There are three tunable parametersα,λ1, and λ2 in our framework. We evaluate the effect of parameters on COIL-20 C1. α,λ2 are two parameters to adjust discriminative local alignment and discriminative global alignment. From Figures 7(a) and 7(b), it can be seen that our method gets the best result, when α=100 and λ2=0.01. Furthermore, λ1 is a parameter to constrain the corrupted data and the classification result is optimal around 1.Figure 7
Classification results with different parameters (a)α. (b)λ2. (c)λ1.
(a)(b)(c)In the end, we show the convergence analysis of our method on different datasets: the original, the 10% corrupted COIL-20 C1, and the CMU-PIE C1. The maximum value ofX−XZc,t+1+Zv,t+1−Et+1∞, Jc,t+1−Zc,t+1∞, and Jv,t+1−Zv,t+1∞is used as convergence criterion in each iteration. The variation of the maximum value with the increase of the number of iterations is shown in Figure 8. The curves point out that the proposed algorithm converges steadily and efficiently after 20 iterations.Figure 8
Convergence curves on different datasets.


## 5. Conclusions

We proposed a subspace learning algorithm with discriminative constraint via low-rank representation to solve the cross-view recognition task. Our method can learn a distribution-invariant subspace from cross-view data by designing two substantial structures with dual low-rank constraints. We also integrate the local alignment and the global alignment into our framework to eliminate the interference caused by the view manifold in the subspace. Meanwhile, we also design a feasible iterative scheme to ensure that the model converges and obtains the optimal solution. Extensive experiments on several public datasets prove that our proposed method has strong robustness and stability for cross-view classification tasks.



---

*Source: 8872348-2020-11-04.xml*