# A New Feature Extraction Algorithm Based on Orthogonal Regularized Kernel CCA and Its Application

Authors: Xinchen Guo; Xiuling Fan; Xiantian Xi; Fugeng Zeng
Journal: Journal of Electrical and Computer Engineering
 (2018)

---

## Abstract

In this paper, an orthogonal regularized kernel canonical correlation analysis algorithm (ORKCCA) is proposed. ORCCA algorithm can deal with the linear relationships between two groups of random variables. But if the linear relationships between two groups of random variables do not exist, the performance of ORCCA algorithm will not work well. Linear orthogonal regularized CCA algorithm is extended to nonlinear space by introducing the kernel method into CCA. Simulation experimental results on both artificial and handwritten numerals databases show that the proposed method outperforms ORCCA for the nonlinear problems.


---

## Body

## 1. Introduction

Canonical correlation analysis (CCA) is a technique of multivariate statistical analysis, which deals with the mutual relationships of two sets of variables [1–3]. This method extracts the representative variables which are the linear combination of the variables in each group. The relationships between new variables can reflect the overall relationships between two groups of variables [4].The orthogonal regularization canonical correlation analysis (ORCCA) algorithm [5] is that the original formula of CCA algorithm with orthogonal constraints is substituted for CCA conjugate orthogonalization [6, 7]. When the number of samples is less and the sample distribution patterns of different classifications are different, the ORCCA algorithm has the better ability of classification. A suboptimal solution to eigenvalue decomposition problem can be obtained by introducing two regularization parameters [8]. So, the complexity of time and space for the quadratic optimization problem should be considered at the same time. ORCCA algorithm is the same as CCA algorithm that both their goals look for the linear combinations of the variables in each group. But when the nonlinear relationships between the variables exist, ORCCA algorithm cannot extract effectively the comprehensive variables.In this paper, the kernel method [9–11] is introduced into ORCCA algorithm, and ORKCCA algorithm is presented. The kernel method maps the linear inseparable data in the low-dimensional space into a higher-dimensional space [12, 13]. In the higher-dimensional space, the characteristics of the data can be extracted and analyzed through the linear method. By introducing kernel function, the computation of the orthogonal regularization canonical correlation analysis extends to a nonlinear feature space. Experimental results show that the accuracies of classification of our method in the nonlinear space are significantly improved. The experimental results show ORKCCA is feasible.


## 2. Orthogonal Regularized CCA Algorithm

Givenn pairs of pairwise samples X=x1,x2,…,xnT and Y=y1,y2,…,ynT, where xi∈Rp, yi∈Rqi=1,2,…,n. We assume that the samples have been centered. ORCCA algorithm aims at finding a pair of projection directions a and b which satisfy the following optimal problem [5].(1)mina,b1n∑i=1naTxi−bTyi2,s.t.aTa=1,bTb=1.The objective function in Equations (1) can be expanded as follows:(2)1n∑i=1naTxi−bTyi2=aTSxxa+bTSyyb−2aTSxyb,where Sxx=1/nXTX, Syy=1/nYTY, and Sxy=1/nXTY.The optimal model in Equation (1) can be rewritten as(3)maxa,b2aTSxyb−aTSxxa−bTSyyb,s.t.aTa=1,bTb=1.According to the Lagrange multipliers method, Lagrange function is as follows:(4)La,b=2aTSxyb−aTSxxa−bTSyyb−λ1aTa−1−λ2bTb−1,where both λ1 and λ2 are Lagrange multipliers.The solutions to Equation (4) are given as follows:(5)SxySyy+λ2Iq−1SxyT−Sxxa=λ1a,(6)SxyTSxx+λ1Ip−1Sxy−Syyb=λ2b,where Ip and Iq denote identity matrices of size p ∗ p and q ∗ q, respectively.Bothλ1 and λ2 in Equations (5) and (6) are called regularization parameters. By solving Equation (5), the eigenvalues λ11,λ12,…,λ1p and their corresponding eigenvectors a1,a2,…,ap can be obtained. The eigenvalues λ21,λ22,…,λ2q and their corresponding eigenvectors b1,b2,…,bq can be obtained from Equation (6).


## 3. Orthogonal Regularized Kernel CCA Algorithm (ORKCCA)

ORCCA algorithm can give the linear relationships between two groups of random variables. But if the linear relationships between two groups of random variables do not exist, the performance of ORCCA will not work well. The kernel method is an effective way to analyze the nonlinear pattern problem. So, the kernel method is introduced into ORCCA algorithm, and ORKCCA algorithm is proposed.BothΦx and Φy are nonlinear mappings which map original random variables xi and yi into Φxxi and Φyyi in P-dimensional space Fx(P > p) and Q-dimensional space Fy(Q > q), i=1,2,…,n. Let a=ΦxXTα, b=ΦyYTβ, where ΦxX∈Rn×P, ΦyY∈Rn×Q α,β∈Rn.ORCCA is implemented in higher-dimensional spacesFx and Fy. So, Equation (7) can be obtained by substituting a, b, Φxxi, and Φyyi into Equation (1) as follows:(7)minα,β1n∑i=1nαTΦxXΦxxi−βTΦyYΦyyi2,s.t.αTΦxXΦxXTα=1,βTΦyYΦyYTβ=1.Expanding the objective function in Equation (7), we get(8)1n∑i=1nαTΦxXΦxxi−βTΦyYΦyyi2=αT1nΦxXΦxXTΦxXΦxXTα+βTΦyYΦyYTΦyYΦyYTβ−2αT1nΦxXΦxXTΦyYΦyYTβ.Applying the kernel trick to Equation (8), Kx and Ky∈Rn×n can be computed, namely, Kx=ΦxXΦxXT=ΦxxiTΦxxjn×n=kxi,xjn×n Ky=ΦyYΦyYT=ΦyyiTΦyyjn×n=kyi,yjn×n, where k⋅,⋅ is kernel function. Centralization is exerted on Kx and Ky. The optimal model in which the kernel method is introduced can be given by using Equation (9):(9)maxα,β2αTMxyβ−αTMxxα−βTMyyβ,s.t.αTKxα=1,βTKyβ=1,where Mxy=1/nKxTKy, Mxx=1/nKxTKx, and Myy=1/nKyTKy.According to the Lagrange multiplier method, the Lagrange function is as follows(10)L′α,β=2αTMxyβ−αTMxxα−βTMyyβ−ζ1αTKxα−1−ζ2βTKyβ−1,where ζ1 and ζ2 are Lagrange multipliers. Taking the partial derivatives of L′α,β with respect to α and β and letting them zero, we get(11)∂L′∂α=2Mxyβ−Mxxα−ζ1α=0,∂L′∂β=2MxyTα−Myyβ−ζ2β=0,where Mxx and Myy are positive semidefinite matrices and ζ1 and ζ2 are positive numbers.So,α and β can be obtained from Equation (11):(12)α=Mxx+ζ1IP−1Mxyβ,(13)β=Myy+ζ2IQ−1MxyTα,where IP and IQ are the identity matrices of size P ∗ P and Q ∗ Q, respectively.Equations (14) and (15) can be obtained through replacing α and β with their expressions in Equations (12) and (13), respectively.(14)MxyMyy+ζ2IQ−1MxyT−Mxxα=ζ1α,(15)MxyTMxx+ζ1IP−1Mxy−Myyβ=ζ2β.As like before, bothλ1 and λ2 in Equations (14) and (15) are called regularization parameters. By solving Equation (14), the eigenvalues ζ11,ζ12,…,ζ1n and their corresponding eigenvectors α1,α2,…,αP can be obtained. The eigenvalues ζ21,ζ22,…,ζ2Q and their corresponding eigenvectors β1,β2,…,βQ can be obtained from Equation (15).


## 4. Simulation Experiments

In this section, we evaluate our method compared with ORCCA on artificial and handwritten numerals databases.


### 4.1. Experiment on Artifical Databases

The pairwise samplesX and Y are generated from the expressions in Equations (16) and (17), respectively.(16)x=θcosθ2sin3θsinθcosθ+ε1,(17)y=e2θcosθ2sinθ3+ε2,where θ obeys uniform distribution on −π,π and ε1 and ε2 are Gaussian noise with standard deviation 0.05. The radial basis function kx,y=exp−x−y2/2σ2 is chosen as kernel function, where σ=1.0.


#### 4.1.1. Determining Regularization Parameters

For the selection of the regularization parameters, by far there is no reliable method to determine the optimal values. In this paper, in order to simplify the calculation, letλ=λ1=λ2 and ζ=ζ1=ζ2. The regularization parameters were chosen from 10−5, 10−4, 10−3, 10−2, 10−1, and 1. This method is used in the literature [5].According to Equations (16) and (17), 100 pairs of data are randomly generated as the training samples. Canonical variables are calculated from the ORCCA and ORKCCA algorithms for the different values of regularization parameters. The correlation coefficients of canonical variables are sorted by the descending order. Many pairs of canonical variables can be gained from the two algorithms. For the sake of simplicity, the most representative of the former two groups of canonical variables are examined.The average value of the correlation coefficients of the former two groups of canonical variables is regarded as criterion that judges the regularization parameters is good or not. The larger the average value is, the better the regularization parameters are.Table1 lists the average value of the correlation coefficients of the former two groups of canonical variables for the different values of the regularization parameters.Table 1
The mean values of the correlation coefficients of the former two groups of canonical variables for the different values of the regularization parameters from ORCCA and ORKCCA.
Regularization parameters
Mean values of the correlation coefficients
ORCCA
ORKCCA
10−5
0.74
0.80
10−4
0.77
0.84
10−3
0.82
0.89
10−2
0.81
0.92
10−1
0.79
0.93
1
0.80
0.92Table1 shows that the optimal values of the regularization parameters for the ORCCA and ORKCCA algorithms are 10−3 and 10−1, respectively. The optimal regularization parameters are used to perform simulations in the next section.


#### 4.1.2. Simulation Experiment 1

According to Equations (16) and (17), 200 pairs of data are randomly generated as the test samples. For the regularization parameters λ=10−3 and ζ=10−1 in the ORCCA and ORKCCA algorithms, the canonical variables are obtained for test samples, respectively. The correlation coefficients of the canonical variables are sorted in the descending order.Tables2 and 3 list the correlation coefficients of the first two groups of canonical variables for ORCCA and ORKCCA algorithms. u1 and v1 denote the first group of canonical variables. u2 and v2 are the second group of canonical variables.Table 2
The correlation coefficients of the first two groups of canonical variables for ORCCA.
v
1
v
2
u
1
0.58
0.14
u
2
0.21
0.36Table 3
The correlation coefficients of the first two groups of canonical variables for ORkCCA.
v
1
v
2
u
1
0.91
0.08
u
2
0.06
0.83The experimental results in Tables2 and 3 show that the correlationships between the same pair of the canonical variables are better than that between the different pairs of canonical variables, especially for nonlinear data.


#### 4.1.3. Simulation Experiment 2

According to Equations (16) and (17), 5 pairs of data are randomly generated as the sample data. Each pair of sample data represents the center data of each class. 100 pairs of data for each class are given by adding Gaussian noise with standard deviation of 0.05 to each class center data. So we have five class data, which contains 100 samples for each class.100, 175, and 250 pairs of data are chosen from the 500 pairs of the whole data as the training samples, respectively. The rest 400, 325, and 250 pairs of data are the test samples, respectively. The classification experiments based on K-neighbors algorithm are carried out on the test samples data which are preprocessed in the above way. And, the accuracies of classification are given. For the test samples with 400, 325, and 250 pairs of data, the experiments are performed 15 times, respectively. The accuracies of classification for 400, 325, and 250 pairs of data are the averages of the accuracies of classification for the 15 experiments results, respectively. Table4 gives the accuracies of classification for ORCCA and ORKCCA for the test samples with the different number.Table 4
Comparisons of the accuracies of classification for ORCCA and ORKCCA.
Numbers of the training samples
ORCCA (%)
ORKCCA (%)
100
65.0
73.1
175
72.4
77.8
250
76.5
83.6In Table4, the first column is the numbers of the training samples and the second column and the third column are the accuracies of classification for ORCCA and ORKCCA for the training samples with the different number. The experimental results show that the accuracies of classification for ORKCCA are higher than those for ORCCA. So, the performance of ORKCCA outperforms that of ORCCA for the nonlinear problem. The comparison curves of the accuracies of classification for ORCCA and ORKCCA are given in Figure 1.Figure 1
Comparison curves of the accuracies of classification for ORCCA and ORKCCA.


### 4.2. Experiments on Handwritten Numerals Databases

The Concordia University CENPARMI database of handwritten Arabic numerals have 10 classes, that is, 10 digits (from 0 to 9), and 600 samples for each. The first 400 samples are used as the training set, and the remaining samples as the test set in each class. Then, the training samples and the test samples are 4000 and 2000, respectively. The handwritten digital images are preprocessed by the method given in [14]. Four kinds of features are extracted as follows: XG (256-dimensional Gabor transformation feature), XL (121-dimensional Legendre moment feature), XP (36-dimensional Pseudo-Zernike moment feature), and XZ (30-dimensional Zernike moment feature).For the choice of the regularization parameters, letλ=λ1=λ2 and ζ=ζ1=ζ2. The regularization parameters were chosen from 10−5, 10−3, and 1. The results of our method are compared with the results of ORCCA in order to verify the effectiveness of ORKCCA. Table 5 lists the accuracies of classification for ORCCA and ORKCCA in different feature combinations and regularization parameters. Experimental results show that (1) the classification effect of the two methods is the best as the regularization parameter is 1; (2) the classification accuracies of ORKCCA are higher than that of ORCCA for different features combinations; (3) the classification accuracies of ORKCCA in the regularization parameters 10−5 and 10−3 are higher than those of ORCCA in the regularization parameters 1.Table 5
Comparisons of the accuracies of classification for ORCCA and ORKCCA in different feature combinations and regularization parameters.
Feature combinations
ORCCA
ORKCCA
λ = 10−5
λ = 10−3
λ = 1
ζ = 10−5
ζ = 10−3
ζ = 1
X
G-XL
0.9314
0.9300
0.9375
0.9625
0.9681
0.9687
X
G-XP
0.9230
0.9228
0.9248
0.9511
0.9525
0.9536
X
G-XZ
0.9180
0.9196
0.9196
0.9482
0.9518
0.9520
X
L-XP
0.9187
0.9187
0.9190
0.9500
0.9533
0.9545
X
L-XZ
0.9200
0.9205
0.9235
0.9574
0.9600
0.9615
X
P-XZ
0.7413
0.7413
0.7525
0.8436
0.8450
0.8450


## 4.1. Experiment on Artifical Databases

The pairwise samplesX and Y are generated from the expressions in Equations (16) and (17), respectively.(16)x=θcosθ2sin3θsinθcosθ+ε1,(17)y=e2θcosθ2sinθ3+ε2,where θ obeys uniform distribution on −π,π and ε1 and ε2 are Gaussian noise with standard deviation 0.05. The radial basis function kx,y=exp−x−y2/2σ2 is chosen as kernel function, where σ=1.0.


### 4.1.1. Determining Regularization Parameters

For the selection of the regularization parameters, by far there is no reliable method to determine the optimal values. In this paper, in order to simplify the calculation, letλ=λ1=λ2 and ζ=ζ1=ζ2. The regularization parameters were chosen from 10−5, 10−4, 10−3, 10−2, 10−1, and 1. This method is used in the literature [5].According to Equations (16) and (17), 100 pairs of data are randomly generated as the training samples. Canonical variables are calculated from the ORCCA and ORKCCA algorithms for the different values of regularization parameters. The correlation coefficients of canonical variables are sorted by the descending order. Many pairs of canonical variables can be gained from the two algorithms. For the sake of simplicity, the most representative of the former two groups of canonical variables are examined.The average value of the correlation coefficients of the former two groups of canonical variables is regarded as criterion that judges the regularization parameters is good or not. The larger the average value is, the better the regularization parameters are.Table1 lists the average value of the correlation coefficients of the former two groups of canonical variables for the different values of the regularization parameters.Table 1
The mean values of the correlation coefficients of the former two groups of canonical variables for the different values of the regularization parameters from ORCCA and ORKCCA.
Regularization parameters
Mean values of the correlation coefficients
ORCCA
ORKCCA
10−5
0.74
0.80
10−4
0.77
0.84
10−3
0.82
0.89
10−2
0.81
0.92
10−1
0.79
0.93
1
0.80
0.92Table1 shows that the optimal values of the regularization parameters for the ORCCA and ORKCCA algorithms are 10−3 and 10−1, respectively. The optimal regularization parameters are used to perform simulations in the next section.


### 4.1.2. Simulation Experiment 1

According to Equations (16) and (17), 200 pairs of data are randomly generated as the test samples. For the regularization parameters λ=10−3 and ζ=10−1 in the ORCCA and ORKCCA algorithms, the canonical variables are obtained for test samples, respectively. The correlation coefficients of the canonical variables are sorted in the descending order.Tables2 and 3 list the correlation coefficients of the first two groups of canonical variables for ORCCA and ORKCCA algorithms. u1 and v1 denote the first group of canonical variables. u2 and v2 are the second group of canonical variables.Table 2
The correlation coefficients of the first two groups of canonical variables for ORCCA.
v
1
v
2
u
1
0.58
0.14
u
2
0.21
0.36Table 3
The correlation coefficients of the first two groups of canonical variables for ORkCCA.
v
1
v
2
u
1
0.91
0.08
u
2
0.06
0.83The experimental results in Tables2 and 3 show that the correlationships between the same pair of the canonical variables are better than that between the different pairs of canonical variables, especially for nonlinear data.


### 4.1.3. Simulation Experiment 2

According to Equations (16) and (17), 5 pairs of data are randomly generated as the sample data. Each pair of sample data represents the center data of each class. 100 pairs of data for each class are given by adding Gaussian noise with standard deviation of 0.05 to each class center data. So we have five class data, which contains 100 samples for each class.100, 175, and 250 pairs of data are chosen from the 500 pairs of the whole data as the training samples, respectively. The rest 400, 325, and 250 pairs of data are the test samples, respectively. The classification experiments based on K-neighbors algorithm are carried out on the test samples data which are preprocessed in the above way. And, the accuracies of classification are given. For the test samples with 400, 325, and 250 pairs of data, the experiments are performed 15 times, respectively. The accuracies of classification for 400, 325, and 250 pairs of data are the averages of the accuracies of classification for the 15 experiments results, respectively. Table4 gives the accuracies of classification for ORCCA and ORKCCA for the test samples with the different number.Table 4
Comparisons of the accuracies of classification for ORCCA and ORKCCA.
Numbers of the training samples
ORCCA (%)
ORKCCA (%)
100
65.0
73.1
175
72.4
77.8
250
76.5
83.6In Table4, the first column is the numbers of the training samples and the second column and the third column are the accuracies of classification for ORCCA and ORKCCA for the training samples with the different number. The experimental results show that the accuracies of classification for ORKCCA are higher than those for ORCCA. So, the performance of ORKCCA outperforms that of ORCCA for the nonlinear problem. The comparison curves of the accuracies of classification for ORCCA and ORKCCA are given in Figure 1.Figure 1
Comparison curves of the accuracies of classification for ORCCA and ORKCCA.


## 4.1.1. Determining Regularization Parameters

For the selection of the regularization parameters, by far there is no reliable method to determine the optimal values. In this paper, in order to simplify the calculation, letλ=λ1=λ2 and ζ=ζ1=ζ2. The regularization parameters were chosen from 10−5, 10−4, 10−3, 10−2, 10−1, and 1. This method is used in the literature [5].According to Equations (16) and (17), 100 pairs of data are randomly generated as the training samples. Canonical variables are calculated from the ORCCA and ORKCCA algorithms for the different values of regularization parameters. The correlation coefficients of canonical variables are sorted by the descending order. Many pairs of canonical variables can be gained from the two algorithms. For the sake of simplicity, the most representative of the former two groups of canonical variables are examined.The average value of the correlation coefficients of the former two groups of canonical variables is regarded as criterion that judges the regularization parameters is good or not. The larger the average value is, the better the regularization parameters are.Table1 lists the average value of the correlation coefficients of the former two groups of canonical variables for the different values of the regularization parameters.Table 1
The mean values of the correlation coefficients of the former two groups of canonical variables for the different values of the regularization parameters from ORCCA and ORKCCA.
Regularization parameters
Mean values of the correlation coefficients
ORCCA
ORKCCA
10−5
0.74
0.80
10−4
0.77
0.84
10−3
0.82
0.89
10−2
0.81
0.92
10−1
0.79
0.93
1
0.80
0.92Table1 shows that the optimal values of the regularization parameters for the ORCCA and ORKCCA algorithms are 10−3 and 10−1, respectively. The optimal regularization parameters are used to perform simulations in the next section.


## 4.1.2. Simulation Experiment 1

According to Equations (16) and (17), 200 pairs of data are randomly generated as the test samples. For the regularization parameters λ=10−3 and ζ=10−1 in the ORCCA and ORKCCA algorithms, the canonical variables are obtained for test samples, respectively. The correlation coefficients of the canonical variables are sorted in the descending order.Tables2 and 3 list the correlation coefficients of the first two groups of canonical variables for ORCCA and ORKCCA algorithms. u1 and v1 denote the first group of canonical variables. u2 and v2 are the second group of canonical variables.Table 2
The correlation coefficients of the first two groups of canonical variables for ORCCA.
v
1
v
2
u
1
0.58
0.14
u
2
0.21
0.36Table 3
The correlation coefficients of the first two groups of canonical variables for ORkCCA.
v
1
v
2
u
1
0.91
0.08
u
2
0.06
0.83The experimental results in Tables2 and 3 show that the correlationships between the same pair of the canonical variables are better than that between the different pairs of canonical variables, especially for nonlinear data.


## 4.1.3. Simulation Experiment 2

According to Equations (16) and (17), 5 pairs of data are randomly generated as the sample data. Each pair of sample data represents the center data of each class. 100 pairs of data for each class are given by adding Gaussian noise with standard deviation of 0.05 to each class center data. So we have five class data, which contains 100 samples for each class.100, 175, and 250 pairs of data are chosen from the 500 pairs of the whole data as the training samples, respectively. The rest 400, 325, and 250 pairs of data are the test samples, respectively. The classification experiments based on K-neighbors algorithm are carried out on the test samples data which are preprocessed in the above way. And, the accuracies of classification are given. For the test samples with 400, 325, and 250 pairs of data, the experiments are performed 15 times, respectively. The accuracies of classification for 400, 325, and 250 pairs of data are the averages of the accuracies of classification for the 15 experiments results, respectively. Table4 gives the accuracies of classification for ORCCA and ORKCCA for the test samples with the different number.Table 4
Comparisons of the accuracies of classification for ORCCA and ORKCCA.
Numbers of the training samples
ORCCA (%)
ORKCCA (%)
100
65.0
73.1
175
72.4
77.8
250
76.5
83.6In Table4, the first column is the numbers of the training samples and the second column and the third column are the accuracies of classification for ORCCA and ORKCCA for the training samples with the different number. The experimental results show that the accuracies of classification for ORKCCA are higher than those for ORCCA. So, the performance of ORKCCA outperforms that of ORCCA for the nonlinear problem. The comparison curves of the accuracies of classification for ORCCA and ORKCCA are given in Figure 1.Figure 1
Comparison curves of the accuracies of classification for ORCCA and ORKCCA.


## 4.2. Experiments on Handwritten Numerals Databases

The Concordia University CENPARMI database of handwritten Arabic numerals have 10 classes, that is, 10 digits (from 0 to 9), and 600 samples for each. The first 400 samples are used as the training set, and the remaining samples as the test set in each class. Then, the training samples and the test samples are 4000 and 2000, respectively. The handwritten digital images are preprocessed by the method given in [14]. Four kinds of features are extracted as follows: XG (256-dimensional Gabor transformation feature), XL (121-dimensional Legendre moment feature), XP (36-dimensional Pseudo-Zernike moment feature), and XZ (30-dimensional Zernike moment feature).For the choice of the regularization parameters, letλ=λ1=λ2 and ζ=ζ1=ζ2. The regularization parameters were chosen from 10−5, 10−3, and 1. The results of our method are compared with the results of ORCCA in order to verify the effectiveness of ORKCCA. Table 5 lists the accuracies of classification for ORCCA and ORKCCA in different feature combinations and regularization parameters. Experimental results show that (1) the classification effect of the two methods is the best as the regularization parameter is 1; (2) the classification accuracies of ORKCCA are higher than that of ORCCA for different features combinations; (3) the classification accuracies of ORKCCA in the regularization parameters 10−5 and 10−3 are higher than those of ORCCA in the regularization parameters 1.Table 5
Comparisons of the accuracies of classification for ORCCA and ORKCCA in different feature combinations and regularization parameters.
Feature combinations
ORCCA
ORKCCA
λ = 10−5
λ = 10−3
λ = 1
ζ = 10−5
ζ = 10−3
ζ = 1
X
G-XL
0.9314
0.9300
0.9375
0.9625
0.9681
0.9687
X
G-XP
0.9230
0.9228
0.9248
0.9511
0.9525
0.9536
X
G-XZ
0.9180
0.9196
0.9196
0.9482
0.9518
0.9520
X
L-XP
0.9187
0.9187
0.9190
0.9500
0.9533
0.9545
X
L-XZ
0.9200
0.9205
0.9235
0.9574
0.9600
0.9615
X
P-XZ
0.7413
0.7413
0.7525
0.8436
0.8450
0.8450


## 5. Conclusions

An orthogonal regularized kernel CCA algorithm for nonlinear problem is presented. By introducing the kernel function, our proposed algorithm is more suitable for solving nonlinear problem. Contrast experiments of ORCCA and ORKCCA are performed on artificial and handwritten numerals databases. Experimental results show that the proposed method outperforms ORCCA for the correlation coefficients of canonical variables and the accuracies of classification on the test data. The experimental results show ORKCCA is feasible.



---

*Source: 8745251-2018-10-29.xml*