ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13

Face Recognition based on Whitening Transformation of Distribution of
Subspaces
Tomokazu Kawahara, Masashi Nishiyama, Tatsuo Kozakaya, Osamu Yamaguchi
Corporate Research & Development Center Toshiba Corporation
1, Komukai-Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa 212-8582, Japan{tomokazu.kawahara, masashi.nishiyama, tatsuo.kozakaya, osamu1.yamaguchi}@toshiba.co.jp
Abstract

input patterns as an input subspace. To compare an input
subspace with the reference subspace representing an individual, a similarity of MSM is deﬁned by an angle between
the input subspace and the reference subspace. MSM has aproblem in that reference subspaces crowd since all facial
patterns have the same structure of facial parts and MSM
does not have a function that separates the subspaces of individuals.
To improve the recognition accuracy by separating subspaces, Fukui et al. have extended MSM to the Constrained
Mutual Subspace Method (CMSM)[2]. In CMSM, reference subspaces are projected onto a constraint subspace,which is designed to emphasize the difference between individuals. Fukui et al. conﬁrmed empirically that the projection to the constraint subspace creates a larger angle between multiple reference subspaces and explained that subspaces are separated because the common components ofsubspaces are removed.
We propose a new method, the Whitened Mutual Subspace Method (WMSM), based on a more mathematical analysis of angles between subspaces, which uses thewhitening transformation of the distribution of subspaces
for separation of subspaces. Whitening is a process to makea distribution uniform. First, we derive the equation that
relates angles between multiple subspaces to a standard deviation of eigenvalues of an autocorrelation matrix of thesesubspaces. This equation describes that uniformizing a distribution of multiple subspaces makes angles between thesesubspaces larger. In other words, the whitening transformation emphasize the difference between individuals. In
particular, the whitening transformation of a distributionof subspaces orthogonalizes reference subspaces when thenumber of reference subspaces is small. We show the proposed method works well even in a practical case throughevaluation experiments on the FRGC 1.0 and the FERET
databases and outperforms other methods.
The remainder of this paper is organized as follows.
First, to explain the reason for using the whitening transformation of a distribution of subspaces mathematically,

We propose a new face recognition method that separates subspaces representing individuals based on the mathematical analysis of angles between multiple subspaces. A
low-dimensional subspace representation by principal component analysis is known to be an effective approach for describing variation of facial patterns. A similarity between
individuals is deﬁned by an angle between their subspaces.
Since all facial patterns have the same structure of facial
parts, it is signiﬁcant to extract individual characteristics
from each subspace by considering the cross-relationshipbetween categories. Our method applies “whitening transformation of distribution of subspaces”, which can uniformize the distribution according to eigenvalues of the autocorrelation matrix of the subspaces. We derive the equation relating angles between subspaces to uniformity of the
distribution of these subspaces. From this equation, thewhitening transformation is effective for separation of thesubspaces. Under the ideal condition, the whitening transformation orthogonalizes all subspaces. In other words, allsimilarities between each other are equal to 0. We show
the proposed method works well even in a practical casethrough evaluation experiments on the FRGC 1.0 and the
FERET databases and outperforms other methods.

1

Introduction

Many face identiﬁcation methods have been proposed,
which represent variation of patterns for an individual as a
low-dimensional subspace generated from a set of patterns
by principal component analysis (PCA) [2, 7, 11]. Since
these methods are able to cope with variation in appearance,a robust face identiﬁcation application can be built.
Yamaguchi et al. have proposed face recognition using
the Mutual Subspace Method (MSM)[11]. They represent
not only reference patterns as a reference subspace but also97

ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13

we analyze angles between multiple subspaces in section
2. Next, we describe the proposed method of face recognition in section 3. We demonstrate the effectiveness of our
method by face recognition experiments in section 4.

where i = 1, . . . d and | · | denotes the norm. The subspaces(1)(d)
Vj , . . . , Vj have the following relation;2

where j = 1, 2. In particular, θ(1) is equal to the angle
between V1 and V2 . Therefore, we use canonical angles
instead of a single angle because more detailed analysis ofseparability is possible. When two subspaces are identicaland orthogonal, all canonical angles are equal to 0 and π/2,
respectively. From the deﬁnition of canonical angles, we
obtain the inequation θ(1) ≤ . . . ≤ θ(d) .
The canonical angles between these subspaces becomelarge when two subspaces separate. Therefore, we deﬁne
a measure of separability of two subspaces V1 and V2 asfollows:(1)

Vj

Mathematical analysis of angles between
subspaces

In this section, we explain the mathematical reason forusing the whitening transformation of a distribution of subspaces for separation of these subspaces. For the purposeof the explanation, two mathematical objects that are calculated from multiple subspaces are prepared and the equation describing the relationship between these mathematical objects are derived. One of the mathematical objects isa measure of separability of multiple subspaces, that consists of canonical angles between these subspaces [1]. The
other is an autocorrelation matrix of multiple subspaces[2].
We derived the equation that consists of a measure of separability of multiple subspaces and a standard deviationof eigenvalues of an autocorrelation matrix of these subspaces. This equation describes that a measure of separability of subspaces becomes large when a standard deviationof eigenvalues of this matrix becomes small. In other word,
uniformizing distribution of subspaces separates these subspaces. Based on this mathematical analysis of angles between subspaces, we propose the method using whitening
transformation for separation of multiple subspaces.
In MSM, a similarity between two subspaces is deﬁned
by an angle between these subspace. In this paper, therefore, we represent that subspaces separate when angles between these subspaces are large.

(2)

⊃ Vj

(d)

⊃ . . . ⊃ Vj1cos2 θ(i) ,
d i=1

(1)

d

Sep(V1 , V2 ) = 1 −(2)where θ(1) , . . . θ(d) are canonical angles between V1 andV2 . If two subspaces are identical and orthogonal, measures of separability of these subspaces are equal to 0 and 1,
respectively. When this measure of two subspaces is large,
these two subspaces separate.
For calculation of the measure of two subspaces (2) using orthonormal bases of these subspaces, we derive theequation between a measure of separability of two subspaces and projection matrices of these subspaces. The projection matrix P of subspace V is deﬁned by equation (3)
[8].
d

P=
ψi ψiT ,
(3)
i=1

2.1

A measure of separability of subspaces

where {ψ1 , . . . ψd } is an orthonormal basis of V. Generally,a projection matrix is deﬁned by d×D matrix (ψ1 , . . . ψd )T
where D is dimension of the vector space. However, we use
the former deﬁnition since the latter deﬁnition does not havethe information of position of the subspace on the vector
subspace. Let Pj be the projection matrix of Vj , wherej = 1, 2. By calculation of the trace of P1 P2 , the equation(4) is obtained.

In this section, we deﬁne a measure of separability of two
subspaces based on canonical angles between these subspaces and extend it to multiple subspaces.
To prepare the deﬁnition of a measure of separability ofsubspaces, we explain canonical angles between two subspaces, which are described in [1]. d canonical angles
θ(1) , . . . θ(d) between the d-dimensional subspaces V1 andV2 in a vector space are deﬁned as follows;
•

(1)
(1)
V1 = V1 and V2 = V2 .

•

(i)
(i)
(i)
θ(i) is the angle between v1 and v2 , where v1 ∈
(i)
(i)
(i)
V1 and v2 ∈ V2 are the nearest vectors under the(i)(i)
condition |v1 | = |v2 | = 1.
(i+1)

(i)

(i)

(i+1)

= {v ∈ V1 |v ⊥ v1 } and V2
• V1
(i)
(i)
V2 |v ⊥ v2 }.

1
Sep(V1 , V2 ) = 1 − tr(P1 P2 ),d(4)where tr(·) is a trace of a matrix, which is a sum of diagonalcomponents of the matrix. (See Appendix for a detailed
calculation of (4)).
We extend a measure of two subspaces (2) to a measureof separability of multiple subspaces. Let V1 , . . . , VN bed-dimensional subspaces in a D-dimensional vector space.
A measure of separability of subspaces V1 , . . . , VN is deﬁned as an average of measures of Vk and Vl (1 ≤ k <= {v ∈

98

ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13

l ≤ N)

autocorrelation matrix using (8) as follows,

Sep(V1 , . . . , VN ) =2
N (N − 1)σλ1≤k<l≤N

2
N (N − 1)1≤k<l≤N

2.3

1
tr(Pk Pl ),
d

S

(7)

where B is the matrix whose columns are the orthonormal
eigenvectors of A and Λ is the diagonal matrix of the corresponding eigenvalues λ1 ≥ . . . ≥ λD .
We calculate an average and a standard deviation of
eigenvalues of an autocorrelation matrix. Let mλ andσλ be an average and a standard deviation of eigenvalues
λ1 , . . . , λD , respectively. In the ﬁrst step, we calculate an
average of eigenvalues of an autocorrelation matrix. An
average of eigenvalues of the autocorrelation matrix mλ isequal to the constant value d/D regardless of arrangementof subspaces V1 , . . . , VN from the following calculation,

=

2.4

1
DN

N

k=1

k=1

tr(Pk ) =

1
DN

N

k=1

d=

d
.
D

1−

1≤k<l≤Ntr(Pk Pl ),1≤k=l≤N1
Pk Pl −Pk ),tr(
dN (N − 1)
N

N

k,l=1

k=1

(10)

Whitening transformation of distribution of subspaces

We propose whitening transformation of distribution ofsubspaces for separation of multiple subspaces based on theanalysis in the previous section. From the analysis in section 2.3, a transformation that decreases standard deviationof eigenvalues of autocorrelation matrix of subspaces separates these subspaces. In other words, whitening transformation of distribution of subspaces is effective to separate
these subspaces (Fig. 1). “Whitening” is a process to makeall eigenvalues of an autocorrelation matrix the same. The

N

l=1

=

1
1−
dN (N − 1)1tr(Pk Pl ),dFrom the equation (10), a transformation that decreases
the standard deviation of eigenvalues σλ separates the subspaces V1 , . . . VN . In particular, all subspaces are separated most when all eigenvalues λ1 , . . . λD are the samevalues.

1
1 
1 
1
λl = trA = tr(
Pk ),
D
D
D N
D



1−

1
= 1−
tr(N 2 A2 − N A),
dN (N − 1)
DN
N (D − d)
= −
σλ2 +
.
d(N − 1)(N − 1)D

k=1=

2
N (N − 1)

=

=

N

mλ

Equation between a measure of separability and an autocorrelation matrix

In this section, we show that a separability of multiple subspaces is decided from only a standard deviation of
eigenvalues of an autocorrelation matrix from the equationthat consists of a measure of separability of multiple subspaces, a standard deviation of eigenvalues of an autocorrelation matrix and a constant term.
Using (6), (8) and (9), a measure of separability of multiple subspaces S = Sep (V1 , . . . VN ) is calculated as follows,

To prepare calculation of a measure of multiple subspaces (5) we explain an autocorrelation matrix of distribution of subspaces, which is described in [2], and calculate an average and a standard deviation of its eigenvalues.
An autocorrelation matrix of distribution of subspaces A
is deﬁned as an average of all projection matrices, like an
autocorrelation matrix of distribution of vectors [8], and its
eigenvalue problem is solved as follows,
1 
Pk = BΛBT ,
N

(9)

l=1

An autocorrelation matrix of subspaces

A=

1  2
1
d
λl − m2λ = trA2 − ( )2 .
D
D
D
D

=

(6)
where Pk is the projection matrix of Vk deﬁned by (3).
Therefore, we calculate a measure of separability of multiple subspaces using orthonormal bases of these subspaces.

2.2

=

l=1

(5)
When all subspaces are identical and orthogonal, measuresof separability of these subspaces are equal to 0 and 1, respectively. The more this measure of multiple subspaces is,
the more these subspaces separate. We obtain the equation(6) from (5) and (4).
Sep(V1 , . . . , VN ) = 1 −1 (λl − mλ )2 ,D
D

Sep(Vk , Vl ).

(8)

Next, we calculate a standard deviation of eigenvalues of an99

ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13

Figure 1. The ellipse and the circle in the center of the ﬁgure represent the distribution ofsubspaces. “Whitening” makes the distribution uniform.

Figure 2. Similarity matrix: angles between
ten reference subspaces in MSM, CMSM and
WMSM in the case that the condition (14) issatisﬁed. The darker a pixel is, the larger theangle between subspaces is.

matrix W that represents whitening transformation of distribution of subspaces V1 , . . . , VN is deﬁned and makes an
autocorrelation matrix the identity matrix I as follows:
W
WAW

T

=

Λ−1/2 BT ,

=

−1/2

(Λ

in the ideal case that the number of these subspaces is small.
In the ﬁrst step, we prove the following proposition. Let u1 , . . . , uN be bases of 1-dimensional subspaces in D-dimensional vector space. A matrix U denotes(u1 , . . . , uN ) and Λ, Λ−1/2 , B, W are deﬁned as in (7) and(11). Let ui be Wui for all i.

(11)
−1/2

B )BΛB (BΛTT) = I,(12)where B and Λ are deﬁned in (7).
Another method, named the Orthogonal Subspace
Method (OSM), in which whitening orthogonalizes subspaces, has been proposed by Fukunaga et al.[3] andKittler[4]. In OSM, an autocorrelation matrix of each classis transformed by the whitening of the autocorrelation matrix generated from all samples in all classes before a subspace of each class is generated from the eigenvectors of
the autocorrelation matrix of this class, the eigenvalues ofwhich are large. In other words, a set of samples in eachclass is represented as a low-dimensional subspace after the
distribution of all samples in all classes is made uniform.
In this method, the eigenvector of an autocorrelation matrix
of a class, the eigenvalue of which is 1, is orthogonal to all
samples in other classes since all eigenvalues of the autocorrelation matrix generated from all samples in all classes
are equal to 1.
In our method and OSM, subspaces are orthogonalizedusing whitening. The difference between our method and
OSM is the order of the linearization and the transformation. In other words, an input subspace and a reference subspace are generated from a set of patterns before whitening
in our method, but after whitening in OSM. Therefore, ourmethod does not use eigenvectors of the autocorrelation matrix whose eigenvalues are small. Furthermore, when thenumber of subspaces is small, these subspaces are always
orthogonalized in our method but not always orthogonalized in OSM (section 2.5).

2.5

Proposition 1 if u1 , . . . , uN is linearly independent,u1 , . . . , uN is orthonormal.
Proof Let U be WU. Since UUT = A = BΛBT in theequation (7) and u1 , . . . , uN is linearly independent,U UT = WUUT WT = Λ−1/2 BT BΛBT BΛ−1/2 = ĨN ,(13)
where ĨN is a diagonal matrix in which the number of 1
on the diagonal is N and others are 0. The symmetric matrix UT U is an identity matrix because all eigenvalues ofUT U are the same as those of U UT without 0 and therank of UT U is N . Therefore, u1 , . . . , uN is orthonormal
because a component of UT U is an inner product of ukand ul . 
Generally, we can orthogonalize all subspaces using thewhitening transformation of the distribution of these subspaces if the following inequation is satisﬁed;dN ≤ D,(14)

where D is the dimension of the vector space includingthese subspaces, because we apply Proposition 1 to basesof these subspaces. The equation (14) requires the dimension or the number of these subspaces to be small. In thecase that the condition (14) is satisﬁed, the measure of separability of subspaces transformed by the whitening transformation is equal to 1 since the autocorrelation matrix ofthese subspaces is ĨdN from the same calculation (13).
Fig. 2 shows similarity matrix images whose pixel values represent an angle between pairs of subspaces in MSM,

Transformation under the ideal condition

We show that subspaces can be orthogonalized by thewhitening transformation of distribution of these subspaces

100

ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13

3.2

Transformation of a subspace and calculation of a similarity

In our proposed method, to transform the input subspace
Vinput and the reference subspace Vref by whitening of
distribution of reference subspaces, we carry out the following steps:
1. Transform a basis of a subspace by the whitening
transformation W.
Figure 3. The ﬂow chart of WMSM.
2. Apply Gram-Schmidt orthogonalization to them.
The orthonormal basis is a basis of the transformed subspace.
We deﬁne a similarity s between the d-dimensional subspaces Vinput and Vref as s = cos2 θ, where θ is the angle
between Vinput and Vref . The angle θ is equal to the 1-th

CMSM, and our method in the case that the condition (14)is satisﬁed. This ﬁgure shows that our method orthogonalizes all these subspaces.

3

canonical angle θ(1) between Vinput and Vref . If Vinput
and Vref are identical, the angle θ is equal to 0. The angle
is calculated using the MSM[11]. The similarity s equals
the largest eigenvalue λmax of X = (xmn ) using

Face Recognition using the whitening
transformation of the distribution of subspaces

In this section, we describe the procedure of WMSM
(Fig. 3).

xmn =

d


(ψm , φl )(φl , ψn ) (m, n = 1 . . . d) ,

(15)

l=1

3.1

Algorithm for face recognition

where {ψi }i=1,...,d and {φj }j=1,...,d are the orthonormalbases of Vinput and Vref , respectively; (ψm , φl ) is the inner product of ψm and φl .

First, we located the face pattern from the positions of
the feature points and cropped to 32 × 32 pixels using 3D
normalization[5] and preprocessing[6]. In order to adapt
localization error of feature points, we represent variationof face patterns due to the localization error as a subspace inthe feature space by perturbation of the feature points andobtaining multiple face patterns from a single face image.
We apply PCA to the vectors to generate an input subspace.
Let {xi }i=1,...n be a set of vectors. The basis of the input
subspace is the eigenvectors of the autocorrelation matrixnZ = 1/n i=1 xi xTi [8].
The whitening transformation (11) is generated from an
autocorrelation matrix of reference subspaces. To allow for
the variation in appearance for each individual, it is effective
to increase the dimension of the reference subspace by addition of other bases that are generated from reference patterns and not used for comparison with an input subspace.
To compare the input subspace with the reference subspace registered in a database for each individual, we calculate their similarities after transforming the input subspace
and the reference subspaces by the whitening transformation of a distribution of reference subspaces. The person in
the image is identiﬁed as the person who corresponds to thereference subspace with the highest similarity.

4

Evaluation with the FRGC 1.0 and FERET
databases

We show the proposed method works well even in a practical case. We performed experiments using the controlled
still images (exp1) in the FRGC 1.0 database [9] and the fa
and the fb data sets in the FERET database[10]. The controlled still images in FRGC 1.0 consisted of 152 gallery
images and 608 probe images. The fa and the fb in FERET
consisted of images of 1196 people with one image per person and 1195 people with one image per person, respectively.
We compare ﬁve methods, namely, MSM, CMSM,
Multiple CMSM (MCMSM) [7], WMSM and Multiple
WMSM (MWMSM). MCMSM and MWMSM apply ensemble learning with bagging to CMSM and WMSM, respectively. In MCMSM, multiple constraint subspaces are
generated from reference subspaces selected randomly inthe same way of bagging. The input subspace and the reference subspaces are projected onto each constraint subspace and a similarity is determined with the similarities

101

ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13
1

Table 1. The methods and their parameters. d
is the dimension of input and reference subspaces. L is the number of constraint subspaces and whitening transformations. d isthe dimension of reference subspaces thatgenerate constraint subspaces and whitening transformations. C is the dimension of
constraint subspaces.
d L d
C
MSM
7 –
–
–
CMSM
7 1 15 210
MCMSM 7 10 15 210
WMSM
7 1 15–
MWMSM 7 10 15–

0.99

Cumulative Match Rate

0.98

0.97

0.96

0.95
MOMSM
OMSM
MCMSM
CMSM
MSM
UMD97

0.94

0.93

5

10

15

20

25
Rank

30

35

40

45

50

Figure 4. Experimental results using FERET
database in terms of Cumulative Match Rate.

Table 2. Experimental results using FRGC 1.0
in terms of Correct Match Rate (CMR) andEqual Error Rate (EER).
CMR (%) EER (%)
MSM
96.4
3.45
CMSM
96.5
2.47
MCMSM
97.2
2.28
WMSM
97.0
1.81
MWMSM
97.2
1.81

5

Conclusions

This paper presented a face recognition method basedon mathematical analysis of angles between subspaces inwhich we apply whitening of a distribution of subspaces toemphasize the difference between individuals. We derived
the equation (10) that relates angles between subspaces toa distribution of these subspace. This equation describesthat the whitening transformation is effective for separationof these subspaces. In the experiment, we obtained highperformance compared with other methods on the FRGC
1.0 and the FERET database .

calculated on each constraint subspace. In MWMSM, multiple whitening transformations are generated from reference subspaces selected randomly and the similarity is determined with an average of the similarities calculated after
transformation by each whitening transformation. Their parameters in the experiments are listed in Table 1.
Table 2 shows the evaluation results in FRGC 1.0 foreach method in terms of Correct Match Rate (CMR) andEqual Error Rate (EER). Correct Match Rate is the probability that an input of the right person is correctly accepted.
Equal Error Rate is the probability that false acceptance rate(FAR) equals the false rejection rate (FRR). It can be seenthat the proposed method and the proposed method with ensemble learning are equivalent to MCMSM with regard to
Correct Match Rate and superior to the other methods withregard to Equal Error Rate on FRGC 1.0.
To evaluate the generalization ability of our method, we
performed experiments using another database. Fig. 4
shows the evaluation results for each method and the best
result (UMD97) of the partially automatic algorithms reported in FERET’97 [10] in terms of Cumulative Match
Rate. It can be seen that the proposed method and the proposed method with ensemble learning are superior to theother methods.

References[1] F. Chaitin-Chatelin. Eigenvalues of Matrices. Johe Wiley &Sons Ltd, 1993. (Enlarged Translation of the French Publication with Masson).
[2] K. Fukui and O. Yamaguchi. Face recognition using multiviewpoint patterns for robot vision. In 11th International
Symposium of Robotics Research, pages 192–201, 2003.
[3] K. Fukunaga and W. Koontz. Application of the KarhunenLoeve expansion to feature extraction and ordering. IEEE
Transactions on Computers, 19(5):311–318, 1970.
[4] J. Kittler. The subspace approach to pattern recognition. In
R. Trappl, G. J. Klir, and L. Ricciardi, editors, Progress incybernetics and systems research, Washington, 1978. Hemisphere Publishing Corporation.
[5] T. Kozakaya and O. Yamaguchi. Face recognition by
projection-based 3d normalization and shading subspace orthogonalization. In Proceedings IEEE 7th International
Conference on Automatic Face and Gesture Recognition,
pages 163–168, 2006.
[6] M. Nishiyama and O. Yamaguchi. Face recognition using
the classiﬁed appearance-based quotient image. In Proceed-102

ACCV'07 Workshop Subspace 2007, Tokyo, Nov., 2007

SS13d(i) (i)T(1)(d)
is deﬁned by i=1 vj vj
since {vj , . . . vj } is an orthonormal basis, where j = 1, 2. By calculation of traceof P1 P2 using the equations (19) and (20) as follows, theequation (16) is obtained.

ings IEEE 7th International Conference on Automatic Faceand Gesture Recognition, pages 49–54, 2006.
[7] M. Nishiyama, O. Yamaguchi, and K. Fukui. Face recognition with the multiple constrained mutual subspace method.
In Audio- and Video-based Biometric Person Authentication,
pages 71–80, 2005.
[8] E. Oja. Subspace Methods of Pattern Recognition. ResearchStudies Press, 1983.
[9] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer,
J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek.
Overview of the face recognition grand challenge. In IEEE
Conference on Computer Vision and Pattern Recognition,
2005.
[10] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss. The
FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and MachineIntelligence, 22(10):1090–1104, 2000.
[11] O. Yamaguchi, K. Fukui, and K. Maeda. Face recognition
using temporal image sequence. In Proceedings of IEEE
3rd International Conference on Automatic Face and Gesture Recognition, pages 318–323, 1998.

A

1
tr(P1 P2 ) =d==Calculation of a measure of separability ofsubspaces
To obtain equation (4), we prove the following equation,1
tr(P1 P2 ) = Sep(V1 , V2 ).
d(16)

To calculate the trace of the product of projection matri(1)(d)
ces, we describe several facts about vectors v1 , . . . , v1(1)(d)
and v2 , . . . , v2 in section 2.1.
A set of vectors
(1)
(d)
(i)
{vj , . . . , vj } is an orthonormal basis of Vj since vj(i+1)

is orthogonal to Vj

(l)

(k)

, where j = 1, 2. Furthermore, v1

is orthogonal to v2 (k = l) since the equations (17) and(i)(i)(18) are derived from the deﬁnition of v1 and v2 ,(i) (i)= cos θ(i) v2 ,(17)(i) (i)P1 v2(i)cos θ(i) v1 ,

(18)

P2 v1

(i)

=

(i)

(i)

where Pj is projection matrix of Vj (j = 1, 2). Fromthese facts, the following equations (19) and (20) is acquired.
1(k = l),(k) (l)(k) (l)(v1 , v1 ) = (v2 , v2 ) =(19)0(k = l),(k) (l)(v1 , v2 ) =cos θ(k)0(k = l),(k = l),(20)

where (·, ·) is the inner product of vectors.
We calculate the trace of the product of projection matrices in the equation (16). The projection matrix Pj of Vj

103

d
d


1
(k) (k)T(l) (l)Tv1 v1 )(
v2 v2 )),
tr((
d
k=1

1
tr(
d

d


(l)T (k) (k)T (l)v1 v1 v2 ),

v2

k,l=1

d
1

d

l=1

k=1

cos2 θ(k) = Sep(V1 , V2 ).