# Neural-Network-Based Approach for Extracting Eigenvectors and Eigenvalues of Real Normal Matrices and Some Extension to Real Matrices

Authors: Xiongfei Zou; Ying Tang; Shirong Bu; Zhengxiang Luo; Shouming Zhong
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

This paper introduces a novel neural-network-based approach for extracting some eigenpairs of real normal matrices of ordern. Based on the proposed algorithm, the eigenvalues that have the largest and smallest modulus, real parts, or absolute values of imaginary parts can be extracted, respectively, as well as the corresponding eigenvectors. Although the ordinary differential equation on which our proposed algorithm is built is only n-dimensional, it can succeed to extract n-dimensional complex eigenvectors that are indeed 2n-dimensional real vectors. Moreover, we show that extracting eigen-pairs of general real matrices can be reduced to those of real normal matrices by employing the norm-reducing skill. Numerical experiments verified the computational capability of the proposed algorithm.


---

## Body

## 1. Introduction

The problem of extracting special eigenpairs of real matrices has attracted much attention both in theory [1–4] and in many engineering fields such as real-time signal processing [5–8] and principal or minor component analysis [9–12]. For example, we may wish to get eigenvectors and the corresponding eigenvalues that (1) have the largest or smallest modulus; (2) have the largest or smallest real parts; (3) have the largest or smallest imaginary parts in absolute value. Two most popular methods for this problem are the power method and the Rayleigh quotient method in their direct forms or in the context of inverse iteration [13]. Recently, many neural-network-based methods have also been proposed to solve this problem [14–23]. However, most of those neural network based methods focused on computing eigenpairs of real symmetric matrices. The following two ordinary differential equations (ODEs):
(1)dx(t)dt=Ax(t)-x(t)TAx(t)x(t),(2)dx(t)dt=x(t)Tx(t)Ax(t)-x(t)TAx(t)x(t)
were proposed by [19, 23], respectively, where A is a real symmetric matrix. Both (1) and (2) are efficient to compute the largest eigenvalue of A, as well as the corresponding eigenvector. In addition, they can succeed to compute the smallest eigenvalue of A and the corresponding eigenvector by simply replacing A with -A, for example,
(3)dx(t)dt=-x(t)Tx(t)Ax(t)+x(t)TAx(t)x(t).The following ODE for solving the generalized eigenvalue problem was proposed by [22]
(4)dx(t)dt=Ax(t)-f(x(t))Bx(t),
where A and B are two real symmetric matrices, and f can be a general form to some degree. Particularly, if B is the identity matrix, (4) can be used to solve the standard eigenvalue problem as (1) and (2).References [16–18] extended those neural network based approaches to the case of real antisymmetric or special real matrices of order n, where the proposed neural networks can be summarized by 2n-dimensional ODEs since eigenvectors in those cases may be n-dimensional complex vectors, that is, 2n-dimensional real vectors.In this paper, we propose an approach for extracting six types of eigenvalues ofn-by-n real normal matrices and the corresponding eigenvectors based on (2) or (3). Although eigenvectors of real normal matrices may be n-dimensional complex vectors, the computation of our proposed method can be achieved in n-dimensional real vector space, which can reduce the scale of networks a lot. Then, we show that any real matrix can be made arbitrarily close to a normal matrix by a series of similarity transformations, based on which our proposed algorithm can be extended to the case of arbitrary real matrices.


## 2. Main Results

Leti=-1 be the imaginary unit, x¯ the conjugate of x, and diag[A1,…,Ap] a block diagonal matrix, where Ai, i=1,…,p, is a square matrix at the ith diagonal block. Unless specially stated, A is a real normal matrix in this paper.Lemma 1.
In [13], A is a real normal matrix of order n if and only if there exists an orthogonal matrix U such that
(5)UTAU=diag[λ1,…,λr︸r,Ar+1,…,Ar+k︸k],r+2k=n,0≤r≤n,
where
(6)U=[u1,…,ur︸rrealeigenvectors,ur+1R,ur+1I,…,ur+kR,ur+kI︸kpairsofcomplexeigenvectors],Ar+j=(ar+j|br+j|-|br+j|ar+j),j=1,…,k.
Here,λj, j=1,…,r, are r real eigenvalues of A corresponding to the r real eigenvectors uj, and ar+s±ibr+s,  s=1,…,k, are k pairs of complex eigenvalues of A corresponding to the k pairs of complex eigenvectors ur+sR±iur+sI.For simplicity, letaj=λj, bj=0 for j=1,…,r. Based on (5) and (6), it is straightforward to verify
(7)UT(A-AT)T(A-AT)U=4diag[b12,…,br2,br+12,br+12,…,br+k2,br+k2](8)UT(A+AT)U=2diag[a1,…,ar,ar+1,ar+1,…,ar+k,ar+k](9)UT(AAT)U=diag[a12,…,ar2,ar+12+br+12,ar+12+br+12,…,ar+k2+br+k2,ar+k2+br+k2].Then, the following six definitions and two lemmas are presented, which will be involved much in the sequel.(C1)
Let𝒥1={s∣|bs|=max{|b1|,…,|br+k|}. Then for all j,k∈𝒥1,  aj=ak.(C2)
Let𝒥2={s∣|bs|=min{|b1|,…,|br+k|}. Then for all j,k∈𝒥2,  aj=ak.(C3)
Let𝒥3={s∣as=max{a1,…,ar+k}. Then for all j,k∈𝒥3,  |bj|=|bk|.(C4)
Let𝒥4={s∣as=min{a1,…,ar+k}. Then for all  j,k∈𝒥4,  |bj|=|bk|.(C5)
Let𝒥5={s∣as2+bs2=max{a12+b12,…,ar+k2+br+k2}. Then for all j,k∈𝒥5,  aj=ak.(C6)
Let𝒥6={s∣as2+bs2=min{a12+b12,…,ar+k2+br+k2}. Then for all j,k∈𝒥6,  aj=ak.Lemma 2 (Theorem  4 in [23]).
Assume that nonzerox(0)∈Rn is not orthogonal to the eigensubspace corresponding to the largest eigenvalue of A. Then, the solution of (2) starting from x(0) converges to an eigenvector corresponding to the largest eigenvalue of A that is equal to limt→+∞(x(t)TAx(t)/x(t)Tx(t)).Lemma 3 (Theorem  5 in [23]).
Assume that nonzerox(0)∈Rn is not orthogonal to the eigensubspace corresponding to the smallest eigenvalue of A. Then, the solution of (3) starting from x(0) converges to an eigenvector corresponding to the smallest eigenvalue of A that is equal to limt→+∞(x(t)TAx(t)/x(t)Tx(t)).Remark 4.
If we randomly choosex(0), the projection of x(0) on the eigensubspace corresponding to the largest or smallest eigenvalue of A will be nonzero with high probability. Hence, (2) and (3) can almost work well with randomly generated x(0).


### 2.1. Computing the Eigenvalues with the Largest or Smallest Imaginary Parts in Absolute Value, as well as the Corresponding Eigenvectors

Without loss of generality, in this subsection we assume thatb1=⋯=br=0<br+12≤⋯≤br+k2 in (7). Note that (A-AT)T(A-AT)=-(A-AT)2.Based on (7), we know that 0 (if any) is the eigenvalue of -(A-AT)2 corresponding to the eigenvector uj, j=1,…,r, and that 4br+s2, s=1,…,k, is the eigenvalue of -(A-AT)2 corresponding to the eigenvectors ur+sR or ur+sI. If replacing A with the symmetric matrix -(A-AT)2 in (2) as
(10)dx(t)dt=-x(t)Tx(t)(A-AT)2x(t)+x(t)T(A-AT)2x(t)x(t)
based on Lemma 2, we know that α=limt→∞x(t) is an eigenvector of -(A-AT)2 corresponding to 4br+k2 and 4br+k2=limt→∞(-x(t)T(A-AT)2x(t)/x(t)Tx(t)) (thus, |br+k| has been gotten), where x(t) is a solution of (10). If br+k=0, bj’s are all zero, meaning that A is symmetric. In this case, we can directly use (2) or (3) to get the largest or smallest eigenvalue of A and the corresponding eigenvector, respectively. Hence, we assume br+k≠0.The following lemma introduces an approach for computingar+k under the condition (C1).Lemma 5.
Assume that (C1) holds. Letα=limt→∞x(t), where x(t) is a solution of (10). Then, ar+k=αT(A+AT)α/2αTα. In addition, there exists ar+k2+br+k2=αT(AAT)α/αTα.Proof.
Assume that4br+j2,  j=s,s+1,…,k,  s≥1, are the largest eigenvalues of -(A-AT)2, that is; |br+s|=|br+s+1|=⋯=|br+k|. Therefore,
(11)𝒥1={r+s,…,r+k}.
Based on Lemma2 and (7), we know that α should be a linear combination of ujR and ujI,  j∈𝒥1. Let
(12)α=∑j∈𝒥1(γjujR+τjujI),γj,τj∈R.
In addition, by (8) we have
(13)(A+AT)ujR=2ajujR,(A+AT)ujI=2ajujI,j∈𝒥1.
And by (9), we have
(14)(AAT)ujR=(aj2+bj2)ujR,(AAT)ujI=(aj2+bj2)ujI,j∈𝒥1.
BecauseU is an orthogonal matrix and for all j∈𝒥1, aj=ar+k holds due to (C1), it is straightforward to verify
(15)αT(A+AT)α2αTα=2ar+k∑j∈𝒥1(γj2+τj2)2∑j∈𝒥1(γj2+τj2)=ar+k,αT(AAT)ααTα=(ar+k2+br+k2)∑j∈𝒥1(γj2+τj2)∑j∈𝒥1(γj2+τj2)=ar+k2+br+k2,
thus proving the lemma.Remark 6.
Based on Lemma5, if  ar+k2+br+k2≠αT(AAT)α/αTα, (C1) does not hold surely, which can be used to check whether (C1) holds or not.
The following lemma introduces an approach for computing a pair of conjugated eigenvectors ofA corresponding to the eigenvalues ar+k±i|br+k| under the condition (C1).Lemma 7.
Assume that (C1) holds. Given any nonzero4br+k2, the largest eigenvalue of -(A-AT)2, and the corresponding eigenvector α obtained by (10), let ar+k=αT(A+AT)α/2αTα and β=(A-AT)α/2|br+k|. Then, ar+k±i|br+k| are two eigenvalues of A corresponding to the eigenvectors β±iα, respectively.Proof.
Let𝒥1 and α take the form as (11) and (12), respectively. Based on (13), we can write
(16)(A+AT)2α=ar+kα.
Following the decomposition ofA as (5) and the definition of β, we have
(17)|br+k|β=(A-AT)α2=UTdiag[0,…,0︸r,Br+1,…,Br+k]Uα2,
where
(18)Bj={(02|bj|-2|bj|0),j∈{r+1,…,r+k}∖𝒥1(02|br+k|-2|br+k|0),j∈𝒥1.
Taking (12) into (17), we get
(19)β=∑j∈𝒥1‍(τjujR-γjujI).
Based on (13) and (19), it is straightforward to verify
(20)(A+AT)2β=ar+kβ.
In addition, since4br+k2 is the eigenvalue of -(A-AT)2 corresponding to the eigenvector α, we have -(A-AT)2α=4br+k2α. Hence, -(A-AT)|br+k|β=2br+k2α. Since br+k≠0, we have
(21)(A-AT)2β=-|br+k|α.
By (20) and (21), we have
(22)Aβ=A+AT2β+A-AT2β=ar+kβ-|br+k|α.
By (16) and (17), we have
(23)Aα=A+AT2α+A-AT2α=ar+kα+|br+k|β.
Then, it is straightforward to verify(24)A(β+iα)=(ar+k+i|br+k|)(β+iα),A(β-iα)=(ar+k-i|br+k|)(β-iα),
thus proving the lemma.To get|b1|, we can in advance get 4b12, the smallest eigenvalue of -(A-AT)2 by replacing A with -(A-AT)2 in (3) as follows:
(25)dx(t)dt=x(t)Tx(t)(A-AT)2x(t)-x(t)T(A-AT)2x(t)x(t).Letα*=limt→∞x(t), where x(t) is a solution of (25). From Lemma 3, we know 4b12=-(α*)T(A-AT)2α*/(α*)Tα*. Then, the following lemma similar to Lemma 5 can be used to compute a1.Lemma 8.
Assume that (C2) holds. Then,a1=(α*)T(A+AT)α*/2(α*)Tα*. In addition, there exists a12+b12=(α*)T(AAT)α*/(α*)Tα*.Proof.
The proof is almost the same to that in Lemma5.Note thatb1 may be zero; that is, -(A-AT)2 has real eigenvalues. In this case, we have the following lemma.Lemma 9.
Assume that (C2) holds and 0 is the smallest eigenvalue of-(A-AT)2 corresponding to the eigenvector α*. Then, a1 is the eigenvalue of A corresponding to the eigenvector α*, where a1=(α*)T(A+AT)α*/2(α*)Tα*.Proof.
Following the conditions, we have-(A-AT)2α*=0. Hence, (α*)T(A-AT)T(A-AT)α*=∥(A-AT)α*∥22=0; that is,
(26)(A-AT)α*=0.
Note thatb1=⋯=br=0,  r≥1, because 0 is the smallest eigenvalue of -(A-AT)2. Based on the definition of 𝒥2, we have
(27)a1=⋯=ar,𝒥2={1,…,r},r≥1.
Applying Lemma3 to (25), we know that α* should be a linear combination of u1,…,ur. Let
(28)α*=∑j∈𝒥2‍γjuj,γj∈R.
By (8), we have
(29)(A+AT)uj=2ajuj=2a1uj,j∈𝒥2.
Therefore,(30)A+AT2α*=12(∑j∈𝒥2γj(A+AT)uj)=a1α*.
Then, by (26) and (30), it is straightforward to verify
(31)Aα*=A+AT2α*+A-AT2α*=a1α*,
thus proving the lemma.In the case ofb1≠0, that is, all the eigenvalues of -(A-AT)2 are complex numbers, we have the following lemma similar to Lemma 7.Lemma 10.
Assume that (C2) holds. Given any nonzero4b12, the smallest eigenvalue of -(A-AT)2, and the corresponding eigenvector α* obtained by (25), let a1=(α*)T(A+AT)α*/2(α*)Tα* and β*=(A-AT)α*/2|b1|. Then, a1±i|b1| are two eigenvalues of A corresponding to the eigenvectors β*±iα*, respectively.Proof.
The proof is almost the same to that in Lemma7.Remark 11.
Among the following four real normal matrices(32)(70000050000050000062000-26),(70000051000-150000062000-26),(7000005-2000250000052000-25),(70000052000-250000062000-26),
only the first three meet (C1), but the last one does not. And only the last three matrices meet (C2), but the first one does not.


### 2.2. Computing the Eigenvalues with the Largest or Smallest Real Parts, as well as the Corresponding Eigenvectors

As shown in (8), 2aj,  j=1,…,r+k, are the eigenvalues of the symmetric matrix (A+AT). In this subsection, we assume that a1≤⋯≤ar and ar+1≤⋯≤ar+k, which can be achieved by reordering aj and the corresponding columns of U.If replacingA with (A+AT) in (2), we get
(33)dx(t)dt=x(t)Tx(t)(A+AT)x(t)-x(t)T(A+AT)x(t)x(t).Without loss of generality, assume thatar+k is the largest real part of the eigenvalues of A (it may be ar. However, Lemmas 12 and 13 have no difference in that case). Applying Lemma 2 to (33), we can get 2ar+k=limt→∞(x(t)T(A+AT)x(t)/x(t)Tx(t)), the largest eigenvalue of (A+AT), and the corresponding eigenvector ζ=limt→∞x(t), where x(t) is a solution of (33).The following lemma introduces an approach for computing|br+k| under the condition (C3).Lemma 12.
Assume that (C3) holds. Letζ=limt→∞x(t), where x(t) is a solution of (33). Then, |br+k|=(ζT(AAT)ζ/ζTζ)-ar+k2.Proof.
By Lemma2, we know that ζ should be a linear combination of ujR and ujI,  j∈𝒥3. Let
(34)ζ=∑j∈𝒥3‍(γjujR+τjujI),γj,τj∈R.
Then, based on (9), we get
(35)(AAT)ujR=(aj2+bj2)ujR,(AAT)ujI=(aj2+bj2)ujI,j∈𝒥3.
BecauseU is an orthogonal matrix and for all j∈𝒥3,  aj=ar+k and |bj|=|br+k| hold, that is, aj2+bj2=ar+k2+br+k2 for all j∈𝒥3, we have
(36)ζT(AAT)ζζTζ=(ar+k2+br+k2)∑j∈𝒥3(γj2+τj2)∑j∈𝒥3(γj2+τj2)=ar+k2+br+k2,
thus proving the lemma.The following lemma introduces an approach for computing a pair of conjugated eigenvectors ofA corresponding to the eigenvalues ar+k±i|br+k| under the condition (C3).Lemma 13.
Assume that (C3) holds. Given any nonzero2ar+k, the largest eigenvalue of (A+AT), and the corresponding eigenvector ζ obtained by (33), let |br+k|=(ζT(AAT)ζ/ζTζ)-ar+k2. If br+k=0, ar+k is an eigenvalue of A corresponding to the eigenvector ζ. If br+k≠0, let η=(A-AT)ζ/2|br+k|. Then, ar+k±i|br+k| are two eigenvalues of A corresponding to the eigenvectors η±iζ, respectively.Proof.
Combining the proofs of Lemmas9 and 7, we can prove this lemma.If replacingA with (A+AT) in (3), we get
(37)dx(t)dt=-x(t)Tx(t)(A+AT)x(t)+x(t)T(A+AT)x(t)x(t).Without loss of generality, assume thatar+1 is the smallest real part of the eigenvalues of A (it may be a1. However, Lemma 14 has no difference in that case). Applying Lemma 3 to (37), we can obtain 2ar+1, the smallest eigenvalue of (A+AT), as well as the corresponding eigenvector, denoted by ζ*. Then, we have the following lemma.Lemma 14.
Assume that (C4) holds. Given any nonzero2ar+1, the smallest eigenvalue of (A+AT), and the corresponding eigenvector ζ* obtained by (37), let |br+1|=((ζ*)T(AAT)ζ*/(ζ*)Tζ*)-ar+12. If br+1=0, ar+1 is the eigenvalue of A corresponding to the eigenvector ζ*. If br+1≠0, let η*=(A-AT)ζ*/2|br+1|. Then, ar+1±i|br+1| are two eigenvalues of A corresponding to the eigenvectors η*±iζ*, respectively.Proof.
Combining the proofs of Lemmas12, 9, and 7, we can prove this lemma.Remark 15.
Among the following four real normal matrices(38)(10000040000050000012000-21),(30000040000040000012000-21),(30000042000-240000042000-24),(30000040000050000052000-25),
only the first three meet (C3), but the last one does not. And only the last three matrices meet (C4), but the first one does not.


### 2.3. Computing the Eigenvalues with the Largest or Smallest Modulus, as well as the Corresponding Eigenvectors

Reorder the eigenvalues of the symmetric matrixAAT in (9) and the corresponding columns of U such that a12≤⋯≤ar2 and ar+12+br+12≤⋯≤ar+k2+br+k2. Without loss of generality, assume that ar+k±i|br+k| are the eigenvalues of A that have the largest modulus (it may be ar. However, Lemma 16 has no difference in that case), and that ar+1±i|br+1| are the eigenvalues of A that have the smallest modulus (it may be a1. However, Lemma 17 has no difference in that case).ReplacingA with AAT in (2), we get
(39)dx(t)dt=x(t)Tx(t)AATx(t)-x(t)TAATx(t)x(t).Applying Lemma2 to (39), we can obtain ar+k2+br+k2=limt→∞(x(t)TATAx(t)/x(t)Tx(t)), the largest eigenvalue of AAT, and the corresponding eigenvector θ=limt→∞x(t), where x(t) is the solution of (39). Then, we have the following lemma.Lemma 16.
Assume that (C5) holds. Given anyar+k2+br+k2, the largest eigenvalue of AAT, and the corresponding eigenvector θ obtained by (39). Then, ar+k=θT(A+AT)θ/2θTθ. Thus, |br+k| can be gotten. If br+k=0, ar+k is an eigenvalue of A corresponding to the eigenvector θ. If br+k≠0, let ω=(A-AT)θ/|2br+k|. Then, ar+k±i|br+k| are two eigenvalues of A corresponding to the eigenvectors ω±iθ, respectively.Proof.
Combining the proofs of Lemmas5, 7, and 9, we can prove this lemma.ReplacingA with AAT in (3), we get
(40)dx(t)dt=-x(t)Tx(t)AATx(t)+x(t)TAATx(t)x(t).Applying Lemma3 to (40), we can obtain ar+12+br+12, the smallest eigenvalue of AAT, and the corresponding eigenvector, denoted by θ*. Then, we have the following lemma.Lemma 17.
Assume (C6) holds. Given anyar+12+br+12, the smallest eigenvalue of AAT, and the corresponding eigenvector θ* obtained by (40). Then, ar+1=(θ*)T(A+AT)θ*/2(θ*)Tθ*. So one can get that |br+1|. If br+1=0, ar+1 is an eigenvalue of A corresponding to the eigenvector θ*. If br+1≠0, let ω*=(A-AT)θ*/2|br+1|. Then, ar+1±i|br+1| are two eigenvalues of A corresponding to the eigenvectors ω*±iθ*, respectively.Proof.
Combining the proofs of Lemmas5, 7, and 9, we can prove this lemma.Remark 18.
Among the following four real normal matrices(41)(200000-20000060000012000-21),(30000060000060000012000-21),(300003-4004300003000-4),(30000040000050000034000-43),
only the first three meet (C5), but the last one does not. And only the last three meet (C6), but the first one does not.
However, there exists some specially constructedA that meet none of (C1) to (C6), for example, A=diag[A1,A2,A3,A4], where
(42)A1=(-41-1-4),A2=(-43-3-4),A3=(43-34),A4=(41-14).
Nevertheless, a randomly generated real normal matrix can meet (C1) to (C6) with high probability.


### 2.4. Extension to Arbitrary Real Matrices

In this subsection,A is an arbitrary real matrix of order n. Let ∥A∥ be the Frobenius norm of A, and, λj,  j=1,…,n, be the eigenvalue of A. Denote the set of all complex nonsingular matrices by 𝒯.By the Schur inequality [13], we know
(43)∑j=1n‍|λj2|≤∥A∥2
with equality if and only if A is a normal matrix. Since the spectrum of A does not change by a similarity transformation, the inequality
(44)∑j=1n‍|λj2|≤∥P-1AP∥2,P∈𝒯
holds with equality if and only if P-1AP is a normal matrix. In addition, [24] proved
(45)∑i=1n|λi2|=infP∈𝒯∥P-1AP∥2.Based on (45), if we can find a sequence Aj as follows:
(46)A1=A,Aj+1=Pj-1AjPj,∥Aj+1∥<∥Aj∥,Pj∈𝒯,j=1,2,…
such that
(47)∑i=1n‍|λi2|=limj→∞∥Aj∥2,whereA∞=limj→∞Aj is to be a normal matrix with the same eigenvalues as A. Such skill, termed as the norm-reducing technique, has been proposed by [25–27]. Moreover, following the idea presented by [26], it is easy to find that when A is real, A∞ can be chosen to be a real normal matrix.In a word, any real matrixA can be translated into a real normal matrix A∞=P∞-1AP∞ by a similarity transformation P∞=P1P2⋯. Typical approaches for constructing Pj can be found in [26]. Note that if λ is the eigenvalue of A∞ corresponding to the eigenvector u, λ is the eigenvalue of A corresponding to the eigenvector P∞u. Hence, our proposed algorithm can be extended to extract eigenpairs of arbitrary real matrices by employing the norm-reducing technique.Without loss of generality, we use the following random matrixA as an example to describe the norm-reducing technique:
(48)A=(1.05410.4052-1.0191-0.57710.4158-1.9797-0.7025-1.3852-0.83640.0430-1.86741.49900.95490.8530-0.9489-1.83240.1378-0.60110.47730.54160.8486-1.5868-1.17190.3023-0.8211).The Frobenius norm ofA is ∥A∥=27.7629, and the eigenvalues of matrix A are λ1=2.8797, λ2=λ3¯=-0.6900+1.8770i, λ4=-1.5144, and λ5=0.9774; obviously ∥A∥>∑i=15‍λi2=19.5401, so A is a nonnormal real matrix. According to the approach that presented in [26], we can construct Pj, j=1,2,…. If the condition ∥Ai-Ai+1∥<10-10 can be satisfied, we break the iteration. After 129 iterations, we can obtain a approximate normal matrix A130:
(49)A130=(-0.74410.00071.63340.82100.0010-0.00352.8797-0.00380.0000-0.0000-1.4162-0.0048-0.91320.82230.0034-1.1560-0.0019-0.1181-1.2370-0.00140.0011-0.00000-0.00360.9774),
from which we have ∥A130*A130-A130A130*∥=0.00121231723027107 and ∥A130∥=19.540133042214 that is very close to ∑i=15‍λi2=19.5400526824864; so the matrix A130 can be regarded as a normal matrix in practical application, the corresponding P∞=P1P2P3⋯P129 are as follows:
(50)P∞=(-0.3499-0.5290-0.00900.2228-0.0151-0.4809-0.0259-0.6935-0.40790.1559-0.29140.72110.67430.03950.3437-0.44840.1475-0.36680.2444-0.87040.0611-0.32650.4294-1.1028-0.5166),
which satisfy A130=P∞-1AP∞. We also can see that the eigenvalues of A130 are λ1=-1.5144,  λ2=λ3¯=-0.6900+1.8770i, λ4=2.8797, and λ5=0.9774, which are just the eigenvalues of original nonnormal matrix A. We presented here the transient behavior of ∥Ai∥2 in Figure 1, where i=1,2,3,… is the iterations and A1=A.Figure 1
Trajectory of∥Ai∥2.In the following, we useN=1000 random matrices to verify the average performance of the norm-reducing technique. Let
(51)MeanΔi2=1N∑j=1N‍Δji2
denotes the average measure of a large number of nonnormal matrices in a statistical sense at ith iteration, where Δji2=∥AjiTAji-AjiAjiT∥2 is the measure of the nonnormal matrix Aj at ith iteration, Δji2=0 if and only if Aj is normal matrix at ith iteration. We also presented the dynamic behavior trajectory of MeanΔ2 in Figure 2, from which we can see that for most of nonnormal matrices, after 350 iterations, the MeanΔ2 is very close to zero.Figure 2
Average performance trajectory of norm-reducing technique.


## 2.1. Computing the Eigenvalues with the Largest or Smallest Imaginary Parts in Absolute Value, as well as the Corresponding Eigenvectors

Without loss of generality, in this subsection we assume thatb1=⋯=br=0<br+12≤⋯≤br+k2 in (7). Note that (A-AT)T(A-AT)=-(A-AT)2.Based on (7), we know that 0 (if any) is the eigenvalue of -(A-AT)2 corresponding to the eigenvector uj, j=1,…,r, and that 4br+s2, s=1,…,k, is the eigenvalue of -(A-AT)2 corresponding to the eigenvectors ur+sR or ur+sI. If replacing A with the symmetric matrix -(A-AT)2 in (2) as
(10)dx(t)dt=-x(t)Tx(t)(A-AT)2x(t)+x(t)T(A-AT)2x(t)x(t)
based on Lemma 2, we know that α=limt→∞x(t) is an eigenvector of -(A-AT)2 corresponding to 4br+k2 and 4br+k2=limt→∞(-x(t)T(A-AT)2x(t)/x(t)Tx(t)) (thus, |br+k| has been gotten), where x(t) is a solution of (10). If br+k=0, bj’s are all zero, meaning that A is symmetric. In this case, we can directly use (2) or (3) to get the largest or smallest eigenvalue of A and the corresponding eigenvector, respectively. Hence, we assume br+k≠0.The following lemma introduces an approach for computingar+k under the condition (C1).Lemma 5.
Assume that (C1) holds. Letα=limt→∞x(t), where x(t) is a solution of (10). Then, ar+k=αT(A+AT)α/2αTα. In addition, there exists ar+k2+br+k2=αT(AAT)α/αTα.Proof.
Assume that4br+j2,  j=s,s+1,…,k,  s≥1, are the largest eigenvalues of -(A-AT)2, that is; |br+s|=|br+s+1|=⋯=|br+k|. Therefore,
(11)𝒥1={r+s,…,r+k}.
Based on Lemma2 and (7), we know that α should be a linear combination of ujR and ujI,  j∈𝒥1. Let
(12)α=∑j∈𝒥1(γjujR+τjujI),γj,τj∈R.
In addition, by (8) we have
(13)(A+AT)ujR=2ajujR,(A+AT)ujI=2ajujI,j∈𝒥1.
And by (9), we have
(14)(AAT)ujR=(aj2+bj2)ujR,(AAT)ujI=(aj2+bj2)ujI,j∈𝒥1.
BecauseU is an orthogonal matrix and for all j∈𝒥1, aj=ar+k holds due to (C1), it is straightforward to verify
(15)αT(A+AT)α2αTα=2ar+k∑j∈𝒥1(γj2+τj2)2∑j∈𝒥1(γj2+τj2)=ar+k,αT(AAT)ααTα=(ar+k2+br+k2)∑j∈𝒥1(γj2+τj2)∑j∈𝒥1(γj2+τj2)=ar+k2+br+k2,
thus proving the lemma.Remark 6.
Based on Lemma5, if  ar+k2+br+k2≠αT(AAT)α/αTα, (C1) does not hold surely, which can be used to check whether (C1) holds or not.
The following lemma introduces an approach for computing a pair of conjugated eigenvectors ofA corresponding to the eigenvalues ar+k±i|br+k| under the condition (C1).Lemma 7.
Assume that (C1) holds. Given any nonzero4br+k2, the largest eigenvalue of -(A-AT)2, and the corresponding eigenvector α obtained by (10), let ar+k=αT(A+AT)α/2αTα and β=(A-AT)α/2|br+k|. Then, ar+k±i|br+k| are two eigenvalues of A corresponding to the eigenvectors β±iα, respectively.Proof.
Let𝒥1 and α take the form as (11) and (12), respectively. Based on (13), we can write
(16)(A+AT)2α=ar+kα.
Following the decomposition ofA as (5) and the definition of β, we have
(17)|br+k|β=(A-AT)α2=UTdiag[0,…,0︸r,Br+1,…,Br+k]Uα2,
where
(18)Bj={(02|bj|-2|bj|0),j∈{r+1,…,r+k}∖𝒥1(02|br+k|-2|br+k|0),j∈𝒥1.
Taking (12) into (17), we get
(19)β=∑j∈𝒥1‍(τjujR-γjujI).
Based on (13) and (19), it is straightforward to verify
(20)(A+AT)2β=ar+kβ.
In addition, since4br+k2 is the eigenvalue of -(A-AT)2 corresponding to the eigenvector α, we have -(A-AT)2α=4br+k2α. Hence, -(A-AT)|br+k|β=2br+k2α. Since br+k≠0, we have
(21)(A-AT)2β=-|br+k|α.
By (20) and (21), we have
(22)Aβ=A+AT2β+A-AT2β=ar+kβ-|br+k|α.
By (16) and (17), we have
(23)Aα=A+AT2α+A-AT2α=ar+kα+|br+k|β.
Then, it is straightforward to verify(24)A(β+iα)=(ar+k+i|br+k|)(β+iα),A(β-iα)=(ar+k-i|br+k|)(β-iα),
thus proving the lemma.To get|b1|, we can in advance get 4b12, the smallest eigenvalue of -(A-AT)2 by replacing A with -(A-AT)2 in (3) as follows:
(25)dx(t)dt=x(t)Tx(t)(A-AT)2x(t)-x(t)T(A-AT)2x(t)x(t).Letα*=limt→∞x(t), where x(t) is a solution of (25). From Lemma 3, we know 4b12=-(α*)T(A-AT)2α*/(α*)Tα*. Then, the following lemma similar to Lemma 5 can be used to compute a1.Lemma 8.
Assume that (C2) holds. Then,a1=(α*)T(A+AT)α*/2(α*)Tα*. In addition, there exists a12+b12=(α*)T(AAT)α*/(α*)Tα*.Proof.
The proof is almost the same to that in Lemma5.Note thatb1 may be zero; that is, -(A-AT)2 has real eigenvalues. In this case, we have the following lemma.Lemma 9.
Assume that (C2) holds and 0 is the smallest eigenvalue of-(A-AT)2 corresponding to the eigenvector α*. Then, a1 is the eigenvalue of A corresponding to the eigenvector α*, where a1=(α*)T(A+AT)α*/2(α*)Tα*.Proof.
Following the conditions, we have-(A-AT)2α*=0. Hence, (α*)T(A-AT)T(A-AT)α*=∥(A-AT)α*∥22=0; that is,
(26)(A-AT)α*=0.
Note thatb1=⋯=br=0,  r≥1, because 0 is the smallest eigenvalue of -(A-AT)2. Based on the definition of 𝒥2, we have
(27)a1=⋯=ar,𝒥2={1,…,r},r≥1.
Applying Lemma3 to (25), we know that α* should be a linear combination of u1,…,ur. Let
(28)α*=∑j∈𝒥2‍γjuj,γj∈R.
By (8), we have
(29)(A+AT)uj=2ajuj=2a1uj,j∈𝒥2.
Therefore,(30)A+AT2α*=12(∑j∈𝒥2γj(A+AT)uj)=a1α*.
Then, by (26) and (30), it is straightforward to verify
(31)Aα*=A+AT2α*+A-AT2α*=a1α*,
thus proving the lemma.In the case ofb1≠0, that is, all the eigenvalues of -(A-AT)2 are complex numbers, we have the following lemma similar to Lemma 7.Lemma 10.
Assume that (C2) holds. Given any nonzero4b12, the smallest eigenvalue of -(A-AT)2, and the corresponding eigenvector α* obtained by (25), let a1=(α*)T(A+AT)α*/2(α*)Tα* and β*=(A-AT)α*/2|b1|. Then, a1±i|b1| are two eigenvalues of A corresponding to the eigenvectors β*±iα*, respectively.Proof.
The proof is almost the same to that in Lemma7.Remark 11.
Among the following four real normal matrices(32)(70000050000050000062000-26),(70000051000-150000062000-26),(7000005-2000250000052000-25),(70000052000-250000062000-26),
only the first three meet (C1), but the last one does not. And only the last three matrices meet (C2), but the first one does not.


## 2.2. Computing the Eigenvalues with the Largest or Smallest Real Parts, as well as the Corresponding Eigenvectors

As shown in (8), 2aj,  j=1,…,r+k, are the eigenvalues of the symmetric matrix (A+AT). In this subsection, we assume that a1≤⋯≤ar and ar+1≤⋯≤ar+k, which can be achieved by reordering aj and the corresponding columns of U.If replacingA with (A+AT) in (2), we get
(33)dx(t)dt=x(t)Tx(t)(A+AT)x(t)-x(t)T(A+AT)x(t)x(t).Without loss of generality, assume thatar+k is the largest real part of the eigenvalues of A (it may be ar. However, Lemmas 12 and 13 have no difference in that case). Applying Lemma 2 to (33), we can get 2ar+k=limt→∞(x(t)T(A+AT)x(t)/x(t)Tx(t)), the largest eigenvalue of (A+AT), and the corresponding eigenvector ζ=limt→∞x(t), where x(t) is a solution of (33).The following lemma introduces an approach for computing|br+k| under the condition (C3).Lemma 12.
Assume that (C3) holds. Letζ=limt→∞x(t), where x(t) is a solution of (33). Then, |br+k|=(ζT(AAT)ζ/ζTζ)-ar+k2.Proof.
By Lemma2, we know that ζ should be a linear combination of ujR and ujI,  j∈𝒥3. Let
(34)ζ=∑j∈𝒥3‍(γjujR+τjujI),γj,τj∈R.
Then, based on (9), we get
(35)(AAT)ujR=(aj2+bj2)ujR,(AAT)ujI=(aj2+bj2)ujI,j∈𝒥3.
BecauseU is an orthogonal matrix and for all j∈𝒥3,  aj=ar+k and |bj|=|br+k| hold, that is, aj2+bj2=ar+k2+br+k2 for all j∈𝒥3, we have
(36)ζT(AAT)ζζTζ=(ar+k2+br+k2)∑j∈𝒥3(γj2+τj2)∑j∈𝒥3(γj2+τj2)=ar+k2+br+k2,
thus proving the lemma.The following lemma introduces an approach for computing a pair of conjugated eigenvectors ofA corresponding to the eigenvalues ar+k±i|br+k| under the condition (C3).Lemma 13.
Assume that (C3) holds. Given any nonzero2ar+k, the largest eigenvalue of (A+AT), and the corresponding eigenvector ζ obtained by (33), let |br+k|=(ζT(AAT)ζ/ζTζ)-ar+k2. If br+k=0, ar+k is an eigenvalue of A corresponding to the eigenvector ζ. If br+k≠0, let η=(A-AT)ζ/2|br+k|. Then, ar+k±i|br+k| are two eigenvalues of A corresponding to the eigenvectors η±iζ, respectively.Proof.
Combining the proofs of Lemmas9 and 7, we can prove this lemma.If replacingA with (A+AT) in (3), we get
(37)dx(t)dt=-x(t)Tx(t)(A+AT)x(t)+x(t)T(A+AT)x(t)x(t).Without loss of generality, assume thatar+1 is the smallest real part of the eigenvalues of A (it may be a1. However, Lemma 14 has no difference in that case). Applying Lemma 3 to (37), we can obtain 2ar+1, the smallest eigenvalue of (A+AT), as well as the corresponding eigenvector, denoted by ζ*. Then, we have the following lemma.Lemma 14.
Assume that (C4) holds. Given any nonzero2ar+1, the smallest eigenvalue of (A+AT), and the corresponding eigenvector ζ* obtained by (37), let |br+1|=((ζ*)T(AAT)ζ*/(ζ*)Tζ*)-ar+12. If br+1=0, ar+1 is the eigenvalue of A corresponding to the eigenvector ζ*. If br+1≠0, let η*=(A-AT)ζ*/2|br+1|. Then, ar+1±i|br+1| are two eigenvalues of A corresponding to the eigenvectors η*±iζ*, respectively.Proof.
Combining the proofs of Lemmas12, 9, and 7, we can prove this lemma.Remark 15.
Among the following four real normal matrices(38)(10000040000050000012000-21),(30000040000040000012000-21),(30000042000-240000042000-24),(30000040000050000052000-25),
only the first three meet (C3), but the last one does not. And only the last three matrices meet (C4), but the first one does not.


## 2.3. Computing the Eigenvalues with the Largest or Smallest Modulus, as well as the Corresponding Eigenvectors

Reorder the eigenvalues of the symmetric matrixAAT in (9) and the corresponding columns of U such that a12≤⋯≤ar2 and ar+12+br+12≤⋯≤ar+k2+br+k2. Without loss of generality, assume that ar+k±i|br+k| are the eigenvalues of A that have the largest modulus (it may be ar. However, Lemma 16 has no difference in that case), and that ar+1±i|br+1| are the eigenvalues of A that have the smallest modulus (it may be a1. However, Lemma 17 has no difference in that case).ReplacingA with AAT in (2), we get
(39)dx(t)dt=x(t)Tx(t)AATx(t)-x(t)TAATx(t)x(t).Applying Lemma2 to (39), we can obtain ar+k2+br+k2=limt→∞(x(t)TATAx(t)/x(t)Tx(t)), the largest eigenvalue of AAT, and the corresponding eigenvector θ=limt→∞x(t), where x(t) is the solution of (39). Then, we have the following lemma.Lemma 16.
Assume that (C5) holds. Given anyar+k2+br+k2, the largest eigenvalue of AAT, and the corresponding eigenvector θ obtained by (39). Then, ar+k=θT(A+AT)θ/2θTθ. Thus, |br+k| can be gotten. If br+k=0, ar+k is an eigenvalue of A corresponding to the eigenvector θ. If br+k≠0, let ω=(A-AT)θ/|2br+k|. Then, ar+k±i|br+k| are two eigenvalues of A corresponding to the eigenvectors ω±iθ, respectively.Proof.
Combining the proofs of Lemmas5, 7, and 9, we can prove this lemma.ReplacingA with AAT in (3), we get
(40)dx(t)dt=-x(t)Tx(t)AATx(t)+x(t)TAATx(t)x(t).Applying Lemma3 to (40), we can obtain ar+12+br+12, the smallest eigenvalue of AAT, and the corresponding eigenvector, denoted by θ*. Then, we have the following lemma.Lemma 17.
Assume (C6) holds. Given anyar+12+br+12, the smallest eigenvalue of AAT, and the corresponding eigenvector θ* obtained by (40). Then, ar+1=(θ*)T(A+AT)θ*/2(θ*)Tθ*. So one can get that |br+1|. If br+1=0, ar+1 is an eigenvalue of A corresponding to the eigenvector θ*. If br+1≠0, let ω*=(A-AT)θ*/2|br+1|. Then, ar+1±i|br+1| are two eigenvalues of A corresponding to the eigenvectors ω*±iθ*, respectively.Proof.
Combining the proofs of Lemmas5, 7, and 9, we can prove this lemma.Remark 18.
Among the following four real normal matrices(41)(200000-20000060000012000-21),(30000060000060000012000-21),(300003-4004300003000-4),(30000040000050000034000-43),
only the first three meet (C5), but the last one does not. And only the last three meet (C6), but the first one does not.
However, there exists some specially constructedA that meet none of (C1) to (C6), for example, A=diag[A1,A2,A3,A4], where
(42)A1=(-41-1-4),A2=(-43-3-4),A3=(43-34),A4=(41-14).
Nevertheless, a randomly generated real normal matrix can meet (C1) to (C6) with high probability.


## 2.4. Extension to Arbitrary Real Matrices

In this subsection,A is an arbitrary real matrix of order n. Let ∥A∥ be the Frobenius norm of A, and, λj,  j=1,…,n, be the eigenvalue of A. Denote the set of all complex nonsingular matrices by 𝒯.By the Schur inequality [13], we know
(43)∑j=1n‍|λj2|≤∥A∥2
with equality if and only if A is a normal matrix. Since the spectrum of A does not change by a similarity transformation, the inequality
(44)∑j=1n‍|λj2|≤∥P-1AP∥2,P∈𝒯
holds with equality if and only if P-1AP is a normal matrix. In addition, [24] proved
(45)∑i=1n|λi2|=infP∈𝒯∥P-1AP∥2.Based on (45), if we can find a sequence Aj as follows:
(46)A1=A,Aj+1=Pj-1AjPj,∥Aj+1∥<∥Aj∥,Pj∈𝒯,j=1,2,…
such that
(47)∑i=1n‍|λi2|=limj→∞∥Aj∥2,whereA∞=limj→∞Aj is to be a normal matrix with the same eigenvalues as A. Such skill, termed as the norm-reducing technique, has been proposed by [25–27]. Moreover, following the idea presented by [26], it is easy to find that when A is real, A∞ can be chosen to be a real normal matrix.In a word, any real matrixA can be translated into a real normal matrix A∞=P∞-1AP∞ by a similarity transformation P∞=P1P2⋯. Typical approaches for constructing Pj can be found in [26]. Note that if λ is the eigenvalue of A∞ corresponding to the eigenvector u, λ is the eigenvalue of A corresponding to the eigenvector P∞u. Hence, our proposed algorithm can be extended to extract eigenpairs of arbitrary real matrices by employing the norm-reducing technique.Without loss of generality, we use the following random matrixA as an example to describe the norm-reducing technique:
(48)A=(1.05410.4052-1.0191-0.57710.4158-1.9797-0.7025-1.3852-0.83640.0430-1.86741.49900.95490.8530-0.9489-1.83240.1378-0.60110.47730.54160.8486-1.5868-1.17190.3023-0.8211).The Frobenius norm ofA is ∥A∥=27.7629, and the eigenvalues of matrix A are λ1=2.8797, λ2=λ3¯=-0.6900+1.8770i, λ4=-1.5144, and λ5=0.9774; obviously ∥A∥>∑i=15‍λi2=19.5401, so A is a nonnormal real matrix. According to the approach that presented in [26], we can construct Pj, j=1,2,…. If the condition ∥Ai-Ai+1∥<10-10 can be satisfied, we break the iteration. After 129 iterations, we can obtain a approximate normal matrix A130:
(49)A130=(-0.74410.00071.63340.82100.0010-0.00352.8797-0.00380.0000-0.0000-1.4162-0.0048-0.91320.82230.0034-1.1560-0.0019-0.1181-1.2370-0.00140.0011-0.00000-0.00360.9774),
from which we have ∥A130*A130-A130A130*∥=0.00121231723027107 and ∥A130∥=19.540133042214 that is very close to ∑i=15‍λi2=19.5400526824864; so the matrix A130 can be regarded as a normal matrix in practical application, the corresponding P∞=P1P2P3⋯P129 are as follows:
(50)P∞=(-0.3499-0.5290-0.00900.2228-0.0151-0.4809-0.0259-0.6935-0.40790.1559-0.29140.72110.67430.03950.3437-0.44840.1475-0.36680.2444-0.87040.0611-0.32650.4294-1.1028-0.5166),
which satisfy A130=P∞-1AP∞. We also can see that the eigenvalues of A130 are λ1=-1.5144,  λ2=λ3¯=-0.6900+1.8770i, λ4=2.8797, and λ5=0.9774, which are just the eigenvalues of original nonnormal matrix A. We presented here the transient behavior of ∥Ai∥2 in Figure 1, where i=1,2,3,… is the iterations and A1=A.Figure 1
Trajectory of∥Ai∥2.In the following, we useN=1000 random matrices to verify the average performance of the norm-reducing technique. Let
(51)MeanΔi2=1N∑j=1N‍Δji2
denotes the average measure of a large number of nonnormal matrices in a statistical sense at ith iteration, where Δji2=∥AjiTAji-AjiAjiT∥2 is the measure of the nonnormal matrix Aj at ith iteration, Δji2=0 if and only if Aj is normal matrix at ith iteration. We also presented the dynamic behavior trajectory of MeanΔ2 in Figure 2, from which we can see that for most of nonnormal matrices, after 350 iterations, the MeanΔ2 is very close to zero.Figure 2
Average performance trajectory of norm-reducing technique.


## 3. Neural Implementation Description

In the presented paper, we mainly focus on the classical neural network differential equation as shown in (2), where A=(aij), i,j=1,2,…,n are symmetric matrices that need to calculate eigenvalues and the corresponding eigenvectors, x(t)=[x1(t),x2(t),…,xn(t)]T is a column vector which denotes the states of neurons in the neural network dynamic system, and the elements of symmetric matrix A denote the connection weights between those neurons. We presented the schematic diagram of the neural network in Figure 3, from which we can see that it is a recurrent neural network since the input is just the output of the system.Figure 3
The schematic diagram of the neural network equation (2) for solving the eigenvector corresponding to the modulus largest eigenvalues of real symmetric matrix A.In the practical applications, we often only need a nonzero column vectorx(0)=[x1(0),x2(0),…,xn(0)]T to start the neural network system by the following update rule:
(52)x(j+1)=x(j)+τ(x(j)Tx(j)Ax(j)-x(j)TAx(j)x(j)),
where j denote the jth iteration and τ is a small time step. The iteration stops once ∥x(j+1)-x(j)∥<ε, where ε is a small constraint error that can be set in advance. If ∥x(j+1)-x(j)∥<ε, we could regard that ∥x(j+1)-x(j)∥=0, that is, x(j)Tx(j)Ax(j)-x(j)TAx(j)x(j)=0; so we have Ax(j)=(x(j)TAx(j)/x(j)Tx(j))x(j), according to the theory in [23], x(j) is the eigenvector corresponding to the modulus largest eigenvalue which can be denoted as x(j)TAx(j)/x(j)Tx(j).


## 4. Examples and Discussion

Three experiments are presented to verify our results. The following real normal matrixA (randomly generated) was used in those three experiments:
(53)A=(1.8818-2.1563-0.5979-0.47680.72700.4133-1.42670.52581.49460.71250.47002.53840.42730.77842.86450.85871.3792-1.0153-0.19291.3273-1.31253.16290.25730.63291.5966-0.6593-0.37990.20582.98450.64041.06271.9973-0.0249-1.57270.92892.3259).Using the[V,D]=eig(A) function in Matlab, we got the eigenvalues of A as λ1=-2.3420, λ2=1.005, λ3=λ4¯=3.1967+1.7860i, and λ5=λ6¯=4.3445+1.2532i, as well as the corresponding eigenvectors u1,…,u6 as follows:
(54)u1=(0.37540.7510-0.2146-0.16640.0280-0.4696),u2=(0.6657-0.02420.29940.2594-0.58880.2295),u3=(-0.0294+0.1297i-0.0243-0.1385i0.6165-0.1917+0.5472i0.0913+0.2785i-0.2708-0.2950i),u5=(0.3207+0.2956i0.0472-0.4424i0.0711-0.2167i-0.1146-0.3213i0.49030.3692-0.2583i),u4=u¯3,u6=u¯5.We can see that all of (C1) to (C6) hold except (C2). For simplicity, denotelimt→∞x(t) by x(∞).Example 19 (for Section2.1).
We used (10) with the following initial condition (randomly generalized)
(55)x(0)=[0.2329,-0.0023,-0.8601,-0.2452,-0.1761,-0.1577]T
to get |b3| (the largest absolute value of the imaginary part among λi) that -x(t)T(A-AT)2x(t)/4x(t)Tx(t) should converge to. From Lemma 5, x(t)T(A+AT)x(t)/2x(t)Tx(t)→a3 should hold. By Lemma 7, x(t) and (A-AT)x(t)/2|b3| should converge to the imaginary and real parts of an eigenvector corresponding to λ3, respectively. The transient behaviors of the above four variables are shown in Figures 4, 5, and 6, respectively. After convergence, we saw
(56)(A-AT)x(∞)2|b3|+x(∞)i=(0.1795-0.0005i-0.1747+0.0737i-0.1857-0.8107i0.7772+0.0872i0.3386-0.2039i-0.3064+0.4450i)=(-0.3012-1.3149i)u3.
Thus, the estimated complex vector is an eigenvector ofA corresponding to λ3.
Although we can use (25) to get the smallest absolute value of the imaginary part among λi (it is zero in this experiment), neither the corresponding real part nor the eigenvector can be obtained from Lemmas 8 or 9 since (C2) does not hold.Figure 4
Trajectories ofϕ(t)=-x(t)T(A-AT)2x(t)/4x(t)Tx(t) and ψ(t)=x(t)T(A+AT)x(t)/2x(t)Tx(t) based on (10) with initial x(0) as (55), which should converge to |b3| and a3, respectively.Figure 5
Trajectories ofx(t), the solution of (10) with initial x(0) as (55), which should converge to the imaginary part of an eigenvector of A corresponding to the eigenvalue λ3.Figure 6
Trajectories ofy(t)=(A-AT)x(t)/2b3 based on (10) with initial x(0) as (55), which should converge to the real part of an eigenvector of A corresponding to the eigenvalue λ3.Example 20 (for Section2.2).
We used (33) with the same x(0) as (55) to get a5 (the largest real part among λi) that x(t)T(A+AT)x(t)/2x(t)Tx(t) should converge to. Based on Lemma 12, (x(t)T(AAT)x(t)/x(t)Tx(t))-a52→|b5| should hold. The transient behaviors of such two variables are shown in Figure 7. Since b5≠0, from Lemma 13, x(t) and (A-AT)x(t)/2|b5| should converge to the imaginary and real parts of an eigenvector corresponding to λ5, as shown in Figures 8 and 9. After convergence, we saw
(57)(A-AT)x(∞)2|b5|+x(∞)i=(-0.5695-0.1474i0.2254+0.5561i0.0534+0.3029i0.3407+0.3091i-0.5830+0.3120i-0.2746+0.5420i)=(-1.1890+0.6363i)u5.
Hence, the estimated complex vector is an eigenvector ofA corresponding to λ5.
Based on (37), we got a1 (the smallest real part among λi). After convergence, we saw that x(∞)T(A+AT)x(∞)/2x(∞)Tx(∞)=a1=λ1=-2.3420, (x(∞)T(AAT)x(∞)/x(∞)Tx(∞))-a12=|b1|=0, and x(∞) was equal to
(58)x(∞)=(-0.3581-0.71650.20480.1588-0.02670.4480)=-0.9541u1,
just as expected from Lemma 14.Figure 7
Trajectories ofψ(t)=x(t)T(A+AT)x(t)/2x(t)Tx(t) and ϕ(t)=(x(t)T(AAT)x(t)/x(t)Tx(t))-a52 based on (33) with initial x(0) as (55), which should converge to a5 and |b5|, respectively.Figure 8
Trajectories ofx(t), the solution of (33) with initial x(0) as (55), which should converge to the imaginary part of an eigenvector of A corresponding to the eigenvalue λ5.Figure 9
Trajectories ofy(t)=(A-AT)x(t)/2|b5| based on (33) with initial x(0) as (55), which should converge to the real part of an eigenvector of A corresponding to the eigenvalue λ5.Example 21 (for Section2.3).
Based on (39) and Lemma 16, we can get λ5 and one corresponding eigenvector again because |λ5| is the largest modulus of λi. In addition, we used (40) with the same x(0) as (55) to get |λ2|, the smallest modulus of λi. By Lemma 17, a2=x(∞)T(A+AT)x(∞)/2x(∞)Tx(∞) and |b2|=(x(∞)T(AAT)x(∞)/x(∞)Tx(∞))-a22 should hold. The transient behaviors of such two variables are shown in Figure 10. After convergence, we saw that b2=0. Hence, from Lemma 17, x(t) should converge to a constant multiple of u2, which is shown in Figure 11. As expected, x(∞) was equal to
(59)x(∞)=(-0.63370.0230-0.2850-0.24690.5605-0.2185)=-0.9519u2.Figure 10
Trajectories ofψ(t)=x(t)T(A+AT)x(t)/2x(t)Tx(t) and ϕ(t)=(x(t)T(AAT)x(t)/x(t)Tx(t))-a22 based on (40) with initial x(0) as (55), which should converge to a2 and |b2|, respectively.Figure 11
Trajectories ofx(t), the solution of (40), with initial x(0) as (55), which should converge to an eigenvector of A corresponding to the eigenvalue λ2.Example 22 (for extension to arbitrary real matrices).
According to the theory in Section2.1, we present here an experiment for arbitrary real matrices to verify the effectiveness of the norm-reducing technique in Section 2.4. Considering the following nonnormal matrix A:
(60)A=(3.4633-3.74872.9281-0.5106-0.04554.48603.4259-3.20105.06251.21200.5268-1.03111.5386-0.4766-1.4478-2.47681.25852.05252.3539-0.84870.0498-3.1267-6.5297-1.51062.2083),
the Frobenius norm of A is ∥A∥=187.8869>∑i=15‍λi2=97.1780, where λi,  i=1,2,…,5 are the eigenvalues of matrix A, so matrix A is a nonnormal matrix. According to the theory in Section 2.4, the eigenpairs problem of nonnormal real matrix can be converted into the eigenpairs problem of the corresponding normal matrix Ap, which can be calculated by norm-reducing technique:
(61)Ap=(2.34380.79151.10711.8624-2.6867-1.59182.10161.72492.20671.98430.8659-1.74344.5103-1.27080.04081.7777-2.3015-0.98983.58150.7201-2.4688-2.3136-0.47460.05450.4528),
and the corresponding P∞ as follows:
(62)P∞=(28.2974-28.8442115.669816.476513.5575-35.968948.54754.346558.13849.0642-10.2373-14.297614.116823.6416-40.2882-3.4360-25.4122-114.241715.019115.1720-79.1532-35.346575.4347-24.8455-48.2893),
which satisfies the relationship Ap=P∞-1AP∞. In order to solve the eigenpairs of matrix A, we have to calculate the eigenpairs of the matrix Ap at first. Direct calculations of the eigenvalues of matrix Ap are λ1=-1.8342, λ2=λ3¯=2.1114+3.786i,  λ4=λ5¯=5.3007+0.1328i and those are just the eigenvalues of nonnormal matrix A and the corresponding eigenvectors as follows:
(63)u1=(0.60790.0886-0.1195-0.28050.7278),u2=(0.0035-0.2240i0.70200.0189+0.3280i0.0502+0.4231i-0.0660+0.4040i)=u3¯,u4=(0.1769-0.4835i-0.0129+0.0561i0.6204-0.2522-0.4643i-0.1415+0.2181i)=u5¯.
The eigenvectors of nonnormal matrixA, which corresponding to the eigenvalues with largest imaginary parts in absolute are
(64)u2A=(-0.2353-0.5331i-0.64160.0449-0.0581i0.3856+0.0270i0.0747-0.2973i)=u3A¯.
According to the results above, the eigenvalues with largest imaginary in absolute areλ2 and λ3, and let a2±|b2|i denote them.
We used (10) with the following initial condition (randomly generalized):
(65)x(0)=[0.3891,0.0856,0.2215,0.3151,0.4929]T
to get |b2| (the largest absolute value of the imaginary part among λi) that -x(t)T(Ap-ApT)2x(t)/4x(t)Tx(t) should converge to. From Lemma 5, x(t)T(Ap+ApT)x(t)/2x(t)Tx(t)→a2 should hold. By Lemma 7, x(t) and (Ap-ApT)x(t)/2|b2| should converge to the imaginary and real parts of an eigenvector corresponding to λ2, respectively. The transient behaviors of the above four variables are shown in Figures 12, 13, and 14, respectively.
After convergence, we saw(66)(Ap-ApT)x(∞)2|b2|+x(∞)i=(0.0394-0.2317i0.7277+0.1121i-0.0327+0.3430i-0.0155+0.4466i-0.1329+0.4083i)=(1.0367+0.1597i)u2.
Thus, the estimated complex vector is an eigenvector ofAp corresponding to λ2. According to the theory in Section 2.4, the estimated complex eigenvector of nonnormal matrix A corresponding to the eigenvalue λ2 should be
(67)P∞((Ap-ApT)x(∞)2|b2|+x(∞)i)=(-25.7189+42.7862i31.6606+44.9313i-6.2833-0.2807i-17.1372-28.3402i-24.5113+9.4386i)=(-49.3476-70.0318i)u2A,
from which we can see that the estimated complex vector is an eigenvector of A corresponding to λ2.Figure 12
Trajectories ofϕ(t)=-x(t)T(Ap-ApT)2x(t)/4x(t)Tx(t) and ψ(t)=x(t)T(Ap+ApT)x(t)/2x(t)Tx(t) based on (10) with initial x(0) as (65), which should converge to |b2| and a2, respectively.Figure 13
Trajectories ofx(t), the solution of (10) with initial x(0) as (65), which should converge to the imaginary part of an eigenvector of Ap corresponding to the eigenvalue λ2.Figure 14
Trajectories ofy(t)=(Ap-ApT)x(t)/2b2 based on (10) with initial x(0) as (65), which should converge to the real part of an eigenvector of Ap corresponding to the eigenvalue λ2.


## 5. Conclusion

This paper introduces a neural network based approach for computing eigenvectors of real normal matrices and the corresponding eigenvalues that have the largest or smallest modulus, have the largest or smallest real part, and have the largest or smallest imaginary part in absolute value. All the computation can be carried out in real vector space although eigenpairs may be complex, which can reduce the scale of networks a lot. We also shed light on extending this method to the case of general real matrices by employing the norm-reducing technique proposed in other literatures. Four simulation examples verified the validity of our proposed algorithm.



---

*Source: 597628-2013-03-20.xml*