Two-stage acceleration for non-linear
PCA
Masahiro Kuroda, Okayama University of Science, kuroda@soci.ous.ac.jp
Michio Sakakihara, Okayama University of Science, sakaki@mis.ous.ac.jp
Yuichi Mori, Okayama University of Science, mori@soci.ous.ac.jp
Masaya Iizuka, Okayama University, iizuka@ems.okayama-u.ac.jp

Abstract.
Principal components analysis (PCA) is a descriptive multivariate method foranalyzing quantitative data. For PCA of a mixture of quantitative and qualitative data, quantiﬁcation of qualitative data requires obtaining optimal scaling data and using ordinary PCA.
The extended PCA, including such quantiﬁcation, is called non-linear PCA. Then, the alternating least squares (ALS) algorithm is used as the quantiﬁcation method. However, the ALS
algorithm for non-linear PCA of large data requires many iterations and much computation
time due to its linear convergence. We provide a new acceleration method for the ALS algorithm using the vector ε (vε) and Graves-Morris (GM) algorithms. Both acceleration algorithms
speed up the convergence of a linearly convergent sequence generated by the ALS algorithm.
Acceleration of the ALS algorithm can be performed in two stages: 1) the vε algorithm generates an accelerated sequence of the ALS sequence and 2) the convergence of the vε accelerated
sequence is accelerated using the GM algorithm. Thus, we expect that, by accelerating theconvergence of the vε accelerated sequence, the GM algorithm improves the overall computational eﬃciency of the ALS algorithm. Numerical experiments examine the performance of thetwo-stage acceleration for non-linear PCA.
Keywords. non-linear PCA, alternating least squares algorithm, acceleration of convergence,
vector ε algorithm, Graves-Morris algorithm

1

Introduction

Principal components analysis (PCA) is a descriptive multivariate method commonly used foranalyzing quantitative data. For PCA of a mixture of quantitative and qualitative data, quantiﬁcation of qualitative data requires obtaining optimal scaling data and using ordinary PCA.
The extended PCA, including such quantiﬁcation, is called non-linear PCA, see Giﬁ [4]. The
existing algorithms for non-linear PCA are PRINCIPALS of Young et al. [9] and PRINCALS
of Giﬁ [4], in which the alternating least squares (ALS) algorithm is utilized. This algorithm

2

Two-stage acceleration for non-linear PCA

alternates between quantiﬁcation of qualitative data for optimal scaling and computation ofordinary PCA of optimal scaling data.
Application of non-linear PCA to very large data sets and variable selection problems requires
numerous iterations and large computation time for convergence of the ALS algorithm, because
the algorithm’s convergence speed is linear. For example, in PCA based on a subset of variables
for qualitative data of Mori et al. [7], the ALS algorithm requires a much larger number of
iterations and longer computation time to search for a reasonable subset. For an iterativealgorithm that generates a linearly convergent sequence such as the ALS algorithm, there exist
several convergence acceleration algorithms, see Brezinski and Zaglia [3]. Kuroda et al. [6]proposed an acceleration algorithm for the convergence of the ALS sequence using the vector
ε (vε) algorithm of Wynn [10]. During iterations of the vε accelerated ALS algorithm, thevε algorithm generates an accelerated sequence of optimal scaling data estimated by the ALS
algorithm. Numerical experiments demonstrated that the vε acceleration greatly speeds up theconvergence of the ALS sequence of the estimated optimal scaling data.
In this paper, we provide a new acceleration method for the ALS algorithm using the vε andGraves-Morris (GM) algorithms [5]. Both algorithms accelerate the convergence of a linearly
convergent sequence. Acceleration of the ALS algorithm can be performed in two stages: theﬁrst stage using the vε algorithm generates an accelerated sequence of the ALS sequence and thesecond stage accelerates the convergence of the vε accelerated sequence using the GM algorithm.
The paper is organized as follows. We brieﬂy describe the ALS algorithm for non-linear
PCA and introduce the vε acceleration for the ALS algorithm proposed by Kuroda et al. [6]in Section 2. Section 3 gives the GM algorithm and its convergence properties. The two-stage
acceleration for the ALS algorithm is described in Section 4. Numerical experiments in Section
5 examine the performance and properties of the two-stage acceleration for PRINCIPALS. In
Section 6, we present our concluding remarks.

2

ALS algorithm for non-linear PCA and vε acceleration

Computation of non-linear PCA
Let X = (X1 X2 · · · Xp ) be an n × p matrix of n observations on p quantitative variables andbe columnwise standardized. In PCA, X is linearly transformed into a substantially smaller setof uncorrelated variables and are approximated by the following bilinear form:
X̂ = ZA ,(1)

where Z = (Z1 Z2 · · · Zr ) is an n×r matrix of n component scores on r (1 ≤ r ≤ p) components,
and A = (A1 A2 · · · Ar ) is a p × r matrix consisting of the eigenvectors of X X/n andA A = Ir . Then, we determine model parameters Z and A such that
θ = tr(X − X̂) (X − X̂) = tr(X − ZA ) (X − ZA )(2)

is minimized for the prescribed r components.
For no-linear PCA with optimal scaling, quantiﬁcation of qualitative data requires obtaining
optimal scaling data and using ordinary PCA. To quantify Xj of qualitative variable j with
Kj categories, the vector is coded using an n × Kj indicator matrix Gj with entries g(j)ik = 1
if object i belongs to category k, and g(j)ik = 0 if object i belongs to some other category

COMPSTAT 2012 Proceedings

Masahiro Kuroda, Michio Sakakihara, Yuichi Mori and Masaya Iizuka

3k  (= k), i = 1, . . . , n and k = 1, . . . , Kj . Then, the optimally scaled vector X∗j of Xj is given byX∗j = Gj αj , where αj is a Kj × 1 score vector for categories of Xj . Let X∗ = (X∗1 X∗2 · · · X∗p )be an n × p matrix of optimally scaled observations to satisfy restrictionsX∗X∗ X∗= Ip ,diagn

1n = 0pand(3)where 1n and 0p are vectors of ones and zeros of length n and p, respectively. In the presenceof qualitative variables, the optimization criterion (2) is replaced byθ∗ = tr(X∗ − X̂) (X∗ − X̂) = tr(X∗ − ZA ) (X∗ − ZA ).

(4)

In non-linear PCA, we determine the optimal scaling parameter X∗ , in addition to estimating
Z and A.

ALS algorithm for non-linear PCA: PRINCIPALS of Young et al.
The ALS algorithm for non-linear PCA alternates between ordinary PCA and optimal scaling,
and minimizes θ∗ of Equation (4) under restriction (3). Then, θ∗ is to be determined by model
parameters Z and A and optimal scaling parameter X∗ , by updating each of the parameters inturn, keeping the others ﬁxed. We use PRINCIPALS of Young et al. [9] as the ALS algorithm
for non-linear PCA.
For the initialization, the observed data X may be used as initial data X∗(0) after it isstandardized to satisfy restriction (3). For given initial data X∗(0) , PRINCIPALS iterates thefollowing steps:
Algorithm 2.1 (PRINCIPALS).
Step1 Model parameter estimation step: Obtain A(t) by solvingX∗(t) X∗(t)
A = ADr ,n(5)where A A = Ir and Dr is an r × r diagonal matrix of eigenvalues, and the superscript(t) indicates the t-th iteration. Compute Z(t) from Z(t) = X∗(t) A(t) .
Step2 Optimal scaling step: Calculate X̂(t+1) = Z(t) A(t) from Equation (1). Find X∗(t+1)
such thatX∗(t+1) = arg mintr(X∗ − X̂(t+1) ) (X∗ − X̂(t+1) )∗X

for ﬁxed X̂(t+1) under measurement restrictions on each of the variables. Scale X∗(t+1) bycolumnwise centering and normalizing.

@ COMPSTAT 2012

4

Two-stage acceleration for non-linear PCAvε acceleration for PRINCIPALS
The vε algorithm of Wynn [10] is a method for accelerating the convergence of a slowly convergent vector sequence and works eﬀective for linearly convergent sequences. It is known that the
maximum speed of convergence of the vε algorithm is superlinear.
Let {Y(t) }t≥0 = {Y(0) , Y(1) , Y(2) , . . . } be a linearly convergent sequence generated by an
iterative computational procedure and let {Ẏ(t) }t≥0 = {Ẏ(0) , Ẏ(1) , Ẏ(2) , . . . } be the accelerated
sequence of {Y(t) }t≥0 . We denote Y(t) = Y(t+1) − Y(t) and deﬁne the inverse of vector Y
by [Y]−1 = Y /Y, Y , where ·, · is the inner product of vectors. Then, the vε algorithm
generates {Ẏ(t) }t≥0 by using
−1 
−1 −1
(t−1)
(t)
(t)
(t−1)
= Y + Y
− Y
.
(6)
Ẏ
When {Y(t) }t≥0 converges to a limit point Y(∞) of {Y(t) }t≥0 , it is known that, in many cases,{Ẏ(t) }t≥0 generated by the vε algorithm converges to Y(∞) faster than {Y(t) }t≥0 .
Kuroda et al. [6] proposed the vε acceleration for the ALS algorithm for non-linear PCA.
The vε acceleration algorithm speeds up the convergence of the ALS sequence. Numerical experiments demonstrated that its speed of convergence is signiﬁcantly faster than that of the ALS
algorithm. We assume that {X∗(t) }t≥0 generated by PRINCIPALS converges to a limit point
X∗(∞) . Then, vε accelerated PRINCIPALS (vε-PRINCIPALS) produces a fast convergent sequence {Ẋ∗(t) }t≥0 of {X∗(t) }t≥0 . The procedure of vε-PRINCIPALS of Kuroda et al. [6] iterates
the following two steps:
Algorithm 2.2 (vε-PRINCIPALS).
Step1 PRINCIPALS step: Compute model parameters A(t) and Z(t) and determine optimal
scaling parameter X∗(t+1) .
Step2 vε acceleration step: Calculate Ẋ∗(t−1) using {X∗(t−1) , X∗(t) , X∗(t+1) } from the vε algorithm:
−1 
−1 −1
vecẊ∗(t−1) = vecX∗(t) + vecX∗(t)− vecX∗(t−1)
,
∗
∗ 
where vecX∗ = (X∗
1 X2 · · · Xp ) and check the convergence by2vecẊ∗(t−2)  < δ,

where δ is a desired accuracy.
Before starting of the iteration of vε-PRINCIPALS, we perform the PRINCIPALS step twice
to obtain X∗(0) and X∗(1) . When {Ẋ∗(t) }t≥0 generated by vε-PRINCIPALS converges to X∗(∞) ,the estimate of X∗ can be obtained from the ﬁnal value of {Ẋ∗(t) }t≥0 . The estimates of Z and A
can then be calculated immediately from the estimate of X∗ in the Model parameter estimation
step of PRINCIPALS.
Note that Ẋ∗(t−1) obtained in the vε acceleration step is not used as the estimate X∗(t+1) atthe (t + 1)-th iteration of the PRINCIPALS step. Thus, vε-PRINCIPALS speeds up the convergence of {X∗(t) }t≥0 without aﬀecting the convergence properties of ordinary PRINCIPALS.

COMPSTAT 2012 Proceedings

Masahiro Kuroda, Michio Sakakihara, Yuichi Mori and Masaya Iizuka

3

5

Graves-Morris acceleration algorithm

Graves-Morris [5] studied generalization of Aitken’s δ 2 of Aitken [1] for vector cases. The development of the GM algorithm is motivated by its numerical performance.
For a linearly convergent sequence {Y(t) }t≥0 , the GM algorithm generates an accelerated
sequence {Ÿ(t) }t≥0 of {Y(t) }t≥0 by using the following equation:Ÿ(t−1) = Y(t+1) −Y(t) , Y(t)Y(t+1) ,Y(t) , 2 Y(t)(7)

where 2 Y(t) = Y(t+1) − Y(t) . Then, Equation (7) takes a hybrid form of the vector-valued
Padé approximants. The Padé approximants are a particular type of rational approximationof functions and are a very important tool for deriving a fast convergent sequence. Baker andGraves-Morris [2] provided the detailed description of Padé approximants and the derivation of
the GM algorithm.
We study the convergence of the GM algorithm. We consider the transformation such asŸ(t−1) = Y(t+1) +Y(t+1).
Y(t+1) , Y(t) 1−
Y(t) , Y(t) (8)We suppose the following:
ASSUMPTION A: Y(t) → Y(∞) as t → ∞ in the sense of the norm.
The assumption means that there exists a constant 0 < K < 1 such that
Y(t+1) ≤ K Y(t) .
Lemma 3.1. We suppose ASSUMPTION A. Then, we have Ÿ(t) → Y(∞) as k → ∞.
Proof. From Equation (8), we have








(t+1)
Y
Y(t+1)≤.
Ÿ(t−1) − Y(t+1) = (t+1) , Y (t)  (t+1) , Y (t)  1 − Y
 1 − YY(t) , Y(t)  Y(t) , Y(t) 
From ASSUMPTION A, we have limt→∞ Y(t+1) = 0 and |Y(t+1) , Y(t) /Y(t) , Y(t) | =
1, since|Y(t+1) , Y(t) | ≤ Y(t+1)

Y(t) ≤ K Y(t) 2 = KY(t) , Y(t) .

Then, we have the lemma.
Below, we show the convergence of the GM algorithm.
Theorem 3.2. Suppose that a vector sequence {Y(t) }t≥0 fulﬁlls ASSUMPTION A. Then, thevector sequence {Ÿ(t) }t≥0 generated by Equation (7) has the same accumulate point as {Y(t) }t≥0 .

@ COMPSTAT 2012

6

Two-stage acceleration for non-linear PCA

Proof. FromY(t) , 2 Y(t)  = Y(t) , Y(t+1) − Y(t) = Y(t) , Y(t+1)  − Y(t) , Y(t) ,
we haveY(t) , Y(t)
1
.
=
(t)
Y(t) , 2 Y(t)Y , Y(t+1) −1Y(t) , Y(t) 
Equation (7) can be written by Equation (8). From Lemma 3.1, we complete the proof.

4

Two-stage acceleration for PRINCIPALSWe show the two-stage acceleration for PRINCIPALS. The acceleration can be performed toobtain a fast convergent sequence of {X∗(t) }t≥0 : the ﬁrst stage generates an accelerated sequence{Ẋ∗(t) }t≥0 of {X(t) }t≥0 by the vε algorithm, and the GM algorithm in the second stage ﬁnds anaccelerated sequence of {Ẋ∗(t) }t≥0 . We denote the accelerated sequence obtained from the GM
algorithm by {Ẍ∗(t) }t≥0 . Note that the ﬁrst stage acceleration corresponds to vε-PRINCIPALS.
In our experiments, the vε algorithm generates a fast linearly convergent sequence by improving
the rate of convergence but rarely converges superlinearly. When {Ẋ∗(t) }t≥0 is a linearly convergent sequence, we expect that the GM algorithm generates the accelerated sequence {Ẍ∗(t) }t≥0
of {Ẋ∗(t) }t≥0 such that it converges faster than {Ẋ∗(t) }t≥0 .
We refer to the procedure of the two-stage acceleration as vεGM-PRINCIPALS. By alternating among the PRINCIPAL and two acceleration steps, vεGM-PRINCIPALS generates{X∗(t) }t≥0 , {Ẋ∗(t) }t≥0 , and {Ẍ∗(t) }t≥0 independently and alternatively.
Algorithm 4.1 (vεGM-PRINCIPALS).
Step1 PRINCIPALS step: Compute model parameters A(t) and Z(t) and determine optimal
scaling parameter X∗(t+1) .
Step2 vε acceleration step: Calculate Ẋ∗(t−1) using {X∗(t−1) , X∗(t) , X∗(t+1) } from the vε algorithm:
−1 
−1 −1− vecX∗(t−1).
vecẊ∗(t−1) = vecX∗(t) + vecX∗(t)
Step3 GM acceleration step: Calculate Ẍ∗(t−1) using {Ẋ∗(t−1) , Ẋ∗(t) , Ẋ∗(t+1) , Ẋ∗(t+2) } from theGM algorithm:
vecẌ∗(t−1) = vecẊ∗(t+1) −and check the convergence by

vecẊ(t) , vecẊ(t)vecẊ(t) , 2 vecẊ(t)

2



vecẌ∗(t−2)  < δ,

where δ is a desired accuracy.

COMPSTAT 2012 Proceedings

vecẊ∗(t+1) ,Masahiro Kuroda, Michio Sakakihara, Yuichi Mori and Masaya Iizuka

7

We show the sequences generated by three steps and the numbers of iterations when the vεand GM acceleration steps start:
PRINCIPALS step{X∗(t) }t≥0 : X∗(0) X∗(1) X∗(2) X∗(3) X∗(4) X∗(5) X∗(6) · · ·Ẋ∗(0) Ẋ∗(1) Ẋ∗(2) Ẋ∗(3) Ẋ∗(4) · · ·vε acceleration step{Ẋ∗(t) }t≥0 :∗(t)Ẍ∗(0) Ẍ∗(1) · · ·GM acceleration step {Ẍ }t≥0 :The GM acceleration step starts after executing the vε acceleration step thrice. Thus, to obtainẌ∗(0) , the GM acceleration step requires the subsequence {Ẋ∗(t) }0≤t≤3 in the vε acceleration
step and {X∗(t) }0≤t≤5 in the PRINCIPALS step.
During iterations of vεGM-PRINCIPALS, the value of Ẍ obtained in the GM acceleration
step is not used as the estimate at the next PRINCIPALS step similar to vε-PRINCIPALS.
Therefore, the GM acceleration step does not aﬀect the convergence of PRINCIPALS. We obtain the estimates of X∗ , Z, and A in the same manner as vε-PRINCIPALS when vεGMPRINCIPALS terminates.

5

Numerical experiment

We study how much faster vεGM-PRINCIPALS converges than PRINCIPALS and vε-PRINCIPALS.
All computations are performed with the statistical package R [8] executing on Intel Core i5 3.3
GHz with 4 GB RAM. CPU times (in seconds) taken are measured by the function proc.time1 .
For all experiments, δ for convergence of vε-PRINCIPALS and vεGM-PRINCIPALS is set to10−8 , and PRINCIPALS terminates when |θ(t+1) − θ(t) | < 10−8 , where θ(t) is the t-th updateof θ calculated from Equation (4). Each algorithm also stops when the number of iterations
exceeds 100,000. We apply these algorithms to a random data matrix of 100 observations on
20 variables with 10 levels and measure the number of iterations and CPU time for r = 2. The
procedure is replicated 100 times.
Table 1 shows the summary of statistics of the numbers of iterations and CPU times of thesealgorithms from 100 simulated data. The table shows that vεGM-PRINCIPALS considerably
reduces the number of iterations and the CPU time. Table 2 shows the summary of statistics of
the iteration and CPU time speed-ups for comparing the speed of convergence of PRINCIPALS
with those of two acceleration algorithms. The iteration speed-up is deﬁned as the numberof iterations required for PRINCIPALS divided by the number of iterations required for theacceleration algorithm. The CPU time speed-up is calculated similarly to the iteration speedup. The values of the iteration and CPU time speed-ups indicate that PRINCIPALS requires
3.2 to 4.4 times greater number of iterations and 2.7 to 3.9 times longer CPU time than thoseof vεGM-PRINCIPALS.
We compare the performance of vεGM-PRINCIPALS with that of vε-PRINCIPALS. Figure
1 shows the boxplots of the iteration and CPU time speed-ups of two acceleration algorithms.
Table 2 and Figure 1 indicate that the GM acceleration step ﬁnds an accelerated sequence of{Ẋ∗(t) }t≥0 in the vε acceleration step, and thus the two-stage acceleration improves the overall
computational eﬃciency of PRINCIPALS. Figure 2 presents the scatter plots of the iteration
and CPU time speed-ups of vεGM-PRINCIPALS for those of vε-PRINCIPALS. The ﬁgureshows that vεGM-PRINCIPALS always converges faster than vε-PRINCIPALS and accelerates
the convergence of {X∗(t) }t≥0 more than vε-PRINCIPALS whenever vε-PRINCIPALS converges
1Times are typically available to 10 msec.

@ COMPSTAT 2012

8

Two-stage acceleration for non-linear PCA

faster. For the case that the values of the iteration and CPU time speed-ups of vε-PRINCIPALS
are more than 4, vεGM-PRINCIPALS gives higher values. Figure 3 shows the scatter plots of
the iteration and CPU time speed-ups of vεGM-PRINCIPALS for the number of iterationsof PRINCIPALS. The ﬁgure indicates that vεGM-PRINCIPALS works well to accelerate theconvergence more than vε-PRINCIPALS for the larger number of iterations of PRINCIPALS.
Thus, it is clear that the two-stage acceleration is very advantageous.

6

Concluding remarks

In this paper, we proposed two-stage acceleration for the ALS algorithm for non-linear PCA. The
ﬁrst stage generates the accelerated sequence {Ẋ∗(t) }t≥0 of {X∗(t) }t≥0 using the vε algorithm andthe next stage accelerates the convergence of {Ẋ∗(t) }t≥0 using the GM algorithm. Then, both
acceleration algorithms ﬁnd fast convergent sequences without modiﬁcation of the estimation
equations of the ALS algorithm. Therefore, the two-stage acceleration speeds up the convergenceof {Ẋ∗(t) }t≥0 while still preserving the stable convergence property of the ALS algorithm. Thevε and GM algorithms are fairly simple computational procedures, and their computationalcosts are less expensive than those for matrix inversion and for solving the eigenvalue problem
in the ALS algorithm for non-linear PCA.
The numerical experiments employing simulated data demonstrated that the two-stage acceleration improves the computational eﬃciency of the vε acceleration of Kuroda et al. [6],and then signiﬁcantly speeds up the convergence of {X∗(t) }t≥0 in terms of the number of iterations and computation time. In this paper, we described the two-stage acceleration only for
PRINCIPALS, but it is applicable to PRINCALS as well.

Acknowledgement
This research is supported by the Japan Society for the Promotion of Science (JSPS), Grant-inAid for Scientiﬁc Research (C), No 22500265 and No 24500353.

Bibliography[1]

Aitken, A. (1926). On Bernoulli’s numerical solution of algebraic equations. Proceedings of
the Royal Society of Edinburgh, 46, 289-305.

[2]

Baker, G. A., Jr. and Graves-Morris, P.R. (1996). Padé Approximants. Cambridge University Press, Cambridge.

[3]

Brezinski, C. and Zaglia, M. (1991). Extrapolation methods: theory and practice. Elsevier
Science Ltd. North-Holland, Amsterdam.

[4]Giﬁ, A. (1990). Nonlinear multivariate analysis. John Wiley & Sons, Ltd., Chichester.tem[5]

Graves-Morris, P.R. (1992). Extrapolation methods for vector sequences. Numerische Mathematik, 61, 475-487.

COMPSTAT 2012 Proceedings

Masahiro Kuroda, Michio Sakakihara, Yuichi Mori and Masaya Iizuka

9[6]

Kuroda, M., Mori, Y., Iizuka, M. and Sakakihara, M. (2011). Acceleration of the alternating
least squares algorithm for principal components analysis. Computational Statistics andData Analysis, 55, 143-153.

[7]

Mori, Y., Tanaka, Y. and Tarumi, T. (1997). Principal component analysis based on asubset of variables for qualitative data. Data Science, Classiﬁcation, and Related Methods(Proceedings of IFCS-96), 547-554, Springer-Verlag.

[8]

R Development Core Team (2011). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL
http://www.R-project.org/.

[9]Young, F.W., Takane, Y., and de Leeuw, J. (1978). Principal components of mixed measurement level multivariate data: An alternating least squares method with optimal scaling
features. Psychometrika, 43, 279-281.

[10] Wynn, P. (1962). Acceleration techniques for iterated vector and matrix problems. Mathematics of Computation, 16, 301-322.

Min
1st Qu.
MedianMean
3rd Qu.
Max.

The number of iterations
PRINCIPALSvε vεGM
181.0
65.0
57.00
332.5 103.8
91.25
475.5 154.5 134.00
605.9 193.8 166.37
740.0 241.0 207.75
3595.0 820.0 615.00

CPU time
PRINCIPALS
vε
3.510
1.410
6.258
2.145
8.780
3.035
11.1303.789
13.3404.620
67.250 15.690

vεGM
1.310
1.992
2.730
3.361
4.062
12.170

Table 1. Summary of statistics of the numbers of iterations and CPU times of PRINCIPALS,
vε-PRINCIPLAS (vε) and vεGM-PRINCIPALS (vεGM) from 100 simulated data (r = 2).

Min
1st Qu.
MedianMean
3rd Qu.
Max.

Iteration speed-up
vε
vεGM
1.383
1.414
2.729
3.175
3.175
3.677
3.201
3.758
3.773
4.377
6.076
9.038

CPU time speed-up
vε
vεGM
1.356
1.356
2.577
2.748
2.895
3.209
2.953
3.332
3.422
3.895
5.548
8.451

Table 2. Summary of statistics of the number of iterations and CPU time speed-ups of vεPRINCIPLAS (vε) and vεGM-PRINCIPALS (vεGM) from 100 simulated data (r = 2).

@ COMPSTAT 2012

Two-stage acceleration for non-linear PCA

2

6
5
2

3

4

CPU time speed−up

64

Iteration speed−up

7

8

8

10

vε

vεGM

vε

vεGM

Figure 1. Boxplots of iteration and CPU time speed-ups of vε-PRINCIPALS (vε) and vεGMPRINCIPALS (vεGM) from 100 simulated data (r = 2).

CPU time speed−up

2

5
2

3

4

4

vεGM

vεGM

6

6

7

8

8

Iteration speed−up

2

3

4

5

6

2

vε

3

4

5

vε

Figure 2. Scatter plots of iteration and CPU time speed-ups of vεGM-PRINCIPALS (vεGM)for vε-PRINCIPALS(vε) from 100 simulated data (r = 2).

COMPSTAT 2012 Proceedings

11

7
6
5
4
2

3

CPU time speed−up

642

Iteration speed−up

88Masahiro Kuroda, Michio Sakakihara, Yuichi Mori and Masaya Iizuka

500

1500

2500

3500

The number of iterations of PRINCIPALS

10

20

30

40

50

60

CPU time of PRINCIPALS

Figure 3. Scatter plots of iteration and CPU time speed-ups of vε-PRINCIPALS ( ) andvεGM-PRINCIPALS () for the number of iterations of PRINCIPALS from 100 simulated data(r = 2).

@ COMPSTAT 2012