Fair principal component analysis (PCA): minorization-maximization algorithms for Fair PCA, Fair Robust PCA and Fair Sparse PCA
Authors: Prabhu babu
Petre Stoica
Astha Saini
Venue: Accepted by TMLR ()

Abstract: In this paper we propose a new iterative algorithm to solve the fair PCA (FPCA) problem. We start with the max-min fair PCA formulation originally proposed in \cite{samadi1} and derive a simple and efficient iterative algorithm which is based on the minorization-maximization (MM) approach. The proposed algorithm relies on the relaxation of a semi-orthogonality constraint  which is proved to be tight at every iteration of the algorithm. The vanilla version of the proposed algorithm requires solving a semi-definite program (SDP) at every iteration, which can be further simplified to a quadratic program by formulating the dual of the surrogate maximization problem. We also propose two important reformulations of the fair PCA problem: a) fair robust PCA - which can handle outliers in the data, and b) fair sparse PCA - which can enforce sparsity on the estimated fair principal components.
The proposed algorithms are computationally efficient and monotonically increase their respective design objectives at every iteration. An added feature of the proposed algorithms is that they do not require the selection of any hyperparameter (except for the fair sparse PCA case where a penalty parameter that controls the sparsity has to be chosen by the user). We numerically compare the performance of the proposed methods with two of the state-of-the-art approaches on synthetic data sets and real-life data sets.

Published in Transactions on Machine Learning Research (02/2025)
Fair principal component analysis (PCA): minorization- maximization algorithms for Fair PCA, Fair Robust PCA and Fair Sparse PCA
Prabhu Babu
Prabbu. Babu Scare. itd. ac. in
Centre for Applied Research in Electronics Indian Institute of Technology, Delhi 110016, India
Petre Stoica
Ps uu.se
Division of Systems and Control, Department of Information Technology Uppsala University, Uppsala, Sweden 75237
Astha Saini
astha. saini wtt.
Centre for Applied Research in Electronics Indian Institute of Technology, Delhi 110016, India
Reviewed on OpenReview: https: // openreview. net/ formm? id= 6fTQrr3APY
Abstract
In this paper propose a iterative algorithm to solve the fair PCA (FPCA) problem. We start with the max-min fair PCA formulation originally proposed in Samadi et al. (0118) and derive simple and efficient iterative algorithm which is based on the minorization- maximization (MM) approach. The proposed algorithm relies on the relaxation of a semi- orthogonality constraint. which is proved to be tight. every iteration the algorithm. The vanilla version of the proposed algorithm requires solving a semi-definite program (SDP) at every itcration, which can be further simplified to quadratic program by formulating the dual of the surrogate maximization problem. We also propose two important reformulations of the fair PCA problem: a) fair robust PCA which can handle outliers in the data, and b) fair sparse PCA - which can enforce sparsity on the estimated fair principal components. The proposed algorithms are computationally efficient and monotonically increase their respective design objectives at every iteration. An added feature the proposed algorithms that they do not roquire the selection of any hyperparameter (except for the fair sparse PCA where penalty parameter that controls the sparsity has to be chosen by the user). We numerically compare the performance of the proposed methods with two of the state-of-the-art approaches on synthetic data sets and real-life data sets.
1 Introduction and problem formulations
Principal component analysis (PCA) is one of the most widely used unsupervised dimensionality reduction technique Jolliffe (2002), which has found applications, among others, in image processing Clausen & Wech-
sler (2000] Du & Fowler (2008 signal processing Castells et al. (2007); Subasi & Gursoy 22010), finance Tsay (2055) and biology Giuliani (2017); Lever et al. (2017). Given the data, PCA learns the underlying low dimensional subspace by minimizing the reconstruction error. Let Y = [y1,y2. .Nn] E R denote
the data samples, where n and N (n < N) represent the dimension and the number of data samples, re-
N
spectively. Without loss of generality, we assume that the data samples are mean centered, i.c. Σ y: = -x=00
i=l

---

Published in Transactions on Machine Learning Research (02/2025)
Mathematically, PCA solves the following problem to find the low rank matrix Y:
Y = arg min | Z| 8.1. rank (Z) = r rrs n
(1)
where ||-|| denotes the Frobenius norm and r denotes the rank of the low-dimensional subspace, which is usually pre-specified. Although the problem in (1] is non-couvex (due to the presence of the rank constraint),, its solution can be computed analytically using the singular value decomposition of Y
0
V V
Y=
n-r
Σ 0 Σ
where the diagonal matrix
contains the singular values in decrcasing order and the unitary ma-
0
trices [
Ú
comprise the corresponding left and right singular vectors. Then, the low rank and V
approximation Y is given by
UUUY
The reconstruction error is given by
Y Y1P = (I Y = Tr - M
(2)
const. RU)
where R 2 YY is the sample covariance matrix. Using 2 the problem in [ can be reformulated as:
Tr(URRU) s.t. UT- I,
(3)
The maximizer of (3) is given by the principal eigenvectors of R, and the cost function in is the Rayleigh quotient which can also be interpreted as the projection variance. However, in real-life applications the data samples come from diverse classes, so maximizing the average variance as in 3 cannot ensure fairness among the classes, In other words, the matrix U determined by solving (3) may be a good fit for some data samples and a poor fit for others. To mitigate this problem, a worst case approach named Fair PCA (FPCA) was recently proposed, which solves the following problem. Let us assume that the data samples are from different classes with the matrix YA containing the samples belonging to class k. The corresponding sample covariance matrix is R YRY Then the FPCA problem can be stated as Samadi et al. (2018)
max U
min KE[L,K]
Tr(URRAU) s.t. UU-
(4)
The solution to 4 will give the best fit for the least favourable class, and in general will balance the fitting with potentially many classes having identical variances (Tr(U"RR U)). The FPCA problem in (4) is hard
to solve due to the presence of the inner minimization operator and the constraint. In Samadi et al. (2018)
and [Tantipongpipat et al. (2099), the authors have proposed algorithms for solving 4 using semidefinite relaxation (SDR) technique. However the relaxation employed by SDR may not. be tight for all problem dimensions and therefore may yield suboptimal solutions (except in the case K = where the relaxation is always tight as proved in Samadi et al. (2018)). In Kamani et al. (2022), the authors have proposod an alternative formulation of FPCA as multi-objective optimization problem and suggested an iterative method to arrive at solutions spanning the Pareto frontier. Finally Zalcberg & Wiesel (2021) has proposed sub-gradient based algorithm to solve the FPCA problem. However, for any gradient approach, the algorithm of ZZacberg & Wiesel (2021) requires a careful choice of the step-size: failing to choose a good step-size may result. in either convergence to a suboptimal solution or a very slow convergence.
In this paper, we propose an iterative algorithm to solve the FPCA problem, which is based on the minorization-masti maximization (MM) approach. Unlike some of the aforementioned algorithms, the proposed algorithm monotonically increase the cost. function (i.e. min (Tn(T)(URRR and it is guaranteed to converge. k An added feature of algorithm is that it does not require the selection of any tuning parameter.
2

---

Published in Transactions on Machine Learning Research (02/2025)
In the presence of outliers, the solution to the problem formulated as in 4 can be significantly affected. To tackle the outliers, we propose the following robust version of the fair PCA problem (hereafter called FRPCA):
max U
min KEILA]
Y s.t. U
(5)
where ||UYR denotes the sum of the absolute values of the clements of U k- The La norm basod
fitting criterion in ( is inspired by the L1 PCA formulation proposed in Kwak (2008) which is the robust
counterpart of the vanilla PCA and has been found to be quite effective handling outliers. The proposed
MM algorithm for 4 can be relatively easily adapted solve the FRPCA problem in (6).
Finally, also consider extension of the proposed MM based algorithm for 44 to cases in which the sparsity of the fair PCA solution is important. To do this, we consider the following penalized problem:
max
min kE[L.K]
TrU"RR - A||U]| s.t. U²U= L
(6)
where À is a prespecified penalty that controls the sparsity of U. The formulation in 66 is motivated by the d dlAAsremoot, et al. (2004) where the & norm of U is usually added as a penalty to the objective in (3) to promote sparsity and hence help identify the significant contributing elements of the estimated principal components. As we will show later the proposed
literature on sparse PCA Zou et al. (2006); Zou & Xue
(2018);
;
method for 4 can be extended to deal with the problem (6) in straightforward manner.
The main contributions of this paper can be summarized as follows:
1) We propose a hyperparameter free algorithm based on the MM approach for solving the FPCA problem. The proposed algorithm has a monotonic convergence behavior and can be efficiently implemented. 2) We present modification of the proposed algorithm that can deal with outliers in the data. More prociscly, we formulate the fair robust PCA (FRPCA) problem and show how the MM algorithm for the FPCA approach can be modified to solve the FRPCA problem.
3) We also present an extension of the proposed algorithm (abbreviated as FSPCA) that can handle a sparsity promoting penalty in the FPCA problem.
4) Finally, present several mumerical results both synthetic data sets and real-life data sets and compare the performance of the proposed algorithm with that of two state-of-the-art. approaches.
Notations: We use bold uppercase (e.g., Y), bold lower case (e.g., x) for matrices and vectors, respectively and italic letters (e.g., æ) for scalars. The superscript t of x' denotes the iteration index. For any symmetric matrices A and B, A > means that A B is positive semidefinite. The variables I, and I, denote the identity matrices of dimension x r and n x n. A denotes the Hermitian square-root of the positive definite matrix A and sgn(x)) denotes the sign of the scalar x. Finally, ||U|| denotes the sum of the absolute values of the elements of the matrix U.
A brief outline of the paper is as follows. In Section 2 we first discuss the general MM approach for a maximization problem, and then the MM approach for max-min problems. In Section 3 we present the proposed algorithm for the FPCA problem and discuss an efficient way to implement it. In Section 4 we consider an extension of the proposed method to deal with fair robust PCA and fair sparse PCA problems. In Section 5, we present several numerical simulation results on both synthetic and real-life data and compare the proposed method with two state-of-the-art methods. Finally, we conclude the paper in Section 6.
2
MM Primer
In this section, we first present the main steps of the MM approach for maximization problems. Later we discuss how the MM approach can be extended to deal with max-min problems, which are of interest in this paper.
2.1 MM for max problems
Consider the following constrained maximization problem:
max f(x) xex
(7)
3

---

Published in Transactions on Machine Learning Research (02/2025)
where x is the optimization variable, fxx denotes the objective function and X is constraint set. An MM- based algorithm solves the above problem by first constructing surrogate function g(x)x*) which strictly lower bounds the objective function f(x) at the current iterate x'. The function g (xxx*) qualifies to be a surrogate function satisfics the following conditions:
(x(x*) S (x) VXEX
(8)
g (xxx²² (x')
(9)
Then in the second step, the surrogate function is maximized to get the next iterate i.e.,
x ++1 E arg max (x(x')
(10)
XEX
The above two steps are repeated until the algorithm converges to a stationary point of the problem The surrogate function ideally should follow the shape of the objective function and be easy to maximize. To summarize, the major steps of the MM approach are:
7
Start with feasible x and set t
Construct a minorizing function g (x(x') of [(x) at x!
3. Obtain xf+ ++1 E arg max (x(x*).
xEx
((xx)=f((x+1)) < e, where c some prespecified convergence threshold, exit; otherwisc t t+ [f(*))
and go to Step 2.
It is straightforward to show that the MM steps monotonically increase the objective at every iteration i.c.
f(x++1) 2 g (x'+) 2 (x' xx*) f(x')
The first inequality and the third equality follow from 8 and (9 and second inequality from 10
2.2 MM for max-min problems
Consider the following max-min optimization problem:
max xEX
[(x)
A min
1=1,2.- K f((x) }
(11)
A possible surrogate function g(x(x') for the max-min problem is as follows:
s(x[x*)
min (x(x(x))
(12)
i=1.2.
where each (x(x')) is tight lower bound on fx( at x'. The individual surrogates (x) satisfy the following conditions:
(x = fi(x')
(13)
(xxxx²)) fx()
(14)
One can easily show that the surrogate function x(x(x') defined in 12 satisfies the conditions S and 9
g. (x(x') S fx)
min (x(x*)
1.2 ,k
< min fx(x)
(15)
11,
g(xixt) < f(x).
and
9(x1 0(x1|x')= = min g:(x'|x') (x') = min fi(x!) f(xt)
(16)
1=1,2.. K
1=1,2 K
4

---

Published in Transactions on Machine Learning Research (02/2025)
Similar to the general MM case, here also one show that the iterates {x'} increase the objective function (x) monotonically and converge to stationary point. We refer the reader to Naghsh et al. (2019) Zhao & Palomar (216)) for application of MM to max-min problems in communications and radar, and to Sun
ct al. (0166) for a detailed discussion the MM approach including different ways of finding the surrogate
lunction the context. of different applications.
3 FPCA
In this section, we will derive the MM algorithm for the problem in (3). For the sake of convenience, we restate the FPCA problem:
max (f(U)
& min kE[I.K]
fa(U)) s.t. UU-
(17)
U
where (U) & R,) Before we venture into solving we state and prove a lemma which will help in the development of the proposed algorithm.
Lemma 1. The non-convex semi-orthogonality constraint U = I, in 17 can be relaxed to U < I,
and the global maximizer of the relaxed problem will satisfy the constraint in 17
Proof. Let the eigenvalue decomposition of UU be as follows:
UU VAVT
(18)
where contains the principal cigenvectors of UU and VTV I, Using we have
TTUURR = =T(((V' RV V)=Aa
Rv
Therefore all functions in 17 are larger for A = I, than for A < I, and this implics that the global maximizer of 17) under the constraint UTU x I, will satisfy the constraint in Therefore the relaxation UTU &f I, does not effect the solution (17).
Using Lemma replace the problem in 17 with the following relaxed problem:
max AE[L,]]
min
Tr(U'RAU) s.t. U I
(19)
The constraint in 19 is convex it can be reformulated linear matrix inequality (Boyd & Vandenberghe (2004h):
I. U I 0
U
However, the maximization problem in 19 is non-convex the objective (for any k) is convex quadratic function in U and the presence of min operator further adds to complications. We resort to the MM approach to solve 19h. Following the MM steps listed in Section 2.2, each convex quadratic function in (19 can be lower bounded via its tangent hyperplanc at U Doing so gives us an MM surrogate for the objective in 19h. For given U and for any k. we have
(u) = TT 2
+ RR - -U)) = - ( U*) & (U)
Then the surrogate problem is given by:
I U
max U
min 9 (U) s.t.
U I,
> 0
(20)
5

---

Published in Transactions on Machine Learning Research (02/2025)
Using the expression for g&( (U) in (0 get
I, U
min (TT(AA + ca s.t.
=
(21)
U In
where
A A (U')TRs, RR, C = -TT((U))RRR)))) U*)
(22)
Problem 21 is convex and can be reformulated as an SDP:
max a a.U
s.t. 2 Tr
(A
+
Za
(23)
I
U
U
I
which can be solved using off-the-shelf solvers like CVX Grant & Boyd (0144).
Consider the inner minimization problem in 20 which can be written as follows:
K
min (U) = min Σ (U)
(24)
s.t. ph 1
Solving the problem on R.H.S. of (44. the solution elements of the vector μ are such that pa corresponding to the minimum value of O (U) and rest 0, giving the minimum value of (u)) So, the inner
minimization problem in 21 can be reformulated using the auxiliary variables μ = [ur, : = 1 as shown below:
max min
Σ
(U)
k=
K
s.t. 1 2 0,
Σ
14 =1
(25)
I
U
b O.
U
I
Lemma 2. The maximizer of 21 (or equivalently of the algorithm (not only at convergence).
23 satisfies the constraint U = I,. at each ileration
Proof. This interesting property will be proved in what follows. Note that 25 is a notationally simpler reformulation of 23 The objective in (25) linear function of U for given μt, and it linear in l for for both and μ are compact and convex. Consequently, using
fixed U. Furthermore the constraint
the minimax theorem Sion (1958), the max and min operators in 25 can be interchanged and we get the following equivalent problem:
K Σ
min 2Tr(ATU)
P&Ck
k=1
(26)
K
I
U2
s.l. ph Σ
p4
U
I
>
0
where
K
A(#) A
(27)
where A(μ) indicates the dependency A on The inner maximization problem in 226 can be solved
closed form. Consider the first term in the objective of 26 By Von-Neumann incquality Marshall et al.
1979 have
Tr((A(H))' & Σ (A(H))))

---

Published in Transactions on Machine Learning Research (02/2025)
where (A(A))) and (U) denote the non-zero singular values of A and U respectively. Because o:(U) S 1, it follows that
Tr((A(A)) Σ (A(()))
with the equality attained for
U - AWW((A()""(W)))
(28)
Indeed,
(TC(A)))AAAAA)))))C(A))) A(<4)) = ((((A)))))A()))))))))))))))) Σ (.(((())
Note that U satisfies the constraint in ie. (U")TU* I... Therefore, as claimed, the maximizer of satisfies the constraint U I, each iteration.
23
Now inserting
28
26
yields the following problem that remains to be solved:
K Σ t=1
K
min 2
(A(H)) + Σ
PaCk
(29)
s.t. ph 0
1k #1
where we have stressed by notation that A function of μ. The first term 129 equal to times the matrix nuclear norm of Ap)), denoted [|([)]]],, and it a convex function A and hence {ux]- SDP (see Appendix A) Recht et al.) (2010 However compared to 223 the number of variables and constraints in (29 is smaller, which can be
Thus, 29 is convex problem like (23) and can be reformulated
an advantage. In fact for K- = 2. 29 can be solved as 1-dimensional problem via bisection method.
Once the minimizer μ³ obtained by solving 23 can obtained as:
29 the corresponding U (which also the maximizer of
U(++1) = (A""))
(30)
and it will serve as the next iterate. The MM-procedure will contimue in the same manner till the iterates converge. The steps of the proposed algorithm are summarized in pseudocode form in Algorithm 1.
Algorithm FPCA algorithm
Input Initial estimate U" K and convergence threshold = 10-5
Set =
repeat
Compute {Ak, ) in 22
Compute 1¹ by solving
29
Obtain Ue+1 from 30
Set t= 1+1. S(U)
until S(U S(U))
S.
f(U')
UPPCA = - at convergence.
Output UFPCA-
The main burden of the proposed algorithm lies in the computation of {A*] solving the SDP in 29
and computing U++1 in (30h. Computation [Ax] 1 involves evaluation of matrix-matris products which
can be done in (Krn2) flops. Solving the SDP in (29 requires roughly O((n + r)3.) flops Shen et al. (2008h; Zheng et al. (202)). and evaluating (30 requires O(nr²) + O(r3) flops, thus the total number of
computations per iteration is the order of O((n 1))) flops. The complexity of O + r)1.5) seems

---

Published in Transactions on Machine Learning Research (02/2025)
to be on the higher side especially when compared to that of recent state-of-the-art algorithm proposed in Zalcberg & Wiesel (2021), which has complexity of (n²). Algorithm can be run on workstation for fairly large dimensions such as = 1000, 100, K = 100. Regarding the space complexity, at every iteration, the matrices {Ax 1 need to be stored which requires O(Kr)) memory words, and the space complexity of the SDP is around (()+)))), thus the total space complexity is on the order of (((
In the following, we propose an alternative approach to solve
(9 that helps reduce the computational
burden. To do 80, we starl with the problem in [29]
K + Σ
min {f(w) - 2 Ap))I
p2ck)
k=1
(31)
K
s.b. μs 2 0
14 =1
Let us introduce an auxillary variable & (Φ > 0) and rewrite
31 in the form:
min 1
Tr( 1)(5))) 1)+ + (T((A())))A()))) + PACK
(32)
s.t.
M& 0
p4 =1
k=1
The problems 31 and 32 are equivalent, which can be seen follows. II we minimize
32 over & for
fixed 14, we get the minimizer T- ((A())))A())~~ and substituting it back in (32) we get the problem
in 31 Thus, instead of solving (31) one can solve 32 to obtain the minimizer of (31]. The problem in (32) can be solved using an alternating minimization approach for given p. say μ², we get the minimizer
as explained above. For fixed + Φτ, the minimizer l can be obtained by solving the following convex problem:
min (((AA()))A(()()))) "
Σ P&Ck
(33)
K
s.t. 19
Σ
pk
=1
k=1
which can be reformulated as a quadratic program (QP):
K
min
1r LTQQu
+
Σ
(34)
K Σ
s.t. ph 2 0
14 =
where [Q], A Tr(AT' Ay). The QP in 34 can be solved using standard solvers whose computational
complexity including the update of क' is the order of O(K3) + O(n²) flops. This is much less than (((++))4 but note that, unlike the problem in (29 the problem in (33) has to be solved multiple times;
fortunately usually only few iterations are needed (lypically 10). There is hardly any theoretical result on this aspect, nonetheless from our numerical simulation experience we have observed that solving (33) usually indeed requires around 10 iterations (even for problem dimension n = 100) to achieve convergence (as can also be seen from the figures Section 5). The iterative steps of the above alternating minimization algorithm for computing the solution fz" (29h are summarized in Algorithm
4 FSPCA and FRPCA
4.1 Fair Sparse PCA
In this section we present an exlension of the proposed method which can handle penalty that promotes the sparsity of the fair principal components. For convenience we repeat here the statement of the sparse FPCA problem (see 6
max (M(U) و min U kE|1,K]
(35)
s.t. UU-II,
8

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 2 Alternating minimization approach for solving 29
Input Initial estimate μº, {ck, A and convergence threshold 10-5.
Set O.
repeat
Compute Φ6 = (A' (A" A()))
Obtain pelt1 by solving 34
Set i == 11
until f(1
c.
f(wt)
μ¹" μt at convergence. Output pr'.
where a penalty factor that controls the sparsity U. Selecting a basic problem in sparse opti- mization literature and there is host of methods that have been suggested for making this selection, with cross validation being often the method of choice (see, e.g. Zou et al. (0006) for selecting in sparse PCA applications). Similarly to the FPCA case, we relax the semi-orthogonality constraint:
max U
min KE[LKK]
[R(U) - A||U||
I
(36)
s.t.
<L
0
U
I
For fixed U UA we can minorize the quadratic functions (f&(U)} using their langent hyperplanes, and obtain the following surrogate problem
min
(U) A||U||
U
I,
U U
(37)
s.t.
0
where before (U) -T)( + cu. The above problem convex (more exactly an SDP) and can be solved via CVX. Similar to the FPCA case, the maximizer of (37) can be shown to satisfy the constraint
UTU-N = I, To do so, we reformulate the problem in 377 using auxiliary variables B and μ follows:
max min
PkIK (U) + (TT
(38)
I, U U
s.t. 14 2 0 p4 =1
0, B [ S Vi
I
Minimizing 138 over B and a we get the objective in the problem
37 Thus
37 and (38
equiva-
lent problems. Using the minimax theorem (the objective and the constraints of satisfy the required conditions), the max and min operators can be swapped:
38
K max Σ U
min B.B
(L + A (T
1=1
(39)
I
Uº
s.t.
Mk 2 0
#b =1
0.
Biyl $ Vi,j
U
I
Using the expression for gb (U) (T ck:
39
can be rewritten as:
K
max 2T<((A + B)
min
PkCk
B.
k=1
(40)
K
I U
s.t. # 2
Σ 5
=
>
0,
B m S 1 Vi,j
I
k=
9

---

Published in Transactions on Machine Learning Research (02/2025)
Similar to 28 the maximizer U of 40 can be obtained in closed form:
U = -A + A B) + A 2 -A B) 2
which satisfies the constraint U I, Thus, also in the FSPCA case, the MM iterates satisfy the semi-orthogonalitt constraint.
The pseudocode of FSPCA is summarized in Algorithm 3.
Algorithm 3 FSPCA algorithm
Input Initial estimate U
{RR}. X, and convergence threshold 10-5
Set 0.
repeat
Compute {Ak, c in 22
Obtain U by solving (37
Set 1= 1=1+1.
until (( e. h(U))
USFPCA U at convergence.
Output USFPCA
4.2 Fair Robust PCA
We next discuss how the proposed approach can be modified solve the fair robust PCA formulation stated
in
(U)
min
| s.t. -
U
kE[L,K]
Similar to what we have done before, we relax the semn--rtthoonallity constraint:
max
min RE[LK]
y
U
I
U
(41)
s.t.
0
I
Compared to the previous cases, we have a different data fitting function here. Nonetheless, Y can be minorized as follows. For a given U
Yu = Σ EKUYY 2 Y
TU Yolo! = ((gn"))ssg
1o
(42)
where
W - (U')
(43)
Thus the surrogate problem corresponding to 41 is given by:
max min T
I
(44)
s.t. U
I 0
which is once again an SDP (after rewriting in epigraph form) that can be solved via CVX. Using similar argument to that employed in the cases of FPCA and FSPCA, here too one can show that the solution of 44h satisfies the semi-orthogonality constraint. The pseudocode of the FRPCA is summarized in Algorithm 4.
We conclude this section with number of remarks on the proposed algorithms.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 4 FRPCA algorithı
Input Initial estimate U {Rt X, and convergence threshold e = 10-5.
Set t 0.
repeat
Compute {W&) in (33
Compute U by solving (44).
Set t= ++11
until 14p (Uttl) (U9) h,,
c.
hr (U")
UPRPCA U at convergence.
Output UFRPCA
The value in the case of FSPCA and the rank for all the algorithms are assumed to be prespecified. When À and r are not specified they can be chosen using model selection approaches or cross validation.
In the case of FSPCA and FRPCA, although we did not prove that we can relax the semi- orthogonality constraint in the original problem formulations (like we did for FPCA), we showed that the corresponding surrogate problems tightly satisfy the semi-on coonioothogonallty constraint, which means that their solutions are feasible for the original problems.
If desired sparsity inducing penalty can also be added to FRPCA problem and MM algorithm can be developed.
In the case of FSPCA and FRPCA, QPs similar to 334 can be derived and solved instead of solving
the SDPs in 37 and
Regarding the convergence of the proposed methods, by the properties of MM the objective function is monot mnicaln increasing at each iteration. Then, because the cost functions of FPCA, FSPCA and FRPCA are bounded above, the convergence of the proposed algorithms follows from the general results proved in Sun et al. (2016).
Regarding the initialization (U") for the proposed methods, in the case of FPCA and FSPCA we observed in the numerical experiments that the objective functions often unimodal. So, we have used random semi-orthogonal matrices to initialize the algorithms. On the other hand, the objective function of FRPCA appeared to be multi-modal (sce the discussion in Section 5.4), thus we have initialized the FRPCA algorithm with the FPCA solution.
Finally we note that RPCA and SPCA are important problems in their own right and multitude of algorithms have been suggested for solving them (see, e.g. Kwak (2008): Zou al. (2006); Zou &
Xue (2018); d d'Aspremont et al. (2004] ). Our algorithms FRPCA and FPSCA with K = 1 can also
be used lo solve the RPCA and SPCA problems.
5 Numerical simulation results
In this section, we will compare the performance of the proposed FPCA method with the SDR method of (Samadi et al. (2018) and the sub-gradient method from Zaleberg & Wiesel (021)) The SDR approach and the semi-
uses the parametrization P = UUT and relaxes the rank(P) orthogonality constraint to 0 & P I,. For K = 2, the aforcmentioned relaxation is tight but for K 2 3 this is not. necessarily true and the method often yields solutions with rank(P) > r. Being a relaxation the objective value attained by SDR is an upper bound which is not always achievable. We first perform simulations basod on synthetic data sets and later include simulation results for real-life data sets. We will also present numerical simulation results showing the performance of the proposed FSPCA and FRPCA methods.
constraint to Tr(P)
11

---

Published in Transactions on Machine Learning Research (02/2025)
280
260
220
Proposed method
min (,00)
su subgrad (small stop) subgrad (medium stap) subgrad (large stop)
200
200
Proposed method
min
subgrad (small step) subgrad (medlum step)
267.756356
267.796354
270
subgrad (large
267.75
150
SDA
140
100
200
300
280
265
120
100
100
150
200
300
100
150
200
250
300
lterations
a)
K =
b) K =
Figure Synthetic data set: Objective vs iteration plots for n 10, N = 100, and r == In the case of SDR, for the simulation settings figure 1b, the rank of optimal P was found to be 6.
350
400
300
350
min ((U)
min (
300
SDR
SDR
Proposed method
Proposed method
250
subgrad
subgrad
200
200
150
150
100
2
8
10
2
4
8
10
a) = 2
b) K =
Figure 2: Synthotic data Objective vs plots for n = 10, N 100.
5.1 Synthetic data
The data samples for any class k is generated as {y; R where R was randomly generated, and the elements of e; are i.i.d. random variables sampled from a Gaussian distribution with zero mean and variance 100, and r = 4, and each class has the same number of data samples equal to K
one. Let n = 10, We first show the objective vs. iteration plots for the three methods Figure for two different values of K. It can be seen from this figure that the proposed FPCA method converges quickly the SDR upper bound (see the zoom insert in Figure la), while the method of Zalcberg & Wiesel [2021] converges slowly to suboptimal value. For K > 2, the SDR upper bound may not be achievable, see for example Figure lb (zoom insert). For fairness, we ran the method of Zalcberg & Wiesel (2021) for different choices of the step length (small, medium and large) and from Figure one can see that even for different step-lengths the method still converges to suboptimal values. In Figures 2a and 2b, we show the objective values obtained by the three methods for different values of and K. From these figures can be soen that the proposod
FPCA approach always finds better objective value than the method from Zalcberg & Wiesel (2021) and reaches the SDR bound whenever the latter is achievable.
12

---

Published in Transactions on Machine Learning Research (02/2025)
In Table we show the average computational time (in seconds) of the algorithms.
(n, r, K), N 1000
subgrad
Proposed Method
(30,2,2)
1.2
2.2
(30,4,2)
1.8
5.8
(30,8,2)
2.6
16.7
(30,16,2)
4.2
48.2
(25,5,5)
3.1
20.2
(50,5,5)
75.6
118.3
(100,5,5)
450.2
740.6
(40,2,2)
15.1
48.6
(40,2,4)
16.7
54.9
(40,2,8)
20.3
60.5
Table 1: Average computational time (in seconds) for the algorithms
5.2 Credit data
In this simulation evaluate the performance of the methods credit data set from Samadi et al. (2018). First the dala is mean centered. The parameters of the credit data are = 21, K = 2, N 30000 (with class 1 having 5385 data samples and class 2 having 24615 data samples). In Figure 3a, we plot the objective vs iteration for the proposed FPCA method and the sub-gradient method of Zalcberg & Wiesel (2021). The proposed method quickly converges to the SDR bound. On the other hand, even for a careful choice of the step size, the method Zalcberg & Wiesel (2021) takes nearly a million iterations converge suboptimal value. In Figure 3b show the optimal objective values obtained by the three methods for in the interval [1. 20). It can be seen from this figure that the FPCA method reaches the same objective
as SDR.
10
20
8-
15
SDR Proposed mothod subgrad
min ((U)
min (U))
subgrad
10
SDR Proposed method
4
2
10
2
10
106
0
5
10
15
20
lteration index
a) Objective v8 iteration (r= 2)
b) Objective vs
Figure 3: Credit data set with n = 21, N 30000, K = 2.
5.3 LFW data
In this example we compare the performances of the three methods, namely SDR, proposed method and the sub-gradient based method for high dimensional real data. set labeled faces in the wild (LFW) Huang et al. (2008), which contains the facial images of both men and women (K 2). Each image is of size 42 42 1764) and there 2962 images of women and 10270 images of men. In Fig. 4 we show the fair PCA
13

---

Published in Transactions on Machine Learning Research (02/2025)
500
400
min, M0
300
SDR proposed method
200
subgrad
100
0
15
20
Figure 4: LFW data set with n = 1764, N = 13232, = 2.
objective obtained by the methods for different values of rank (r). It can be seen that the proposed method tightly follows the SDR whereas the sub-gradient method is sub-optimal.
5.4 Fair robust PCA
In this simulation, we generate the data as in Section 5.1 but now some data samples for randomly chosen classes corrupted by outliers. The outliers, which roughly 10% of the total number of samples, are generated from Gaussian distribution with mean and variance one. In Figure 5a we plot the objective iteration for the proposed FPCA algorithm for ten different initializations. The objective in (5) has several local maxima, which is evident from the plot as the method converges to different local maxima depending on the intialization. In Figure 5b, we vary a in the interval [0, 20] and compute the average normalized error between the estimated subspaces (calculated using 100 Monte-Catte runs):
Normalized subspace error |00 -UPCCA Oreaal
(45)
|UrCCA OPCcl
where ÜPPCA denotes the FPCA estimate of the principal components obtained from outlier-free data, and Ö denotes either UFPCA or UFRPCA obtained in the presence of outliers. It can be seen from the figure that the FRPCA performs well in the presence of the outliers and has comparatively low errors in the presence of large outliers.
5.5 Fair sparse PCA
In this sinulation generate the data as in Section 5.1 with the choice of parameters: 40, 10, N 200, K = 2. In Figure 6a, we show the evolution of the objective in (35) vs iterations for À = 5. The proposed algorithm monotonically increases the penalized objective as expected. The choice of À determines the sparsness of U. Figure 6b we show the number of non-zero elements increases the sparsity of the estimated U increases.
À (as expected) when À
6 Conclusion and Future Work
In this paper we have proposed new MM algorithm for solving the fair PCA (FPCA) problem. The proposed algorithm is computationally efficient and monotonically increases the FPCA objective al each iteration. Furthermore, the proposed FPCA algorithm does not require the selection of any hyperparameter. We have also proposod two MM algorithms to solve the fair robust PCA and fair sparse PCA problems. Finally, we performed numerical simulations on both synthetic data and real-life data and compared the performance of the proposed approach with that of two state-of-the-art approaches. In future work,
14

---

Published in Transactions on Machine Learning Research (02/2025)
90
FRPCA FPGA
85
Normated (subepace antemate)
80
5
75
70
0.5
0.4
65
0.3
60
02
0.1
55
5
10
15
20
25
30
iteration index
Cuttie magnitude
a) Objective vs iteration (for 10 random initializations)
b) Normalized subspace error vs Outlier magnitude (a).
Figure 5: Performance of FRPCA algorithm for n 10, N 100, K 2, r = 4 and 10% outliers.
70
350
Number of non-zero elements in
300
250
Ponalized objectiv
55
200
150
45
100
50
30
0
5
10
15
20
25
30
50
100
150
Iteration
a) The Objective in 35
vs iteration
b) Number of non-zero clements vs
Figure 6: FSPCA: n = 40, r r= 10, N 200, K= 2.
we plan to study the convergence properties of the proposed algorithm and derive its convergence rate, with particular focus on problem 33) and the number of iterations ncoded to attain practical convergence.
References
Stephen Boyd and Lieven Vandenberghe. Conver optimization. Cambridge university press, 2004.
Francisco Castells, Pablo Laguna, Leif Sörmmo, Andreas Bollmann, and José Millet Roig. Principal compo- nent analysis ECG signal processing. EURASIP Journal on Advonces in Signal Processing, 2007:1-21, 2007.
Clifford Clauson and Harry Wechsler. Color image compression using PCA and backpropagation learning. Pattern Recognition, 33(9):1555-1160, 2000.
Alexandre d' d'Aspremont, Laurent Ghaoui, Michael Jordan, and Gert Lanckrict. direct formulation for sparse PCA using semidefinite programming. Advances in Neural Information Processing Systems, 17, 2004.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Qian Du and James E Fowler. Low-complexity principal component analysis for hyperspectral image pression. The International Journal of High Performance Compuling Applicalions, 22(4))438 4488 2008.
Alessandro Giuliani. The application of principal component analysis to drug discovery and biomedical data. Drug discovery today, 22(7):1069 1076, 2017.
Michacl Grant and Stephen Boyd. CVX: Matlab software for disciplined convex programming, version 2.1, 2014.
Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller, Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on Faces in Real- Life Images:: Detection, Alignment, and Recognition, 2008.
lan Jolliffe. Principal component analysis for special types of data. Springer, 2002.
Mohammad Mahdi Kamani, Farzin Haddadpourr Rana Forsati, and Mehrdad Mahdavi. Efficient fair prin- cipal component analysis. Machine Learning, pp. 1-32, 2022.
Nojun Kwak. Principal component analysis based on L1-norm maximization. IEEE Iransactions on Pallern Analysis and Machine Intelligence, 30(9):1672-1666 2008.
Jake Lever, Martin Krzywinski, and Naomi Altman. Points of significance: Principal component analysis. Nature methods, 14(7):641 643, 2017.
Albert W Marshall, Ingram Olkin, and Barry C Arnold. Inequalities: theory of majorization and its appli- cations. Springer, 1979.
Mohammad Mahdi Naghsh, Maryam Masjedi, Arman Adibi, and Petre Stoica. Max min fairness design for MIMO interference channels: A minorization maximization approach. IEEE Transactions on Signal Processing, 67(18)::407-44119 2019.
Benjamin Recht, Maryam Fazel, and Pablo Parrilo. Guaranteed minimum-rant rank solutions of lincar matrix equations via nuclear
minimization. STAM revieu, 52(3))471-500) 501, 2010.
Samira Samadi, Uthaipon Tantipongpipat, Jamie H Morgenstern, Mohit Singh, and Santosh Vempala. The price of fair PCA: One extra dimension. Advances in neural information processing systems, 31, 2018.
Chunhua Shen, Alan Welsh, and Lei Wang. PSDBoost: Matrixx-eneration lincar programming for positive semidefinite matrices learning. Advances in Neural Information Processing Systems, 21, 2008.
Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171-176, 1958.
Abdulhamit Subasi and M Ismail Gursoy. EEG signal classification using PCA, ICA, LDA and support vector machines. Expert systems with applications, (1112)))659-666 2010.
Ying Sun, Prabhu Babu, and Daniel P Palomar. Majorization-miniizzatii algorithms in signal processing, communications, and machine learning. IEEE Transactions on Signal Processing, 65(3):794-816, 2016.
Uthaipon Tantipongpipat, Samira Samadi, Mohit Singh, Jamie H Morgenstern, and Santosh Vempala. Multi- criteria dimensionality reduction with applications fairness. Advances in neural informalion processing systems, 32, 2019.
Rucy S Tsay. Analysis of financial time series. John wiley & sons, 2005.
Gad Zalcberg and Ami Wiesel. Fair principal component analysis and filter design. IEEE Transactions on Signal Processing, 69:4835 4842, 2021.
Licheng Zhao and Daniel P Palomar. Maximin joint optimization of transmitting code and receiving filter in radar and communications. IEFF Transactions Signal Processing, 65(4):850 863, 2016.
16

---

Published in Transactions on Machine Learning Research (02/2025)
Yongbin Zheng, Yuzhuang Yan, Sheng Liu, Xinsheng Huang, and Wanying Xu. An Efficient Approach to Solve the Large-Scale Semidefinite Programming Problems. Mathemalical Problems in Engineering, 2012 (1):764760, 2012.
Hui Zou and Lingzhou Xue. A Selective Overview of Sparse Principal Component Analysis. Proceedings of the IEEE, 106(8):1311-1320, 2018.
Hui Zou, Trevor Hastie, and Robert Tibshirani. Sparse principal component analysis. Journal of Computa- tional and Graphical Statistics, 15(2):263 -26, 2006.
A Appendix
The minimization problem of FPCA in terms of singular values of (given in 29 is as follows:
K
min 2 Σ
(A())) + Mcck
==1
R=1
(46)
K
s.t. Mi 20 0
Σ
11
Since the sum of singular values of matrix is the nuclear norm of the matrix, we have:
min 2 2||A(H)II- + Σ MgCs
(47)
s.t.
0 #4 1,
where ||-||x denotes the muclear norm of the matrix. The problem in 47 can be re-written semi-definite
program (SDP) form Recht et al. (2010) as follows:
min Wa.Wa.
Tr(W,) Tr(W2)
PaCk
(48)
W1
A(
Pk Σ
1,
> 0.
A(U)
W2
k=1
17


Reviews:
[{"review_id": "aasDjSkmbU", "reviewer": "Reviewer_YYj6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n- The paper is very well-written with sufficient background and discussion.\n- The proposed MM method is robust and natural for the max-min type of the objective function of fair PCA.\n- As a strength of MM, it is parameter-free, which means no tuning is needed.\n- The math seems clear and sound to me, I did check some proofs and arguments and they seem correct.\n- Numerical experiments are sound: they do show that the MM method (which is exact in terms of rank) closely matches the method proposed in  [Samadi et al. 2018] (which is -- in general -- not exact in terms of rank due to the parametrization and relaxation) and outperforms the subgradient method. \n\n**Weaknesses:**\n\n- The paper does not contribute any added insight in terms of convergence of the proposed method, but rather, it uses general convergence of the MM.\n- Iteration complexity is of order $n^{4.5}$ where $n$ is the data dimension. Although they managed to propose an iterative cheaper version with a complexity of $n^3$, this version is sub-optimal (hard to say theoretically to stop the inner-loop algorithm).\n- Plotting objective against iterations may be unfair for the subgradient method since the iteration complexity of the MM is higher.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper developed an MM method to solve the fair PCA problem proposed in [Samadi et al. 2018]. The authors further proposed two variants of the fair PCA problem, namely fair robust PCA (FRPCA) and fair sparse PCA (FSPCA) by replacing the (loosely speaking) quadratic objective function with the L1 counterpart and adding the (negative) L1 norm of the optimization variable to the objective function. They show that the mentioned MM method can be straightforwardly extended to tackle those newly proposed problems. They perform several numerical experiments to solve those three problems, and the results are quite sound.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Those recommendations are mainly to improve the paper:\n\n1) Please mention that the crucial requirement of MM is that the surrogates are easy to maximise.\n\n2) Some claims about the MM should be made more accurate. On page 4, it says \"convergence to a stationary point.\" But as far as I know, in general, MM does not converge to any stationary points.  The limiting set should be \"no-progress\" points [Lange21] where MM can not improve the solution anymore, it does not mean the gradient of the objective function vanishes there. Furthermore, there are cases where MM -- at its limit -- jumps around different no-progress points, so it is not convergent [see Lange21]. The objective function of fair PCA has some structure, I would expect a stronger conclusion to hold. The authors cited [Zhao & Palomar 2016], please indicate precisely which theorems in [Zhao & Palomar 2016] apply here.\n\n[Lange21] Lange, Kenneth, et al. \"Nonconvex optimization via MM algorithms: Convergence theory.\" arXiv preprint arXiv:2106.02805 (2021).\n\n3) All the identity matrices should be associated with their dimensions.\n\n4) The first equation of the Proof in 5 seems redundant, I suggest omitting it to make the proof neater.\n\n5) Right after equation (26), please recall that $c_k$ does not depend on $U$ so we can safely ignore the second term.\n\n6) In Algorithm 1 (and other algorithms as well), please explain why the stopping condition is well-defined. For example, as I mentioned, there are cases where MM jumps between no-progress points at its limit, therefore $U^{l+1}$ is not close to $U^t$.\n\n7) In the complexity discussion, a citation is needed for the complexity of $(n+r)^{4.5}$ of the SDP solver.\n\n8) Please give more explanation for the choice of the surrogate in (41). Is it a common choice for L1?\n\n9) The last claim in subsection 5.1. is not obvious to me: \"The proposed FPCA approach always finds a better objective value than the method from Zalcberg & Wiesel (2021) and reaches the SDR bound whenever the latter is achievable.\" I believe this claim is not sufficiently supported.\n\n10) In section 5.4., the description of the added noise is not clear. Please provide more explanations.\n\n11) The claim: \"Unlike the FPCA and the FSPCA case, the objective in (5) has several local maxima\" implies the objective of FPCA and FSPCA do not have any local maxima. This does not seem to be the case because the objectives of FPCA and FSPCA are nonconvex. Please provide explanations.\n\n12) Where is the SDR curve in Figure 2(a)?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OHMguqtNyR", "reviewer": "Reviewer_KaBw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. The paper is well-written, easy to follow, the ideas are explained clearly, and the algorithms are easy to understand.\n2. The paper contains extensive numerical results to support the findings.\n\n## Weakness\n1. While the writing style is clear, the technical content is not well-organized.\n2. The paper mentions several times that the MM algorithm is convergence, but it didn't formally state these convergence results with proper citations.\n\nOverall, the work presents a valuable contribution to the field of FPCA and related extensions, which is of interest to the TMLR audience.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new iterative algorithm based on the minorization-maximization (MM) approach to solve the fair principal component analysis (FPCA) problem. The proposed algorithm is computationally efficient, monotonically increases the objective at each iteration, and does not require any hyperparameter tuning. The authors also present extensions of the algorithm to handle robust PCA (FRPCA) and sparse PCA (FSPCA) formulations of the fair PCA problem. Numerical experiments on synthetic and real-world datasets show the effectiveness of the proposed approach compared to existing state-of-the-art methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I suggest the authors to address the following comments:\n\n1. Please use a unified equation referencing style: equation 4 vs (4), equation 6 vs (6).\n2. The definition of $\\ell$-1 norm of a matrix below (6) should be placed at just after (5).\n3. The definitions in (9) and (10) can be placed before (8) for consistency.\n4. Please consider summarizing the ``interesting properties'' for (23) into a lemma and put the following proof into a former proof.\n5. (28) can be formulated as an SDP, what is the SDP? This is necessary since we need to see the SDP to understand the discussion of computational complexity.\n6. Explicitly state the dependency $A(\\mu)$, otherwise it is quite misleading to ignore $\\mu$.\n7. The problem (33) has to be solved for multiple times, and usually only a few iterations are needed. Is there any theoretical result including the iteration complexity to support this point? \n8. Please also include some discussions regarding the convergence properties (in more details) and the computational cost with memory consumption. \n9. It is also suggested to include comparison of computational time in the experiments?\n10. Are all data set used mean centered?\n11. What is the Section V.A? Please correct similar typos.\n12. Please provide more details on how to get the reformulation (24)?", "broader_impact_concerns": "There aren't any major ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hqwvtK6XmT", "reviewer": "Reviewer_6sKv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is written with clear structure and high quality language. The authors proposed a new MM algorithm for solving the fair PCA (FPCA) problem. The proposed algorithm is computationally efficient and monotonically increases the FPCA objective at each iteration. The authors also proposed two MM algorithms to solve the fair robust PCA and fair sparse PCA problems. They conducted numerical experiments on both synthetic data sets and a real-life data set and compared the performance of the proposed approach with that of two state-of-the-art approaches. The authors provided details about all the theoretical analysis and mathematical proof and the empirical results from the numerical experiments are consistent with the theoretical results.\n\nOne interesting advantage of this submission is that the idea of the application of MM algorithms to solve the PCA problem. This is one novel idea and method. The main idea of the MM algorithm is very similar to the expectation–maximization (EM) algorithm, which is an iterative method to find (local) maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, Both of them constructed one upper/lower bound of the objective function and then maximize/minimize this upper/lower bound and alternates between these two steps until the convergence error is small enough. This idea is more likely an application of the MM/EM/k-means algorithms into the calculation of the PCA problem. Hence, the main idea of this submission is somehow impressive.\n\nThere are no major weakness of this submission.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this submission, the authors we propose a new iterative algorithm to solve the fair PCA (FPCA) problem. First, they discuss the general MM approach for a maximization problem, and then the MM approach for max-min problems. The authors present the proposed algorithm for the FPCA problem and discuss an efficient way to implement it. They consider an extension of the proposed method to deal with fair robust PCA and fair sparse PCA problems. The authors also conduct several numerical simulation results on both synthetic and real-life data and compare the proposed method with two state-of-the-art methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "There are no significant changes. The authors should add more contents/previous works in the introductory section about the history of the application of MM method to sole the. problems like PCA. Also Forbenius norm is usually denoted as $||A||_F$ instead of $||A||$ for some matrix $A$.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: While the authors have addressed all reviewers' concerns, some of the requested changes from reviewer KaBw should be better implemented in the paper, to improve clarity and claims/evidence:
- question 4: Lemma 2 proof is missing the Latex commands begin{proof} and end{proof}
- question 5: Please include the SDP in the appendix.
- question 7: Please mention that while problem (33) is solved in less than 10 iterations experimentally for problems of size up to n=100, there is no theoretical result on this respect.
- question 9: Please include a table of the computational time. Varying only one parameter (n, r or K) while keeping the rest fixed might be a way to do a more proper comparison of the computational time.
- question 12: Please add this small explanation to the paper for didactical purposes.

Finally, it is very important to add some ideas for future work in Section 6. Perhaps question 7 from reviewer KaBw is one candidate, but other ideas will be highly welcome by TMLR's readership.
Decision: Accept (poster)