Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents an optimization based formulation of fair principal components analysis problem with a particular focus on the robustness of the optimal models. In the case when the desired robustness is specified by Gaussian distributions, the authors show how to adapt a manifold subgradient descent to solve a non-convex reformulation of the proposed fair pca formulation.

main_review: Strength: + Robustness criterion for linear transformation of fairness can be formulated as a single optimization problem as long as the ambiguity set is defined using a single normal distribution.
+ A smooth reformulation is presented which can be solved using standard riemannian gradient descent algorithm.

Weakness: - Generalization to multiclass sensitive attribute, and other fairness metrics is missing.
- Figure 1 illustrating different approaches is nice, although it fails to capture the distributional robustness aspect of the formulation presented in the paper.
- It is hard to follow the paper since the notations are not standard.
- Experimental results are preliminary and conclusions are not clear.

summary_of_the_review: Justification:  The technical content is interesting, although in its current form, it is hard to follow. For example, why is problem (4)  infinite dimensional? As far as I can see, the decision variables in optimization problem (4) is a d by k matrix which is finite dimensional, although the zeroth order and first order oracles maybe noisy. This is evident in their convergence analysis, where they can only bound the gradient norm of the reformulated variable U (which in its original formulation is V). The paper contains some technical insights in the formulation as a SDP which happens to have a nice connection to Riemannian geometry as illustrated. However, at times, the paper reads as if one has to refer multiple textbooks just to make sure that paper indeed is technically correct. For example, many quantities appear without definitions (\hat{p}_a is not defined), and sometimes with somewhat unclear notations (Q(dx\times da). To me, the biggest technical question that the paper fails to answer is the generalization to nonbinary features. This is important in the context of the paper's contribution because we know that standard SDP solvers do not scale to settings with big or high dimensional data. The convergence rate in this setting is also not clear since a naive extension would indicate at least polynomial slowdown. The experiments are also inconclusive since the paper focuses exclusively on the binary sensitive attribute setting, and with just one metric. While the approach here may not be directly extendable to other metrics, the paper completely misses on discussing when the chosen formulation is appropriate for practitioners.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the formulation, reformulation and algorithm for distributionally robust fairness-aware PCA. The reformulation exploits techniques from distributionally robust optimization, and the algorithm is based on reimannian sub-gradient descent. The theory is tested on UCI datasets.
main_review: Strength:

1. The paper is nicely written and easy to follow. The motivation is clear and the problem formulation is novel.

2. Results on reformulation and riemannian gradient descent render tractable algorithms for solving the proposed problem. I do not find major technical errors.

3. Numerical experiments are based on real data set and are comprehensive.

Weakness:

1. The formulation simply combines distributioanlly robust PCA with a fairness penalty. Although this formulation is new, it seems a natural extension of existing frameworks.

2. The theoretical results are relatively easy to derive based on existing results on distributionally robust optimization and riemannian optimization. Hence, the originality is not methodological, but just applying well-established tools to a new formulation.

3. The dimensions of variables in UCI datasets are still relatively low (<=40 according to Table 2). The results can be strengthened by testing on datasets of larger dimensions.

summary_of_the_review: This paper proposes a new formulation for PCA and derives a complete set of results with regard to the computational issues of the new results. My only reservation is that most theory are relatively straightforward compared to existing literature on distributionally robust optimization and Riemannian optimization.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper aims to improve fairness and distributional robustness in dimensionality reduction techniques. More specifically, it proposes a regularization term to principal compoenent analysis (PCA) such that the expected reconstruction errors for each groups (conditioned on a binary sensitive variable) are similar, and relaxes constraint on the empirical probability distribution by maximizing over a Wasserstein-type ambiguity set. This is optimized with subgradient descent on the Stiefel manifold, and numerical experiments comparing the proposed method to previous fair PCA are conducted.
main_review: This paper tackles an interesting problem, and is mostly well-written and easy to follow. The problem formulation and re-formulation steps are not trivial.

However, there are places in this paper that needs more explanation and justification. 

- There needs to be a discussion on the assumptions for Theorem 3.2. How restrictive are they? Were they satisfied in the experiments? For example, (i) implies that the regularization parameter is less than 0.5, which will restrict the flexibility shown in Figure 1. 
- The ambiguity set B(P) is a central part of the proposed method. What is the justification for using Wasserstein-type divergence as opposed to others? Also, the second line of equation (5) needs to be explained more carefully.
- In defining $gap_\mu(U)$, is $||U'-U||_F$ the best choice to measure the difference between two orthonormal matrices? One can think of many examples with rotations where $U$ and $U'$ span the same subspace but will have very large frobenius norm difference. A more typical choice in the literature would be $||UU^T - U' U' ^T||$. 
- Is the RHS of Theorem 4.3. necessarily decreasing with $t$? It's not obvious to me why this guarantees convergence.
- Experiments and discussions on the robustness of the proposed method are lacking, compared to the fairness aspect. Are there any comparisons to other robust PCA methods, or simulation experiments?
- In the experiments, how were the sensitive attributes chosen? Were they based on what are considered sensitive attributes in the real world, e.g. race, gender, or were they chosen randomly?
- This paper is lacking a conclusion section. It would have made it stronger if the authors mentioned how they would extend this work to non-binary sensitive attributes.
- On a very high level, it's not *as* obvious why reconstruction error should be balanced compared to, for example, in supervised learning. The toy example in Figure 1 is helpful but I'm wondering if the authors can describe a real-world example where bias in reconstruction error will be harmful - where PCA is the end result rather than a preprocessing step for supervised learning problem.

Other minor problems that can be fixed with careful editing, such as:
- Page 2: "signal process" -> signal processing
- Page 5: "By the definition the ambiguity..."
- Page 7: "At each iteration, the point..." sentence is hard to parse.
- Page 9: "In Each dataset, ..." "other features is..." "the the generalization..."
summary_of_the_review: It's a good paper, but is missing some interpretation of its theoretical results and justifications. It is likely that I will change my recommendation after the discussion period, if the authors provide more details.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper considers the fair PCA with distribution ally robust optimization algorithm. The problem is of great interest. The key idea of the proposed method is to consider a penalized loss function with the penalty on the fairness criterion. The distributionally robust optimization and its reformulation is developed for parameter estimation. 
main_review: The strength of the paper is the rigorous optimization framework to solve the proposed fair PCA, with a nice reformulation and theoretical property. 

The weakness of this paper is in the following folds. First, it is not very clear on the significance of the novelty. The authors cast the PCA problem as the minimization of the penalized reconstructed error. Such a formulation is widely common in the machine learning community. Because of the L2-type loss function, the use of the distributionally robust optimization is not that surprising with the known complication on the penalty part. Second, the proposed method involves the selection of tuning parameter. While the authors seem to avoid the discussion tuning parameter selection. Moreover, the numerical experiments just consider the use of fixed value of these tuning parameters, which makes reader hard to judge the quality of the numerical comparison. Third, due the use of the penalty to engage the fairness, it is not how good the proposed method enables the fairness, especially there is no criterion on the selection of the tuning parameter \lambda. Fourth, under the context of fairness, the authors also avoid the discussion on how the choice of k, the number of PCs, would affect the fairness.  
summary_of_the_review: 
Based on the aforementioned strengths and weakness, I would consider the proposed method contain interest merits, with the conservation of the sufficiency on the methodology novelty, framework rigor, and the applicability in practice.
review_title: null

Meta-review: This paper considers the problem of distributionally robust fair PCA for binary sensitive variables. The main modeling contribution of the paper is the consideration of fairness and robustness of the PCA simultaneously, and the main technical contribution of the paper is the provision of a Riemannian subgradient descent algorithm for this problem and proof that it reaches local optima of this non-convex optimization problem. The results will be of interest to those working at the intersection of fair and robust learning.
Decision: Accept (Poster)