Towards Marginal Fairness Sliced Wasserstein Barycenter
Authors: Khai Nguyen
Hai Nguyen
Nhat Ho
Venue: ICLR 2025 Spotlight (2025)
Keywords: Sliced Wasserstein Barycenter
Optimal Transport
Sliced Wasserstein
Averaging Measures.
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We discuss the notion of marginal fairness sliced Wasserstein barycenter which is a special case of sliced Wasserstein barycenter where the distances to marginals are approximately equal.

Abstract: The Sliced Wasserstein barycenter (SWB) is a widely acknowledged method for efficiently generalizing the averaging operation within probability measure spaces. However, achieving marginal fairness SWB, ensuring approximately equal distances from the barycenter to marginals, remains unexplored. The uniform weighted SWB is not necessarily the optimal choice to obtain the desired marginal fairness barycenter due to the heterogeneous structure of marginals and the non-optimality of the optimization. As the first attempt to tackle the problem, we define the marginal fairness sliced Wasserstein barycenter (MFSWB) as a constrained SWB problem. Due to the computational disadvantages of the formal definition, we propose two hyperparameter-free and computationally tractable surrogate MFSWB problems that implicitly minimize the distances to marginals and encourage marginal fairness at the same time. To further improve the efficiency, we perform slicing distribution selection and obtain the third surrogate definition by introducing a new slicing distribution that focuses more on marginally unfair projecting directions. We discuss the relationship of the three proposed problems and their relationship to sliced multi-marginal Wasserstein distance. Finally, we conduct experiments on finding 3D point-clouds averaging, color harmonization, and training of sliced Wasserstein autoencoder with class-fairness representation to show the favorable performance of the proposed surrogate MFSWB problems.

Published as a conference paper at ICLR 2025
TOWARDS MARGINAL FAIRNESS SLICED WASSERSTEIN BARYCENTER
Khai Nguyen*
Hai Nguyen"
Department of Statistics and Data Sciences University of Texas Austin Austin, TX 78713, USA
Qualcomm AI Research! hainnêqti. qua comm, com
khainb@utexas.c
Nhat Ho
Department of Statistics and Data Sciences University of Texas Austin Austin, TX 78713, USA
sinhnhattuteeeen edu
ABSTRACT
The Sliced Wasserstein barycenter (SWB) a widely acknowledged method for efficiently generalizing the averaging operation within probability measure spaces. However, achieving marginal fairness SWB, ensuring approximately equal distances from the barycenter to marginals, remains unexplored. The uniform weighted SWB is not necessarily the optimal choice to obtain the desired marginal fairness barycenter due the heterogencous structure of marginals and the non- optimality of the optimization. As the first attempt to tackle the problem, we define the marginal fairness sliced Wasserstein barycenter (MFSWB) strained SWB problem. Due to the computational disadvantages of the formal definition, we propose two hyperparameter--fiee and computationally tractable sur- rogate MFSWB problems that implicitly minimize the distances to marginals and encourage marginal fairness at the same time. To further improve the efficiency, we perform slicing distribution selection and obtain the third surrogate definition by introducing a new slicing distribution that focuses more on marginally unfair projecting directions. We discuss the relationship of the three proposed problems and their relationship to sliced multi-marginal Wasserstein distance. Finally, we conduct experiments finding 3D point-clouds averaging, color harmonization, and training of sliced Wasserstein autoencoder with flass-fairness representation to show the favorable performance of the proposed surrogate MFSWB problems
con-
INTRODUCTION
generalizes "averaging" to the space of probability measures. In particular, a Wasserstein barycenter a probability measure that minimizes a weighted sum of Wasserstein distances between it and some given marginal probability measures. Due to the rich geometry of the Wasserstein distance (Peyré & Cuturi 2020). the Wasserstein barycenter can be seen the Fréchet mean (Grove & Karcher] 1973) the space of probability measures. As result, Wasserstein barycenter has been applied widely to various applications in machine learning such Bayesian inference (Srivastava al. 2018 Staib al., 2017), domain adaptation (Montesumal & Mboula, 2021). clustering (Ho et al. 2017]. sensor fusion (Elvander et al. 2018), text classifica- tion (Kusner et al., 2015), and so on. Moreover, Wasserstein barycenter is also powerful tool for computer graphics since it can be used for texture mixing (Rabin et al. 2012). style transfer (Mrouch) 2020), shape interpolation (Solomon al. 2015), and many other tasks on many other domains.
Wasserstein barycenter (Agueh & Carlier 2011)
* "Equal Contribution
"Qualcomm Vietnam Company Limited
Code for the paper is httpttp:://gihhub..cc/////

---

Published as a conference paper at ICLR 2025
Margtabb
LSWe 1w5272.7, Ww07.32
NESSAD A- 574.55 W-117..0
Figure 1: The uniform SWB and the MFSWB of 4 Gaussian distributions.
Despite being useful, it is very computationally expensive to compute Wasserstein barycenter. In more detail, the computational complexity of Wasserstein barycenter is O(n log n) when using linear programming (Anderes et al.] 20T6) where n is the largest number of supports of marginal probability measures. When using entropic regularization for optimal transport (Cuturi 2013), the
computational complexity is reduced to (nn)) (Kroshnin et al., 2019). Nevertheless, quadratic scaling is not enough when the number of supports approaches a hundred thousand or million. To address the issue, Sliced Wassserstein Barycenter (SWB) is introduced in (Bonneel et al. (013) by replacing Wasserstein distance with its sliced variant i.e., Sliced Wasseretein (SW) distance. Thank to the closed-form of Wasserstein distance in one-dimension, SWB has a low time complexity i.e., (n log n)) which enables fast computation. Combining with the fact that Sliced Wasserstein is equivalent to Wasserstein distance in bounded domains (Bonnotte (2013) and Sliced Wasserstein does not suffer from the curse of dimensionality (Nguyen al. [021] Nadjahi et al. 2020, Manole et al.) 2022
Nietert et al. 2022). SWB becomes a scalable alternative choice of Wasserstein barycenter.
In some applications, we might want to find a barycenter that minimizes the distances marginals while having equal distances to marginals at the same time e.g., constructing shape template for group of shapes (Bongratz et al. 2022 Sun et al. 2023) that can be further used downstream tasks, exact balance style mixing between images (Bonneel et al., 2015), fair generative modeling (Choi al. 2020), and so on. We refer to such a barycenter as marginal fairness barycenter. Both the Wasserstein barycenter and SWB are defined based on a given set of marginal weights (marginal coefficients), and these weights represent the importance levels of marginals toward the barycenter. Nevertheless, uniform (weights) barycenter does not necessarily lead to the desired marginal faimess barycenter as shown in Figure [ Moreover, obtaining the marginal fairness barycenter is challenging since such barycenter might not exist and might not be identifiable given non-global-optine optimization (Karcher mean problem). To the best of our knowledge, there is no prior work that investigates finding a marginal fairness barycenter.
In this work, we make the first attempt to tackle the marginal fairness barycenter problem i.c., we focus on finding Marginal Fairness Sliced Wasserstein Barycenter (MFSWB) to utilize the scalability of SW distance.
Contribution: In summary, our main contributions four-fold:
1. We define the Marginal Fairness Sliced Wasserstein Barycenter (MFSWB) problem, which is constrained barycenter problem where the constraint aims to limit the average pair-wise absolute difference between distances from the barycenter to the marginals. We derive the dual form of MFSWB, discuss its computation, and address its computational challenges.
2. To address this issue, we propose surrogate definitions of MFSWB that are hyperparameter-fie free and computationally tractable. Motivated by Fair PCA (Samadi et al., 2018), we propose the first surrogate MFSWB, which minimizes the largest SW distance from the barycenter to the marginals. To solve the problem of biased gradient estimation of the first surrogate MFSWB, we propose the second surrogate MFSWB, which is the expectation of the largest one-dimensional Wasserstein distance from the projected barycenter to the projected marginals. We show that the second surrogate is upper bound of the first surrogate and can yield unbiased gradient estimator. We further extend the second surrogate to the third surrogate by applying slicing distribution selection and show that the third surrogate is an upper bound of the previous two.
3. We discuss the connection between the proposed surrogate MFSWB problems and the Sliced Multi-marginal Wasserstein (SMW) distance with the maximal ground metric. In particular, solving
2

---

Published as a conference paper at ICLR 2025
the proposed MFSWB problems is equivalent to minimizing lower bound of the SMW. By showing that the SMW with the maximal ground metric is generalized metric, we demonstrate that safe to use the proposed surrogate MFSWB problems.
We conduct simulations with Gaussian data and experiments on various applications, including 3D point-cloud averaging. color harmonization, and sliced Wasserstein autoencoder with class-fat fair representation, to demonstrate the favorable performance of the proposed surrogate definitions.
Organization. We first discuss some preliminaries on SW distance, SWB, its computation, and Sliced Multi-marginal Wasserstein distance in Section 2 We then introduce the formal definition and surrogate definitions of marginal fairness SWB in Section 3 Next, conduct experiments to demonstrate the favorable performance and fairness of the proposed definitions in Section 4 We conclude the paper and provide some future directions in Section [3] Finally, we defer the proof's of key results, the discussion on related works, and additional materials to the Appendices.
2
PRELIMINARIES
Sliced Wasserstein distance. The definition of sliced Wasserstein (SW) distance (Bonneel et al. 2015 between two probability measures 111 E Pp (R.)) and μ22 Pp((R.) is:
(W (2) Eg-u [Wg 0fp2)],
where the Wasserstein distance has closed form in one-dimension which is wp(0tun; 01p2)
So |F FoR (2) 10-FFrrt ( (2)l where füt and 0tv denotes the pushforward measures of μ and through the function f(x) = S x, Feau, and Feiue are the cumulative distribution function (CDF) of 0fj1 and 02 respectively.
Sliced Wasserstein Barvcenter. The definition of the sliced Wasserstein barycenter (SWB) prob- lem (Bonneel et al. 2015) of K 2 2 marginals Pi...., PK (R(R)) with marginal weights
W1 WKK 0 (2 We 1) is defined as:
min (..K ((GG
= Σ wi w (p, m4x)
(2)
R=1
When W1 -...:w. wK 1/K, we obtain an uniform SWB problem.
Computation of parametric SWB. Let Pa be parameterized by & E Φ SWB can be solved by gradient-based optimization. In that case, the interested quantity is the gradient Vo H1:K wI:K) Σx wk SWP pk). However, the gradient
VoSWB pk) V Vo [WP 01px)] Egnurss 1n V WP (O 0ppx)]
for any l: 1, 1,.... K is intractable due to the intractability of SW with the expectation with respect to the uniform distribution over the unit-hypersphere. Therefore, Monte Carlo estimation is used. In particular, projecting directions 0, 0,, gradient estimator is formed:
Or. are sampled i.i.d from U($d-1), and the stochastic
V SWT PR) ~
V Wp NN((Ottu,,0ttpu
1=1
With the stochastic gradient, the SWB can be solved by using a stochastic gradient descent algorithm. We refer the reader to Algorithm i Ap more detail. Specifically, we now discuss the discrete SWB i.c., marginals and the barycenter are discrete measures.
Free supports barycenter. In this setting, we have Ps. 1 Σ' de, pe 1 Σ 8g. and (E1,..., In), we can compute the (sub-)gradient with the time complexity On
(O#Hos 0zpk) ploTa x; 0 yク()) 1-
v (- si
0 TT (o()))
(4)
where o = o og with oi and 0g are any sorted permutation of {ø x and fy......yh.. Here, [n] denotes the set (1,2...., [m] - [n] is the permuation function such that To) S
3

---

Published as a conference paper at ICLR 2025
(2) .... (n) (1) (2) 2 ... 2 (a) Similarly, 02 [n] - [n]] is the
permuation function such that (rg(1) S (2) S ... I (og(n) or (1)(1) 2 (2) 2 2 (r2(n))
and 02 is the argsort operator. The transport map is contructed as o = 01 02
Fixed supports barycenter. In this setting, we have pa =Ei= ded2,, Mk th Brb2r E oi Σ 8i and Φ (d) ( Da). We can compute the gradient as follows:
V Wp 0tpN)) = f",
(5)
where f* the first optimal Kantorovich dual potential of (p flpz) which can be obtained with the time complexity (n log n). We refer the reader to Proposition 1 in (Cuturi Doucet 2014) for the detail and Algorithm in (Séjouréé et al. 2022) for the computational algorithm.
When the supports or weights of the barycenter are the output of parametric function, we can use the chain rule to estimate the gradient of the parameters of the function. For the continuous case, we can approximate the barycenter and marginals by their empirical versions, and then perform the estimation in the discrete case. Since the sample complexity of SW is O(n 1/2) Nadjaai al. 2019; N Nguyen et al. [021] Manole et al. 2022, Nietert ct al., 2022). the approximation eтror will reduce fast with the number of support increases. Another option is use continuous Wasserstein solvers (Fan et al. 2021 Korotin et al., 2022 Claici et al. 2018), however, this option is not as simple
as the first one.
Sliced Multi Multi-marginal Wasserstein Distance. Given K > marginals PI..... Mk (p(R)))
Sliced Multi murti-larrgina Wasserstein Distance (Cohen et al. 2021 (SMW) is defined as:
inf (WWx())::)) = (EII((A......R))
[ c(80 ....... 0 x px) (x)....)
xx)
(6)
where the expectation is under 0 ~ 11(3--1). When using the barycentric cost i.e.,
K
K
c(0 T rı, 0 xx)
3k
0
Ik
0
IK
k=
k==1
for 3x > Vk and Σg 8g = 1. Minimizing SMW? (HI.K. pi; c) with respect to μ is equivalent to
barycenter problem. We refer the reader to Proposition 7 in (Cohen et al. 2021) for more detail.
3 MARGINAL FAIRNESS SLICED WASSERSTEI BARYCENTER
We first formally define the marginal faimeess Sliced Wasserstein barycenter in Section 3.1 We
then propose surrogate problems in Section 3.2 Finally, we discuss the connection of the proposed surrogate problems to sliced multi-marginal Wasserstein in Section 3.3
3.1 FORMAL DEFINITION
Now, we define the Marginal Faimness Sliced Wasserstein Barycenter (MFSWB) problem by adding marginal fairness constraints the SWB problem.
Definition 1. Given K 2 2 marginals pi...., .PK E (R")), admissible 2 0 = -I....A K
and + 1....,KK K, the Marginal Fairness Sliced Wasserstein barycenter (MFSWB) is defined as:
1 K J K
K-1
K
min
Σ
SW. (H,
s.t. (K-
2
Σ
Σ
(W p.) SWT S
(7)
1)K
E=1
1=H+1
Remark 1. We want in Definition to be close to O i.e., pi...., .p. are on the SW SWp-sphere with the center μ. However, for a too-small value ofe, there might not exist a solution p.
Duality objective, For admissible dual form
0, there exist a Lagrange multiplier À such that we have the
K Σ SW (H, + (K-11) 1)K
K 1-1
K
21
C(u, A) = KK
(W,(H))) - SWP(M,))) dc. (8)
1=11
4

---

Published as a conference paper at ICLR 2025
Computational challenges. Firstly, MFSWB in Definition requires admissible 0 to guarantee the existence of the barycenter jr. In practice, it is unknown if a value of e satisfies such a property. Secondly, given it is not trivial to obtain the optimal Lagrange multiplier X in Equation equa- tion 8 minimize the duality gap, which can be non-zero (weak duality). Thirdly, directly using the dual objective in Equation equation reauurees hyperparameter tuning for À and might not provide a good landscape for optimization. Moreover, we cannot obtain an unbiased gradient estimate of in the case of the parametric barycenter pé. In greater detail, the Monte Carlo estimation of the absolute distance between two SW distances is biased. Finally, Equation equation 8 has a quadratic time complexity and space complexity in terms of the number of marginals, i.e., O(K3).
3.2 SURROGATE DEFINITIONS
Since it is not convenient to use the formal MFSWB in applications, propose three surrogate definitions of MFSWB that are free of hyperparameters and computationally friendly.
First Surrogate Definition. Motivated by Fair PCA (Samadi et al. 2018), we propose practical surrogate MFSWB problem that is hyperparameter-free free.
marginals (K,,...,KK E (RRR), the surrogate Marginal Fairness Sliced Wasserstein Barycenter (s-MFSWB) problem is defined as:
Definition Given K 2
min Pr:k) SF(;; (1:k) = max SWP(H, (), ky
The S-MFSWB problem tries minimize the maximal distance from the barycenter to the marginals. Therefore, it can minimize indirectly the overall distances between the barycenter to the marginals and implicitly make the distances to marginals approximately the same.
Gradient estimator. Let Pa be paramterized by ø E Φ and F(Ø, k) (W' MR). we would like to compute V maxke(1..... K} F(O, k)). By Danskin's envelope theorem (Danskin 2012), we have:
max ke(.....
F(o, VoF( V (SS ps=).
for k* arg max K} F(o, k). Nevertheless, k* is intractable due to the intractablity of SWT (K)) for k = K. Hence, can form the estimation
arg max SW (Hep pki L) KE
where SW (1os Pai L. ΣL_ Wy O(tr)) with 0 0 iid U(S Then, we can estimate V S (Ro, Pg.) as in Equation 3 We refer the reader to Algorithm in Appendix B for the gradient estimation and optimization procedure. The downside of this estimator is that it is biased.
Second Surrogate Definition. To address the biased gradient issue of the first surrogate problem, propose the second surrogate MESWB problem.
Definition Given K > 2 marginals Pi..... U4 E (R.)), the unbiased surrogate Marginal Fairness Sliced Wasserstein Barycenter (usMMFSWB) problem is defined as:
minuSF(ur USF(u;)))) = BB-
WP WF(0in, (5px)) ME(I,..,
max
(10)
In contrast to M-BSSWB which minimizes the maximal SW distance among marginals, uF-MFSWB minimizes the expected value of the maximal one-dimensional Wasserstein distance among marginals. By considering fairness on one-dimensional projections, us-MFSWB can yield an unbiased gradient estimate which is the reason why named as unbiased s-MFSWB.
Gradient estimator. Let Ps be paramterized by Φ E 4, and F(O, 0, = WP 0luw). we would like to compute VaBa-gu-1 [maxke(L..... F(0, F(O,, 6, k)] which is equivalent to
Egaaga- (V maxte(1.....t F(B, o. k)] due to the Leibniz's rule. By Danskin's 's envelope theorem, have:
Φ max kE(l....K)
F(0, k) = V VoF(0,, = V Wg 01yx=)
5

---

Published as a conference paper at ICLR 2025
for k3 = arg ma (l.nk) F(0, k.k) where we can estimate V WD((Hpp 0tuzz) can be com- puted as in Equation 4 5 Overall, with .,,...,0r, Or. iid U($L-1), we can form the final estimation
1 E V Wapp Ortuug ) which is an unbiased estimate. We refer the reader to Algorithm in
Appendix for the gradient estimation and optimization procedure.
Proposition 1. Given K 2 2 marginals P1.K Ppp(R)) we have SF(E:ME:) S UUFF(; P1.k)
Proof of Proposition is given in Appendix A.1 From the proposition, we see that minimizing the objective of Us-MFSWB also reduces the objective of s-MFSWB implicitly.
Proposition Given K 2 marginals , E (R()), .1,...,0,.
Os. i.d u(Sd-1), we have:
1 Var ( 0dux;)} (11)
E
Wg(Orduss Octwug) V VOUSF(uoiHIN S
VI
where kg arg me WBp 0fuu); and the expectation and variance are under the random projecting direction 0 ~ (
Proof of Proposition 2 is given in Appendix A.2 From the proposition, we know that the approxi- mation error of the gradient estimator of us- WFB reduces at the order of (I.-1/2). Therefore, increasing L leads to better gradient approximation. The approximation could be further improved via Quasi-Monte Carlo methods (Nguyen et al. 2024a).
Third Surrogate Definition. The us-MFSWB in Definition 3 utilizes the uniform distribution as the slicing distribution, which is empirically shown to be non-optimal in statistical estimation (Nguyen et al.) 2021). Following the slicing distribution selection approach in (Nguyen & Ho (202)). propose the third surrogate with a new slicing distribution that focuses on unfair projecting directions.
Marginal Fairness energy-based Slicing distribution. Since we want to encourage marginal faimess, is natural to construct the slicing distribution based on fairness energy.
Definition 4. Given K3 2 marginals uI....,,u. MKK E (K.( the Marginal Fairness energy-based Slicing distribution a (0; pb (1:k)= E P(Sá-1) is defined with the density function as follow:
x exp
max (KE(I....
(Pi0n 0tws))
(12)
We see that the marginal fairness energy-based slicing distribution in Definition 4 put more mass to a projecting direction 0 that has the larger maximal one-dimensional Wasserstein distance marginals. Therefore, it will penalize more marginally unfair projecting directions.
Energy-based surrogate MFSWB. From the new proposed slicing distribution, we can define new surrogate MFSWB problem, named Energy-based surrogate MFSWB.
Definition 5, Given K 2 2 marginals (.,,..., PK E (TR.), the energy-based surrogate Marginal Fairness Sliced Wasserstein Barycenter (es-MFSWB) problem is defined as:
max AE(LL...K) WP(0sp, 0Fpp) WF(02)
min EFF(FFFFPPP.. ESF(M (1.K) (0~0((ppp)).)) P
(13)
Similar to the us-MFSWB, CS-MFSWB also employs the implicit mee-immnniionll marginal fairness. Nevertheless, cS-MFSWB utilizes the marginal fairness energy-based slicing distribution to reweight the importance of each projecting direction instead of treating them equally.
Proposition 3. Given K 2 2 marginals μi,k (RR),, we have ((F S CSF(M
that minimizing the objective of es- MFSWB implicitly reduces the objective of us-MFSWB thereby decreasing the objective of 5-MFSWB well (Proposition
Proof of Proposition ii given in Appendix A.3 According to the proposition,
Gradient estimator. Let Aa be parameterized by ó E ④. we want to estimate V V&ESF(ug p.x)). Since the slicing distribution is unnormalized,
importance sampling to form an estimation.

---

Published as a conference paper at ICLR 2025
iid u($ we can form the importance sampling stochastic gradient estimation: With 01....,01 02
exp [Wg(Oddu (O:du,,0EERRR;) )) (0,EM,, Oetien
1 V (7 0rines
(e L.
exp (
which can be further derived by using the chain rule and previously discussed techniques. It is worth noting that the above estimation is only asymptotically unbiased. We refer the reader to Algorithm in Appendix B for the gradient estimation and optimization procedure.
Computational complexities of proposed surrogates. For the number of marginals K, the three proposed surrogates have linear time complexity and space complexity i... ((K) which is the
same as the conventional SWB and is better than ((K)) of the formal MFSWB. For the number of projections L, the number of supports n, and the number of dimensions d, the proposed surrogates
have the time complexity of O (Lnn(log) n + d)) and the space complexity of (L((n + d)) which are similar the formal MFSWB and SWB.
3.3 SLICED MULTI-MARGINAL WASSERSTEIN DISTANCE WITH MAXIMAL GROUND METRIC
To shed some light on the proposed substrates, we connect them to a special variant of Sliced multi-marginal Wasserstein (SMW) (see Equation ( i.e., SMW with the maximal ground metric
c(l Ii....,, 0 xx) =
max (.J....,...E(.......
(0 x; 0 xyl.
We first show that SMW with the maximal ground metric is generalized metric on the space of probability measures.
Proposition 4. Sliced multi-marginal Wasserstein distance with the maximal ground metric is generalized metric i.e., it satisfies non-negativity, marginal exchangeability, generalized triangle inequality, and identity of indiscernibles.
Proof of Proposition 4 is given in Appendix A.4 It is worth noting that SMW with the maximal ground metric has never been defined before. Since our work focuses on the MFSWB problem, we will leave the careful investigation of this variant of SMW to future work.
Proposition 5. Given K marginals p1,..... =KK E (,(R)), the maximal ground metric c(OT 21 2...... T [T/==== (1..)) 10Tx;
x; 0 2z5. we have:
minUSF(p1 (2.)) < min SMWP (M1, (K
(14)
Proof of Proposition is given in Appendix A.5 and the inequality holds when changing μl to any μ4 with = 2......... K. Combining Proposition we have the corollary of min,, SF(m; (2.k) S ming, SMWP (1,, M2.-. MKK; c). From the proposition, we see that minimizing the us-MFSWB is equivalent to minimizing lower bound of SMW with the maximal ground metric. Therefore, this proposition implies the us-MFSWB could try to minimize the multi-marginal distance. Moreover, this proposition can help understand the proposed surrogates through the gradient flow of SMW. We can further extend the proposition to show the minimizing es-MFSWB objective is the same as minimizing a lower bound of energy- based SMW with the maximal ground metric, new special variant of SMW. We refer the reader to Propositon 6 in Appendix B for more detail.
4 EXPERIMENTS
In this section, we compare the barycenter found by our proposed surrogate problems i.e., s-MFSWB, us- US-MFSWB, and es cs-MFSWB with the barycenter found by USWB and the formal MFSWB. For evaluation, we use two metries i.., the F-metric (F) and the W-metric (W) which are defined as follows:
K-1 K
1 K
= K(K
Σ
Σ 5=
(F(,, - W (g)),, W = K Σ
W (p,
where z is the barycenter, MI..... uK are the given marginals, and WT is the Wasserstein dis- tance (Flamary al.) (021) of the order p. Here, the F-metric represents the marginal fairness degree of the barycenter and the W-metric represents the centrality of the barycenter. For all following experiments, p 2 for the Wasserstein distance and barycenter problems.

---

Published as a conference paper at ICLR 2025
200 NORT-NON-----
BATT WHALLL
Figure 2: Barycenters from USWB, MFSWB with I, s-FFSW. us-MFSWB, and es-MFSWB along gradient iterations with the corresponding F-metric and W-metrie.
4.1 BARYCENTER OF GAUSSIANS
We first start with simple simulation with 4 marginals which empirical distributions with 100 i.i.d samples from 4 Gaussian distributions i.e., (N0, I) ((20,, (), 8), I). and (((11,-8), I). We then find the barycenter which is represented 100 supports initialized by sampling i.i.d from (((0, -), I). We use stochastic gradient descent with 50000 iterations of learning rate 0.01, the number of projections 100. We show the visualization of the found barycenters with the corresponding F-metric and W-metric by using USWB, s-MFSWB, us-MFSWB, and es- MFBFSWW at iterations O, 1000, 5000, and 50000 in Figure 2 the USWB does not lead to marginal fairness barycenter. The three proposed surrogate problems help to find better barycenter faster in both two metrics than USWB. At convergence i.c., iteration 50000, we that USWB does not give a fair barycenter while the three proposed surrogates lead to a more fair barycenter. Among the proposed surrogates, CE-MESWB gives the most marginal faimeess barycenter with competitive centerness. The formal MFSWB (dual form with À = leads to the most fair barycenter. However, the performance of the formal MESWB is quite sensitive to A. We also observe the same phenomenon for different choices of learning rate in Figure 5 in Appendix D We show the visualization for = 0.1 and X = 10 in Figure in Appendix D
empirical distribution with
We observe that
4.2 3D POINT-CLOUD AVERAGING
We aim to find the mean shape of point-cloud shapes by casting a point cloud X = { Tn into empirical probability measures Px= - dz, We select two point-cloud shapes which consist 2048 points in ShapeNet Core-55 dataset (Chang et al. 2013). We initialize the barycenter with spherical point-cloud. We use stochastic gradient descent with 10000 iterations of learning rate 0.01, the number of projections 10. We report the found barycenters for two car shapes in Figure 3 at the final iteration and the corresponding F-metric and W-metric at iterations 1000, 5000, and 10000
8

---

Published as a conference paper at ICLR 2025
Figure 3: Averaging point-clouds with USWB, MESWB (A = 1), M-MFSW us-MFSWB, and es-MFSWB.
Table F-metric W-metric along iterations in point-cloud averaging application.
Deration
1000
W(1)
F12
USWB
3746
65.72
0.33
45.37
1,55
29.81
MESW
152.20
3746
4.76
84.85
3.76 2.0.2
45.2 =.12
1,32
39.73 0
MESWI
250.94
0.0
3746
0.39:8
7900 0.35
48.1
140 0:10
39.03 9
MFSWW
252.21
0.0
3746-05 05
0.9
7.30++2.
15:11+000
0.15+22.2 +.5.6
02.56 + -0:6
MESWD
252.24
0.0
0748.05
1: 0.0
2.22::0.7
81:84 0:344
4.01 0.28
4.15=0113
1.15 2: 0,00
22,88 à 0.17
us MPSWB
252.24
0.0
3748.05
0.0
0.1 0.11
78.09 2.00..0..3
0.24
44.08 2.0.2.0.33
0.57
2: 0.15
30.21 2.1
65 MESWB
252.24
0.0
3748.05
8.0
0.21..019
2:.16
3.5.2.0.0
44.37 2: 0.08
0.84 0.22
39.35 :: 0.08
in Table T from three independent runs. As in the Gaussian simulation, S-MFSWB, us-MFSWB, and es-MFSWB help to reduce the metrics faster than the USWB. With the slicing distribution selection, es BS-MFSWB performs the best at every iteration, even better than the formal MFSWB with three choices of À i.e., 0.1, 10, We also observe a similar phenomenon for two plane shapes in Figure 7 and Table 3 Appendix D We refer the reader to Appendix D for a detailed discussion.
4.3 COLOR HARMONIZATION
We want transform the color palette of a source image, denoted as X = (21....,, for the number of pixels, to be an exact hybrid between two target images. Similar to the previous point-cloud averaging, we transform the color palette of an image into empirical probability measure over colors (RGB) i.e., Px 1 Ei= &2.- We then minimize barycenter losses i.e., USWB, MFSWB (A E 1, 10)), M-MFSWB, us- MFFSB and cs-MFSWB by using stochastic gradient descent with the learning rate 0.0001 and 20000 iterations. We report both the transformed images and the corresponding F-metric and W-metric in Figure We also report the full results in Figure g Appendix D As in previous experiments, we that the three proposed surrogates yield better barycenter faster than USWB. The proposed es-MFSWB is the best variant among all surrogates since has the lowest F-metric and W-metric all iterations. We refer the reader Figure - Figure [4 in Appendix D for additional flowers-image example, where a similar relative comparison happens. For the formal MFSWB. is worse than S--SSWW in one setting and better than MFFWBB in one setting with the right choice of X. Therefore, it is more convenient to use Us-MFSWB in practice.
TO in
4.4 SLICED WASSERSTEIN AUTOENCODER WITH CLASS- FAIR REPRESENTATION
Problem. We consider training the sliced Wasserstein autoencoder (SWAB) Kolouri et al. 2018 with class-faimes regularization. In particular, we have the data distributions of K > classes i.c.. μ4 E ((Rd)) for = 1...., , K and we would like to estimate an encoder network fø Rd + R² (ø E ④) and a decoder network 9p : h + Rdd (2b v with R²th a low-dimensional latent space.
Given prior distribution μ4ر E P( p 2 1, K1 E R+, K2 E R+, and a minibatch size 2 1, we perform the following optimization problem:
mini 0.v
dl (X (x + P(GIXNEE) +o)B(P) P(o(X.) P(ocxxe)
KM
A=
1=1
where (X1.... Xx) RM 8. &.. RM ~ 1 RM cis a reconstruction loss, Pz
M ΣM 82,-
2k=1 DE (x)) P))(XN) M EM 8),(x,) for k = 1..... K, and B
denotes barycenter loss i.e., USWB, MESWB, s-MFSWB, us-MFSWB, and es-MESWB. This

---

Published as a conference paper at ICLR 2025
Source Image
Togpe Image
Tirger Image
MFSMAA- www-LLL-1111100.000 1464
Figure 4: Harmonized images from USWB, MFSWB 1), M-FSWB us- MFSWFF and es-MFSWB.
Table 2: Results of grid search for learning rates in (0.0001, 0.0005, 0.001} for training SWAE.
Methods
RL (J)
W2. 10 (L)
W Vnirage 10 (L)
F 10 (U)
W 10 (U) Finges (1)
SWAB USWB
3.002
9,949
26.572
28.512
7.787
3.195
9.174
27.446
5.190
12.448
7.140
MESWB A 0.1
2,812
8,981
26.636
17.206
28.734
7.846
MFSWB
1.0
2,883
7.978
26.355
18.069
29.701
7.367
MFSWB 10.0
3,801
8.497
26.658
18.501
28.768
7.950
M-MFSWB
3.170
7.806
28.277
2.037
8.699
7.419
us BBBB
2,833
8,720
27.939
2.072
7.780
6.898
c5-MFSWB
3,056
9.154
28.012
1.760
7.268
7.485
setting can be seen as an inverse barycenter problem i.c., the barycenter is fixed and the marginals are learnt under some constraints (c.g., the reconstruction loss and the aggregated distribution loss).
Results. We train the autoencoder on MNIST dataset (LeCun et al. 1998) (d = 28 x 28) with 2D ball (h 2) 14 with differnt
K1 = 8.0, A2 0.5, 250 epochs, using a uniform distribution learning rates: f0.0001, 0.0005, 0.0008, 0.001} and do grid search on each method, reporting their best score for each metric. Following the training phase, we evaluate the trained autoencoders on the test set. Similar to previous experiments, we use the metrics F (Fimen) and W (Watem)) in the latent space distributions fotju, ..... Sotrk and the barycenter /40- We use the reconstruction loss (binary cross-entropy, denoted as RL), the Wasserstein-2 distance between the prior and aggregated
posterior distribution in latent space W 2, == W3 (Po K Ex=Ifoljp as well as in image
2,image == W? god/uu, 1 Σk Ek Furthermore, we quantify the practical effect of the K method by measuring Fairness metric in Image space. During evaluation, we approximate #4 by its empirical version of 10000 samples. We report the quantitative result of grid search in Table 2, and reconstructed images, generated images, and images of latent codes Figure 13 Appendix D From the results, the proposed surrogate MFSWB generally yield better scores than USWB, except for the generative score i.e, W 2.image The formal MFSWB performs well reconstruction loss and
space W
w 2iimage though its F and W scores are high. The W 2.latent varies slightly across runs, with minor differences in performance order, indicating relatively similar results. While us-MFSWB achieves the best F Finages score, indicating the best fairness performance in image space, es-MFSWB excels in fairness within the latent space. Compared to conventional SWAE, using barycenter loss results in more class-fair latent representation but sacrifices image reconstruction and generative quality.
5 CONCLUSION
We introduced marginal fairness sliced Wasserstein barycenter (MFSWB), a special case of sliced Wasserstein barycenter (SWB) which has approximately the same distance to marginals. We first defined the MFSWB as constrainted uniform SWB problem. After that, to overcome the com- putational drawbacks of the original problem, we propose three surrogate definitions of MFSWB which are hyperparameter-ffee and easy to compute. We discussed the relationship of the proposed surrogate problems and their connection to the sliced Multi-marginal Wasserstein distance with the maximal ground metric. Finally, we conduct simulations with Gaussian and experiments 3D point-cloud averaging, color harmonization, and sliced Wasserstein autoencoder with class-fairness representation to show the benefits of the proposed surrogate MFSWB definitions. Future works will focus on replacing SW with other metrics such as generalized sliced Wasserstein (Kolouri et al. 2019) and augmented sliced Wasserstein (Chen al. 2022
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
We would like to thank Joydeep Ghosh for his insightful discussion during the course of this project.
REFERENCES
Martial Aguch and Guillaume Carlier. Barycenters in the Wasserstein space. SIAM Journal on Mathematical Analysis, 43(2):004-224, 2011.
Ethan Anderes, Steffen Borgwardt, and Jacob Miller. Discrete Wasserstein barycenters: Optimal transport for discrete data. Mathematical Methods of Operations Research, 84:389 409, 2016.
Fabian Bongratz, Aune-Marie Rickmann, Sebastian Pölsterl, and Christian Wachinger. Vox2cortex: Fast explicit reconstruction of cortical surfaces from 3d mri scans with geometric deep neural net- works. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20773-20777 2022.
Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and Radon Wasserstein barycenters of measures. Journal of Mathematical Imaging and Vision, 1(51):22 45, 2015.
Nicolas Bonnotte. Unidimensional and evolution methods for optimal transportation. PhD thesis, Paris 11, 2013.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-ries 3d model repository. arXiv preprint arXiv:1512.03012, 2015.
Xiongjie Chen, Yongxin Yang, and Yunpeng Li. Augmented sliced Wasserstein distances. Interna- tional Conference on Learning Representations, 2022.
Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair generative modeling via weak supervision. In International Conference Machine Learning, pp. 1887-1898. PMLR, 2020.
Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Fair regression with Was Wasserstein barycenters. Advances in Neural Information Processing Systems, 33: 7321-7331, 2020.
Sebastian Claici, Edward Chien, and Justin Solomon. Stochastic Wasserstein barycenters. In International Conference Machine Learning, pp. 999-1008. PMLR, 2018.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Samuel Cohen, Alexander Terenin, Yannik Pitcan, Brandon Amos, Marc Peter Deisenroth, and KS Kumar. Sliced multi-marginal optimal transport. arXiv preprint 2rXiv:2102.077 2021.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems, pp. 2292-2300, 2013.
Marco Cuturi and Arnaud Doucet. Fast computation of Wasserstein barycenters. In International conference on machine learning, pp. 685-693, PMLR, 2014.
John M Danskin. The theory of max-min and its application to weapons allocation problems, volume 5. Springer Science & Business Media, 2012.
Filip Elvander, Isabel Haasler, Andreas Jakobsson, and Johan Karlsson. Tracking and sensor fusion in direction of arrival estimation using optimal mass transport. 2018 26th European Signal Processing Conference (EUSIPCO), pp. 1617-1621. IEEE, 2018.
Jiaojaa Fan, Amirhossein Taghvaei, and Yongxin Chen. Scalable computations of Wasserstein barycenter via input convex neural networks. In International Conference on Machine Learning, pp. 1571- PMLR, 2021.
11

---

Published as a conference paper at ICLR 2025
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Lactitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalic T.H. Gayraud, Hicham Janati, Alain Rakotomamoniy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1-8, 2021. URL htttp://jmlr wwtpppppps//v222222 20 451 html
Paula Gordaliza, Eustasio Del Barrio, Gamboa Fabrice, and Jean-Michel Loubes. Obtaining fairness using optimal transport theory. In International conference on machine learning, pp. 2357-2365. PMLR, 2019.
Karsten Grove and Hermann Karcher. How to conjugate 1-close group actions. Mathematische Zeitschrift, 132(1):11-00, 1973.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge a local nash equílibrium. Advances in neural information processing systems, 30, 2017.
Nhat Ho, XuanLong Nguyen, Mikhail Yurochkin, Hung Hai Bui, Viet Huynh, and Dinh Phung. Multilevel clustering via Wasserstein means. In International Conference on Machine Learning. pp. 1501-1509, 2017.
François Hu, Philipp Ratz, and Arthur Charpentier. Fairness in multi-task learning via W asserstein barycenters. In Joint European Conference on Machine Learning and Knowledge Discovery in Databuses, pp. 295-312. Springer, 2023.
Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in artificial intelligence, pp. 862-872. PMLR, 2020.
Soheil Kolouri, Phillip Pope, Charles Martin, and Gustavo K Rohde. Sliced Wasserstein auto-encoders. In International Conference on Learning Representations, 2018.
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo Rohde. Generalized sliced Wasserstein distances. In Advances in Neural Information Processing Systems, pp. 261-272,
2019.
Alexander Korotin, Vage Egiazarian, Lingxiao Li, and Evgeny Burnaev. Wasserstein iterative networks for barycenter estimation. Advances in Neurul Information Processing Systems, 35: 15672-15686, 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Master's 's thesis, Department of Computer Science, University of Toronto, 2009.
Alexey Kroshnin, Nazarii Tupitsa, Darina Dvinskikh, Pavel Dvurechensky, Alexander Gasnikov, and Cesar Uribe. On the complexity of approximating Wasserstein barycenters. In International conference on machine learning, pp. 3530-3540. PMLR, 2019.
Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. From word embeddings to document distances. In International conference on machine learning, pp. 957-966. PMLR, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2333 1998.
Tudor Manole, Sivaraman Balakrishnan, and Larry Wasserman. Minimax confidence intervals for the sliced Wasserstein distance. Electronic Journal of Statistics, 16(1):2252-233 2022.
Eduardo Fernandes Montesuma and Fred Maurice Ngole Mboula. Wasserstein barycenter for multi- source domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and paftern recognition, pp. 16785-16793, 2021.
Youssef Mrouch. Wasserstein style transfer. In International Conference on Artificial Intelligence
and Statistics, pp. 842-852. PMLR, 2020.
12

---

Published as a conference paper at ICLR 2025
Kimia Nadjahi, Alain Durmus, Umut Simsekli, and Roland Badeau. Asymptotic guarantees for learning generative models with the sliced- Wasserstein distance. In Advances in Neural Information Processing Systems, pp. 250 250-260,200 2019.
Kimia Nadjahi, Alain Durmus, Lénaic Chizat, Soheil Kolouri, Shahin Shahrampour, and Umut Simsekli. Statistical and topological properties of sliced probability divergences. Advances in Neural Information Processing Systems, 33:20802-208/ 2020.
Khai Nguyen and Nhat Ho. Energy-base based sliced Wasserstein distance. Advances in Neural Informa- lion Processing Systems, 2023.
Khai Nguyen and Nhat Ho. Hierarchical hybrid sliced Wasserstein: A scalable metric for heteroge- neous joint distributions. arXiv preprint 24K::2404.15378 2024.
Khai Nguyen, Nhat Ho, Tung Pham, and Hung Bui. Distributional werredd-Wssertteinn and applications to generative modeling. International Conference on Learning Representations, 2021.
Khai Nguyen, Nicola Bariletto, and Nhat Ho. Quasi-monte carlo for 3d sliced Wasserstein. The Twelfth International Conference Learning Representations, 2024a.
Khai Nguyen, Shujian Zhang, Tam Le, and Nhat Ho. Sliced Wasserstein with random-patt -atth projecting directions. International Conference on Machine Learning, 2024b.
Sloan Nietert, Ritwik Sadhu, Ziv Goldfeld, and Kengo Kato. Statistical, robustness, and computational guarantees for sliced Wasserstein distances. Advances in Neural Information Processing Systems, 2022.
Gabriel Peyré and Marco Cuturi. Computational optimal transport, 2020.
Julien Rabin, Gabriel Peyré, Julie Delon, and Mare Bernot. Wasserstein barycenter and its application to texture mixing. In Scale Space and Variational Methods in Computer Vision: Third International Conference, SSVM 201 I, Ein-Gedi, Israel, May 29-June 2, 2011, Revised Selected Papers 3, pp- Springer, 2012.
Samira Samadi, Uthaipon Tantipongpipat, Jamie H Morgenstern Mohit Singh, and Santosh Vempala. The price of fair pea: One extra dimension. Advances in neural information processing systems, 31, 2018.
Thibault Séjouréé, Trançois-Xavier Vialard, and Gabriel Peyré. Faster unbalanced optimal transport: Translation invariant sinkhorn and 1-d frank-wolfe. International Conference on Artificial Intelligence and Statistics, pp. 4995-5021. PMLR, 2022.
Chiappa Silvia, Jiang Ray, Stepleton Tom, Pacchiano Aldo, Jiang Heinrich, and Aslanides John. A general approach to fairness with optimal transport. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3633-3640, 2020.
Justin Solomon, Fernando De Goes, Gabriel Peyré, Mareo Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, and Leonidas Guibas. Convolutional Wasserstein distances: Efficient optimal transporta- tion on geometric domains. ACM Trunsactions on Graphics (ToG), 34(4):1-11, 2015.
Sanvesh Srivastava, Cheng Li, and David B Dunson. Scalable bayes via barycenter Wasserstein space. Journal of Machine Learning Research, 19(8):1-35, 2018.
Matthew Staib, Sebastian Claici, Justin M Solomon, and Stefanie Jegelka. Parallel streaming Wasserstein barycenters. Advances in Neural Information Processing Systems, 30, 2017.
Shanlin Sun, Thanh-Tume Tung Le, Chenyu You, Hao Tang, Kun Han, Haoyu Ma, Deying Kong, Xiangyi Yan, and Xiaohui Xie. Hybrid-csr: Coupling explicit and implicit shape representation for cortical surface reconstruction. arXiv preprint arXiv:2307. 12299, 2023.
Yubo Zhuang, Xiaohui Chen, and Yun Yang. Wasserstein k-means for clustering probability distribu- tions. Advances in Neural Information Processing Systems, 35:1 1182-11399993 2022.
13

---

Published as a conference paper at ICLR 2025
Supplement to "Marginal Fairness Sliced Wasserstein Barycenter" "
We present skipped proofs in Appendix A We then provide some additional materials which are mentioned in the main paper in Appendix BJ After that, related works are discussed in Appendix We then provide additional experimental results Appendix D Finally, we report the used computational devices in Appendix B
A PROOFS
A.1 PROOF OF PROPOSITION
I
Proof. From Definition
we have
SF(H,Mi))= = max SWP mk))
ke
max ke(l....)
BB 1 (WP(Otuu 0dun)l
Let k- = arg max B [W (11,, 0b we have
SF(H,M1:K) = (s- [W (0iu, 01px-)
S (
max kE(1.....K)
WP 01pss)
=USF(p, (1.K)))))
as from Definition which completes the proof.
A.2 PROOF OF PROPOSITION 2
Using the Holder's inequality, we have:
Oiduxa)) V.USF(( (1:k)
L
L
2
E [WP 0tux;))
2
WP (Pi (iHi,,
WP 0HmNz))
V WE(00tuo. Oituns) 1)
Var
L
1=1
Var [Vo WP 0pm4;))]
VL
which completes the proof.
A.3
PROOF OF PROPOSITION 3
We first restate the following Lemma from Nguuyen et al. 2024b) and provide the proof for com- pleteness.
14

---

Published as a conference paper at ICLR 2025
Lemma 1. For any L 1, 0 02 S ... S az and 0 < <b S by ... S by, we have:
...
a,by
(15)
Proof. For L 1, we directly have a,b; a,b, Assuming that for the inequality holds i.e. + ( b,) - Eiz a;,, which is equivalent to (E a.) b.) I LE a,b, Now, show that (E - 4:) (E= b,) Ei= a&b; i.e., the inequality holds for T. + We have
L
L+1
L+1
b,)
Σ
a.)
b.) +
a ab)bb11 +
b.Ja + actibL+1
t=1
i=1
1=1
#=1
i=1
$ L Σ
a,d; +
ag)bL+1 E b,)a + aL+IDL++
Since a1+1b1+1 + a₂b; 2 16 + bL+Ia, for all 1 S S L by rearrangement inequality. By taking the sum of these inequalities over from 1 to L, we obtain:
a b.. +
b,)a S Σ
ab. + Lar4ibr41-
2m
Then, we have
L+1
L+1
L
bj)
< L
a;b; +
a ai)b +
bi)a + a
1=1
i=1
i=1
$ T.
a₂b; + Σ asbi Lottibetz + a
i=1
L+1
(L (1++1)
azb;),
which completes the proof.
Now, we go back the main inequality which is (s: S (E)())))).)) From Defini- tion 5 we have:
ESF(M = Egra (8mm.)))))
max KE () WP 02pls) (ūfmx)
WP 0zun) fo (0; μ. (1:K)
= =Eo-1188--)
max ke(l...,K)
r(d/2)
2m4/4
where (0: P1:K) exp (maxne().....) WP(Otn, 0tyk)) Now, we consider Monte Carlo estimation of ESF(H (1:K) by importance sampling:
ESF(up L) = L Σ
exp (max (Ottou 0:tje:)
(P 0/1/4)) k)(L.....)
max
E exp (max)e().....K) W3' (0, 0,t1u))
U($d-1). Similarly, consider Monte Carlo estimation of USS (4SF(1 (1:)):)) where 0,, 01...., Or. sid
uSF(u PL.K L) T
Σ
max W (Ortt, Octuu)) kE(.....)
for the same set of oy...., Or.-
Without losing generality. we assume that max.e(1.....)) (Or Ort))). Let
max K} Wp (r, 01pp))
15

---

Published as a conference paper at ICLR 2025
m ().... k) W (0,dp, 0.tpk)
and exp (1.... WB (0:0i, 0:ipx)) by ap-
plying Lemma I
we have:
LS (SF(I)MLR,L A7 1.
By letting L + oc and applying the law of large numbers, we obtain:
USF (.: EEF((u ((:
which completes the proof.
A.4 PROOF OF PROPOSITION 4
We first recall the definition of the SMW with the maximal ground metric:
PK:c) [xEll(p.....x)) iE(1, [JE(1.....
inf
max
10 xs 0 By (d .....,..........................
Non-negativity. Since (ax.g ().,....)...((....))... 10 x; x111 2 for any 2 $1...., 2k and for any 0, can obtain the desired property (M 2 which implies (.. 2 O.
Marginal Exchangeability. For any permutation o [[]]] + [[]]] we have:
inf SM (kic)) [xIIl(p.....xx) sE(1....,KK.3 ().E(L.....))
max
10T 0J Teay,dee Pd (......,, xk)
inf (EII(p)))).....okkkk)) SMWP (Ma(()())))))))))))))))))))))))) (Uo(I))...Mo(())) (o(K)ic).
max
10Tx; x; 0 xz 1 Pd (Z)...... xk)
iE(1...... ().........K)
Generalized Triangle Inequality. For μ E (TR)), we have
SMWP ( (AKC)
E
inf
10 TT 00 22,11 dr (x ,r1,..,, .xk)
()..E(l.....
K
< EE
inf
max
10T AT x1,' dm (21,,.
iE (l....
K
E
inf
max (G(L....KK)())JJ.E
10T x - o "", ..,)....,))))..,,.
Hlll p (K)
k=
E
max (E()....
10 x; 0 TT 1 (x), ..... pp- Fa+ ..)
for T is the optimal multi-marginal transportation plan and л x211 ..... 2g-1 - 2K is the marginal joint distribution by integrating out Zt. By the gluing lemma (Peyré & Cuturi 2020), there
exists optimal plans n (z1:-..; 23- 1, y, xx +1, FK for any k [[K]] and follows n.. We further
16

---

Published as a conference paper at ICLR 2025
have:
E
max
max
10Tx; T0 xz
KMMA),E u................. K)NR)
max
10Tx; x4 - T u|? d (x1, Ik
xk)
ie K (P(c))
K
E
inf
10 x; 0 (z))
PA 1.m iE(I,.,K)a (1,,,,
Applying the Minkowski's inequality, we obtain the desired property:
K
SMW, (MM.p &
SMW,(ui....... Mu MKi c).
Identity of Indiscernibles. From the proof in Appendix A.5 we have:
SMW (4 ( (KiC) 2 E
max
WP 01p;)
iiE( JEJE (1 (I.....) K}
2
max
E
[B (Otps 05p4)]
iE()...... K)jE (I.........
max EE(1 K)..(L.....))
SW (4s.
Therefore, when SMW, (MI,WW()...../14... = O, we have SWP(u,, (15) = 0 which implies μ4 for any i,j E [[]]] As a result, (4) =
from the metricity of the SW distance. For the other direction, easy see that if μ4 ... HK. we have SMW, (w1............................. MK; c) 0 based on the definition and the metricity of the Wasserstein distance.
A.5 PROOF OF PROPOSITION 5
Given the maximal ground metric c(OT æ1...., (Tœx)= = maxie (1... 10 Tx - 0T xzy. from Equation 6
[cI(p)....kK)) / c(o si,..., ..,..... ......,.' , TT Pm (di, (.,, .x)
SMWF (mi,...... ... PK3C) E inf
EE inf =ETT(( iE(I.... () (1...,,K)
max
10T 00 Jxxy Pd (x) xk)
By Jensen inequality i.e., (II,..., xKk) - ma.,E((..., 10Tx; - TT xz11 convex function, have:
inf sETT( uK] ie(1...... K)JE( (1....... / 10T o x, (d TK)
SMWP (K3e) 2 E
Using max-min inequality, we have:
/ 10T TT xy "di xxx)
(MWP (x;;) 2 E
inf iE ..... K).j(I...... reM()
max
inf (E K)Jj((L.... rElloums)) / 10T 10Tz,- 10xx;-00 -0 o0 I (d)(
E
max
E
max (E(1......
K).jE(L.....K WP(04p:: 02pp))
17

---

Published as a conference paper at ICLR 2025
Algorithm I Computational algorithm of the SWB problem
Input: Marginals f1:.... MK. 2 1, weights wI..... wk. the number of projections L, step size 7,, the number of iterations T.
Initialize the barycenter M
for t = to T do
Set V & = 0
Sample 0; ~ 1)
for = 1 to L do
for k 1 to K do
Set V = V
WB (Ordus 0rtpa)
end for end for ④ = Φ V end for Return: μ
Algorithm 2 Computational algorithm of the s-MFSWB problem
Input: Marginals M1....,KK, p 2 the number of projections L, step size f, the number of
iterations T.
Initialize the barycenter Ms
for /. = to T do
Set V
Sample (8(-11) k*
for K do
for = 1 to do 1111 W 0(tte)) Σt=
Wp(Oitus,0ttuu then
k
end if end for end for
V + Ei= (OW OIEPK))
Ø V
end for Return: N
Therefore, minimizing two sides with respect to μl, have:
min SMWP (MI,......................... (KGe) 2 min E
iE(1... ...... ()....... WT(00 02pz)
min E
max (E((,...KK)
min USF(PF
which completes the proof.
B ADDITIONAL MATERIALS
Algorithms. As mentioned in the main paper, present the computational algorithm for SWB in Algorithm for s-MFSWB in Algorithm 2 for us-MFSWB in Algorithm 3 and for cs-MESWB in
Algorithm
Energy-based Sliced Multi-marginal Wasserstein. As shown Proposition us-MESWB is equivalent to minimizing lower bound of SMW with the maximal ground metrie. We now show that MFSWB is also equivalent to minimizing a lower bound of a variant of SMW i.c. Energy- based sliced Multi-marginal Wasserstein with the maximal ground metric. We refer the reader
18

---

Published as a conference paper at ICLR 2025
Algorithm 3 Computational algorithm of the us-MFSWB problem
Input: Marginals MI,.... 2 the number of projections L, step size 1), the number of
iterations T.
Initialize the barycenter U
for t = to T do
Set V 9 = 0 Sample 0; ~ (((=-11)
for = 1 to L do
kj
for k: = to K do
if ( 0stpux) wo Ortpx;)) then
kj = k
end if
end for
+ V W 01ūNN;
end for
Φ V s
end for Return: u
Algorithm 4 Computational algorithm the es-MFSWB problem
Input: Marginals M1..... P 2 the number of projections L, step size y, the number of
iterations T.
Initialize the barycenter H
for f. to T do
Set V =0
Sample ~~~ (( for 1 1 to L do
ki =
for k: 2 K do if WP (1: Oiduu) > (p 01tpz;) then
kj k
end if end for end for
for I to L
exp( (f
W1
exp(W, ( (0, Bi (23))
end for
whe WP Ocdu);
end for Return: us
to Proposition 6 for a detailed definition. The proof of Proposition 6 is similar to the proof of Proposition 5 in Appendix A.5
Proposition 6. Given K marginals Pı.... uK E P, (RR), the maximal ground metric c(0 TT [xxx (I...,
10T x, @Txyl. we have:
min (inE(F(()))) S min ESMWP (u1, ........ Mkie), in
(16)
where
/ c(0Tan (i.T.,....,, TT Txxx x "d ( [K)
ESMWY(PL. (2,, (kic) =E
inf sT (EI(......
19

---

Published as a conference paper at ICLR 2025
UINO1 APOAR PARE
MORRA 21- STR.0 2-1001
PAOKUC AMOMSSS
Figure 5: Barycenters from USWB. MFSWB with À = 1, s-MFSWB, us-MFSWB, and es-MESWB with learning rate 0.001 (first row), 0.005 (second row), and 0.05 (third row). and the expectation is with respect to a (0) i.e.,
(0 (1, M2:k) x exp
max ke (2..,...
C RELATED WORKS
Fair Learning with Wasserstein Barycenter. A connection between fair regression and one- dimensional Wasserstein barycenter established by deriving the expression for the optimal function minimizing squared risk under Demographic Parity constraints (Chzhen et al. (020). Similarly, Demographic Parity fair classification is connected to one-dimensional Wasserstein- distance barycenter in (Jiang et al.) (020). The work (Hu et al.] (2023) extends the Demographic Parity constraint to multi-task problems for regression and classification and connects them to the one- dimensional Wasserstein-2 distance barycenters. A method to augment the input so that predictability of the protected attribute is impossible, by using Wasserstein-22 distance Barycenters to repair the data proposed in (Gordaliza et al. (2019). A general approach for using one-dimensional Wasserstein- 1 distance harycenter to obtain Demographic Parity in classification and regression is proposed in (Silvia al. 2020). Overall, all discussed works define fairnsss in terms of Demographic Parity constraints in app applications with a response variable (classification and regression) in one dimension. In contrast, we focus on marginal fairness barycenter i.e., using a of measures only, any dimensions,
Other possible applications. Wasserstein barycenter has been used to cluster measures in Zhuang al. 2022). In particular, a K-mean algorithm for measures is proposed with Wasserstein barycenter as the averaging operator. Therefore, our MFSWB can be directly used to enforce the faimesss for averaging inside each cluster. The proposed MFSWB can be also used average meshes by changing the SW to H2SW which is proposed in (Nguyen & Ho! 2024).
D ADDITIONAL EXPERIMENTS
Gaussians barycenter with the formal MFSWB. We report the result of finding barycenters from USWB, MFSWB with X 1, M-MFSWB, US-MFSWB, and es-MFSWB with learning rate 0. 0.001, 0.005, and 0.05 in Figure 5 We present the result finding barycenters of Gaussian distributions with MFSWB À 0.1 and 10 in Figure
20

---

Published as a conference paper at ICLR 2025
-ES.E--CA--EELUU N-22027
HSMMA-121 112--13122 W-272.
F- W-0025
Figure 6: Barycenters from MESWB with À 0.1 and X 10 along gradient iterations with the corresponding F-metric and W-metric.
Space
Sager
buge?
USWB
Figure 7: Averaging point-clouds with USWB, MFSWB 1), M-MFSWB, us-MFSWB, and es-MFSWB.
Table F-metric and W-metric along iterations in point-cloud averaging application.
Matiod
Itenation 0
Epoch 1000
Epoch 5000
Epoch 10000
F(J)
W.(
(()
W (1)
F(]
W
F(J)
(1)
USWB
748.67 0.0
4814.71
0.0
35.22
161.11 1 0.544
7,82 0.22
100.52 0.26
11.08 1. 0.08
108.52
0.17
MESSE
746.87
4814.71
35.15
0.55
107.88
1.16
MFSWR
746.67
4814.71
1.30
28.19
13.57
123.65
1.52
-MPSWB
748.67
4816.71
0.0
36.
154.4
0.68 0.44
109.17
0.34
2.54 2.06
107.57
0.10
us MPSW
706,07 = 0.0
4814.71
0.0
28.05
1.37
161.27 2: 0.85
1,02 =: 0.8
108:07 = 0.1
1,45 0.77
108.2 0.13
MESWB
708.07 3 0.0
4810.71 1: 0J0
28.00 1 1.16
143.24 11 0.76
0.99 0.332
109.08 0.14
1,36 0.02
108.28 -: 0.07
Point-cloud averaging. We report the averaging results of two point-clouds of plane shapes Figure 7a the corresponding F-metrics and W-metric along iterations in Table We see that the proposed surrogates achieve better F-metric and W-metric than the USWB. In this case, us-MFSWB gives the best F-metric at the final epoch, however, es- es-MFSWB also gives comparable performance and performs better earlier epochs. For the formal MFSWB, it does not perform well with the chosen set of
Color Harmonization. We first present the harmonized images of different methods including USWB, MFSWB (A=1). S-MFSWB, us-MESWB, and Cs-MESWB at iteration 5000 and 10000 for the demonstrated images in the main text in Figure 8 Figure 9 Moreover,
report the results of MFSWB (A 0.1, 10) at iteration 5000, 10000, and 20000 in Figure [0] Similarly, we repeat the that es-MFSWB helps to
same experiments with flower images in Figure
14
Overall,
21

---

Published as a conference paper at ICLR 2025
Image
Target Image
Target Image:
Figure 8: Harmonized images from USWB, MFSWB (A 1), s-MFSWB, MS-MFSWB and es-MFSWB at
iteration 5000.
Source Image
Target Image
Target Image 2
Figure 9: Harmonized images from USWB, MFSWB (A 1), M-MFSWW, us MF-MF, and es cs-MFSWB at iteration 10000.
HFSWS 0.1, 1347 530, $390.205
H15WB - 20 05.55..20 3140-4456
MEFOAAT A 1.0.1. TORK 355, W 3A75 965
MERWI 2 10.5- 70221 -0
Figure 10: Harmonized images from MFSWB with À = 0.1 and À = 10 at iterations 5000, 10000, and 20000.
reduce both F-metric and W- W-metric faster than USWB and other surrogates. For the formal MFSWB. the performance depends significantly the choice of X.
Sliced Wasserstein autoencoder with class-fairness representation. We use the RMSprop opti- mizer with learning rate 0.01, alpha=0.99, eps=le 8. As mentioned in the main text, report the used neural network architectures:
We report randomly selected reconstructed images, some randomly generated images, and the test latent codes of trained autoencoders in Figure 15 Overall, we observe that the qualitative results consistent with the quantitive results in Table From the latent spaces, that the proposed
22

---

Published as a conference paper at ICLR 2025
Source Image
Target Image
Target Image
2620
Figure 11: Harmonized images from USWB, MESWB (A 1) s-MFSWB, us-MESWB, and cs-MFSWB at iteration 5000.
Source Image
Target Image
Target Image 2
USMB SELLL Wi $449.3
MOSIB An 1996. 3... 40806011 SMMPMM 2.22 580 H S1.2
MEOWE W 4013.447
1073 BO5,
Figure 12: Harmonized images from USWB, MESWB (A = 1), M-MESWB, MFSWW, and es-MPSWB at iteration 10000.
surrogates helps to make the codes classes have approximately the same structure which do appear in the conventional SWAE's latent codes.
23

---

Published as a conference paper at ICLR 2025
Source Image
Target Image
Target Image
USAB A 2044 273 3717777
NISAB A- 107 311 WR 272320 DNSIO F 1:15 226 N 2000
MEVEN 2567 183 MY 20TA EXT
AEPSING 7 LO3TOO 20 1804393
Figure 13: Harmonized images from USWB, MESWB (A 1), M-MFSWB, MFSWB, and es MFSWB iterations 20000.
MEROO 301
NFSBD 11 221442 2372.12
NISNO W 140444
Figure 14: Color harmonized images from MFSWB with À 0.1 and À 10 at iterations 5000, 10000, and 20000.
E COMPUTATIONAL DEVICES
For the Gaussian simulation, point-cloud averaging, and color harmonization, we HP Omen 25L desktop for conducting experiments. Additionally, for the Sliced Wasserstein Autoencoder with class-fair representation experiment, we employ the NVIDIA Tesla V100 GPU.
24

---

Published as a conference paper at ICLR 2025
Method
Reconstructed Images
Generated Images
Latent Space
3
3
5
2
4
9
6
4
3
9
0
5
5
5
8 5
7
2
6
7
/
9
5
8
7
9
2
7
O
4
7
8
9
2
5 $
8
9
2
5
0
6
9
1
5
/
5
3
1
2
2
SWAE
9
5
7
2
6
2
2
2
7
9
0
4
7
9
7
5
4
9
9
8
2
7
0
9
8
3
7
2
9
5
9
3
4
S
O
7
/
0
9
2
0
9
0
O
6
3
8
4
6
5
9
5
4
5
4
5
1
9
0
3
5
4
3
9
/
3
8
3
3
7
2
/
2
/
B
I
2
5
2
2
8
3
9
9
2
9
6
2
6
8
8
5
7
8
5
O
6
9
8
6
7
5
0
3
USWB
9
5
7
7
2
5
2
G
3
4
8
/
9
0
5
8
9
9
6
1
5
O
9
0
2
3
6
8
9
9
3
3 /
5
7
5
2
9
5
9
4
3
4
0
3
2
2
0
9
0
O
8
8
4
5
5
O
3
3
$
2 9
7
5
7 9
4
5
5
9
6
8
0
O
8
7
4
9
8
7
2
2
/
2
8
/ 2
9
7
9 6 9
4
8
0
8
3
2
$ 6
4
0
2
4
8
9
9
MFSWB À = 0.1
4
5
7
7
8
3 0
8
/
8
5
7
7
9
5
5
P
8
2
3
4
(
5
6
8
0
9
8
2
/
6
9
S
9
9
4
8
8
0
9
O
2
7
4
9
8
5
6
5
2
4 2 5
5
4
5
2
5
4
9
2
5
5
4
7
7
/
9
/ 7
9 5
4
8
G
9
7
7
5 2
5
3
8
7
4
2
2
9
6
2
5
2
/
8
5
3
2
O
6
9
2
2
0
4
3
MESWB A = 1.0
9
7
7
/
1
8
5
7
7
9
1
150 2
O
2
9
6
2
O
5
0
9
9
4
8
9
6
2
6
5
0
5
2
9
2
6
9
/
2
/
6
I
6
5
5
2
2
6
25

---

Published as a conference paper at ICLR 2025
Method
Reconstructed Images
Generated Images
Latent Space
9
/
4
8
4
4
5
9
3
0
3
5
5
5
3
4
0
6
1
0
9
3
2
3
8
7
L
0
6
6
5
1
6
2
5
6
3
8
9
0
2
3
0
6
MFSWB À 10.0
9
5
7
2
6
2
4
0
0
/
8
8
1
0
3
5
5
8
6
0
5
6
9
7
3
/
7
8
2
9
0
O
O
8
4
2
0
3
9
9
9
7
7
8
9
0
3
5 4
5 1
5 1
9
7
4
0
7
2
8
P
3
5
6
5
9
0
6
3
8
9
2
2
1
5
0
8
7
0
2
M-MFWW
9
7
7
2
2
4
2
9
0
9
8
5
7
7
0
5
7
5
9
S
5
5
0
2
6
6
3
4
5
5
0
0
0
5
0
4
6
/
3
9
8
2
2
2.
5
4
3
8
7
7
7
2
7
2
3
9
7
3
0
4
9
9
5
5
3
8
8
9
2
2
1
4
5
9
9
0
2
0
S
8
6
0
0
6
5
5
0
9
US-MFSWB
9
7
1
O
5
0
8
5
7
7
5
8
8
0
4
4
9
0
9
9
8
2
0
3
7
4
2
6
6
3
3
7
O
0
7
7
6
/
3
9
2 9
7
9
5
5
2
9
5
9
0
3
0
9
9
a
2
7
S
7
7
4
0
8
3
S
8
9
2
1
5
0
7
9
5
7
4
8
es-MFSWB
5
5
a
0
2
3
9
8
7
7
8
2
2
9
2
2 6
6
9
9
9
8
7
7
0
7
/
9
9
7 7
0
4
4
2
9
3
0
6
0
5
6
2
9
7
2
6
4
S
5
2
7
0
4
O
8
9
2
0
2
0
8
Figure 15: Reconstructed images, generated images and latent space of all methods.
26

---

Published as a conference paper at ICLR 2025
Layer
Description
MNISTAutoencoder
Encoder
Conv2d
(1, 16, kernel size=3, Stdide=1,paading--
LeakyReLU
(negative slope=0.2, inplace='True)
Conv2d
(16, 16, kernel size=3, stride=1, padding=1)
LeakyReLU
(negative slope=0.2, inplace=True)
AvgPool2d
(kernel size=2)
Conv2d
(16, 32, kernel size=3, stride=l,, padding-))
LeakyRel.U
(negative slope=0.2,, inplace=ree))))
Conv2d
(32, 32, kernel size=3, stride=1, paddingg] padding=1)
LeakyReLU
(negative slope=0.2, inelac==eee)) (kernel size=2)
AvgPool2d
Conv2d
(32, 64, kernel size=3, padide=l,ppdddin----
LeakyReLU
(negative slope=0.2, implace=True)
Conv2d
(64, 64, kernel size=3, stride=1, padding=1))
LeakyRel.U
(negative slope=0.2, inplace=Trrue)
AvgPool2d
(kernel size=2, padding=1)
Linear
(in ii__feature==122, out_features=128)
ReLU
(inplace=True)
Linear
(in 128 out_taaturs==2)
Decoder
Linear
(in_features=2, out_feature==128) (in features=128, out features=1024)
Linear
ReLU
(inplace=True)
Upsample
(scale factor=2, mode=ncarest)
Conv2d
(64, 64, kernel size=3, stride=1,
padding=1) stride=1,,
LeakyReI.U
(negative slope=0.2, inplace=/True)
Conv2d
(64, 64, kernel size=3, stridc=l,, padding)))
padding=1)
LeakyReLU
(negative slope=0.2, inplace=True) (scale factor=2, mode-nearest) (64, 64, kernel size=3,
Upsample
Conv2d
LeakyReLU
(negative slope=0.2,, inpla (64, 64, kernel size=3, stride=l,, padding:
Conv2d
LeakyRel.U
(negative slope=0.22, impla (scale factor=2, mode=ncarest)
Upsample
Conv2d
(64, 32, kernel size=3, stridd=
LeakyReLU
(negative slope=0.2, inplace=Tee))
Conv2d
(32, 32, kernel size=3, stride=1, padding=1)
LeakyReLU
(negative s.ope=0.2, inplace=Trrue) (32, 1, kernel size=3, stride=1, padding=1)
Conv2d
Table 4: MNIST Autoencoder Architecture
27

---

Published as a conference paper at ICLR 2025
Table 5: Comparison of methods with R2
0.5 on CIFARIO after 500 epochs.
Methods
RL (4)
W 2.lssen (L)
W 2,imagg (4)
Flateot (4)
W lnseen
F Finages (4)
W imaggs (4)
SWAE
0.640
6.101
141.984
0.280
4.585
46.006
178.798
UBSW
0.640
6.104
135.944
0.228
4.572
44.024
174.322
MFSWB À 0.1
0.640
6,097
142.530
0.281
4.585
46.080
179.210
MESWB
1.0
0.641
6.092
142.289
0.279
4.578
46.076
179.135
MESWB = 10.0
0.640
6,100
141.503
0.282
4.585
46.088
178.373
S-MFBSW
0.640
6,103
134.766
0,218
4,569
42.503
173,530
us MS-M 0.642
6.088
131.934
0.209
4.546
39.329
171.204
es-MFBSW
0.642
6.060
132.170
0.212
4.534
40.642
171.573
Table 6: Comparison of methods with K2 = 0.5 on STL10 after 500 epochs.
Methods
(T)
2,tnem (L)
W 2,inage (4)
Fintent (4)
W Wineet (1)
Finsages (4)
W Wimages (L)
SWAE
0.613
16,826
301.397
0.647
15.699
25.827
199.175
UBSW
0.616
16,908
301.143
0.585
15.719
24.905
199.918
MESWB À 0.1
0.614
16,823
301.704
0.647
15.698
25.637
199.662
MFSWB À = 1.0
0.614
16,814
301,505
0,647
15.688
25.790
199.307
MFSWB
10.0
0.613
16.831
301.370
0.648
15.705
25.546
199.168
S-MFBSW
0.613
16.842
302.632
0.580
15.658
23.520
200.262
us-MFBSW
0.616
16.830
297.952
0.586
15.645
23.638
197.057
es-MFBSW
0.616
16.796
296.548
0.557
15.658
22.551
199.117
Results. We evaluate the scalability of our method using two well-established datasets: CI- FARIO (Krizhevsky et al. 2009) (d = 32 x 32 x 3) and STL10 (Coates et al. (211] 1) (d = 64 x 64 3).
For these experiments, we set 11 8.0, M2 = 0.5, and train for 500 epochs with a learning rate of 0.0005. The CIFAR10 10 experiment uses uniform distribution on 48- 48-dimensional ball (h 48), while the STL10 experiment uses a 18-dimensional ball (h- 128).
We assess fairness and averaging distance in the latent space, denoted Fatent and Watennt respectively, Additionally, we measure the reconstruction loss (RL) and the Wasserstein-2 distance between the prior and aggregated posterior distribution in the latent space, W 2, Unlike the MNIST experiments, where the Wasserstein distance was used to measure metrics related image space, we employ the FID score (Heusel al. (207)) for CIFAR10 10 and STL.10 due to its widespread and reliability in measuring distances. Specifically, the F-metric and W-metric in the image domain and the gap between generated images and the dataset W
2mimaee calculated as:
K-1
K
2
F Finages
FID(H, - FID(U, 1g))
(17)
K(K -
i=1 1=4
K
W images
FID(U,
(18)
K
1=1
2 imagee
FID
P40:
K
14
(19)
where y is the empirical distribution of generated images. /1...... .. MK are the images for each label the dataset, and FTD() the FID score (Heusel et al. 2017). We report the quantitative results in Table the CIFAR10 experiment and Table 6 for the STLIO experiment.
The proposed methods outperform baselines across nearly all metrics. For CIFARIO, Us-MFBSW and es-MFBSW deliver the best results, with us-MFBSW excelling in image domain metrics like
W 2m Fimage- and W Wimage- On STL10, es-MEBSW stands out, achieving the best W 22 W2 2iimage
and F Finage, while also improving fairness in the latent space with the lowest Fiatents while MF-S does its best at reducing the averaging distance both in latent and image domain, which are Watent and W image, respectively.
28

---

Published as a conference paper at ICLR 2025
Overall, compared to the baselines, the proposed methods achieve greater geometric faimesss and bring the generated images closer to the dataset distribution in both latent and image spaces, though this comes at the expense of reduced image reconstruction quality.
29


Reviews:
[{"review_id": "n5lYXlNUcJ", "reviewer": "Reviewer_1X1Z", "summary": "The traditional Sliced Wasserstein Barycenter (SWB) employs uniform weights between the barycenter and its marginals, leading to imbalances in the distances between them. This paper introduces the Marginal Fairness Sliced Wasserstein Barycenter (MFSWB), which aims to equalize these distances, ensuring a fairer representation. The authors propose three algorithms to address the MFSWB formulation and conduct experiments on synthetic Gaussian distributions, 3D point clouds, color harmonization, and generative models. The results demonstrate that the proposed method is highly effective, validating the authors' claims.", "questions": "Did you have a typo in line 506? Should $P_{f(X_k)}$ be equal to $\\frac{1}{M} \\sum_{i=1}^M \\delta_{f(X_{ki})}$?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper addresses an important problem that has been overlooked. In traditional SWB, the weights between barycenter and marginals are uniform. However, this uniform weights actually leads to non-uniform distances between the barycenter and the marginals. The authors have a simple experiment demonstrating that in the paper. To ensure equal distances, they propose the MFSWB to address this problem. They define MFSWB such that the average distance to be within an $\\epsilon$.\n\nThe proposed MFSWB is hard to solve. The authors proposed three surrogate definitions for MFSWB. The first one, called s-MFSWB, aims to minimize the maximal distance from the barycenter to the marginals. Yet, the s-MFSWB is biased. To rectify this, they proposed the second unbiased surrogate, us-MFSWB. They show that us-MFSWB's objective is an upper bound of s-MFSWB, meaning that minimizing us-MFSWB implicitly minimizes s-MFSWB. Furthermore, They also show that the approximation error of the gradient estimator of us-MFSWB reduces at the order of $\\mathcal{O}(L^{−1/2})$. Since s-MFSWB and us-MFSWB both use the uniform distribution as the slicing distribution, which can empirically result in non-optimal in statistical estimation, the authors proposed the energy-based surrogate MFSWB (es-MFSWB) which considers the importance of the sampling directions. \n\nThe authors conducted extensive experiments evaluating the proposed methods and comparing the methods against the conventional SWB. The results are convincing. Using the synthetic Gaussian data, they showed that SWB produced the barycenter with non-uniform distances from the marginals, whereas the proposed methods yielded barycenters with approximately equal distances from the marginals. The authors also performed experiments on the 3D points clouds, Color Harmonization, and generative models with both quantitative results and qualitative results showing the effectiveness of the proposed method. The proposed es-MFSWB appears to be the best among all three proposed methods. \n\nThe authors shows the potential of using the proposed method in learning conditional generative models as an inverse barycenter problem.", "weaknesses": "I didn't really catch any weakness of this paper. The paper is complete with clear motivation, effective solutions and sufficient experimental results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RfnJPh92rS", "reviewer": "Reviewer_6MLn", "summary": "This paper aims at achieving marginal fairness sliced wasserstein barycenter (MFSWB). To tackle the problem, the authors define the MFSWB as a constraint SWB problem. Due to the computational disadvantages of the formal definition, the authors propose two hyperparameter-free and computationally tractable surrogate MFSWB problems that implicitly minimize the distances to marginals and encourage marginal fairness at the same time. To further improve the efficiency, the authors perform slicing distribution selection and obtain the third surrogate definition by introducing a new slicing distribution that focuses more on marginally unfair projecting directions. The experimental results demonstrate the favorable performance of the proposed MFSWB performance.", "questions": "1. Can the authors elaborate on the significance of MFSWB? The description in current version is somewhat high-level.\n2. Can the authors provide a clearer description about the formulation in line 148?\n3. What does the symbol $\\sharp$ in Eq.(1) denote?\n4. What does the $x_i(y_i)$ in line 144 denote?\n5. What does the $\\pi$ in line 144 denote?\n6. What modification to the methodology presented in this paper are necessary to replace SWD with GSWD [1] or ASWD [2]?\n\n[1] Kolouri, Soheil, et al. \"Generalized sliced wasserstein distances.\" Advances in neural information processing systems 32 (2019).\n\n[2] Chen, Xiongjie, Yongxin Yang, and Yunpeng Li. \"Augmented sliced Wasserstein distances.\" arXiv preprint arXiv:2006.08812 (2020).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-organized and the writing is good.\n2. The paper has solid theoretical proofs.\n3. The story line is easy to follow.", "weaknesses": "Please refer to questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WywX4fOj0F", "reviewer": "Reviewer_LqjG", "summary": "This paper presents the marginal fairness sliced Wasserstein barycenter (MFSWB) problem. The goal of MFSWB is to achieve fairness of barycenter to marginals. To solve the problem, the Lagaragian format and three surrogate formulations are proposed, from biased estimation to unbiased estimation. Both time and space complexities are analyzed. \n\nFor the experiments, Gaussian toy data is utilized for verification. Then, point cloud, color hamonization, and SWAE experiments are conducted on more realistic data.   \nBoth fairness metirc and barycenter metrics are reported.", "questions": "- What does USWB stand for in the paper? \n- Can the method extended to settings with more than two images or point cloud?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The presented problem is novel and reasonable. If fairness of barycenter can be achieved w.r.t. marginals, then varous future applicaiton can be conducted e.g. ensuring fairness of ML algorithms. \n- The paper organization, problem definition and writing is good and easy to follow. If audience has some Sliced Wasserstein knowledge, then it will be easy to follow the idea. \n- The Lagaragian formulation as well as three surrogates are proposed to deal with the Fairness barycenter estimation with unbiased gradient estimator.   \nThe space and time complexity analysis show that the proposed e-MFSWB has good space and time complexity compared with conventional SWB.", "weaknesses": "- Most of the experiments conducted are simple or on simple dataset, such as Gaussian, Simple Color Hamonization of two images or point cloud of only two shapes, AE on simple MNIST dataset.   \nI am wondering the scability and real application of the proposed method. For example, if more than two (e.g. >5) images or point clouds are adopted, what might be the results?   \nOr are there more realistic potential appplications of the proposed method? If related work or potential applications are discussed, it would be better. For example, CelebA dataset results or Fairness generation [R1]. \n- USWB first appears in Figure 1 and used in several places. But it is not defined. \n- I can understand the F-metric is significantly improved with the proposed solution. But for W-metric why it is also improved. Are there feasible explanation or insights? Can you provide an explanation or discussion on why the W-metric improves along with the F-metric? This would be valuableto better understand the proposed method. \n\n[R1] Fair generative modeling via weak supervision.\n\n–––––––––––––––\nAfter Rebuttal: \nThe authors' rebuttal addressed those questions comprehensively, I would like to increase my overall score to 8.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)