Original: T HE U NBALANCED G ROMOV W ASSERSTEIN D ISTANCE : C ONIC F ORMULATION AND R ELAXATION
Revised: T HE U NBALANCED G ROMOV W ASSERSTEIN D ISTANCE : C ONIC F ORMULATION AND R ELAXATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Comparing metric measure spaces (i.e. a metric space endowed with a probability distribution) is at the heart of many machine learning problems.
Revised: Comparing metric measure spaces (i e. a metric space endowed with a probability distribution) is at the heart of many machine learning problems.

Original: This includes for instance predicting properties of molecules in quantum chemistry or generating graphs with varying connectivity.
Revised: This includes for instance predicting properties of molecules in quantum chemistry or generating graphs with varying connectivity.

Original: The most popular distance between such metric measure spaces is the Gromov-Wasserstein (GW) distance, which is the solution of a quadratic assignment problem.
Revised: The most popular distance between such metric measure spaces is the Gromov-Wasserstein (GW) distance, which is the solution of a quadratic assignment problem.

Original: This distance has been successfully applied to supervised learning and generative modeling, for applications as diverse as quantum chemistry or natural language processing.
Revised: This distance has been successfully applied to supervised learning and generative modeling, for applications as di- verse as quantum chemistry or natural language processing.

Original: The GW distance is however limited to the comparison of metric measure spaces endowed with a probability distribution.
Revised: The GW distance is however limited to the comparison of metric measure spaces endowed with a probability distribution.

Original: This strong limitation is problematic for many applications in ML where there is no a priori natural normalization on the total mass of the data.
Revised: This strong limitation is problematic for many applications in ML where there is no a priori natural normalization on the total mass of the data.

Original: Furthermore, imposing an exact conservation of mass across spaces is not robust to outliers and often leads to irregular matching.
Revised: Furthermore, imposing an exact conservation of mass across spaces is not robust to outliers and often leads to irregular matching.

Original: To alleviate these issues, we introduce two Unbalanced Gromov-Wasserstein formulations: a distance and a more tractable upper-bounding relaxation.
Revised: To alleviate these issues we introduce two Unbalanced Gromov-Wasserstein formulations: a distance and a more tractable upper-bounding relaxation.

Original: They both allow the comparison of metric spaces equipped with arbitrary positive measures up to isometries.
Revised: They both allow the comparison of metric spaces equipped with arbitrary positive measures up to isometries.

Original: The ﬁrst formulation is a positive and deﬁnite divergence based on a relaxation of the mass conservation constraint using a novel type of quadratically-homogeneous divergence.
Revised: The ﬁrst formulation is a positive and deﬁnite divergence based on a relaxation of the mass conservation constraint using a novel type of quadratically-homogeneous divergence.

Original: This divergence works hand in hand with the entropic regularization approach which is popular to solve large scale optimal transport problems.
Revised: This divergence works hand in hand with the entropic regularization approach which is popular to solve large scale optimal transport problems.

Original: We show that the underlying non-convex optimization problem can be efﬁciently tackled using a highly parallelizable and GPU-friendly iterative scheme.
Revised: We show that the underlying non-convex optimization problem can be efﬁciently tackled using a highly parallelizable and GPU-friendly iterative scheme.

Original: The second formulation is a distance between mm-spaces up to isometries based on a conic lifting.
Revised: The second formulation is a distance between mm-spaces up to isometries based on a conic lifting.

Original: Lastly, we provide numerical simulations to highlight the salient features of the unbalanced divergence and its potential applications in ML.
Revised: Lastly, we provide numerical simulations to highlight the salient features of the unbalanced divergence and its potential applications in ML

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Comparing data distributions on different metric spaces is a basic problem in machine learning.
Revised: Comparing data distributions on different metric spaces is a basic problem in machine learning.

Original: This class of problems is for instance at the heart of surfaces (Bronstein et al., 2006) or graph matching (Xu et al., 2019) (equipping the surface or graph with its associated geodesic distance), regression problems in quantum chemistry (Gilmer et al., 2017) (viewing the molecules as distributions of points in R 3 ) and natural language processing
Revised: This class of problems is for instance at the heart of surfaces (Bronstein et al., 2006) or graph matching (Xu et al., 2019) (equipping the surface or graph with its associated geodesic distance), regression problems in quantum chemistry (Gilmer et al., 2017) (viewing the molecules as distributions of points in R 3 ) and natural language processing

Original: (Grave et al., 2019; Alvarez-Melis &
Revised: (Grave et al., 2019; Alvarez-Melis &

Original: Jaakkola, 2018) (where texts in different languages are embedded as points distributions in different vector spaces).
Revised: Jaakkola, 2018) (where texts in different languages are embedded as points distributions in different vector spaces).

Original: This paper deﬁnes for the ﬁrst time a class of distances between these objects, which we deﬁne below as metric measure spaces.

Original: Metric measure spaces.
Revised: Metric measure spaces.

Original: The mathematical way to formalize these problems is to model the data as metric measure spaces (mm-spaces).
Revised: The mathematical way to formalize these problems is to model the data as metric measure spaces (mm-spaces).

Original: A mm-space is denoted as X =
Revised: A mm-space is denoted as X =

Original: ( X, d, µ ) where X is a complete separable set endowed with a distance d and a positive Borel measure µ ∈ M + ( X ) .
Revised: ( X, d, µ ) where X is a complete separable set endowed with a distance d and a positive Borel measure µ ∈ M + ( X ) .

Original: For instance, if X = ( x i )
Revised: For instance, if X = ( x i )

Original: i is a ﬁnite set of points, then µ = (cid:80) i m i
Revised: i is a ﬁnite set of points, then µ = (cid:80) i m i

Original: δ x i (here δ x
Revised: δ x i (here δ x

Original: i is the Dirac mass at x i ) is simply a set of positive weights m i
Revised: i is the Dirac mass at x i ) is simply a set of positive weights m i

Original: = µ ( { x i } ) ≥ 0 associated to each point x i , which accounts for its mass or importance.
Revised: = µ ( { x i } ) ≥ 0 associated to each point x i , which accounts for its mass or importance.

Original: For instance, setting some m i to 0 is equivalent to removing the point x i .
Revised: For instance, setting some m i to 0 is equivalent to removing the point x i .

Original: We refer to Sturm (2012) for a mathematical account on the theory of mm-spaces.
Revised: We refer to Sturm (2012) for a mathematical account on the theory of mm-spaces.

Original: In all the applications highlighted above, it makes sense to perform the comparisons up to isometric transformations of the data.
Revised: In all the applications highlighted above, it makes sense to perform the comparisons up to isometric transformations of the data.

Original: Two mm-spaces X =
Revised: Two mm-spaces X =

Original: ( X, d X , µ ) and Y =
Revised: ( X, d X , µ ) and Y =

Original: ( Y, d Y , ν ) are considered to be equal (denoted X ∼ Y ) if they are isometric, meaning that there is a bijection ψ : spt( µ ) → spt( ν ) (where spt( µ ) is the support of µ ) such that d X ( x, y )
Revised: ( Y, d Y , ν ) are considered to be equal (denoted X ∼ Y ) if they are isometric, meaning that there is a bijection ψ : spt( µ ) → spt( ν ) (where spt( µ ) is the support of µ ) such that d X ( x, y )

Original: =
Revised: =

Original: d Y ( ψ ( x ) , ψ ( y )) and ψ (cid:93) µ = ν .
Revised: d Y ( ψ ( x ) , ψ ( y )) and ψ (cid:93) µ = ν .

Original: Here ψ (cid:93) is the push-forward operator, so that ψ (cid:93) µ = ν is equivalent to imposing ν ( A )
Revised: Here ψ (cid:93) is the push-forward operator, so that ψ (cid:93) µ = ν is equivalent to imposing ν ( A )

Original: =
Revised: =

Original: µ ( ψ − 1 ( A )) for any set A ⊂ Y .
Revised: µ ( ψ − 1 ( A )) for any set A ⊂ Y .

Original: For discrete spaces where µ = (cid:80) i m i δ x i , then one should have ν = ψ (cid:93) µ = (cid:80) i m i
Revised: For discrete spaces where µ = (cid:80) i m i δ x i , then one should have ν = ψ (cid:93) µ = (cid:80) i m i

Original: δ ψ ( x i ) .
Revised: δ ψ ( x i ) .

Original: As highlighted by M´emoli (2011), considering mm-spaces up to isometry is a powerful way to formalize and analyze a wide variety of problems such as matching, regression and classiﬁcation of distributions of points belonging to different spaces.
Revised: As highlighted by M´emoli (2011), considering mm-spaces up to isometry is a powerful way to formalize and analyze a wide variety of problems such as matching, regression and classiﬁcation of distributions of points belonging to different spaces.

Original: The key to unlock all these problems is the computation of a distance between mm-spaces up to isometry.
Revised: The key to unlock all these problems is the computation of a distance between mm-spaces up to isometry.

Original: So far, existing distances (reviewed below) assume that µ is a probability distribution, i.e. µ ( X )
Revised: So far, existing distances (reviewed below) assume that µ is a probability distribution, i.e. µ ( X )

Original: = 1 .
Revised: = 1 .

Original: This constraint is not natural and sometimes problematic for most of the practical applications to machine learning.
Revised: This constraint is not natural and sometimes problematic for most of the practical applications to machine learning.

Original: The goal of this paper is to alleviate this restriction.
Revised: The goal of this paper is to alleviate this restriction.

Revised: We deﬁne for the ﬁrst time a class of distances between unbalanced metric measure spaces, these distances being upper-bounded by divergences which can be approximated by an efﬁcient numerical scheme.

Original: Csisz´ar divergences The simplest case is when X
Revised: Csisz´ar divergences The simplest case is when X

Original: = Y and one simply ignores the underlying metric.
Revised: = Y and one simply ignores the underlying metric.

Original: One can then use Csisz´ar divergences (or ϕ -divergences), which perform a pointwise comparison (in contrast with optimal transport distances, which perform a displacement comparison).
Revised: One can then use Csisz´ar divergences (or ϕ -divergences), which perform a pointwise comparison (in contrast with optimal transport distances, which perform a displacement comparison).

Original: It is deﬁned using an entropy function ϕ : R + → [0 , + ∞ ] , which is a convex, lower semi-continuous, positive function with ϕ (1) = 0 .
Revised: It is deﬁned using an entropy function ϕ : R + → [0 , + ∞ ] , which is a convex, lower semi-continuous, positive function with ϕ (1) = 0 .

Original: The Csisz´ar ϕ -divergence reads
Revised: The Csisz´ar ϕ -divergence reads

Original: [Equation]
Revised: D ϕ ( µ | ν ) def. = (cid:82) X ϕ (cid:0) d µ d ν (cid:1) d ν + ϕ

Original:  d µ ⊥ , where µ = d µ d ν ν + µ ⊥ is the Lebesgue decomposition of µ with respect to ν and ϕ (cid:48)∞ = lim
Revised: (cid:48)∞ (cid:82) X d µ ⊥ , where µ = d µ d ν ν + µ ⊥ is the Lebesgue decomposition of µ with respect to ν and ϕ (cid:48)∞ = lim

Original: r →∞ ϕ
Revised: r →∞ ϕ

Original: ( r ) /r
Revised: ( r ) /r

Original: ∈ R ∪ { + ∞} is called the recession constant.
Revised: ∈ R ∪ { + ∞} is called the recession constant.

Original: This divergence D ϕ is convex, positive, 1-homogeneous and weak* lower-semicontinuous, see Liero et al.
Revised: This divergence D ϕ is convex, positive, 1-homogeneous and weak* lower-semicontinuous, see Liero et al.

Original: (2015) for details.
Revised: (2015) for details.

Original: Particular instances of ϕ -divergences are Kullback-Leibler ( KL ) for ϕ ( r ) =
Revised: Particular instances of ϕ -divergences are Kullback-Leibler ( KL ) for ϕ ( r ) =

Original: r log( r ) − r + 1 (note that ϕ
Revised: r log( r ) − r + 1 (note that ϕ

Original: (cid:48)∞ = ∞ ) and Total Variation ( TV ) for ϕ ( r ) =
Revised: (cid:48)∞ = ∞ ) and Total Variation ( TV ) for ϕ ( r ) =

Original: | r − 1 | .
Revised: | r − 1 | .

Original: Balanced and unbalanced optimal transport.
Revised: Balanced and unbalanced optimal transport.

Original: If the common embedding space X is equipped with a distance d ( x, y ) , one can use more elaborated methods such as optimal transport (OT) distances, which are computed by solving convex optimization problems.
Revised: If the common embedding space X is equipped with a distance d ( x, y ) , one can use more elaborated methods such as optimal transport (OT) distances, which are computed by solving convex optimization problems.

Original: This type of methods has proven useful for ML problems as diverse as domain adaptation (Courty et al., 2014), supervised learning over histograms (Frogner et al., 2015) and unsupervised learning of generative models (Arjovsky et al., 2017).
Revised: This type of methods has proven useful for ML problems as diverse as domain adaptation (Courty et al., 2014), supervised learning over histograms (Frogner et al., 2015) and unsupervised learning of generative models (Arjovsky et al., 2017).

Original: In this case, the extension from probability distributions to arbitrary positive measures ( µ, ν ) ∈ M + ( X ) 2 is now well understood and corresponds to the theory of unbalanced
Revised: In this case, the extension from probability distributions to arbitrary positive measures ( µ, ν ) ∈ M + ( X ) 2 is now well understood and corresponds to the theory of unbalanced

Original: OT.
Revised: OT.

Original: Following Liero et al.
Revised: Following Liero et al.

Original: ; Chizat et al.
Revised: ; Chizat et al.

Original: (2018c)
Revised: (2018c)

Original: , a family of unbalanced Wasserstein distances is deﬁned by solving
Revised: , a family of unbalanced Wasserstein distances is deﬁned by solving

Original: [Equation]
Revised: [Equation]

Original: Here ( π 1 , π 2 ) are the two marginals of the joint distribution π , deﬁned by π 1 ( A )
Revised: Here ( π 1 , π 2 ) are the two marginals of the joint distribution π , deﬁned by π 1 ( A )

Original: =
Revised: =

Original: π ( A × Y ) for
Revised: π ( A × Y ) for

Original: A ⊂ X .
Revised: A ⊂ X .

Original: The mapping λ : R + → R and exponent q ≥ 1 should be chosen wisely to ensure for instance that UW deﬁnes a distance (see Section 2.2.1).
Revised: The mapping λ : R + → R and exponent q ≥ 1 should be chosen wisely to ensure for instance that UW deﬁnes a distance (see Section 2.2.1).

Original: It is frequent to take ρ D ϕ instead of D ϕ
Revised: It is frequent to take ρ D ϕ instead of D ϕ

Original: (i.e. take ψ = ρϕ ) to adjust the strength of the marginals’ penalization.
Revised: (i.e. take ψ = ρϕ ) to adjust the strength of the marginals’ penalization.

Original: Balanced OT is retrieved with the convex indicator ϕ
Revised: Balanced OT is retrieved with the convex indicator ϕ

Original: =
Revised: =

Original: ι { 1 } or by taking the limit ρ → + ∞ , which enforces π 1 = µ and π 2 = ν .
Revised: ι { 1 } or by taking the limit ρ → + ∞ , which enforces π 1 = µ and π 2 = ν .

Original: When 0 < ρ < + ∞ , unbalanced OT operates a trade-off between transportation and creation of mass, which is crucial to be robust to outliers in the data and to cope with mass variations in the modes of the distributions.
Revised: When 0 < ρ < + ∞ , unbalanced OT operates a trade-off between transportation and creation of mass, which is crucial to be robust to outliers in the data and to cope with mass variations in the modes of the distributions.

Original: For supervised tasks, the value of ρ should be cross-validated to obtain the best performances.
Revised: For supervised tasks, the value of ρ should be cross-validated to obtain the best performances.

Original: Its use is gaining popularity in applications, such as medical imaging registration (Feydy et al., 2019a), videos (Lee et al., 2019) and gradient ﬂow to train neural networks (Chizat & Bach, 2018;
Revised: Its use is gaining popularity in applications, such as medical imaging registration (Feydy et al., 2019a), videos (Lee et al., 2019), generative learning (Balaji et al., 2020) and gradient ﬂow to train neural networks (Chizat & Bach, 2018;

Original: Rotskoff et al., 2019).
Revised: Rotskoff et al., 2019).

Original: Furthermore, existing efﬁcient algorithms for balanced OT extend to this unbalanced problem.
Revised: Furthermore, existing efﬁcient algorithms for balanced OT extend to this unbalanced problem.

Original: In particular Sinkhorn’s iterations, introduced in ML for balanced OT by Cuturi (2013), extend to unbalanced OT
Revised: In particular Sinkhorn’s iterations, introduced in ML for balanced OT by Cuturi (2013), extend to unbalanced OT

Original: (Chizat et al., 2018a; S´ejourn´e et al., 2019), as detailed in Section 3.
Revised: (Chizat et al., 2018a; S´ejourn´e et al., 2019), as detailed in Section 3.

Original: The Gromov-Wasserstein distance and its applications.
Revised: The Gromov-Wasserstein distance and its applications.

Original: The Gromov-Wasserstein (GW) distance (M´emoli, 2011; Sturm, 2012) generalizes the notion of OT to the setting of mm-spaces up to isometries.
Revised: The Gromov-Wasserstein (GW) distance (M´emoli, 2011; Sturm, 2012) generalizes the notion of OT to the setting of mm-spaces up to isometries.

Original: It corresponds to replacing the linear cost (cid:82) λ ( d )d π of OT by a quadratic function
Revised: It corresponds to replacing the linear cost (cid:82) λ ( d )d π of OT by a quadratic function

Original: GW 
Revised: GW ( X , Y

Original: [Equation]
Revised: [Equation]

Original: X ,
Revised: =

Original: [Equation]
Revised: [Equation]

Original: It is proved in M´emoli (2011); Sturm (2012) that GW deﬁnes with λ ( t ) =
Revised: It is proved in M´emoli (2011); Sturm (2012) that GW deﬁnes with λ ( t ) =

Original: t q a distance up to isometries on balanced mm-spaces (i.e. the measures are probability distributions).
Revised: t q a distance up to isometries on balanced mm-spaces (i.e. the measures are probability distributions).

Original: In this paper, we extend this construction to arbitrary positive measures, and provide explicit settings in Section 2.2.1.
Revised: In this paper, we extend this construction to arbitrary positive measures, and provide explicit settings in Section 2.2.1.

Original: This distance is applied successfully in natural language processing for unsupervised translation learning (Grave et al., 2019; Alvarez-Melis & Jaakkola, 2018), in generative learning for objects lying in spaces of different dimensions (Bunne et al., 2019) and to build VAE for graphs (Xu et al., 2020).
Revised: This distance is applied successfully in natural language processing for unsupervised translation learning (Grave et al., 2019; Alvarez-Melis & Jaakkola, 2018), in generative learning for objects lying in spaces of different dimensions (Bunne et al., 2019) and to build VAE for graphs (Xu et al., 2020).

Original: It has been adapted for domain adaptation over different spaces (Redko et al., 2020).
Revised: It has been adapted for domain adaptation over different spaces (Redko et al., 2020).

Original: It is also a relevant distance to compute barycenters between graphs or shapes (Vayer et al., 2018; Chowdhury & Needham, 2020).
Revised: It is also a relevant distance to compute barycenters between graphs or shapes (Vayer et al., 2018; Chowdhury & Needham, 2020).

Original: When ( X , Y ) are Euclidean spaces, this distance compares distributions up to rigid isometry, and is closely related (but not equal) to metrics deﬁned by procrustes analysis (Grave et al., 2019; Alvarez-Melis et al., 2019).
Revised: When ( X , Y ) are Euclidean spaces, this distance compares distributions up to rigid isometry, and is closely related (but not equal) to metrics deﬁned by procrustes analysis (Grave et al., 2019; Alvarez-Melis et al., 2019).

Original: The problem (2) is non convex because the quadratic form (cid:82) λ ( | d X − d Y | )
Revised: The problem (2) is non convex because the quadratic form (cid:82) λ ( | d X − d Y | )

Original: d π ⊗ π is not positive in general.
Revised: d π ⊗ π is not positive in general.

Original: It is in fact closely related to quadratic assignment problems (Burkard et al., 1998), which are used for graph matching problems, and are known to be NP-hard in general.
Revised: It is in fact closely related to quadratic assignment problems (Burkard et al., 1998), which are used for graph matching problems, and are known to be NP-hard in general.

Original: Nevertheless, nonconvex optimization methods have been shown to be successful in practice to use GW distances for ML problems.
Revised: Nevertheless, nonconvex optimization methods have been shown to be successful in practice to use GW distances for ML problems.

Original: This includes for instance alternating minimization (M´emoli, 2011; Redko et al., 2020) and entropic regularization (Peyr´e et al., 2016; Gold & Rangarajan, 1996).
Revised: This includes for instance alternating minimization (M´emoli, 2011; Redko et al., 2020) and entropic regularization (Peyr´e et al., 2016; Gold & Rangarajan, 1996).

Original: Related works and contributions.
Revised: Related works and contributions.

Original: The recent work of De Ponti & Mondino (2020) extends the L p transportation distance deﬁned in Sturm et al.
Revised: The concomitant work of De Ponti & Mondino (2020) extends the L p transportation distance deﬁned in Sturm et al.

Original: (2006) to unbalanced mm-spaces and studies its geometric properties.
Revised: (2006) to unbalanced mm-spaces and studies its geometric properties.

Original: This distortion distance is not equivalent to the GW distance, and is more difﬁcult to estimate numerically because it explicitly imposes a triangle inequality constraint in the optimization problem.
Revised: This distortion distance is not equivalent to the GW distance, and is more difﬁcult to estimate numerically because it explicitly imposes a triangle inequality constraint in the optimization problem.

Original: The work of Chapel et al.
Revised: The work of Chapel et al.

Original: (2020) relaxes the GW distance to the unbalanced setting by hybridizing GW with partial OT (Figalli, 2010) for unsupervised labeling.
Revised: (2020) relaxes the GW distance to the unbalanced setting by hybridizing GW with partial OT (Figalli, 2010) for unsupervised labeling.

Original: It ressembles one particular setting of our formulation, but with some important differences, detailed in Section 2.
Revised: It ressembles one particular setting of our formulation, but with some important differences, detailed in Section 2.

Original: Our construction is also connected to partial matching methods, which ﬁnd numerous applications in graphics and vision (Cosmo et al., 2016).
Revised: Our construction is also connected to partial matching methods, which ﬁnd numerous applications in graphics and vision (Cosmo et al., 2016).

Original: In particular, Rodola et al.
Revised: In particular, Rodola et al.

Original: (2012) introduces a mass conservation relaxation of the GW problem.
Revised: (2012) introduces a mass conservation relaxation of the GW problem.

Original: The two main contributions of this paper are the deﬁnition of two formulations relaxing the GW distance.
Revised: The two main contributions of this paper are the deﬁnition of two formulations relaxing the GW distance.

Original: The ﬁrst one is called the Unbalanced Gromov-Wasserstein (UGW) divergence and can be computed efﬁciently on GPUs.
Revised: The ﬁrst one is called the Unbalanced Gromov-Wasserstein (UGW) divergence and can be computed efﬁciently on GPUs.

Original: The second one is called the Conic Gromov-Wasserstein distance (CGW).
Revised: The second one is called the Conic Gromov-Wasserstein distance (CGW).

Original: It is proved to be a distance between mm-spaces endowed with positive measures up to isometries, as stated in Theorem 1 which is the main theoretical result of this paper.
Revised: It is proved to be a distance between mm-spaces endowed with positive measures up to isometries, as stated in Theorem 1 which is the main theoretical result of this paper.

Original: We also prove in Theorem 1 that UGW can be used as a surrogate upper-bounding CGW.
Revised: We also prove in Theorem 1 that UGW can be used as a surrogate upper-bounding CGW.

Original: We present those concepts and their properties in Section 2.
Revised: We present those concepts and their properties in Section 2.

Original: We also detail in Section 3 an efﬁcient computational scheme for a particular setting of UGW.
Revised: We also detail in Section 3 an efﬁcient computational scheme for a particular setting of UGW.

Original: This method computes an approximate stationary point of the nonconvex energy.
Revised: This method computes an approximate stationary point of the nonconvex energy.

Original: It leverages the strength of entropic regularization and the Sinkhorn algorithm, namely that it is GPU-friendly and deﬁnes smooth loss functions amenable to back-propagation for ML applications.
Revised: It leverages the strength of entropic regularization and the Sinkhorn algorithm, namely that it is GPU-friendly and deﬁnes smooth loss functions amenable to back-propagation for ML applications.

Original: Section 4 provides some numerical experiments to highlight the qualitative behavior of this algorithm, which shed some lights on the favorable properties of UGW to cope with outliers and mass variations in the modes of the distributions.
Revised: Section 4 provides some numerical experiments to highlight the qualitative behavior of this algorithm, which shed some lights on the favorable properties of UGW to cope with outliers and mass variations in the modes of the distributions.

Original:  U NBALANCED G ROMOV -W ASSERSTEIN FORMULATIONS
Revised:  U NBALANCED G ROMOV -W ASSERSTEIN FORMULATIONS

Original: We present in this section our two new formulations and their properties.
Revised: We present in this section our two new formulations and their properties.

Original: The ﬁrst one, called UGW, is exploited in Sections 3 and 4 to derive an efﬁcient algorithm used in numerical experiments.
Revised: The ﬁrst one, called UGW, is exploited in Sections 3 and 4 to derive an efﬁcient algorithm used in numerical experiments.

Original: The second one, called CGW, deﬁnes a distance between mm-spaces up to isometries.
Revised: The second one, called CGW, deﬁnes a distance between mm-spaces up to isometries.

Original: Those results build upon the results of Liero et al.
Revised: Those results build upon the results of Liero et al.

Original: (2015), and a summary of the construction of UOT
Revised: (2015), and a summary of the construction of UOT

Original: is detailed in
Revised: is detailed in

Original: Appendix A
Revised: Appendix A

Original: In all what follows, we consider complete separable mm-spaces endowed with a metric and a positive measure.
Revised: In all what follows, we consider complete separable mm-spaces endowed with a metric and a positive measure.

Original: 2. T HE UNBALANCED G ROMOV -W ASSERSTEIN DIVERGENCE
Revised: 2. T HE UNBALANCED G ROMOV -W ASSERSTEIN DIVERGENCE

Original: This new formulation makes use of quadratic ϕ -divergences, deﬁned as D ⊗ ϕ ( ρ | ν ) def.
Revised: This new formulation makes use of quadratic ϕ -divergences, deﬁned as D ⊗ ϕ ( ρ | ν ) def.

Original: =
Revised: =

Original: D ϕ ( ρ ⊗ ρ | ν ⊗ ν ) , where ρ ⊗ ρ ∈ M + ( X 2 ) is the tensor product measure deﬁned by d( ρ ⊗ ρ )( x, y )
Revised: D ϕ ( ρ ⊗ ρ | ν ⊗ ν ) , where ρ ⊗ ρ ∈ M + ( X 2 ) is the tensor product measure deﬁned by d( ρ ⊗ ρ )( x, y )

Original: =
Revised: =

Original: d ρ ( x )d ρ ( y ) .
Revised: d ρ ( x )d ρ ( y ) .

Original: Note that D ⊗ ϕ is not a convex function in general.
Revised: Note that D ⊗ ϕ is not a convex function in general.

Original: Deﬁnition 1 (Unbalanced GW) .
Revised: Deﬁnition 1 (Unbalanced GW) .

Original: The Unbalanced Gromov-Wasserstein divergence is deﬁned as
Revised: The Unbalanced Gromov-Wasserstein divergence is deﬁned as

Original: UGW ( X , Y ) = inf π ∈M + ( X × Y )
Revised: UGW ( X , Y ) = inf π ∈M + ( X × Y )

Original: L ( π ) where
Revised: L ( π ) where

Original: [Equation]
Revised: [Equation]

Original: This deﬁnition can be understood as an hybridation between (1) and (2) but with a twist: one needs to use the quadratic divergence D ⊗ ϕ in place of D ϕ .
Revised: This deﬁnition can be understood as an hybridation between (1) and (2) but with a twist: one needs to use the quadratic divergence D ⊗ ϕ in place of D ϕ .

Revised: To the best of our knowledge, it is the ﬁrst time such quadratic divergences are being used and studied.

Original: In the TV case, this is the most important distinction between UGW and partial GW (Chapel et al., 2020).
Revised: In the TV case, this is the most important distinction between UGW and partial GW (Chapel et al., 2020).

Original: Using quadratic divergences results in UGW being 2-homogeneous: if µ and ν are multiplied by θ ≥ 0 , then UGW ( X , Y ) is multiplied by θ 2 .

Original: Not using such divergences results in a lack of 2-homogeneity of L which is crucial to show the connection between our proposed formulations.

Original: Note also that the balanced GW distance (2) is recovered as a particular case when using ϕ =
Revised: Note also that the balanced GW distance (2) is recovered as a particular case when using ϕ =

Original: ι = or by letting ρ → + ∞ for an entropy ψ = ρϕ .
Revised: ι = or by letting ρ → + ∞ for an entropy ψ = ρϕ .

Revised: Using quadratic divergences results in UGW being 2-homogeneous: for θ ≥ 0 , writing ( X θ , Y θ ) equiped with ( θµ, θν ) , one has θ − 2 UGW ( X θ , Y θ ) =

Revised: UGW ( X , Y ) .

Revised: When using non tensorized ϕ -divergences, the resulting unbalanced Gromov-Wassertein functional between X θ and Y θ have very different and inconsistent behaviors when θ → 0 and θ → + ∞ .

Revised: Indeed, once normalized by θ − 2 and θ − 1 , one obtains respectively balanced GW and a Hellinger-type distance.

Revised: Using tensorized divergences ensure that the behavior does not depends on θ .

Original: We ﬁrst prove the existence of optimal plans π solution to (3), which holds for the three key settings of Section 2.2.1.
Revised: We ﬁrst prove the existence of optimal plans π solution to (3), which holds for the three key settings of Section 2.2.1, namely for KL , TV , and for compact metric spaces (such as ﬁnite pointclouds and graphs).

Original: All proofs are deferred in Appendix B.
Revised: All proofs are deferred in Appendix B.

Original: Proposition 1 (Existence of minimizers) .
Revised: Proposition 1 (Existence of minimizers) .

Original: We assume that ( X, Y ) are compact and that either (i) ϕ superlinear, i.e ϕ (cid:48)∞ = ∞ , or (ii) λ has compact sublevel sets in R + and 2 ϕ
Revised: We assume that ( X, Y ) are compact and that either (i) ϕ superlinear, i.e ϕ (cid:48)∞ = ∞ , or (ii) λ has compact sublevel sets in R + and 2 ϕ

Original: (cid:48)∞ + inf λ > 0 .
Revised: (cid:48)∞ + inf λ > 0 .

Original: Then there exists π ∈ M + ( X × Y ) such that UGW ( X , Y ) =
Revised: Then there exists π ∈ M + ( X × Y ) such that UGW ( X , Y ) =

Original: L ( π ) .
Revised: L ( π ) .

Original: The following proposition ensures that the functional UGW can be used to compare mm-spaces.
Revised: The following proposition ensures that the functional UGW can be used to compare mm-spaces.

Original: Proposition 2 (Deﬁniteness of UGW) .
Revised: Proposition 2 (Deﬁniteness of UGW) .

Original: Assume that ϕ − 1 ( { 0 } )
Revised: Assume that ϕ − 1 ( { 0 } )

Original: =
Revised: =

Original: { 1 } and λ − 1 ( { 0 } ) = { 0 } .
Revised: { 1 } and λ − 1 ( { 0 } ) = { 0 } .

Original: Then
Revised: Then

Original: UGW ( X , Y ) ≥ 0 and is 0 if and only if X ∼ Y .
Revised: UGW ( X , Y ) ≥ 0 and is 0 if and only if X ∼ Y .

Original: We end this section with a reformulation of UGW (3) which allows to connect it with our second formulation in the next section.
Revised: We end this section with a reformulation of UGW (3) which is important to make the connection with the second formulation of the following section.

Original: Its proof is defered to Appendix B. 
Revised: Its proof is deferred to Appendix B. It splits UGW into two parts: the term ϕ (0)( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) accounts for the pure creation/destruction of mass and a new transport cost L c accounts for the remaining part (partial/pure transport and partial creation/destruction of mass).

Original: Lemma 1.
Revised: Lemma 1.

Original: Deﬁning L c ( a, b ) def.
Revised: Deﬁning L c ( a, b ) def.

Original: =
Revised: =

Original: c + aϕ (1 /a ) + bϕ (1 /b ) , and writing ( f def.
Revised: c + aϕ (1 /a ) + bϕ (1 /b ) , and writing ( f def.

Original: = d µ d π 1 , g def.
Revised: = d µ d π 1 , g def.

Original: =
Revised: =

Original: d ν d π 2 ) the
Revised: d ν d π 2 ) the

Original: Lebesgue densities of ( µ, ν ) w.r.t.
Revised: Lebesgue densities of ( µ, ν ) w.r.t.

Original: ( π 1 , π 2 ) such that µ = fπ 1 + µ ⊥ and ν = gπ 2 + ν ⊥ , one has
Revised: ( π 1 , π 2 ) such that µ = fπ 1 + µ ⊥ and ν = gπ 2 + ν ⊥ , one has

Original: L ( π ) =
Revised: L ( π ) =

Revised: (cid:90)

Revised: [Equation]

Revised: ( f

Revised: [Equation]

Revised: f, g

Original: [Equation]
Revised: [Equation]

Revised: )d

Original: [Equation]
Revised: [Equation]

Revised: d π + ϕ (0)( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) .

Original: 2. T HE CONIC G ROMOV -W ASSERSTEIN DISTANCE
Revised: 2. T HE CONIC G ROMOV -W ASSERSTEIN DISTANCE

Original: We introduce a second “conic” formulation of unbalanced GW, which is connected to UGW, and whose construction is inspired by the conic formulation of UOT (see Appendix A for an overview).
Revised: We introduce a second “conic” formulation of unbalanced GW, which is connected to UGW, and whose construction is inspired by the conic formulation of UOT (see Appendix A for an overview).

Original: 2.2. B ACKGROUND ON CONE SETS AND DISTANCES
Revised: 2.2. B ACKGROUND ON CONE SETS AND DISTANCES

Original: The conic formulation lifts a point x ∈ X to a couple ( x, r ) ∈
Revised: The conic formulation lifts a point x ∈ X to a couple ( x, r ) ∈

Original: X × R + where r encodes some (power of) a mass.
Revised: X × R + where r encodes some (power of a) mass.

Original: Then we seek optimal transport plans deﬁned over C
Revised: Then we seek optimal transport plans deﬁned over C

Original: [ X ] def.
Revised: [ X ] def.

Original: =
Revised: =

Original: X × R + / ( X ×{ 0 } ) , where coordinates ( x, r = 0) with no mass are merged into a single point 

Revised: X × R + / ( X ×{ 0 } ) , where coordinates ( x, r = 0) with no mass are merged into a single point

Original: X called the apex of the cone.
Revised: X called the apex of the cone.

Original: In the sequel, points of X × R + are noted ( x, r ) , while [ x, r ] are quotiented points of C [ X ] .
Revised: In the sequel, points of X × R + are noted ( x, r ) , while [ x, r ] are quotiented points of C [ X ] .

Original: While transport plans depend on variables ([ x, r ] , [ y, s ]) and ([ x (cid:48) , r (cid:48) ] , [ y (cid:48) , s (cid:48) ]) in C
Revised: While transport plans depend on variables ([ x, r ] , [ y, s ]) and ([ x (cid:48) , r (cid:48) ] , [ y (cid:48) , s (cid:48) ]) in C

Original: [ X ] × C
Revised: [ X ] × C

Original: [ Y ] , the transportation cost involved in our conic formulation only makes use of the 2-D cone
Revised: [ Y ] , the transportation cost involved in our conic formulation only makes use of the 2-D cone

Original: C
Revised: C

Original: [ R + ] over R + endowed with the distance | u − v | (note that any other distance on R could be used as well).
Revised: [ R + ] over R + endowed with the distance | u − v | (note that any other distance on R could be used as well).

Original: More speciﬁcally, we consider coordinates of the form ([ u, a ] , [ v, b ]) =
Revised: More speciﬁcally, we consider coordinates of the form ([ u, a ] , [ v, b ]) =

Original: ([ d X ( x, x (cid:48) ) , rr (cid:48) ] , [ d Y ( y, y (cid:48) ) , ss (cid:48) ])
Revised: ([ d X ( x, x (cid:48) ) , rr (cid:48) ] , [ d Y ( y, y (cid:48) ) , ss (cid:48) ])

Original: ∈ C
Revised: ∈ C

Original: [ R + ] × C [ R + ] .
Revised: [ R + ] × C [ R + ] .

Original: Thus we now describe conic discrepancies
Revised: Thus we now describe conic discrepancies

Original: D on C [ R + ] , which are deﬁned for ( p, q ) ≥ 0 as 
Revised: D on C [ R + ] , which are deﬁned for ( p, q ) ≥ 0 as D

Original: [Equation]
Revised: [Equation]

Original:  a 
Revised: [ u, a ] , [ v, b ]) q def.

Original: [Equation]
Revised: [Equation]

Original: = inf
Revised: | u

Original: [Equation]
Revised: [Equation]

Original: θL
Revised: [Equation] p , b p ) where H c ( a p , [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original:  is the perspective transform of L c introduced in Lemma 1.
Revised: , b p θ ) is the perspective transform of L c introduced in Lemma 1.

Original: The intuition underpinning the deﬁnition of this cost is that the perspective transform accounts for the possibility to rescale a transport plan  by a scale θ at each points in the cone.
Revised: The intuition underpinning the deﬁnition of this cost is that the perspective transform accounts for the possibility to rescale a transport plan π by a scalar θ but the scaling is performed pointwise instead of globally.

Original: In general D is not a distance, but it is always deﬁnite as stated by this result proved in Appendix A.
Revised: In general D is not a distance, but it is always deﬁnite as stated by this result proved in Appendix A.

Original: Proposition 3.
Revised: Proposition 3.

Original: Assume λ − 1 ( { 0 } )
Revised: Assume λ − 1 ( { 0 } )

Original: = { 0 } , ϕ − 1 ( { 0 } ) =
Revised: = { 0 } , ϕ − 1 ( { 0 } ) =

Original: { 1 } and ϕ is coercive.
Revised: { 1 } and ϕ is coercive.

Original: Then D is deﬁnite on
Revised: Then D is deﬁnite on

Original: C
Revised: C

Original: [ R + ] , i.e. D ([ u, a ] , [ v, b ])
Revised: [ R + ] , i.e. D ([ u, a ] , [ v, b ])

Original: = 

Revised: =

Original: if and only if ( a = b = 0) or ( a = b and u = v ) .
Revised: if and only if ( a = b = 0) or ( a = b and u = v ) .

Original: Of particular interest are those ϕ where D is a distance, which necessitates a careful choice of λ, p and q .
Revised: Of particular interest are those ϕ where D is a distance, which necessitates a careful choice of λ, p and q .

Original: We now detail three examples where this is the case.
Revised: We now detail three examples where this is the case.

Original: Gaussian Hellinger distance (GH).
Revised: Gaussian Hellinger distance (GH).

Original: When D ϕ = KL , λ ( t ) =
Revised: When D ϕ = KL , λ ( t ) =

Original: t 2 and q = p = 2 , then one has
Revised: t 2 and q = p = 2 , then one has

Original: D ([ u, a ] , [ v, b ]) 2 = a 2 + b 2 − 2 abe −| u − v | / 2 .
Revised: D ([ u, a ] , [ v, b ]) 2 = a 2 + b 2 − 2 abe −| u − v | / 2 .

Original: This cone distance (Burago et al., 2001) is further generalized by De Ponti (2019) who shows that D is a distance for power entropies ϕ ( s )
Revised: This cone distance (Burago et al., 2001) is further generalized by De Ponti (2019) who shows that D is a distance for power entropies ϕ ( s )

Original: =
Revised: =

Original: s p − p ( s − 1) − 1 p ( p − 1) if p ≥ 1 (the case p = 1 corresponding to D ϕ = KL ).
Revised: s p − p ( s − 1) − 1 p ( p − 1) if p ≥ 1 (the case p = 1 corresponding to D ϕ = KL ).

Original: Hellinger-Kantorovich (HK) /
Revised: Hellinger-Kantorovich (HK) /

Original: Wasserstein-Fisher-Rao distance (WFR).
Revised: Wasserstein-Fisher-Rao distance (WFR).

Original: When D ϕ = KL , λ ( t ) = − log cos 2 ( t ∧ π 2 ) and q = p = 2 , then one has D ([ u, a ] , [ v, b ]) 2 = a 2 + b 2 − 2 ab cos( π 2 ∧| u − v | ) .
Revised: When D ϕ = KL , λ ( t ) = − log cos 2 ( t ∧ π 2 ) and q = p = 2 , then one has D ([ u, a ] , [ v, b ]) 2 = a 2 + b 2 − 2 ab cos( π 2 ∧| u − v | ) .

Original: This construction, which might seem peculiar, corresponds to the one used to make unbalanced
Revised: This construction, which might seem peculiar, corresponds to the one used to make unbalanced

Original: OT a geodesic distance, as detailed in (Liero et al., 2015; Chizat et al., 2018c).
Revised: OT a geodesic distance, as detailed in (Liero et al., 2015; Chizat et al., 2018c).

Original: Partial optimal transport distance (PT).
Revised: Partial optimal transport distance (PT).

Original: When D ϕ = TV , λ ( t ) =
Revised: When D ϕ = TV , λ ( t ) =

Original: t q , q ≥ 1 and p = 1 , then
Revised: t q , q ≥ 1 and p = 1 , then

Original: D ([ u, a ] , [ v, b ]) q
Revised: D ([ u, a ] , [ v, b ]) q

Original: =
Revised: =

Original: a + b − ( a ∧ b )(2 − | u − v | q ) + deﬁnes a cone distance (Chizat et al., 2018c).
Revised: a + b − ( a ∧ b )(2 − | u − v | q ) + deﬁnes a cone distance (Chizat et al., 2018c).

Original:  D EFINITIONS AND PROPERTIES
Revised: 2.2. D EFINITIONS AND PROPERTIES

Original: The conic formulation consists in solving a GW problem on the cone, with the addition of two linear constraints.
Revised: The conic formulation consists in solving a GW problem on the cone, with the addition of two linear constraints.

Original: Informally speaking, L c from Lemma 1 becomes D , the term ( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) is taken into account by the constraints (5) below, and the variables ( f, g ) are replaced by ( r p , s p ) .
Revised: Informally speaking, L c from Lemma 1 becomes D , the term ( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) is taken into account by the constraints (5) below, and the variables ( f, g ) are replaced by ( r p , s p ) .

Original: It reads CGW ( X , Y ) def.
Revised: It reads CGW ( X , Y ) def.

Original: = inf α ∈U p ( µ,ν ) H ( α ) where
Revised: = inf α ∈U p ( µ,ν ) H ( α ) where

Original: [Equation]
Revised: [Equation]

Original: It is similar to the conic formulation of UW, see Appendix A. Note that similarly to the GW formulation (2) – and in sharp contrast with the conic formulation of UW – here the transport plans are deﬁned on the cone C
Revised: It is similar to the conic formulation of UW, see Appendix A. Note that similarly to the GW formulation (2) – and in sharp contrast with the conic formulation of UW – here the transport plans are deﬁned on the cone C

Original: [ X ] × C [ Y ] but the cost D is a distance on C
Revised: [ X ] × C [ Y ] but the cost D is a distance on C

Original: [ R + ] .
Revised: [ R + ] .

Original: We present now the main contributions of this paper, proved in Appendix C.
Revised: We present now the main contributions of this paper, proved in Appendix C.

Original: We state that CGW deﬁnes a distance under conditions that hold for the settings of Section 2.2.1, and that it is controlled by UGW.
Revised: We state that CGW deﬁnes a distance under conditions that hold for the settings of Section 2.2.1, and that it is upperbounded by UGW.

Original: While the distance CGW 1 /q seems difﬁcult to compute (because it is deﬁned on a lifted space), UGW can be approximated with efﬁcient numerical schemes as detailed in Section 3. 
Revised: While the distance CGW 1 /q cannot be casted as a ﬁnite dimensional program even in discrete settings (because it is deﬁned on a lifted space), UGW can be approximated with efﬁcient numerical schemes as detailed in Section 3. The tightness of the bound between UGW and CGW and the computation of CGW are open questions left for future works.

Original: Theorem 1.
Revised: Theorem 1.

Original: (i)
Revised: (i)

Original: The divergence CGW is symmetric, positive and deﬁnite up to isometries.
Revised: The divergence CGW is symmetric, positive and deﬁnite up to isometries.

Original: (ii)
Revised: (ii)

Original: If D is a distance on C
Revised: If D is a distance on C

Original: [ R + ] , then CGW 1 /q is a distance on the set of mm-spaces up to isometries.
Revised: [ R + ] , then CGW 1 /q is a distance on the set of mm-spaces up to isometries.

Original: (iii)
Revised: (iii)

Original: For any ( D ϕ , λ, p, q ) with associated cost D on the cone, one has UGW ≥ CGW .
Revised: For any ( D ϕ , λ, p, q ) with associated cost D on the cone, one has UGW ≥ CGW .

Original:  proof (i) CGW is positive and symmetric.
Revised: Sketch of proof (i) CGW is positive and symmetric.

Original: Deﬁniteness holds thanks to Proposition 3.
Revised: Deﬁniteness holds thanks to Proposition 3.

Original: (ii)
Revised: (ii)

Original: The triangle inequality is similar to balanced OT, and applies the gluing lemma (Villani, 2003,
Revised: The triangle inequality is similar to balanced OT, and applies the gluing lemma (Villani, 2003,

Original: Lemma 7.6).
Revised: Lemma 7.6).

Original: The non-trivial part is showing that the latter lemma holds, because it glues two plans provided they have a common marginal.
Revised: The non-trivial part is showing that the latter lemma holds, because it glues two plans provided they have a common marginal.

Original: Since CGW is invariant under radial rescalings (called dilations in Appendix C), it is possible to dilate two plans such that they have a common marginal and remain optimal.
Revised: Since CGW is invariant under radial rescalings (called dilations in Appendix C), it is possible to dilate two plans such that they have a common marginal and remain optimal.

Original: (iii) Take an optimal plan π for UGW ( X , Y ) .
Revised: (iii) Take an optimal plan π for UGW ( X , Y ) .

Original: From this π one can build a plan α such that L ( π ) ≥ H ( α ) because L c ≥ H c .
Revised: From this π one can build a plan α such that L ( π ) ≥ H ( α ) because L c ≥ H c .

Original: Furthermore α ∈ U p , and is thus admissible and suboptimal, which yields UGW ( X , Y ) =
Revised: Furthermore α ∈ U p , and is thus admissible and suboptimal, which yields UGW ( X , Y ) =

Original: L ( π ) ≥ H ( α ) ≥ CGW ( X , Y ) .
Revised: L ( π ) ≥ H ( α ) ≥ CGW ( X , Y ) .

Original:  A LGORITHMS
Revised:  A LGORITHMS

Original: The computation of the distance CGW is in practice out-of-reach because it requires an optimization over a lifted conic space, which would need to be discretized.
Revised: The computation of the distance CGW is in practice out-of-reach because it requires an optimization over a lifted conic space, which would need to be discretized.

Original: We focus in this section on the numerical computation of the upper bound UGW, using an alternate minimization coupled with entropic regularization.
Revised: We focus in this section on the numerical computation of the upper bound UGW, using an alternate minimization coupled with entropic regularization.

Original: The algorithm is presented on arbitrary measures, the special case of discrete measures being a particular case.
Revised: The algorithm is presented on arbitrary measures, the special case of discrete measures being a particular case.

Original: The discretized formulas and algorithms are detailed in
Revised: The discretized formulas and algorithms are detailed in

Original: Appendix D, see also Chizat et al.
Revised: Appendix D, see also Chizat et al.

Original: (2018a); Peyr´e et al.
Revised: (2018a); Peyr´e et al.

Original: All implementations are available at https://github.com/anonymous-conference-submission .
Revised: All implementations are available at https://github.com/anonymous-conference-submission .

Original: In order to derive a simple numerical approximation scheme, following M´emoli (2011), we introduce a lower bound obtained by introducing two transportation plans.
Revised: In order to derive a simple numerical approximation scheme, following M´emoli (2011), we introduce a lower bound obtained by introducing two transportation plans.

Original: To further accelerate the method and enable GPU-friendly iterations, similarly to Gold et al.
Revised: To further accelerate the method and enable GPU-friendly iterations, similarly to Gold et al.

Original: ; Solomon et al.
Revised: ; Solomon et al.

Original: , we consider an entropic regularization.
Revised: , we consider an entropic regularization.

Original: It reads, for any ε ≥ 0 ,
Revised: It reads, for any ε ≥ 0 ,

Original: [Equation]
Revised: [Equation]

Original: and F ( π, γ ) def.
Revised: and F [Equation]

Original: = (cid:90)

Original: [Equation]
Revised: [Equation]

Revised: where ( γ 1 , γ 2 ) denote the marginals of the plan γ .

Original: Note that in contrast to the entropic regularization of GW Peyr´e et al.
Revised: Note that in contrast to the entropic regularization of GW Peyr ´ e et al.

Original: (2016), here we use a tensorized entropy to maintain the overall homogeneity of the energy.
Revised: (2016), here we use a tensorized entropy to maintain the overall homogeneity of the energy.

Original: A simple method to approximate this lower bound is to perform an alternate minimization on π and γ , which is known to converge for smooth ϕ to a stationary point since the coupling term in the functional is smooth (Tseng, 2001).
Revised: A simple method to approximate this lower bound is to perform an alternate minimization on π and γ , which is known to converge for smooth ϕ to a stationary point since the coupling term in the functional is smooth (Tseng, 2001).

Original: Note that if π ⊗ γ is optimal then so is ( sπ ) ⊗ ( 1 s γ ) with s ≥ 0 .
Revised: Note that if π ⊗ γ is optimal then so is ( sπ ) ⊗ ( 1 s γ ) with s ≥ 0 .

Original: Thus without loss of generality we optimize under the constraint m ( π ) = m ( γ )  .
Revised: Thus without loss of generality we optimize under the constraint m ( π ) = m ( γ ) by setting s = (cid:112) m ( γ ) /m ( π ) .

Original: In general, this bound is not expected to be tight, but empirically, alternate minimization often converges to a solution with π = γ (as already observed for instance in Rangarajan et al. (1999); Solomon et al.
Revised: In general, this bound is not expected to be tight, but empirically, alternate minimization often converges to a solution with π = γ (as already observed for instance in Rangarajan et al. (1999); Solomon et al.

Original: (2016)), so that the algorithm also ﬁnds a local minimizer of the UGW ε problem.
Revised: (2016)), so that the algorithm also ﬁnds a local minimizer of the UGW ε problem.

Original: In the Balanced-GW case, this can be explained by the fact that this scheme is equivalent to a mirror descent algorithm (Solomon et al., 2016).
Revised: In the Balanced-GW case in Euclidean spaces, the optimum is known to satisfy π = γ (Konno, 1976) and alternate descent is equivalent to a mirror descent algorithm (Solomon et al., 2016).

Original: Minimizing the lower bound of (6) with respect to either π or γ is non-trivial for an arbitrary ϕ .
Revised: Minimizing the lower bound of (6) with respect to either π or γ is non-trivial for an arbitrary ϕ .

Original: We restrict our attention to the Kullback-Leibler case D ϕ =
Revised: We restrict our attention to the Kullback-Leibler case D ϕ =

Original: ρ KL with ρ > 0 , which can be addressed by solving a regularized and convex unbalanced problem as studied in Chizat et al.
Revised: ρ KL with ρ > 0 , which can be addressed by solving a regularized and convex unbalanced problem as studied in Chizat et al.

Original: (2018a)
Revised: (2018a)

Original: ; S ´ ejourn ´
Revised: ; S ´ ejourn ´

Original: e et al.
Revised: e et al.

Original: It is explained in the following proposition.
Revised: It is explained in the following proposition.

Original: Proposition 4.
Revised: Proposition 4.

Original: For a ﬁxed γ , the optimal π ∈ arg min π F ( π, γ ) + ε KL( π ⊗ γ | ( µ ⊗ ν ) ⊗ 2 ) is the solution of min π
Revised: For a ﬁxed γ , the optimal π ∈ arg min π F ( π, γ ) + ε KL( π ⊗ γ | ( µ ⊗ ν ) ⊗ 2 ) is the solution of min π

Original: [Equation]
Revised: [Equation]

Original:  where m ( γ ) def.
Revised: ( π | µ ⊗ ν ) , where m ( γ ) def.

Original: = γ ( X × Y ) is the mass of γ , and where we deﬁne the cost associated to γ as 
Revised: = γ ( X × Y ) is the mass of γ , and where we deﬁne the cost associated to γ as c [Equation]

Original: [Equation]
Revised: [Equation]

Original: Computing the cost c εγ for spaces X and Y of n points has in general a cost O ( n 4 ) in time and memory.
Revised: Computing the cost c εγ for spaces X and Y of n points has in general a cost O ( n 4 ) in time and memory.

Original: However, as explained for instance in Peyr ´ e et al.
Revised: However, as explained for instance in Peyr´e et al.

Original: (2016), for the special case λ ( t ) =
Revised: (2016), for the special case λ ( t ) =

Original: t 2 , this cost is reduced to O ( n 3 ) in time and O ( n 2 ) in memory, which allows to scale the method to larger problems.
Revised: t 2 , this cost is reduced to O ( n 3 ) in time and O ( n 2 ) in memory.

Original: This is the setting we consider in the numerical simulations.
Revised: This is the setting we consider in the numerical simulations.

Original: Algorithm ε ← m ( π ) ε 4: while ( f, g ) has not converged do 5: ∀ x, f ( x ) ← − ˜ ε ˜ ρ ˜ ε +˜ ρ log (cid:16) (cid:82) e ( g ( y ) − c ( x,y )) / ˜ ε d ν ( y ) (cid:17) 6: ∀ y, g ( y ) ← − ˜ ε ˜ ρ ˜ ε
Revised: This makes the method applicable for scales of the order of 10 4 points.

Original: +˜ ρ log (cid:16) (cid:82) e ( f ( x ) − c ( x,y )) /

Original: ˜ ε

Original: d µ ( x ) (cid:17) 7: Update γ ( x, y ) ← exp (cid:104) ( f ( x ) + g ( y ) − c ( x, y )) / ˜ ε (cid:105) µ ( x ) ν ( y ) 8:
Revised: For larger datasets one should use approximation schemes such as hierarchical approaches (Xu et al., 2019) or Nystr¨om compression of the kernel (Altschuler et al., 2018).

Original: Rescale γ ← (cid:112) m

Original: ( π ) /m

Original: ( γ ) γ 9: Return ( π, γ ) .

Original: The resulting alternate minimization method is detailed in Algorithm 1, see Appendix D for a discretized version.
Revised: The resulting alternate minimization method is detailed in Algorithm 1, see Appendix D for a discretized version.

Original: It uses the unbalanced Sinkhorn algorithm of Chizat et al.
Revised: It uses the unbalanced Sinkhorn algorithm of Chizat et al.

Original: (2018a)
Revised: (2018a)

Original: ; S´ejourn´e et al.
Revised: ; S´ejourn´e et al.

Original: (2019) as sub-iterations
Revised: (2019) as sub-iterations

Original: and is initialized using π = µ ⊗ ν/ (cid:112) m ( µ ) m ( ν ) .
Revised: and is initialized using π = µ ⊗ ν/ (cid:112) m ( µ ) m ( ν ) .

Original: This Sinkhorn algorithm operates over a pair of continuous functions (so-called Kantorovitch potentials)
Revised: This Sinkhorn algorithm operates over a pair of continuous functions (so-called Kantorovitch potentials)

Original: f ( x ) and g ( y ) .
Revised: f ( x ) and g ( y ) .

Original: For discrete spaces X and Y of size n , these functions are stored in vectors of size n , and that integral involved in the updates becomes a sum.
Revised: For discrete spaces X and Y of size n , these functions are stored in vectors of size n , and that integral [Table] [Table] [Table]

Original: Each iteration of Sinkhorn thus has a cost n 2 , and all the involved operation can be efﬁciently mapped to parallelizable GPU routines as detailed in Chizat et al.
Revised: [Table] [Table] [Table] [Table]

Original: (2018a)
Revised: [Table] [Table] [Table] [Table]

Revised: involved in the updates becomes a sum.

Original: ; S´ejourn´e et al. 
Revised: Each iteration of Sinkhorn thus has a cost n 2 , and all the involved operation can be efﬁciently mapped to parallelizable GPU routines as detailed in Chizat et al. (2018a); S´ejourn´e et al. Another advantage of using an unbalanced Sinkhorn algorithm is its complexity O ( n 2 /ε ) to compute an ε -approximation, as stated in Pham et al. (2020), which should be compared to O ( n 2 /ε 2 ) operations for balanced Sinkhorn.

Original: Note also that balanced GW is recovered as a special case when setting ρ → + ∞ , so that ˜ ρ/ (˜ ε + ˜ ρ ) → 1 should be used in the iterations.
Revised: Note also that balanced GW is recovered as a special case when setting ρ → + ∞ , so that ˜ ρ/ (˜ ε + ˜ ρ ) → 1 should be used in the iterations.

Original: In order to speed up Sinkhorn inner-loops, especially for small values of ε , one can use linear extrapolation (Thibault et al., 2017) or non-linear Anderson acceleration (Scieur et al., 2016).
Revised: In order to speed up Sinkhorn inner-loops, especially for small values of ε , one can use linear extrapolation (Thibault et al., 2017) or non-linear Anderson acceleration (Scieur et al., 2016).

Original: There is an extra scaling step after computing γ involving the mass m ( π ) .
Revised: There is an extra scaling step after computing γ involving the mass m ( π ) .

Original: It corresponds to the scaling s of π ⊗ γ such that m ( π ) = m ( γ ) , and we observe that this scaling is key not only to impose this mass equality but also to stabilize the algorithm.
Revised: It corresponds to the scaling s of π ⊗ γ such that m ( π ) = m ( γ ) , and we observe that this scaling is key not only to impose this mass equality but also to stabilize the algorithm.

Original: Otherwise we observed that m ( γ ) < 1 < m ( π ) and underﬂows whenever m ( γ ) → 0 and m ( π ) → ∞ .
Revised: Otherwise we observed that m ( γ ) < 1 < m ( π ) and underﬂows whenever m ( γ ) → 0 and m ( π ) → ∞ .

Original:  N UMERICAL EXPERIMENTS
Revised:  N UMERICAL EXPERIMENTS

Original: This section presents numerical simulations on synthetic examples, to highlight the qualitative behavior of UGW with respect to mass variation and outliers.
Revised: This section presents numerical simulations on synthetic examples, to highlight the qualitative behavior of UGW with respect to mass variation and outliers.

Original: In all these experiments, µ and ν are probability distributions, which allows us to compare GW with UGW.
Revised: In all these experiments, µ and ν are probability distributions, which allows us to compare GW with UGW.

Original: Robustness to imbalanced classes.
Revised: Robustness to imbalanced classes.

Original: In this ﬁrst example, we take X =
Revised: In this ﬁrst example, we take X =

Original: Y =
Revised: Y =

Original: R 2 and consider
Revised: R 2 and consider

Original: E , C and S to be uniform distributions on an ellipse, a disk and a square.
Revised: E , C and S to be uniform distributions on an ellipse, a disk and a square.

Original: Figure 1 contrasts the transportation plan obtained by GW and UGW for a ﬁxed µ = 0 .
Revised: Figure 1 contrasts the transportation plan obtained by GW and UGW for a ﬁxed µ = 0 .

Original: 5 E + 0 .
Revised: 5 E + 0 .

Original: 5 C and ν obtained using two different mixtures of E and S .
Revised: 5 C and ν obtained using two different mixtures of E and S .

Original: The black segments show the largest entries of the transportation matrix π , for a sub-sampled set of points (to ease visibility), thus effectively displaying the matching induced by the plan.
Revised: The black segments show the largest entries of the transportation matrix π , for a sub-sampled set of points (to ease visibility), thus effectively displaying the matching induced by the plan.

Original: Furthermore, the width of the dots are scaled according to the mass of the marginals π 1 ≈ µ and π 2 ≈ ν , i.e. the smaller the point, the smaller is the amount of transported mass.
Revised: Furthermore, the width of the dots are scaled according to the mass of the marginals π 1 ≈ µ and π 2 ≈ ν , i.e. the smaller the point, the smaller is the amount of transported mass.

Original: This ﬁgure shows that the exact conservation of mass imposed by GW leads to a poor geometrical matching of the shapes which have different global mass.
Revised: This ﬁgure shows that the exact conservation of mass imposed by GW leads to a poor geometrical matching of the shapes which have different global mass.

Original: As this should be expected, UGW recovers coherent matchings.
Revised: As this should be expected, UGW recovers coherent matchings.

Original: We suspect the alternate minimization algorithm was able to ﬁnd the global minimum in these cases.[Figure]
Revised: We suspect the alternate minimization algorithm was able to ﬁnd the global minimum in these cases.

Original: Inﬂuence of ε and debiasing.
Revised: Inﬂuence of ε and debiasing.

Original: This ﬁgure (and the following ones) does not show the inﬂuence of ε (which is set of a low value ε = 10 − 2 on a domain [0 , 1] 2 ).
Revised: This ﬁgure (and the following ones) does not show the inﬂuence of ε . This parameter is set of a low value ε = 10 − 2 on a domain [0 , 1] 2 so as to approximate the optimal plan of the unregularized UGW problem .

Original: This inﬂuence is similar to those of classical OT, namely that it introduces an extra diffusion bias.
Revised: The impact of ε is similar to those of classical OT, namely that it introduces an extra diffusion bias.

Original: This bias can be corrected by computing a debiased cost

Original: [Equation]

Original: While this debiasing is shown to lead to a valid divergence for W in Feydy et al. (2019b) and UW in S´ejourn´e et al.

Original: (2019), we leave its study for UGW for future works.

Original: Robustness to outlier Figure 2 shows another experiment on a 2-D dataset, using the same display convention as Figure 1.
Revised: Robustness to outlier Figure 2 shows another experiment on a 2-D dataset, using the same display convention as Figure 1.

Original: It corresponds to the two moons dataset with additional outliers (displayed  in cyan).
Revised: It corresponds to the two moons dataset with additional outliers (displayed [Figure] [Figure] in cyan).

Original: Decreasing the value of ρ (thus allowing for more mass creation/destruction in place of transportation) is able to reduce and even remove the inﬂuence of the outliers, as expected.
Revised: Decreasing the value of ρ (thus allowing for more mass creation/destruction in place of transportation) is able to reduce and even remove the inﬂuence of the outliers, as expected.

Original: Furthermore, using small values of ρ tends to favor “local structures”, which is a behavior quite different from UW (1).
Revised: Furthermore, using small values of ρ tends to favor “local structures”, which is a behavior quite different from UW (1).

Original: Indeed, for UW, ρ → 0 sets to zero all the mass of π outside of the diagonal (points are not transported), while for UGW, it is rather pairs of points with dissimilar pairwise distances which cannot be transported together.[Figure] S o u r ce X
Revised: Indeed, for UW, ρ → 0 sets to zero all the mass of π outside of the diagonal (points are not transported), while for UGW, it is rather pairs of points with dissimilar pairwise distances which cannot be transported together.

Original: [Figure]

Original: T a r g e t Y [Figure] [Figure] Graph matching.
Revised: Graph matching and comparison with Partial-GW.

Original: We now consider two graphs ( X, Y ) equipped with their respective geodesic distances.
Revised: We now consider two graphs ( X, Y ) equipped with their respective geodesic distances.

Original: These graphs correspond to points embedded in R 2 , and the length of the edges corresponds to their Euclidean length.
Revised: These graphs correspond to points embedded in R 2 , and the length of the edges corresponds to their Euclidean length.

Original: These two synthetic graphs are close to be isometric, but differ by addition or modiﬁcation of small sub-structures.
Revised: These two synthetic graphs are close to be isometric, but differ by addition or modiﬁcation of small sub-structures.

Original: The colors c ( x ) are deﬁned on the “source” graph X and are mapped by an optimal plan π on y ∈ Y to a color 1 π 1 ( y ) (cid:82) X c ( x )d π ( x, y ) .
Revised: The colors c ( x ) are deﬁned on the “source” graph X and are mapped by an optimal plan π on y ∈ Y to a color 1 π 1 ( y ) (cid:82) X c ( x )d π ( x, y ) .

Original: This allows to visualize the matching induced by GW and UGW for a varying ρ , as displayed in Figure 3.
Revised: This allows to visualize the matching induced by GW and UGW for a varying ρ , as displayed in Figure 3.

Revised: The graphs for GW should be taken as reference since there is no mass creation.

Revised: The POT library (Flamary & Courty, 2017) is used to compute GW.

Original: For large values of ρ , UGW behaves similarly to GW, thus producing irregular matchings which do not preserve the overall geometry of the shapes.
Revised: For large values of ρ , UGW behaves similarly to GW, thus producing irregular matchings which do not preserve the overall geometry of the shapes.

Original: In sharp contrast, for smaller values of ρ (e.g. ρ = 10 − 1 ), some ﬁne scale structures (such as the target’s small circle) are discarded, and UGW is able to produce a meaningful partial matching of the graphs.
Revised: In sharp contrast, for smaller values of ρ (e.g. ρ = 10 − 1 ), some ﬁne scale structures (such as the target’s small circle) are discarded, and UGW is able to produce a meaningful partial matching of the graphs.

Original: For intermediate values ( ρ = 10 0 ), we observe that the two branches and the blue cluster of the source are correctly matched to the target, while for GW the blue points are scattered because of the marginal constraint.
Revised: For intermediate values ( ρ = 10 0 ), we observe that the two branches and the blue cluster of the source are correctly matched to the target, while for GW the blue points are scattered because of the marginal constraint.

Revised: Figure 4 shows a comparison with Partial-GW (Chapel et al., 2020), computed using the POT library.

Revised: It is close to UGW with a TV ⊗ penalty, since partial OT is equivalent to the use of a TV relaxation of the marginal.

Original: C ONCLUSION
Revised: UGW with a KL ⊗ penalty is ﬁrst computed for a given ρ , then the total mass m of the optimal plan is computed, and is used as a parameter for PGW which imposes this total mass as a constraint. Figure 3 and 4 display the transportation strategy associated to both methods. KLUGW operates smooth transitions between transportation and creation of mass, while PGW either performs pure transportation or pure destruction/creation of mass. This can be observed in Figurewhere nodes of the graphs are removed and not taken into account by the matching.

Revised: Note also that since PGW is equivalent to solving GW on sub-graphs, the color distribution of GW and PGW are the same.

Revised: S ou r ce

Revised: X 0.

Revised: T a r g e t Y ρ = 0 .ρ

Revised: =ρ =

Revised: GW ( ρ = ∞ )

Revised: S ou r ce

Revised: X 0.

Revised: T

Revised: [Figure]

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised:  C ONCLUSION AND PERSPECTIVES

Original: This paper deﬁnes two Unbalanced Gromov-Wasserstein formulations.
Revised: This paper deﬁnes two Unbalanced Gromov-Wasserstein formulations.

Original: We prove that they are both positive and deﬁnite.
Revised: We prove that they are both positive and deﬁnite.

Original: We provide a scalable, GPU-friendly algorithm to compute one of them, and show that the other is a distance between mm-spaces up to isometry.
Revised: We provide a scalable, GPU-friendly algorithm to compute one of them, and show that the other is a distance between mm-spaces up to isometry.

Original: These divergences and distances allow for the ﬁrst time to blend in a seamless way the transportation geometry of GW with creation and destruction of mass.
Revised: These divergences and distances allow for the ﬁrst time to blend in a seamless way the transportation geometry of GW with creation and destruction of mass.

Original: This hybridization is the key to unlock both theoretical and practical issues.
Revised: This hybridization is the key to unlock both theoretical and practical issues.

Original: It raises new questions on the metric properties of those divergences and distances, as well as the geodesic structure that it might induce.
Revised: This work opens new questions for futures works. On the theoretical side, the geodesic structures induced by unbalanced GW distances and divergences is an important subject of study.

Revised: On the practical side, removing the bias introduced by the use of entropic regularization is important for applications to ML.

Revised: Note that such a debiasing was successfully applied for Balanced-GW in Bunne et al.

Revised: and is shown to lead to a valid divergence for balanced OT in Feydy et al.

Original: It also offers new perspectives on the extension of GW methods to ML applications.
Revised: (2019b) and UW in S´ejourn´e et al. The design of efﬁcient numerical solvers for the the conic formulation is also an interesting avenue for future works.

Original: A B ACKGROUND ON UNBALANCED OPTIMAL TRANSPORT
Revised: A B ACKGROUND ON UNBALANCED OPTIMAL TRANSPORT

Original: Following Liero et al.
Revised: Following Liero et al.

Original: (2015), this section reviews and generalizes the homogeneous and conic formulations of unbalanced optimal transport.
Revised: (2015), this section reviews and generalizes the homogeneous and conic formulations of unbalanced optimal transport.

Original: These three formulations are equal in the convex setting of UOT.
Revised: These three formulations are equal in the convex setting of UOT.

Original: Our relaxed divergence UGW and conic distance CGW deﬁned in Section 2 build upon those constructions but are not anymore equal due to the non-convexity of GW problems.
Revised: Our relaxed divergence UGW and conic distance CGW deﬁned in Section 2 build upon those constructions but are not anymore equal due to the non-convexity of GW problems.

Original: A. H OMOGENEOUS FORMULATION
Revised: A. H OMOGENEOUS FORMULATION

Original: To ease the description of the homogeneous formulation, we develop and refactor the Csisz`ar divergence terms of (1) in a form analog to Lemma 1.
Revised: To ease the description of the homogeneous formulation, we develop and refactor the Csisz`ar divergence terms of (1) in a form analog to Lemma 1.

Original: It reads
Revised: It reads

Original: [Equation]
Revised: [Equation]

Original: where L c ( r, s ) def.
Revised: where L c ( r, s ) def.

Original: =
Revised: =

Original: c + rϕ (1 /r )
Revised: c + rϕ (1 /r )

Original: + sϕ (1 /s ) , | µ ⊥ | def.
Revised: + sϕ (1 /s ) , | µ ⊥ | def.

Original: = µ ⊥ ( X ) and ( f def.
Revised: = µ ⊥ ( X ) and ( f def.

Original: = d µ d π 1 , g def.
Revised: = d µ d π 1 , g def.

Original: =
Revised: =

Original: d ν d π 2 ) are the densities of the Lebesgue decomposition of ( µ, ν ) with respect to ( π 1 , π 2 ) and µ = fπ 1 + µ ⊥ and ν = gπ 2 + ν ⊥ .
Revised: d ν d π 2 ) are the densities of the Lebesgue decomposition of ( µ, ν ) with respect to ( π 1 , π 2 ) and µ = fπ 1 + µ ⊥ and ν = gπ 2 + ν ⊥ .

Original: Such form is helpful to explicit the terms of pure mass creation/destruction ( | µ ⊥ | + | ν ⊥ | ) and reinterpret the integral under π as a transport term with a new cost L λ ( d ) .
Revised: Such form is helpful to explicit the terms of pure mass creation/destruction ( | µ ⊥ | + | ν ⊥ | ) and reinterpret the integral under π as a transport term with a new cost L λ ( d ) .

Original: Then the authors of Liero et al.
Revised: Then the authors of Liero et al.

Original: (2015) deﬁne the homogeneous formulations HUW as
Revised: (2015) deﬁne the homogeneous formulations HUW as

Original: [Equation]
Revised: [Equation]

Original: where the 1-homogeneous function H c is the perspective transform of L c
Revised: where the 1-homogeneous function H c is the perspective transform of L c

Original: [Equation]
Revised: [Equation]

Original: By deﬁnition one has L c ≥ H c , though after optimization one has UW = HUW.
Revised: By deﬁnition one has L c ≥ H c , though after optimization one has UW = HUW.

Original: A. C ONE SETS , CONE DISTANCES AND EXPLICIT SETTINGS
Revised: A. C ONE SETS , CONE DISTANCES AND EXPLICIT SETTINGS

Original: The conic formulation detailed in Section A.3 is obtained by performing the optimal transport on the cone set C [ X ] def.
Revised: The conic formulation detailed in Section A.3 is obtained by performing the optimal transport on the cone set C [ X ] def.

Original: =
Revised: =

Original: X × R + / ( X × { 0 } ) , where the extra coordinate accounts for the mass of the particle.
Revised: X × R + / ( X × { 0 } ) , where the extra coordinate accounts for the mass of the particle.

Original: Coordinates of the form ( x, 0) are merged into a single point called the apex of the cone, noted 0 X .
Revised: Coordinates of the form ( x, 0) are merged into a single point called the apex of the cone, noted 0 X .

Original: In the sequel, points of X × R + are noted ( x, r ) and those of C
Revised: In the sequel, points of X × R + are noted ( x, r ) and those of C

Original: [ X ] are noted [ x, r ] to emphasize the quotient operation at the apex.
Revised: [ X ] are noted [ x, r ] to emphasize the quotient operation at the apex.

Original: For a pair ( p, q ) ∈ R + , we deﬁne for any [ x, r ] , [ y, s ] ∈ C
Revised: For a pair ( p, q ) ∈ R + , we deﬁne for any [ x, r ] , [ y, s ] ∈ C

Original: [ X ]
Revised: [ X ]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: In general D C [ X ] is not a distance, but it is always deﬁnite as proved by the following result.
Revised: In general D C [ X ] is not a distance, but it is always deﬁnite as proved by the following result.

Original: Proposition 5.
Revised: Proposition 5.

Original: Assume that d is deﬁnite, λ − 1 ( { 0 } ) =
Revised: Assume that d is deﬁnite, λ − 1 ( { 0 } ) =

Original: { 0 } and ϕ − 1 ( { 0 } ) = { 1 } .
Revised: { 0 } and ϕ − 1 ( { 0 } ) = { 1 } .

Original: Assume also that for any ( r, s ) , there always exists θ ∗ such that H c ( r, s )
Revised: Assume also that for any ( r, s ) , there always exists θ ∗ such that H c ( r, s )

Original: = θ ∗ L c ( rθ ∗ , sθ ∗ ) .
Revised: = θ ∗ L c ( rθ ∗ , sθ ∗ ) .

Original: Then D C
Revised: Then D C

Original: [ X ] is deﬁnite on
Revised: [ X ] is deﬁnite on

Original: C [ X ] , i.e. D C
Revised: C [ X ] , i.e. D C

Original: [ X ] ([ x, r ] , [ y, s ])
Revised: [ X ] ([ x, r ] , [ y, s ])

Original: = 

Revised: =

Original: if and only if ( r = s = 0) or ( r = s and x = y ) .
Revised: if and only if ( r = s = 0) or ( r = s and x = y ) .

Original: Proof.
Revised: Proof.

Original: Assume D C
Revised: Assume D C

Original: [ X ] ([ x, r ] , [ y, s ]) = 0 , and write θ ∗ such that
Revised: [ X ] ([ x, r ] , [ y, s ]) = 0 , and write θ ∗ such that

Original: [Equation]
Revised: [Equation]

Original: where the last line is given by the deﬁnition of reverse entropy.
Revised: where the last line is given by the deﬁnition of reverse entropy.

Original: There are two cases.
Revised: There are two cases.

Original: If θ ∗ > 0 , since all terms are positive, there are all equal to 0 .
Revised: If θ ∗ > 0 , since all terms are positive, there are all equal to 0 .

Original: By deﬁniteness of d it yields x = y and because ϕ − 1 ( { 0 } ) = { 1 } we have r p =
Revised: By deﬁniteness of d it yields x = y and because ϕ − 1 ( { 0 } ) = { 1 } we have r p =

Original: s p
Revised: s p

Original: = θ ∗ and r = s .
Revised: = θ ∗ and r = s .

Original: If θ ∗ = 0 then D C [ X ] ([ x, r ] , [ y, s ]) q = ϕ (0)( r p + s p ) .
Revised: If θ ∗ = 0 then D C [ X ] ([ x, r ] , [ y, s ]) q = ϕ (0)( r p + s p ) .

Original: The assumption ϕ − 1 ( { 0 } ) = { 1 } implies ϕ (0) > 0 , thus necessarily r =
Revised: The assumption ϕ − 1 ( { 0 } ) = { 1 } implies ϕ (0) > 0 , thus necessarily r =

Original: s = 0 .
Revised: s = 0 .

Original: The function H c can be computed in closed form for a certain number of common entropies ϕ , and we refer to Liero et al.
Revised: The function H c can be computed in closed form for a certain number of common entropies ϕ , and we refer to Liero et al.

Original: (2015, Section 5) for an overview.
Revised: (2015, Section 5) for an overview.

Original: Of particular interest are those ϕ where D C
Revised: Of particular interest are those ϕ where D C

Original: [ X ] is a distance, which necessitates a careful choice of λ, p and q .
Revised: [ X ] is a distance, which necessitates a careful choice of λ, p and q .

Original: We now detail three particular settings where this is the case.
Revised: We now detail three particular settings where this is the case.

Original: In each setting we provide ( D ϕ , λ, p, q ) and its associated cone distance D C
Revised: In each setting we provide ( D ϕ , λ, p, q ) and its associated cone distance D C

Original: [ X ] .
Revised: [ X ] .

Original: Gaussian Hellinger distance It corresponds to
Revised: Gaussian Hellinger distance It corresponds to

Original: [Equation]
Revised: [Equation]

Original: in which case it is proved in Liero et al.
Revised: in which case it is proved in Liero et al.

Original: that D C
Revised: that D C

Original: [ X ] is a cone distance.
Revised: [ X ] is a cone distance.

Original: Hellinger-Kantorovich / Wasserstein-Fisher-Rao distance It reads
Revised: Hellinger-Kantorovich / Wasserstein-Fisher-Rao distance It reads

Original: [Equation]
Revised: [Equation]

Original: in which case it is proved in Burago et al.
Revised: in which case it is proved in Burago et al.

Original: that D C
Revised: that D C

Original: [ X ] is a cone distance.
Revised: [ X ] is a cone distance.

Original: The weight λ ( t ) = − log cos 2 ( t ∧ π 2 ) , which might seem more peculiar, is in fact the penalty that makes unbalanced OT a length space induced by the Gaussian-Hellinger distance (if the ground metric d is itself geodesic), as proved in Liero et al.
Revised: The weight λ ( t ) = − log cos 2 ( t ∧ π 2 ) , which might seem more peculiar, is in fact the penalty that makes unbalanced OT a length space induced by the Gaussian-Hellinger distance (if the ground metric d is itself geodesic), as proved in Liero et al.

Original: ; Chizat et al.
Revised: ; Chizat et al.

Original: (2018b).
Revised: (2018b).

Original: This weight introduces a cut-off, because λ ( d ( x, y )) =
Revised: This weight introduces a cut-off, because λ ( d ( x, y )) =

Original: + ∞ if d ( x, y ) > π/ 2 .
Revised: + ∞ if d ( x, y ) > π/ 2 .

Original: There is no transport between points too far from each other.
Revised: There is no transport between points too far from each other.

Original: The choice of π/ 2 is arbitrary, and can be modiﬁed by scaling λ (cid:55)→ λ ( · /s ) for some cutoff s .
Revised: The choice of π/ 2 is arbitrary, and can be modiﬁed by scaling λ (cid:55)→ λ ( · /s ) for some cutoff s .

Original: Partial optimal transport It corresponds to
Revised: Partial optimal transport It corresponds to

Original: [Equation]
Revised: [Equation]

Original: in which case it is proved in Chizat et al.
Revised: in which case it is proved in Chizat et al.

Original: (2018c)
Revised: (2018c)

Original: that D C
Revised: that D C

Original: [ X ] is a cone distance.
Revised: [ X ] is a cone distance.

Original: The case D ϕ =
Revised: The case D ϕ =

Original: TV is equivalent to partial unbalanced OT, which produces discontinuities (because of the non-smoothness of the divergence) between regions of the supports which are being transported and regions where mass is being destroyed/created.
Revised: TV is equivalent to partial unbalanced OT, which produces discontinuities (because of the non-smoothness of the divergence) between regions of the supports which are being transported and regions where mass is being destroyed/created.

Original: Note that Liero et al.
Revised: Note that Liero et al.

Original: (2015) do not mention that this D C [ X ] deﬁnes a distance, so this result is new to the best of our knowledge, although it can be proved without a conic lifting that partial OT deﬁnes a distance as explained in Chizat et al.
Revised: (2015) do not mention that this D C [ X ] deﬁnes a distance, so this result is new to the best of our knowledge, although it can be proved without a conic lifting that partial OT deﬁnes a distance as explained in Chizat et al.

Original: (2018c).
Revised: (2018c).

Original: A. C ONIC FORMULATION OF UW
Revised: A. C ONIC FORMULATION OF UW

Original: The last formulation reinterprets UW as an OT problem on the cone, with the addition of two linear constraints.
Revised: The last formulation reinterprets UW as an OT problem on the cone, with the addition of two linear constraints.

Original: Informally speaking, H c becomes D C
Revised: Informally speaking, H c becomes D C

Original: [ X ] , the term ( | µ ⊥ | + | ν ⊥ | ) is taken into account by the constraints (13) below, and the variables ( f, g ) are replaced by ( r p , s p ) .
Revised: [ X ] , the term ( | µ ⊥ | + | ν ⊥ | ) is taken into account by the constraints (13) below, and the variables ( f, g ) are replaced by ( r p , s p ) .

Original: It reads
Revised: It reads

Original: [Equation]
Revised: [Equation]

Original: where the constraint set U p ( µ, ν ) is deﬁned as
Revised: where the constraint set U p ( µ, ν ) is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: Thus CUW consists in minimizing the Wasserstein distance W D C [ X ] ( α 1 , α 2 ) on the cone ( C [ X ] , D C
Revised: Thus CUW consists in minimizing the Wasserstein distance W D C [ X ] ( α 1 , α 2 ) on the cone ( C [ X ] , D C

Original: [ X ] ) .
Revised: [ X ] ) .

Original: The additional constraints on ( α 1 , α 2 ) mean that the lift of the mass on the cone must be consistent with the total mass of ( µ, ν ) .
Revised: The additional constraints on ( α 1 , α 2 ) mean that the lift of the mass on the cone must be consistent with the total mass of ( µ, ν ) .

Original: When D C [ X ] is a distance, CUW inherits the metric properties of W D C
Revised: When D C [ X ] is a distance, CUW inherits the metric properties of W D C

Original: [ X ] .
Revised: [ X ] .

Original: Our theoretical results rely on an analog construction for GW.
Revised: Our theoretical results rely on an analog construction for GW.

Original: The following proposition states the equality of the three formulations and recapitulates its main properties.
Revised: The following proposition states the equality of the three formulations and recapitulates its main properties.

Original: The proofs are detailed in Liero et al.
Revised: The proofs are detailed in Liero et al.

Original: Proposition 6 (From Liero et al.
Revised: Proposition 6 (From Liero et al.

Original: One has UW = HUW = CUW , which are symmetric, positive and deﬁnite.
Revised: One has UW = HUW = CUW , which are symmetric, positive and deﬁnite.

Original: Furthermore, if ( X, d X ) and ( C [ X ] , D C [ X ] ) are metric spaces with X separable, then M + ( X ) endowed with CUW is a metric space.
Revised: Furthermore, if ( X, d X ) and ( C [ X ] , D C [ X ] ) are metric spaces with X separable, then M + ( X ) endowed with CUW is a metric space.

Original: Proof.
Revised: Proof.

Original: The equality UW = HUW is given by Liero et al.
Revised: The equality UW = HUW is given by Liero et al.

Original: (2015, Theorem 5.8), while the equality
Revised: (2015, Theorem 5.8), while the equality

Original: HUW = CUW holds thanks to Liero et al.
Revised: HUW = CUW holds thanks to Liero et al.

Original: (2015, Theorem 6.7 and Remark 7.5), where the latter theorem can be straightforwardly generalized to any cone distance built as in Section 2.2.1.
Revised: (2015, Theorem 6.7 and Remark 7.5), where the latter theorem can be straightforwardly generalized to any cone distance built as in Section 2.2.1.

Original: Since
Revised: Since

Original: D C
Revised: D C

Original: [ X ] is symmetric, positive and deﬁnite (see Proposition 3), then so is CUW.
Revised: [ X ] is symmetric, positive and deﬁnite (see Proposition 3), then so is CUW.

Original: Furthermore, if
Revised: Furthermore, if

Original: D C
Revised: D C

Original: [ X ] satisﬁes the triangle inequality, separability of X allows to apply the gluing lemma (Liero et al., 2015, Corollary 7.14) which generalizes to any exponent p deﬁning U p ( µ, ν ) and any cone distance D C
Revised: [ X ] satisﬁes the triangle inequality, separability of X allows to apply the gluing lemma (Liero et al., 2015, Corollary 7.14) which generalizes to any exponent p deﬁning U p ( µ, ν ) and any cone distance D C

Original: [ X ] .
Revised: [ X ] .

Original: B UGW FORMULATION AND DEFINITENESS
Revised: B UGW FORMULATION AND DEFINITENESS

Original: We present in this section the proofs of the properties of our divergence UGW.
Revised: We present in this section the proofs of the properties of our divergence UGW.

Original: We refer to Section 2 for the deﬁnition of the UGW formulation and its related concepts.
Revised: We refer to Section 2 for the deﬁnition of the UGW formulation and its related concepts.

Original: For conciseness we write
Revised: For conciseness we write

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst start with the existence of minimizers stated in Proposition 1.
Revised: We ﬁrst start with the existence of minimizers stated in Proposition 1.

Original: It illustrates in some sense that our divergence is well-deﬁned.
Revised: It illustrates in some sense that our divergence is well-deﬁned.

Original: Proposition 7 (Existence of minimizers) .
Revised: Proposition 7 (Existence of minimizers) .

Original: Assume ( X , Y ) to be compact mm-spaces and that we either have
Revised: Assume ( X , Y ) to be compact mm-spaces and that we either have

Original: ϕ superlinear, i.e ϕ (cid:48) ∞
Revised: ϕ superlinear, i.e ϕ (cid:48) ∞

Original: = ∞ 2.
Revised: = ∞ 2.

Original: λ has compact sublevel sets in R + and 2 ϕ
Revised: λ has compact sublevel sets in R + and 2 ϕ

Original: (cid:48)∞ + inf λ >
Revised: (cid:48)∞ + inf λ >

Original: Then there exists π ∈ M + ( X × Y ) such that UGW ( X , Y )
Revised: Then there exists π ∈ M + ( X × Y ) such that UGW ( X , Y )

Original: =
Revised: =

Original: L ( π ) .
Revised: L ( π ) .

Original: Proof.
Revised: Proof.

Original: We adapt here from Liero et al.
Revised: We adapt here from Liero et al.

Original: (2015, Theorem 3.3).
Revised: (2015, Theorem 3.3).

Original: The functional is lower semicontinuous as a sum of l.s.c terms.
Revised: The functional is lower semicontinuous as a sum of l.s.c terms.

Original: Thus it sufﬁces to have relative compactness of the set of minimizers.
Revised: Thus it sufﬁces to have relative compactness of the set of minimizers.

Original: Under either one of the assumptions, coercivity of the functional holds thanks to Jensen’s inequality
Revised: Under either one of the assumptions, coercivity of the functional holds thanks to Jensen’s inequality

Original: [Equation]
Revised: [Equation]

Original: As m ( π ) → + ∞ the right hand side converges to 2 ϕ (cid:48)∞ + inf λ > 0 , which under either one of the assumptions yields L ( π ) → + ∞ , hence the coercivity.
Revised: As m ( π ) → + ∞ the right hand side converges to 2 ϕ (cid:48)∞ + inf λ > 0 , which under either one of the assumptions yields L ( π ) → + ∞ , hence the coercivity.

Original: Thus we can assume there exists some M such that m ( π ) <
Revised: Thus we can assume there exists some M such that m ( π ) <

Original: M .
Revised: M .

Original: Since the spaces are assumed to be compact, the Banach-Alaoglu theorem holds and gives relative compactness in M + ( X × Y ) .
Revised: Since the spaces are assumed to be compact, the Banach-Alaoglu theorem holds and gives relative compactness in M + ( X × Y ) .

Original: Take any sequence of plans π n that approaches UGW ( X , Y ) =
Revised: Take any sequence of plans π n that approaches UGW ( X , Y ) =

Original: inf L ( π ) .
Revised: inf L ( π ) .

Original: Compactness gives that a subsequence π n k weak* converges to some π ∗ .
Revised: Compactness gives that a subsequence π n k weak* converges to some π ∗ .

Original: Because L is l.s.c, we have L ( π ∗ ) ≤ inf L ( π ) , thus L ( π ∗ ) =
Revised: Because L is l.s.c, we have L ( π ∗ ) ≤ inf L ( π ) , thus L ( π ∗ ) =

Original: inf L ( π ) .
Revised: inf L ( π ) .

Original: The existence of such limit reaching the inﬁmum gives the existence of a minimizer.
Revised: The existence of such limit reaching the inﬁmum gives the existence of a minimizer.

Original: Note that this formulation is nonegative and symmetric because the functional L is also nonegative and symmetric in its inputs ( X , Y ) .
Revised: Note that this formulation is nonegative and symmetric because the functional L is also nonegative and symmetric in its inputs ( X , Y ) .

Original: This formulation allows straightforwardly to prove the deﬁniteness of UGW.
Revised: This formulation allows straightforwardly to prove the deﬁniteness of UGW.

Original: Proposition 8 (Deﬁniteness of UGW) .
Revised: Proposition 8 (Deﬁniteness of UGW) .

Original: Assume that ϕ − 1 ( { 0 } )
Revised: Assume that ϕ − 1 ( { 0 } )

Original: =
Revised: =

Original: { 1 } and λ − 1 ( { 0 } ) = { 0 } .
Revised: { 1 } and λ − 1 ( { 0 } ) = { 0 } .

Original: The following assertions are equivalent:
Revised: The following assertions are equivalent:

Original: UGW ( X , Y ) =2.
Revised: UGW ( X , Y ) =2.

Original: ∃ π ∈ M + ( X × Y ) whose marginals are ( µ, ν )
Revised: ∃ π ∈ M + ( X × Y ) whose marginals are ( µ, ν )

Original: such that d X ( x, x (cid:48) )
Revised: such that d X ( x, x (cid:48) )

Original: =
Revised: =

Original: d Y ( y, y (cid:48) ) for π ⊗ πa.e.
Revised: d Y ( y, y (cid:48) ) for π ⊗ πa.e.

Original: ( x, x (cid:48) , y, y (cid:48) ) ∈ ( X × Y ) 2 . 3.
Revised: ( x, x (cid:48) , y, y (cid:48) ) ∈ ( X × Y ) 2 . 3.

Original: There exists a mm-space ( Z, d Z , η ) with full support and Borel maps ψ X : Z → X and ψ Y : Z → Y .
Revised: There exists a mm-space ( Z, d Z , η ) with full support and Borel maps ψ X : Z → X and ψ Y : Z → Y .

Original: such that ( ψ X ) (cid:93) η = µ , ( ψ Y ) (cid:93)
Revised: such that ( ψ X ) (cid:93) η = µ , ( ψ Y ) (cid:93)

Original: η
Revised: η

Original: = ν and d Z = ( ψ X ) (cid:93)
Revised: = ν and d Z = ( ψ X ) (cid:93)

Original: d X = ( ψ Y ) (cid:93)
Revised: d X = ( ψ Y ) (cid:93)

Original: d Y 4.
Revised: d Y 4.

Original: There exists a Borel measurable bijection between the measures’ supports ψ : spt ( µ ) → spt ( ν ) with Borel measurable inverse such that ψ (cid:93) µ = ν and d Y = ψ (cid:93) d X .
Revised: There exists a Borel measurable bijection between the measures’ supports ψ : spt ( µ ) → spt ( ν ) with Borel measurable inverse such that ψ (cid:93) µ = ν and d Y = ψ (cid:93) d X .

Original: Proof.
Revised: Proof.

Original: Recall that (2) ⇔ (3) ⇔ (4) from Sturm (2012, Lemma 1.10).
Revised: Recall that (2) ⇔ (3) ⇔ (4) from Sturm (2012, Lemma 1.10).

Original: thus it remains to prove (1) ⇔ (2) .
Revised: thus it remains to prove (1) ⇔ (2) .

Original: If there is such coupling plan π between ( µ, ν ) then one has π ⊗ π -a.e.
Revised: If there is such coupling plan π between ( µ, ν ) then one has π ⊗ π -a.e.

Original: that Γ = 0 , and all ϕdivergences are zero as well, yielding a distance of zero a.e.
Revised: that Γ = 0 , and all ϕdivergences are zero as well, yielding a distance of zero a.e.

Original: Assume now that UGW ( X , Y ) = 0 , and write π an optimal plan.
Revised: Assume now that UGW ( X , Y ) = 0 , and write π an optimal plan.

Original: All terms of L are positive, thus under our assumptions we have Γ = 0 , π 1 ⊗ π 1 = µ ⊗ µ and π 2 ⊗ π 2 = ν
Revised: All terms of L are positive, thus under our assumptions we have Γ = 0 , π 1 ⊗ π 1 = µ ⊗ µ and π 2 ⊗ π 2 = ν

Original: ⊗ ν .
Revised: ⊗ ν .

Original: Thus we get that π has marginals ( µ, ν ) and that d X ( x, x (cid:48) )
Revised: Thus we get that π has marginals ( µ, ν ) and that d X ( x, x (cid:48) )

Original: =
Revised: =

Original: d Y ( y, y (cid:48) )
Revised: d Y ( y, y (cid:48) )

Original: π ⊗ π -a.e.
Revised: π ⊗ π -a.e.

Original: We end with a result on the reformulation of UGW which is the ﬁrst step to connnect it with the conic formulation CGW.
Revised: We end with a result on the reformulation of UGW which is the ﬁrst step to connnect it with the conic formulation CGW.

Original: Lemma 2.
Revised: Lemma 2.

Original: Deﬁning L c ( r, s ) def.
Revised: Deﬁning L c ( r, s ) def.

Original: =
Revised: =

Original: c + rϕ (1 /r )
Revised: c + rϕ (1 /r )

Original: + sϕ (1 /s ) , and writing ( f def.
Revised: + sϕ (1 /s ) , and writing ( f def.

Original: = d µ d π 1 , g def.
Revised: = d µ d π 1 , g def.

Original: =
Revised: =

Original: d ν d π 2 ) the
Revised: d ν d π 2 ) the

Original: Lebesgue densities of ( µ, ν ) w.r.t.
Revised: Lebesgue densities of ( µ, ν ) w.r.t.

Original: ( π 1 , π 2 ) such that µ = fπ 1 + µ ⊥ and ν = gπ 2 + ν ⊥ , one has
Revised: ( π 1 , π 2 ) such that µ = fπ 1 + µ ⊥ and ν = gπ 2 + ν ⊥ , one has

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Using Equation (23), one has
Revised: Using Equation (23), one has

Original: [Equation]
Revised: [Equation]

Original: C C ONIC FORMULATION AND METRIC PROPERTIES
Revised: C C ONIC FORMULATION AND METRIC PROPERTIES

Original: We present in this section the proofs of the properties mentioned in Section 2.
Revised: We present in this section the proofs of the properties mentioned in Section 2.

Original: We refer to Sectionand Appendix A for the deﬁnition of the conic formulation and its related concepts.
Revised: We refer to Sectionand Appendix A for the deﬁnition of the conic formulation and its related concepts.

Original: In this section we frequently use the notion of marginal for neasures.
Revised: In this section we frequently use the notion of marginal for neasures.

Original: For any sets E, F , we write
Revised: For any sets E, F , we write

Original: P ( E ) : E × F → E the canonical projection such that for any ( x, y ) ∈
Revised: P ( E ) : E × F → E the canonical projection such that for any ( x, y ) ∈

Original: E × F, P ( E ) ( x, y ) = x .
Revised: E × F, P ( E ) ( x, y ) = x .

Original: Consider two complete separable mm-spaces X
Revised: Consider two complete separable mm-spaces X

Original: =
Revised: =

Original: ( X, d X , µ ) and Y =
Revised: ( X, d X , µ ) and Y =

Original: ( Y, d Y , ν ) .
Revised: ( Y, d Y , ν ) .

Original: Write π ∈
Revised: Write π ∈

Original: M + ( X × Y ) a coupling plan, and deﬁne its marginals by π 1 = P ( X ) (cid:93) π and π 2 = P ( Y ) (cid:93) π .
Revised: M + ( X × Y ) a coupling plan, and deﬁne its marginals by π 1 = P ( X ) (cid:93) π and π 2 = P ( Y ) (cid:93) π .

Original: The deﬁnition of the marginals can also be seen by the use of test functions.
Revised: The deﬁnition of the marginals can also be seen by the use of test functions.

Original: In the case of π 1 it reads for any test function ξ
Revised: In the case of π 1 it reads for any test function ξ

Original: [Equation]
Revised: [Equation]

Original: C.
Revised: C.

Original: P RELIMINARY RESULTS
Revised: P RELIMINARY RESULTS

Original: We present in this section concepts and properties which are necessary for the proof of Theorem 1.
Revised: We present in this section concepts and properties which are necessary for the proof of Theorem 1.

Original: We introduce a dilation operator whose role is to rescale the radial coordinate of a measure with a given scaling.
Revised: We introduce a dilation operator whose role is to rescale the radial coordinate of a measure with a given scaling.

Original: Deﬁnition 2 (dilations) .
Revised: Deﬁnition 2 (dilations) .

Original: Consider v ([ x, r ] , [ y, s ]) a Borel measurable scaling function depending on [ x, r ] , [ y, s ] ∈ C
Revised: Consider v ([ x, r ] , [ y, s ]) a Borel measurable scaling function depending on [ x, r ] , [ y, s ] ∈ C

Original: [ X ] × C [ Y ] .
Revised: [ X ] × C [ Y ] .

Original: Take a plan α ∈ M + ( C [ X ] × C
Revised: Take a plan α ∈ M + ( C [ X ] × C

Original: [ Y ]) .
Revised: [ Y ]) .

Original: We deﬁne the dilation
Revised: We deﬁne the dilation

Original: Dil v : α (cid:55)→ ( h v ) (cid:93) ( v p α ) where
Revised: Dil v : α (cid:55)→ ( h v ) (cid:93) ( v p α ) where

Original: [Equation]
Revised: [Equation]

Original: where w = v ([ x, r ] , [ y, s ]) .
Revised: where w = v ([ x, r ] , [ y, s ]) .

Original: It reads for any test function ξ
Revised: It reads for any test function ξ

Original: [Equation]
Revised: [Equation]

Original: The importance of dilations is given by the following lemma.
Revised: The importance of dilations is given by the following lemma.

Original: Lemma 3 (Invariance to dilation) .
Revised: Lemma 3 (Invariance to dilation) .

Original: The problem CGW is invariant to dilations, i.e. for any α ∈
Revised: The problem CGW is invariant to dilations, i.e. for any α ∈

Original: U p ( µ, ν ) , we have Dil v ( α ) ∈
Revised: U p ( µ, ν ) , we have Dil v ( α ) ∈

Original: U p ( µ, ν ) and H ( α ) =
Revised: U p ( µ, ν ) and H ( α ) =

Original: H ( Dil v ( α )) .
Revised: H ( Dil v ( α )) .

Original: Proof.
Revised: Proof.

Original: First we prove the stability of U p ( µ, ν ) under dilations.
Revised: First we prove the stability of U p ( µ, ν ) under dilations.

Original: Take α ∈ U p ( µ, ν ) .
Revised: Take α ∈ U p ( µ, ν ) .

Original: For any test function ξ deﬁned on X we have
Revised: For any test function ξ deﬁned on X we have

Original: [Equation]
Revised: [Equation]

Original: Similarly we get P ( Y ) (cid:93) ( s q Dil v ( α ))
Revised: Similarly we get P ( Y ) (cid:93) ( s q Dil v ( α ))

Original: =
Revised: =

Original: ν , thus Dil v ( α ) ∈
Revised: ν , thus Dil v ( α ) ∈

Original: U p ( µ, ν ) .
Revised: U p ( µ, ν ) .

Original: It remains to prove the invariance of the functional.
Revised: It remains to prove the invariance of the functional.

Original: Recall that D q is p-homogeneous.
Revised: Recall that D q is p-homogeneous.

Original: It yields
Revised: It yields

Original: [Equation]
Revised: [Equation]

Original: Both the functional and the constraint set are invariant, thus the whole CGW problem is invariant to dilations.
Revised: Both the functional and the constraint set are invariant, thus the whole CGW problem is invariant to dilations.

Original: The above lemma allows to normalize the plan such that one of its marginal is ﬁxed to some value.
Revised: The above lemma allows to normalize the plan such that one of its marginal is ﬁxed to some value.

Original: Fixing a marginal allows to generalize the gluing lemma which is a key ingredient of the triangle inequality in optimal transport.
Revised: Fixing a marginal allows to generalize the gluing lemma which is a key ingredient of the triangle inequality in optimal transport.

Original: Lemma 4 (Normalization lemma) .
Revised: Lemma 4 (Normalization lemma) .

Original: Assume there exists α ∈ U p ( µ, ν ) such that CGW ( X , Y ) =
Revised: Assume there exists α ∈ U p ( µ, ν ) such that CGW ( X , Y ) =

Original: H ( α ) .
Revised: H ( α ) .

Original: Then there exists ˜ α
Revised: Then there exists ˜ α

Original: such that ˜ α ∈
Revised: such that ˜ α ∈

Original: U p ( µ, ν ) and CGW ( X , Y ) =
Revised: U p ( µ, ν ) and CGW ( X , Y ) =

Original: H (˜ α ) and whose marginal on C [ Y ] is ν C
Revised: H (˜ α ) and whose marginal on C [ Y ] is ν C

Original: [ Y ] =
Revised: [ Y ] =

Original: P ( C [ Y ]) (cid:93) ˜
Revised: P ( C [ Y ]) (cid:93) ˜

Original: α
Revised: α

Original: = δ 

Revised: = δ

Original: Y + p (cid:93) ( ν ⊗ δ 1 ) , where p is the canonical injection from Y × R + to C
Revised: Y + p (cid:93) ( ν ⊗ δ 1 ) , where p is the canonical injection from Y × R + to C

Original: [ Y ] .
Revised: [ Y ] .

Original: Proof.
Revised: Proof.

Original: The proof is exactly the same as Liero et al.
Revised: The proof is exactly the same as Liero et al.

Original: (2015, Lemma 7.10) and is included for completeness.
Revised: (2015, Lemma 7.10) and is included for completeness.

Original: Take an optimal plan α .
Revised: Take an optimal plan α .

Original: Because the functional and the constraints are homogeneous in ( r, s ) , the plan ˆ α
Revised: Because the functional and the constraints are homogeneous in ( r, s ) , the plan ˆ α

Original: =
Revised: =

Original: α +
Revised: α +

Original: δ 0 X ⊗ δ 

Revised: δ 0 X ⊗ δ

Original: Y veriﬁes ˆ α ∈ U p ( µ, ν ) and H (ˆ α ) =
Revised: Y veriﬁes ˆ α ∈ U p ( µ, ν ) and H (ˆ α ) =

Original: H ( α ) .
Revised: H ( α ) .

Original: Indeed, because of this homogeneity the contribution δ 0 X ⊗ δ 

Revised: Indeed, because of this homogeneity the contribution δ 0 X ⊗ δ

Original: Y has ( r, s ) =
Revised: Y has ( r, s ) =

Original: (0 , 0) which has thus no impact.
Revised: (0 , 0) which has thus no impact.

Original: Considering ˆ α instead of α allows to assume without loss of generality that the transport plan charges the apex, i.e. setting
Revised: Considering ˆ α instead of α allows to assume without loss of generality that the transport plan charges the apex, i.e. setting

Original: [Equation]
Revised: [Equation]

Original: one has ω Y def.
Revised: one has ω Y def.

Original: =
Revised: =

Original: ˆ α ( S ) ≥ 1 .
Revised: ˆ α ( S ) ≥ 1 .

Original: Then we can deﬁne the following scaling
Revised: Then we can deﬁne the following scaling

Original: [Equation]
Revised: [Equation]

Original: We prove now that Dil v (ˆ α ) has the desired marginal on C ( Y ) by considering test functions ξ ([ y, s ]) .
Revised: We prove now that Dil v (ˆ α ) has the desired marginal on C ( Y ) by considering test functions ξ ([ y, s ]) .

Original: We separate the integral into two parts with the set S , and write ˆ α =
Revised: We separate the integral into two parts with the set S , and write ˆ α =

Original: ˆ α | S
Revised: ˆ α | S

Original: + ˆ α | S c their restrictions to S and S c respectively.
Revised: + ˆ α | S c their restrictions to S and S c respectively.

Original: It reads
Revised: It reads

Original: [Equation]
Revised: [Equation]

Original: which is the formula of the desired marginal on C
Revised: which is the formula of the desired marginal on C

Original: [ Y ] .
Revised: [ Y ] .

Original: Since ˆ α ∈ U p ( µ, ν ) , its dilation is also in
Revised: Since ˆ α ∈ U p ( µ, ν ) , its dilation is also in

Original: U p ( µ, ν ) , and H ( α ) =
Revised: U p ( µ, ν ) , and H ( α ) =

Original: H (ˆ α ) =
Revised: H (ˆ α ) =

Original: H ( Dil v (ˆ α )) .
Revised: H ( Dil v (ˆ α )) .

Original: C.1. P ROOF OF T HEOREM
Revised: C.1. P ROOF OF T HEOREM

Original: Non-negativity and symmetry hold since H is a sum of non-negative symmetric terms.
Revised: Non-negativity and symmetry hold since H is a sum of non-negative symmetric terms.

Original: To prove
Revised: To prove

Original: Deﬁniteness , assume CGW ( X , Y )
Revised: Deﬁniteness , assume CGW ( X , Y )

Original: = 0 , and write α an optimal plan.
Revised: = 0 , and write α an optimal plan.

Original: We have α ⊗ α -a.e.
Revised: We have α ⊗ α -a.e.

Original: that d X ( x, x (cid:48) )
Revised: that d X ( x, x (cid:48) )

Original: =
Revised: =

Original: d Y ( y, y (cid:48) ) and rr (cid:48) = ss (cid:48) because D is deﬁnite (see Proposition 3).
Revised: d Y ( y, y (cid:48) ) and rr (cid:48) = ss (cid:48) because D is deﬁnite (see Proposition 3).

Original: Thanks to the completeness of ( X , Y ) and a result from Sturm (2012, Lemma 1.10), such property implies the existence of a Borel isometric bijection with Borel inverse between the supports of the measures ψ : Supp( µ ) →
Revised: Thanks to the completeness of ( X , Y ) and a result from Sturm (2012, Lemma 1.10), such property implies the existence of a Borel isometric bijection with Borel inverse between the supports of the measures ψ : Supp( µ ) →

Original: Supp( ν ) , where Supp denotes the support.
Revised: Supp( ν ) , where Supp denotes the support.

Original: The bijection ψ veriﬁes d X ( x, x (cid:48) )
Revised: The bijection ψ veriﬁes d X ( x, x (cid:48) )

Original: =
Revised: =

Original: d Y ( ψ ( x ) , ψ ( x (cid:48) )) .
Revised: d Y ( ψ ( x ) , ψ ( x (cid:48) )) .

Original: To prove X ∼ Y it remains to prove ψ (cid:93) µ = ν .
Revised: To prove X ∼ Y it remains to prove ψ (cid:93) µ = ν .

Original: Due to the density of continuous functions of the form ξ ( x ) ξ ( x (cid:48) ) , the constraints of U p ( µ, ν ) are equivalent to (cid:90)
Revised: Due to the density of continuous functions of the form ξ ( x ) ξ ( x (cid:48) ) , the constraints of U p ( µ, ν ) are equivalent to (cid:90)

Original: [Equation]
Revised: [Equation]

Original: Take a continuous test function ξ deﬁned on Supp( ν ) 2 .
Revised: Take a continuous test function ξ deﬁned on Supp( ν ) 2 .

Original: Writing y =
Revised: Writing y =

Original: ψ ( x ) and y (cid:48)
Revised: ψ ( x ) and y (cid:48)

Original: =
Revised: =

Original: ψ ( x (cid:48) ) , one has
Revised: ψ ( x (cid:48) ) , one has

Original: [Equation]
Revised: [Equation]

Original: Since ψ is a bijection, there is a bijection between continuous functions ξ of Supp( ν ) 2 and functions ˜ ξ of Supp( µ ) 2 .
Revised: Since ψ is a bijection, there is a bijection between continuous functions ξ of Supp( ν ) 2 and functions ˜ ξ of Supp( µ ) 2 .

Original: Thus we obtain ν = ψ (cid:93) µ
Revised: Thus we obtain ν = ψ (cid:93) µ

Original: and we have X ∼ Y .
Revised: and we have X ∼ Y .

Original: It remains to prove the triangle inequality .
Revised: It remains to prove the triangle inequality .

Original: Assume now that D satisﬁes it.
Revised: Assume now that D satisﬁes it.

Original: Given three mm-spaces ( X , Y , Z ) respectively equipped with measures ( µ, ν, η ) , consider α, β which are optimal plans for
Revised: Given three mm-spaces ( X , Y , Z ) respectively equipped with measures ( µ, ν, η ) , consider α, β which are optimal plans for

Original: CGW ( X , Y ) and CGW ( Y , Z ) .
Revised: CGW ( X , Y ) and CGW ( Y , Z ) .

Original: Using Lemma 4 to both α and β , we can consider measures (¯ α, ¯ β ) which are also optimal and have a common marginal ¯ ν on C
Revised: Using Lemma 4 to both α and β , we can consider measures (¯ α, ¯ β ) which are also optimal and have a common marginal ¯ ν on C

Original: [ Y ] .
Revised: [ Y ] .

Original: Thanks to this common marginal and the separability of ( X, Y, Z ) , the standard gluing lemma (Villani, 2003, Lemma 7.6) applies and yields a glued plan γ ∈ M + ( C [ X ] × C
Revised: Thanks to this common marginal and the separability of ( X, Y, Z ) , the standard gluing lemma (Villani, 2003, Lemma 7.6) applies and yields a glued plan γ ∈ M + ( C [ X ] × C

Original: [ Y ] × C
Revised: [ Y ] × C

Original: [ Z ]) whose respective marginals on C [ X ] × C [ Y ] and
Revised: [ Z ]) whose respective marginals on C [ X ] × C [ Y ] and

Original: C [ Y ] × C [ Z ] are (¯ α, ¯ β ) .
Revised: C [ Y ] × C [ Z ] are (¯ α, ¯ β ) .

Original: Furthermore, the marginal ¯ γ of γ on C [ X ] × C [ Z ] is in U p ( µ, η ) .
Revised: Furthermore, the marginal ¯ γ of γ on C [ X ] × C [ Z ] is in U p ( µ, η ) .

Original: Indeed, (¯ γ, ¯ α ) have the same marginal on C [ X ] and same for (¯ γ, ¯ β ) on C [ Z ] , hence this property.
Revised: Indeed, (¯ γ, ¯ α ) have the same marginal on C [ X ] and same for (¯ γ, ¯ β ) on C [ Z ] , hence this property.

Original: Write
Revised: Write

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: Since ¯ γ ∈ U p ( µ, η ) , it is thus suboptimal, which yields Equation (17).
Revised: Since ¯ γ ∈ U p ( µ, η ) , it is thus suboptimal, which yields Equation (17).

Original: Because ¯ γ is the marginal of γ we get Equation (18).
Revised: Because ¯ γ is the marginal of γ we get Equation (18).

Original: Equations (19) and (20) are respectively obtained by the triangle and
Revised: Equations (19) and (20) are respectively obtained by the triangle and

Original: Minkowski inequalities, which hold because D which is a distance.
Revised: Minkowski inequalities, which hold because D which is a distance.

Original: Equation (21) is the marginalization of γ , and Equation (22) is given by the optimality of (¯ α, ¯ β ) , which ends the proof of the triangle inequality.
Revised: Equation (21) is the marginalization of γ , and Equation (22) is given by the optimality of (¯ α, ¯ β ) , which ends the proof of the triangle inequality.

Original: C.1. P ROOF OF THE INEQUALITY BETWEEN UGW AND CGW
Revised: C.1. P ROOF OF THE INEQUALITY BETWEEN UGW AND CGW

Original: The proof consists in considering an optimal plan π for UGW, building a lift α of this plan into the cone such that L ( π ) ≥ H ( α ) , and prove that α is admissible for the program CGW, thus suboptimal.
Revised: The proof consists in considering an optimal plan π for UGW, building a lift α of this plan into the cone such that L ( π ) ≥ H ( α ) , and prove that α is admissible for the program CGW, thus suboptimal.

Original: Using Equation (8), we have
Revised: Using Equation (8), we have

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Recall that the canonic injection p reads p ( x, r ) =
Revised: Recall that the canonic injection p reads p ( x, r ) =

Original: [ x, r ] .
Revised: [ x, r ] .

Original: Based on the above Lebesgue decomposition, we deﬁne the conic plan
Revised: Based on the above Lebesgue decomposition, we deﬁne the conic plan

Original: [Equation]
Revised: [Equation]

Original: We have that α ∈ U p ( µ, ν ) .
Revised: We have that α ∈ U p ( µ, ν ) .

Original: Indeed for the ﬁrst marginal (and similarly for the second) we have for any test function ξ ( x )
Revised: Indeed for the ﬁrst marginal (and similarly for the second) we have for any test function ξ ( x )

Original: [Equation]
Revised: [Equation]

Original: We deﬁne θ ∗ =
Revised: We deﬁne θ ∗ =

Original: θ ∗ c ( r, s ) the parameter which veriﬁes H c ( r, s )
Revised: θ ∗ c ( r, s ) the parameter which veriﬁes H c ( r, s )

Original: =
Revised: =

Original: θ ∗ L c ( r/θ ∗ , s/θ ∗ ) .
Revised: θ ∗ L c ( r/θ ∗ , s/θ ∗ ) .

Original: We restrict α ⊗ α to the set S
Revised: We restrict α ⊗ α to the set S

Original: = { θ ∗ λ (Γ) (( rr (cid:48) )
Revised: = { θ ∗ λ (Γ) (( rr (cid:48) )

Original: p , ( ss (cid:48) ) p ) > 0 } .
Revised: p , ( ss (cid:48) ) p ) > 0 } .

Original: By construction, θ ∗ c ( r, s ) is 1-homogeneous in ( r, s ) .
Revised: By construction, θ ∗ c ( r, s ) is 1-homogeneous in ( r, s ) .

Original: Thus on S we necessarily have r, r (cid:48) , s, s (cid:48) > 0 .
Revised: Thus on S we necessarily have r, r (cid:48) , s, s (cid:48) > 0 .

Original: It yields
Revised: It yields

Original: [Equation]
Revised: [Equation]

Original: Concerning the orthogonal part of the decomposition, note that whenever θ ∗ = 0 , due to the deﬁnition of H the cone distance reads
Revised: Concerning the orthogonal part of the decomposition, note that whenever θ ∗ = 0 , due to the deﬁnition of H the cone distance reads

Original: [Equation]
Revised: [Equation]

Original: It geometrically means that the shortest path between [ x, r ] and [ y, s ] must pass via the apex, which corresponds to a pure mass creation/destruction regime.
Revised: It geometrically means that the shortest path between [ x, r ] and [ y, s ] must pass via the apex, which corresponds to a pure mass creation/destruction regime.

Original: Furthermore we have that
Revised: Furthermore we have that

Original: [Equation]
Revised: [Equation]

Original: Indeed, thanks to Equation (24) we have for the ﬁrst marginal that
Revised: Indeed, thanks to Equation (24) we have for the ﬁrst marginal that

Original: [Equation]
Revised: [Equation]

Original: Note that the last equality holds because each term of α ⊗ α involving a measure δ 

Revised: Note that the last equality holds because each term of α ⊗ α involving a measure δ

Original: X cancels out when integrated against ( rr (cid:48) )
Revised: X cancels out when integrated against ( rr (cid:48) )

Original: p .
Revised: p .

Original: Eventually the computation gives (thanks to Lemma 1)
Revised: Eventually the computation gives (thanks to Lemma 1)

Original: [Equation]
Revised: [Equation]

Original: 2 L λ (Γ) ( f ⊗ f, g ⊗ g )
Revised: 2 L λ (Γ) ( f ⊗ f, g ⊗ g )

Original: d π d π + ϕ (0)( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) ≥ (cid:90) H λ (Γ) ( f ⊗ f, g ⊗ g )
Revised: d π d π + ϕ (0)( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) ≥ (cid:90) H λ (Γ) ( f ⊗ f, g ⊗ g )

Original: d π d π + ϕ (0)( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) ≥ (cid:90) D ([ d X ( x, x (cid:48) ) , ( f ⊗ f ) 1 p ] , [ d Y ( y, y (cid:48) ) , ( g ⊗ g ) 1 p ])
Revised: d π d π + ϕ (0)( | ( µ ⊗ µ ) ⊥ | + | ( ν ⊗ ν ) ⊥ | ) ≥ (cid:90) D ([ d X ( x, x (cid:48) ) , ( f ⊗ f ) 1 p ] , [ d Y ( y, y (cid:48) ) , ( g ⊗ g ) 1 p ])

Original: q d π d π + (cid:90) ϕ
Revised: q d π d π + (cid:90) ϕ

Original: (0)( rr (cid:48) )
Revised: (0)( rr (cid:48) )

Original: p d ( α ⊗ α ) | S c + (cid:90) ϕ (0)( ss (cid:48) )
Revised: p d ( α ⊗ α ) | S c + (cid:90) ϕ (0)( ss (cid:48) )

Original: p d ( α ⊗ α ) | S c ≥ (cid:90) D ([ d X ( x, x (cid:48) ) , rr (cid:48) ] , [ d Y ( y, y (cid:48) ) , ss (cid:48) ])
Revised: p d ( α ⊗ α ) | S c ≥ (cid:90) D ([ d X ( x, x (cid:48) ) , rr (cid:48) ] , [ d Y ( y, y (cid:48) ) , ss (cid:48) ])

Original: q d ( α ⊗ α ) | S + (cid:90) ϕ (0)(( rr (cid:48) )
Revised: q d ( α ⊗ α ) | S + (cid:90) ϕ (0)(( rr (cid:48) )

Original: p + ( ss (cid:48) )
Revised: p + ( ss (cid:48) )

Original: p )d ( α ⊗ α )
Revised: p )d ( α ⊗ α )

Original: | S c ≥ (cid:90) D ([ d X ( x, x (cid:48) ) , rr (cid:48) ] , [ d Y ( y, y (cid:48) ) , ss (cid:48) ])
Revised: | S c ≥ (cid:90) D ([ d X ( x, x (cid:48) ) , rr (cid:48) ] , [ d Y ( y, y (cid:48) ) , ss (cid:48) ])

Original: q d α d α ≥ H ( α ) .
Revised: q d α d α ≥ H ( α ) .

Original: Thus we have UGW ( X , Y )
Revised: Thus we have UGW ( X , Y )

Original: =
Revised: =

Original: L ( π ) ≥ H ( α ) ≥ CGW ( X , Y ) .
Revised: L ( π ) ≥ H ( α ) ≥ CGW ( X , Y ) .

Original: D A LGORITHMIC DETAILS AND FORMULAS D. D ECOMPOSITION OF KL QUADRATIC DIVERGENCE
Revised: D A LGORITHMIC DETAILS AND FORMULAS D. D ECOMPOSITION OF KL QUADRATIC DIVERGENCE

Original: We present in this section an additional property on the quadratic-KL divergence which allows to reduce the computational burden to evaluate it by involving the computation of a standard KL divergence.
Revised: We present in this section an additional property on the quadratic-KL divergence which allows to reduce the computational burden to evaluate it by involving the computation of a standard KL divergence.

Original: Proposition 9.
Revised: Proposition 9.

Original: For any measures ( µ, ν ) ∈ M + ( X ) , one has
Revised: For any measures ( µ, ν ) ∈ M + ( X ) , one has

Original: [Equation]
Revised: [Equation]

Original: In particular,
Revised: In particular,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Assuming KL( µ ⊗ ν | α ⊗ β ) to be ﬁnite, one has µ = fα and ν = gβ .
Revised: Assuming KL( µ ⊗ ν | α ⊗ β ) to be ﬁnite, one has µ = fα and ν = gβ .

Original: It reads
Revised: It reads

Original: [Equation]
Revised: [Equation]

Original: We now prove Proposition 4 which applies the above result.
Revised: We now prove Proposition 4 which applies the above result.

Original: Proposition 10.
Revised: Proposition 10.

Original: For a ﬁxed γ , the optimal π ∈ arg min π F ( π, γ ) + ε KL( π ⊗ γ | ( µ ⊗ ν ) ⊗ 2 ) is the solution of min π
Revised: For a ﬁxed γ , the optimal π ∈ arg min π F ( π, γ ) + ε KL( π ⊗ γ | ( µ ⊗ ν ) ⊗ 2 ) is the solution of min π

Original: [Equation]
Revised: [Equation]

Original: where m ( γ ) def.
Revised: where m ( γ ) def.

Original: = γ ( X × Y ) is the total mass of γ , and where we deﬁne the cost and weight associated to γ as
Revised: = γ ( X × Y ) is the total mass of γ , and where we deﬁne the cost and weight associated to γ as

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: First note that F ( γ, π ) = F ( π, γ ) so that minimizing with the ﬁrst or the second argument gives the same solution.
Revised: First note that F ( γ, π ) = F ( π, γ ) so that minimizing with the ﬁrst or the second argument gives the same solution.

Original: The rest follows from the relation
Revised: The rest follows from the relation

Original: [Equation]
Revised: [Equation]

Original: m ( µ )) , and also from KL( π 1 | µ ) =
Revised: m ( µ )) , and also from KL( π 1 | µ ) =

Original: (cid:82) log( d γ 1 d µ )d γ 1 − ( m ( γ ) − m ( µ )) .
Revised: (cid:82) log( d γ 1 d µ )d γ 1 − ( m ( γ ) − m ( µ )) .

Original: Similar formulas hold for ( π 2 , γ 2 ) and ( π, γ ) .
Revised: Similar formulas hold for ( π 2 , γ 2 ) and ( π, γ ) .

Original: D. D ISCRETE SETTING AND FORMULAS
Revised: D. D ISCRETE SETTING AND FORMULAS

Original: In order to implement those algorithms, one consider discrete mm-spaces X
Revised: In order to implement those algorithms, one consider discrete mm-spaces X

Original: =
Revised: =

Original: ( x i ) ni
Revised: ( x i ) ni

Original: =1 and Y =
Revised: =1 and Y =

Original: ( y j ) mj
Revised: ( y j ) mj

Original: =1 , endowed with discrete measures µ = (cid:80) i µ i δ x i and ν = (cid:80) j ν j δ y j , where µ i , ν j ≥ 0 .
Revised: =1 , endowed with discrete measures µ = (cid:80) i µ i δ x i and ν = (cid:80) j ν j δ y j , where µ i , ν j ≥ 0 .

Original: The distance matrices are D X i,i (cid:48) def.
Revised: The distance matrices are D X i,i (cid:48) def.

Original: =
Revised: =

Original: d X ( x i , x i (cid:48) ) and D X j,j (cid:48) def.
Revised: d X ( x i , x i (cid:48) ) and D X j,j (cid:48) def.

Original: =
Revised: [Equation]

Original: d X ( y j , y j (cid:48) ) .
Revised: d X ( y j , y j (cid:48) ) .

Original: Transport plan are thus also discrete π = (cid:80) i,j π i,j δ ( x i ,y j ) .
Revised: Transport plan are thus also discrete π = (cid:80) i,j π i,j δ ( x i ,y j ) .

Original: The functional L now reads in this discrete setting
Revised: The functional L now reads in this discrete setting

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where we deﬁne the marginals π 1 ,k def.
Revised: where we deﬁne the marginals π 1 ,k def.

Original: = (cid:80) j π k,j , π 2 ,(cid:96) def.
Revised: = (cid:80) j π k,j , π 2 ,(cid:96) def.

Original: = (cid:80) i π i,(cid:96) and m ( π ) =
Revised: = (cid:80) i π i,(cid:96) and m ( π ) =

Original: (cid:80) i,j π i,j .
Revised: (cid:80) i,j π i,j .

Original: When one runs the stabilized implementation of Sinkhorn’s iterations with a ground cost C i,j =
Revised: When one runs the stabilized implementation of Sinkhorn’s iterations with a ground cost C i,j =

Original: C ( x i , y j ) between the points, it is necessary to use a Log-Sum-Exp reduction which reads
Revised: C ( x i , y j ) between the points, it is necessary to use a Log-Sum-Exp reduction which reads

Original: [Equation]
Revised: [Equation]

Original: (28) where LSE j is a reduction performed on the index j .
Revised: (28) where LSE j is a reduction performed on the index j .

Original: It reads
Revised: It reads

Original: [Equation]
Revised: [Equation]

Original: (29) where the logarithm and exponential are pointwise operations.
Revised: (29) where the logarithm and exponential are pointwise operations.

Original: Input: mm-spaces X
Revised: Input: mm-spaces X

Original: =
Revised: =

Original: ( D Xi,j , ( µ i ) i ) and Y =
Revised: ( D Xi,j , ( µ i ) i ) and Y =

Original: ( D Yi,j , ( ν j ) j ) , relaxation ρ , regularization ε
Revised: ( D Yi,j , ( ν j ) j ) , relaxation ρ , regularization ε

Original: Output: approximation ( π, γ ) minimizing1: Initialize
Revised: Output: approximation ( π, γ ) minimizing1: Initialize

Original: ,j = γ i,j = µ i ν j / (cid:113) ( (cid:80) i µ i )( (cid:80) j ν j ) , vector g ( s =0) j = 0 .
Revised: ,j = γ i,j = µ i ν j / (cid:113) ( (cid:80) i µ i )( (cid:80) j ν j ) , vector g ( s =0) j = 0 .

Original: 2: while π has not converged do 3:
Revised: 2: while π has not converged do 3:

Original: Update π ← γ 4:
Revised: Update π ← γ 4:

Original: Deﬁne m ( π ) ← (cid:80) i,j π i,j , ˜ ρ ← m ( π ) ρ , ˜ ε ← m ( π ) ε 5:
Revised: Deﬁne m ( π ) ← (cid:80) i,j π i,j , ˜ ρ ← m ( π ) ρ , ˜ ε ← m ( π ) ε 5:

Original: Deﬁne c ← ComputeCost( X , Y , π , ρ , ε ) 6: while ( f, g ) has not converged do 7:
Revised: Deﬁne c ← ComputeCost( X , Y , π , ρ , ε ) 6: while ( f, g ) has not converged do 7:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Update γ i,j ←
Revised: Update γ i,j ←

Original: [Equation]
Revised: [Equation]

Original: Rescale γ ← (cid:112) m
Revised: Rescale γ ← (cid:112) m

Original: ( π ) /m
Revised: ( π ) /m

Original: ( γ ) γ 11:
Revised: ( γ ) γ 11:

Original: Return ( π, γ ) .
Revised: Return ( π, γ ) .

Original: We also provide an algorithm that computes the cost c επ deﬁned in Proposition (10).
Revised: We also provide an algorithm that computes the cost c επ deﬁned in Proposition (10).

Original: We focus on the case D ϕ
Revised: We focus on the case D ϕ

Original: =
Revised: =

Original: ρ KL and λ ( t ) =
Revised: ρ KL and λ ( t ) =

Original: t 2 which is computable with complexity O ( n 3 ) as shown in Peyr ´ e et al.
Revised: t 2 which is computable with complexity O ( n 3 ) as shown in Peyr ´ e et al.

Original: Indeed, note that one has
Revised: Indeed, note that one has

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Input: spaces X
Revised: Input: spaces X

Original: ,j , ( µ i ) i and
Revised: ,j , ( µ i ) i and

Original: ,(cid:96) ,( ν j )
Revised: ,(cid:96) , ν j )

Original: , transport matrix (
Revised: , transport matrix (

Original: ,k )
Revised: ,k )

Original: , relaxation
Revised: , relaxation

Original: , regularization ε
Revised: , regularization ε

Original: Output: cost c επ deﬁned in Proposition
Revised: Output: cost c επ deﬁned in Proposition