P ARTIAL W ASSERSTEIN A DVERSARIAL N ETWORK FOR N ON - RIGID P OINT S ET R EGISTRATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Given two point sets, the problem of registration is to recover a transformation that matches one set to the other.
This task is challenging due to the presence of a large number of outliers, the unknown non-rigid deformations and the large sizes of point sets.
To obtain strong robustness against outliers, we formulate the registration problem as a partial distribution matching (PDM) problem, where point sets are regarded as discrete distributions, and the goal is to partially match them.
To handle large point sets, we propose a method for large scale PDM problem by utilizing the partial Wasserstein-1 (PW) discrepancy, which we show can be efﬁciently optimized.
Speciﬁcally, we theoretically derive the Kantorovich–Rubinstein duality for the PW discrepancy, and show its gradient can be explicitly computed.
Based on these theoretical results, we propose a partial Wasserstein adversarial network (PWAN), which approximates the PW discrepancy by a neural network, and learns the transformation adversarially with the network.
It also incorporates an efﬁcient coherence regularizer for non-rigid transformations to avoid unrealistic deformations.
We evaluate PWAN on practical point set registration tasks, and show that the proposed PWAN is robust, scalable and performs more favorably than the state-of-the-art methods.
I NTRODUCTION
Point set registration is a fundamental task in many computer vision applications, such as object tracking (Gao & Tedrake, 2018), shape retrieval (Berger et al., 2017) and contour matching (Avots et al., 2019).
As illustrated in Fig.
1, given two point sets, i.e ., a reference set and a source set, representing two partially overlapped shapes, the goal of this task is to recover an appropriate transformation that matches the source set to the reference one.
It is challenging due to the following factors:
The point sets may contain a fraction of
outliers which do not have valid correspondences in the other point set, such as the noise points and the points in the non-overlapped region of the shapes.
The number of points consisted in the point sets may be huge.
The deformation of point sets can be non-rigid.
[Figure]
The registration problem is generally solved in the distribution matching (DM) framework, where the point sets are regarded as probability distributions, and are aligned by minimizing a discrepancy between them.
To reduce the inﬂuence of outliers, practical registration methods utilize the robust discrepancies, such as Kullback-Leibler divergence (Myronenko & Song, 2010; Hirose, 2021a) and
L 2 distance (Jian & Vemuri, 2011).
Thus they are able to greedily align the largest possible fraction of points while being tolerant of a small number of outliers.
For point sets that are dominated by outliers, the greedy behaviors of these methods easily bias toward outliers, thus leading to degraded registration results.
An example of such result is shown in Fig. 1(b).
To obtain stronger robustness against outliers, we propose to formulate the registration problem in a novel partial distribution matching (PDM) framework, where we only seek to partially match the distributions.
Comparing to the DM based methods, the PDM formulation is naturally more robust against outliers.
For example, in Fig.
1(c), the PDM formulation ﬁnd a more natural solution where only a fraction of points are well matched.
However, existing solutions for PDM problems generally require to compute the correspondence between distributions (Chizat et al., 2018; Chapel et al., 2020), thus they are intractable for registration problems which involve large scale distributions.
To handle this issue, we propose the ﬁrst efﬁcient solver for large scale PDM problem.
Our method utilizes the partial Wasserstein-1 (PW) discrepancy (Figalli, 2010), which we show can be efﬁciently optimized without computing the correspondence.
Speciﬁcally, we theoretical derived the Kantorovich–Rubinstein (KR) duality (Villani, 2003) for the PW discrepancy, and show that its gradient can be explicitly computed via its potential.
Based on these results, we propose a partial Wasserstein adversarial network (PWAN), which approximates the potential by a neural network, and the unknown transformation can be trained in an adversarial way with the network.
We also incorporate a coherence regularizer for the non-rigid transformation to enforce its smoothness.
We note that PWAN generalizes the popular Wasserstein generative adversarial net (WGAN)
(Arjovsky et al., 2017) to the PDM problem.
An illustration of the proposed PWAN is presented in Fig.
The contribution of this work is summarized as follows.
- Theoretically, we derive the KR duality for the mass-type PW discrepancy. We investigate the properties of mass-type PW discrepancy and the closely related distance-type PW discrepancy.
In particular, we show their gradients can be efﬁciently computed, thus they can be optimized via gradient descent.
- Based on the theoretical results, we propose the ﬁrst method, called PWAN, for large scale PDM problem.
The well-known WGAN is a special case of our method in the DM setting.
- We apply PWAN to point set registration task, where point sets are regarded as discrete distributions.
We experimentally show that PWAN exhibits stronger robustness against outliers than the existing methods, and can register point sets accurately even when they are dominated by outliers, such as when they contain a large fraction of noise points, or when they are only partially overlapped
R ELATED WORK
There is a large body of literatures on point set registration problem (Besl & McKay, 1992; Zhang, 1994; Chui & Rangarajan, 2000; Myronenko & Song, 2010; Maiseli et al., 2017; Vongkulbhisal et al., 2018; Hirose, 2021a).
The existing methods can be roughly categorized into two classes, i.e ., the correspondence-based methods and the probabilistic methods.
This section only discusses the latter class which is the most related to our method.
More discussions can be found in Appx.
F.1.
The probabilistic methods solve the registration problem as a DM problem.
Speciﬁcally, most of these methods smooth the point sets as Gaussian mixture models (GMMs), and then align the distributions via minimizing the robust discrepancies between them.
Coherent point drift (CPD) (Myronenko &
Song, 2010) and its variants (Hirose, 2021a) are well-known methods is this class, which minimize
the
Kullback-Leibler (KL) divergence between the distributions.
Other robust discrepancies, including the L 2 distance (Ma et al., 2013; Jian & Vemuri, 2011), the kernel density estimator (Tsin & Kanade, 2004) and the scaled Geman–McClure estimator (Zhou et al., 2016) have also been studied.
The proposed PWAN is related to the probabilistic methods because it regards point sets as distributions.
However, it has two major differences from them.
First, PWAN directly processes the point sets as un-normalized discrete distributions instead of smoothing them to GMMs, thus it is more concise and natural.
Second, PWAN solves a PDM problem instead of the DM problem, as it only requires to match a fraction of distributions, thus it is more robust to outliers.
Some recent works have been devoted to the PDM problem using the Wasserstein-type discrepancy.
Bonneel & Coeurjolly (2019) proposed a fast algorithm based on the sliced Wasserstein metric for a special PDM problem, where a small distribution is fully matched to the large one.
Various partial or unbalanced discrepancies (Chizat et al., 2018; Séjourné et al., 2019; Chapel et al., 2020) have been utilized for the PDM problem, however, they are generally not scalable to large distributions as they require to compute the correspondence between distributions.
The distance-type PW discrepancy used in our paper has been utilized for imaging problem (Lellmann et al., 2014; Schmitzer & Wirth, 2019), but these methods do not directly apply to distributions in continuous space.
P RELIMINARIES ON PARTIAL W ASSERSTEIN -1 DISCREPANCIES
This section introduces the major tools used in this work, i.e ., two types of PW discrepancies: the partial-mass Wasserstein-1 discrepancy (Figalli, 2010; Caffarelli & McCann, 2010) and the boundeddistance Wasserstein-1 discrepancy (Lellmann et al., 2014; Bogachev, 2007).
For simplicity, we call them the mass-type and the distance-type PW discrepancy respectively.
Given two discrete distributions α = (cid:80) x i ∈
X a i δ x i and β = (cid:80)
y j ∈ Y b
j δ y j , where point sets X
= { x i } qi =1 and Y
=
{ y j } rj =1 are in a compact metric space Ω , δ is the Dirac function, and a = [ a i ] qi =1 ∈
R q + and b = [ b j ] rj =1 ∈ R r + are the associated mass vectors, the mass-type PW discrepancy (Figalli, 2010) between α and β is deﬁned by the optimal transport problem which seeks to transports at least m ≤ min( m α , m β ) unit mass (where m α = (cid:80) i a i and m β = (cid:80) j b j ) from α to β in the cheapest way.
Formally, we denote L M,m the mass-type PW discrepancy with mass threshold m , and deﬁne the primal form of L M,m ( α, β ) as
[Equation]
where P i,j encodes the amount of mass transported from x i to y j , and the feasible set Γ m ( α, β ) is given by Γ m ( α, β ) =
{ P ∈ R
[Equation]
We also consider the optimal transport problem with bounded transport distance (Lellmann et al., 2014).
Formally, we denote L D,h the distance-type PW discrepancy with a transport distance threshold h , and deﬁne the primal form of L D,h as the Lagrangian of problem (1), where the mass constraint is soften by a Lagrange multiplier h > 0 :
[Equation]
In this work, we use L D,h and L M,m as two types of objective functions to partially match the distributions.
Note these two objective functions are closely related.
Because according to the Lagrange duality, for a ﬁxed ( α, β, m ) , there exists a h ∗ ≥ 0 , such that the solution to L D,h ∗ ( α, β ) recovers to that to L M,m ( α, β ) . Nevertheless, these two objective functions are not equivalent. Because when α or β varies during iterations, the corresponding h ∗ varies accordingly.
In other words, minimizing L M,m is generally not equivalent to minimizing L D,h with any ﬁxed h .
We present more detailed preliminaries in Appx.
A.
T HE P ROPOSED A PPROACH
In this section, we present the details of the proposed PWAN.
We ﬁrst formulate the problem in
Sec. 4.1.
Then we present an efﬁcient method to solve the problem in Sec.
4.2 and 4.3.
We ﬁnally summarize our algorithm in Sec. 4.4.
4. P ROBLEM FORMULATION
Let Y =
{ y j } rj =1 ⊆ Ω and X = { x i } qi =
⊆
Ω be the source and reference point sets, where Ω ⊆ R 3 is a closed bounding box of the points.
We seek to solve the registration problem in a PDM formulation.
Formally, we deﬁne the corresponding reference and source distribution as α = (cid:80) x i ∈
X a i δ x i and β = (cid:80)
y j ∈ Y b
j δ y j respectively, where a i , b j ∈ R + are the mass assigned to each point and are ﬁxed to 1 in this paper.
Let T θ : Ω → Ω denote a non-rigid transformation parametrized by θ and β θ = (cid:80) y
j ∈ Y b j δ T θ ( y j ) denote the transformed β .
Our goal is to ﬁnd an optimal θ so that β θ is partially matched to α , i.e ., we seek to solve
[Equation]
where L is L M,m (1) or L D,h (2), which measures the closeness between the nearest fraction of α and β θ , and the coherence energy C ensures T θ is sufﬁciently smooth.
4. R EGISTRATION WITH THE PW DISCREPANCIES
Using L M,m and L D,h in the objective function (3) provides two natural ways to control the ratio of matching.
Speciﬁcally, by expressing L M,m or L D,h in their respective primal form (1) or (2), we have
[Equation]
where const represents a constant not relevant to θ , and P represents the correspondence given by the solution of the primal form of
L M,m ( α, β θ ) (1) or L D,h ( α, β θ ) (2).
To obtain a sense of this formulation, we visualize the correspondence P between two C-shape point sets in Fig.
As can be seen, P establishes the correspondence between a fraction of points within the mass threshold m or the distance threshold h , and discards the other points, thus minimizing (4) is simply to match these corresponding point pairs.
In other words, minimizing L M,m ( α, β θ ) is to match
m points in β θ that are nearest to α in the optimal way, while minimizing L D,h
( α, β θ ) is to match the points in β θ within the distance h of α in the optimal way.
However, for large scale point sets, optimizing (4) is computationally intractable, since it requires to solve for a matrix P of shape
q × r in a linear programing in each iteration.
Thus, it is natural to ask that how to avoid the computation of P , so that the PW discrepancies can be optimized efﬁciently.
This question is answered by the following two theorems, whose proofs are given in Appx.
E.
Theorem 1.
L D,h ( α, β ) can be equivalently expressed as
[Equation]
The optimizer of problem (5) exists.
Let f denote an optimizer of problem (5) .
When T θ is Lipschitz w.r.t.
θ , L D,h ( α, β θ ) is continuous w.r.t.
θ , and is differentiable almost everywhere.
Furthermore, we have
[Equation]
Theorem 2.
L M,m ( α, β ) can be equivalently expressed as
[Equation]
The optimizer of problem (7) exists.
Let ( f , h ) denote an optimizer of problem (7) .
When T θ is Lipschitz w.r.t.
θ , L M,m ( α, β θ ) is continuous w.r.t.
θ , and is differentiable almost everywhere.
Furthermore, we have
[Equation]
Theorem 1 and Theorem 2 tell us two properties of L D,h and L M,m .
First, they provide the KR dual forms for L D,h and L M,m in euqation (5) and (7) respectively, which no longer require to compute the correspondence P , but only require to ﬁt a function f to q + r data points instead.
We call formulations (5) and (7) KR forms , and call the maximizers of them potentials .
Furthermore, they claim that if T θ satisﬁes a mild regularity assumption, then L D,h ( α, β θ ) and L M,m ( α, β θ ) are differentiable w.r.t.
θ and their gradients can be computed, which implies that they can be optimized via gradient descent.
See Tab. 1 in the appendix for a comparison of equivalent formulations.
Now, we can formulate our PWAN model, which effectively optimizes the PW discrepancies, by implementing Theorem 1 and Theorem 2 into neural networks.
Speciﬁcally, we compute L M,m ( α, β θ ) or L D,h ( α, β θ ) according to (7) or (5), where f is approximated by a neural network called the potential network.
We note that both these two equations require f to be Lipschitz, and the output of f to be bounded in interval [ − h, 0] , where h is a ﬁxed parameter in (5), while is a learnable variable in (7).
Thus we parametrize f as f w,h = l ( g w ; h ) , where g w is a neural network parametrized by w , which maps each point to a real number, and l ( x ; h )
=
max {−| x | , − h } is an activate function acting on the output of g w .
In this way, we ensure that f w,h maps each point to a number in interval [ − h, 0] .
To further ensure f w,h is Lipschitz, we use an additional regularizer to penalize the gradient of its output w.r.t.
its input (Gulrajani et al., 2017).
Then we train f w,h by gradient ascent: for
L M,m ( α, β θ ) , we update { w, h } to maximize
Loss M,m = (cid:88)
[Equation]
while for L D,h ( α, β θ ) , we update { w } to maximize
Loss D,h = (cid:88)
[Equation]
where κ is the strength of the gradient penalty and is set to κ = 100 in all our experiments.
Finally, with the trained potentials network f w,h , the gradients ∇ θ L M,m ( α, β θ ) and ∇ θ L D,h ( α, β θ ) can be computed via back-propagating their respectively losses through θ , i.e .,
[Equation]
where L is L M,m or L D,h .
Thus T θ can be updated via gradient descent.
To provide an intuition of the KR forms, we present a simple example comparing the primal form and the learned KR form in Appx.
F.2.
4. O PTIMIZE THE COHERENCE ENERGY
This section discusses the optimization of the coherence energy, which ensures that the whole β θ , including the points discarded by the PW discrepancy, remains smooth in the matching process.
First of all, we deﬁne the non-rigid transformation T θ parametrized by θ =
( A, t, V ) as
[Equation]
where y j ∈ R 1 × 3 represents the coordinate of point y j ∈ Y , A ∈ R 3 × 3 is the linear transformation matrix, t ∈ R 1 × 3 is the translation vector, V ∈ R r × 3 is the offset vector of all points in Y , and V j represents the j -th row of V .
According to this deﬁnition, T θ is the composition of two simple transformations: a linear transformation and the adding of an offset vector to each point.
Despite its simplicity, T θ includes several useful transformations as its special case.
For example, when V =and A ∈ SO (3) , T θ becomes the rigid transformation, and when A = I and t = 0 , T θ becomes the “drift” transformation in Myronenko & Song (2010).
Note that T θ is Lipschitz w.r.t.
θ (Propositionin the appendix), thus it satisﬁes the requirement of Theorem 1 and 2.
Now we deﬁne the coherence energy similar to Myronenko & Song (2010);
Hirose (2021a), i.e ., we enforce that V varies smoothly in space.
Formally, let G ∈ R r × r be a kernel matrix, e.g .,
the
Gaussian kernel G ρ ( i, j ) =
e −|| y
i − y j
|| 2 /ρ , and σ be a positive number.
The coherence energy is deﬁned as
[Equation]
13) where λ ∈ R + is the strength of constraint, I is the identity matrix, and Tr ( · ) is the trace of a matrix.
Since the matrix inversion ( σ I + G ρ ) − 1 is computationally intractable for large scale point sets, we approximate it via the Nyström method (Williams & Seeger, 2000), and obtain the gradient
[Equation]
(14) where Q ∈ R r × k , Λ ∈ R k × k is a diagonal matrix, and k (cid:28) r .
Note the computational cost of (14) is only O
( r ) if we regard k as a constant.
The detailed derivation is presented in Appx.
F.3.
4. A N ADVERSARIAL ALGORITHM
With the methods detailed in Sec. 4.2 and Sec. 4.3, we can ﬁnally solve problem (3) efﬁciently.
Speciﬁcally, we iterate the following two steps: 1) Estimate the PW discrepancy between α and β θ .
2) Update θ by minimizing the estimated PW discrepancies and the coherent energy.
Formally, we solve the following mini-max problem by alternatively updating f w,h and T θ
[Equation]
where we use Loss M,m (9), (cid:101)
w = { w, h } for L M,m , or Loss D,h (9), (cid:101)
w = { w } for L D,h .
An illustration of our method is shown in Fig.
The detailed algorithm is presented in Alg.
For clearness, we refer to PWAN based on L M,m and L D,h as mass-type PWAN (m-PWAN) or distancetype PWAN (d-PWAN) respectively.
Algorithm 1 PWAN for point set registration
Input: reference set X , source set Y , transform T θ , potential network f w,h , network update frequency u , training type (“ mass ” or “ distance ”), mass threshold m , distance threshold h , training step T , coherence parameters ( λ, ρ, σ ) , Nyström parameter k .
Output: the learned θ .
if training type = “ mass ” then (cid:101) w ← ( w, h ) ; m ← m ;
L ← Loss M,m deﬁned in (9) else if training type = “ distance ” then (cid:101) w ← ( w ) ; h ← h ;
L ← Loss D,h deﬁned in (10) end if for t = 1 , . .
, T do
Obtain the transformed distribution β θ .
for = 1 , . . . , u do
Compute ∂L/∂ (cid:101) w by back-propagating L through (cid:101) w . Update (cid:101) w by ascending the gradient ∂L
Compute ∂ C /∂θ using ( λ, ρ, σ, k ) according to (14). Compute ∂L/∂θ by back-propagating (11) through θ . Update θ by descending the gradient ∂ ( L + C ) /∂θ .
Registration
Finally, we analyse a toy example of registering the 1-dimensional distributions shown in Fig. 4.
For now, we do not consider the coherent energy for simplicity. In the beginning of an iteration, the potential f w,h (shown in Fig. 4) is learned in the potential update step.
Note the learned f w,h may have some ﬂat regions where it attains its maximal or minimal values ( 0 or − h ) (Propositionand 22 in the appendix).
Thus the gradient norm of f w,h is 0 in these regions.
Then, the following registration step will move β θ in the direction of ∇ f w,h . In other words, it will move the points outside the ﬂat regions leftward (toward α ), and it will not move the points inside the ﬂat regions. After a sufﬁcient number of iterations, a fraction of points in β θ will be completely matched to α , while the other points will not be moved.
More discussions about the connections between our method and WGAN can be found in Appx.
F.4.
E XPERIMENTS AND A NALYSIS
In this section, we experimentally evaluate the proposed PWAN on point set registration.
After describing the experiment settings in Sec. 5.1, we ﬁrst present a toy example to highlight the robustness of the PW discrepancies in Sec.
Then we compare PWAN against the state-of-the-art methods in Sec. 5.3, and discuss its scalability in Sec. 5.4.
We ﬁnally evaluate PWAN on two real datasets in Sec. 5.5.
More experimental results are given in Appx.
F.
5. E XPERIMENT SETTINGS
The source point sets (blue) are synthesized by bending the reference point sets (red) in a nonlinear manner.
As shown in Fig.
5, we use three synthesized datasets in our experiments: bunny, armadillo and monkey.
The bunny and armadillo datasets are from the Stanford Repository (Standford), and the monkey dataset is from the internet.
The number of points in these shape are 8 , 171 , 106 , 289 and 7 , 958 respectively.
Following Hirose (2021b), we artiﬁcially deform these sets, and we evaluate the registration results via the mean square error (MSE).
We use a 5 -layer point-wise multi-layer perception as our network (Fig. 11 in the appendix), and the parameters used in the experiments are given in Appx.
F.5.
C OMPARISON OF SEVERAL DISCREPANCIES
To provide an intuition of our PDM formulation for registration, we compare the PW discrepancy with two representative robust discrepancies used in DM based registration methods, i.e ., KL divergence (Hirose, 2021a) and L 2 distance (Jian & Vemuri, 2011), on a toy 1-dimensional example.
For now, we do not consider the coherence energy.
We construct the toy point sets X and Y shown in Fig. 6(a), where we ﬁrst sample 10 equi-spaced data points in interval [0 , 3] , then we deﬁne Y by translating the data points by a distance t , and deﬁne
X by adding N outliers in a narrow interval [7 . 8 , 8 . 2] to the data points.
For N = { 1 , 10 , 10 3 } , we record four discrepancies: KL, L 2 , L M, 10 and L D, 2 between X and Y as a function of t , and present the results from Fig. 6(b) to Fig. 6(e).
As can be seen, there exist two alignment modes in this experiment, i.e ., t = 0 and t = 6 . 5 , which respectively correspond to the correct alignment and the degraded alignment where Y is matched to outliers.
When the number of outliers is small, e.g ., N = 1 , all discrepancies admit a deep local minimum t = 0 .
However, as for KL and L 2 divergence, the local minimum t = 0 gradually vanishes and they gradually bias toward t = 6 .
5 as N increases.
In contrast, L M, 10 and L D, 2 do not suffer from this issue, i.e ., the local minimum t = 0 remains deep regardless of the value of N , and the local minimum t = 6 .
5 is always shallower than the local minimum t = 0 .
The results show a key advantage of PDM against DM for registration:
The correct local minimum t = 0 of the DM formulation (KL and L 2 ) vanishes as N increases.
However, thanks to the threshold of PW discrepancies, the local minimum t = 0 of the PDM formulation is not inﬂuenced by the remote outliers beyond the threshold, i.e ., it maintains a deep local minimum t = 0 even for large N . Thus the PDM formulation is always likely to ﬁnd the correct alignment t = 0 even for the large N . [Figure]
5. E VALUATION OF THE REGISTRATION ACCURACY
We compare the performance of PWAN with four state-of-the-art methods: CPD (Myronenko &
Song, 2010), GMM-REG (Jian & Vemuri, 2011), BCPD (Hirose, 2021a) and TPS-RPM (Chui &
Rangarajan, 2000).
We evaluate them on the following two artiﬁcial datasets.
Point sets with extra noise points.
We ﬁrst sample N = 500 random points from each of the original and the deformed sets as the source and reference sets respectively.
Then we add extra uniformly distributed noise points to the reference set, and we normalize both sets to mean 0 and variance 1 .
We vary the number of outliers from 100 to 600 at an interval of 100 , i.e ., the outlier/nonoutlier ratio varies from 0 .
2 to 1 .
at an interval of 0 . 2 .
Partially overlapped point sets.
We ﬁrst sample N = 1000 random points from each of the original and the deformed sets as the source and reference sets respectively.
Then for each set, we intersect it with a random plane, and we retain the points in one side of the plane and discard all points in the other side.
We vary the retain ratio s from 0 .
7 to 1 .
at an interval of 0 .
for both the source and the reference sets, thus the minimal ratios of overlapping area are (2 s − 1) /s
=0 .
94 and 1 accordingly, and the minimal corresponding mass is m =
(2 s − 1) ∗ 1000 .
We evaluate m-PWAN with m = 500 (equivalently d-PWAN with h = + ∞ ) in the ﬁrst experiment, while we evaluate m-PWAN with m = m and d-PWAN with h = 0 . 05 in the second experiment.
For each setting, we run all methods 100 times and report the median of MSE in Fig.
Fig. 7(a) presents the results of the ﬁrst experiment.
In the bunny and armadillo datasets, PWAN performs comparably with TPS-RPM, and these two methods show much lower registration errors than other methods.
However, for the monkey dataset, TPS-RPM often fails to converge, while
PWAN still shows low registration errors.
This suggests that both TPS-RPM and PWAN are more robust against noise points than other baseline methods, while PWAN is more stable than TPS-RPM.
Fig. 7(b) presents the results of the second experiment.
Two types of PWANs perform comparably, and they outperform the baseline methods by a large margin when the overlap ratio is low, while perform comparably with them when the data is fully overlapped.
This result suggests the proposed
PWAN can effectively register the partially overlapped sets.
5. E VALUATION OF THE EFFICIENCY
To evaluate the efﬁciency of PWAN, we ﬁrst need to investigate the inﬂuence of its parameters.
In particular, the most important parameter that affects the efﬁciency is the network update frequency u , which controls the tradeoff between the efﬁciency and effectiveness.
Speciﬁcally, larger u leads to more accurate estimation of the potential and the gradient, while smaller u allows for faster [Figure] estimations.
We quantify the inﬂuence of u using the bunny dataset and report the results in the left panel of Fig. 8.
As can be seen, both MSE and the variance of MSE decrease as u increases, while the computation time increases proportionally with u .
This result suggests that more accurate and stable results can be achieved at the expense of higher computational cost, i.e ., larger u .
We benchmark the computation time of different methods on a computer with two
Nvidia GTX TITAN GPUs and an Intel i CPU.
We ﬁx u = 20 for PWAN.
We sample q = r points from the bunny shape, where q varies from 10 3 to 7 × 10 5 .
PWAN is run on the GPU while the other methods are run on the CPU.
We also implement a multi-GPU version of PWAN where the potential network is updated in parallel.
We run each method 10 times and report the mean of the computation time in the right panel of Fig. 8.
As can be seen, BCPD is the fastest method when q is small, and PWAN is comparable with BCPD when q is near 10 6 .
In addition, the 2-GPU version PWAN is faster than the single GPU version, and it is faster than BCPD when q is larger than 5 × 10 5 .
5. E VALUATION ON REAL DATA
To demonstrate the capability of PWAN on handling point sets with non-artiﬁcial deformations, we evaluate it on a human face dataset (Zhang et al., 2008) and a 3D human dataset (DataSet).
The details of this experiment are shown in Appx.
F.9 and Appx.
F.10.
C ONCLUSION
In this paper, we formulate the point set registration task as a PDM problem, where point sets are regarded as discrete distributions and are only required to be partially matched.
In order to solve the
PDM problem, we develope a theory for PW discrepancy, which allows us to estimate it efﬁciently.
Based on the theory, we proposed a PWAN method for PDM problem, and apply it to practical point sets registration task.
We experimentally show that PWAN can effectively handle the point sets dominated by outliers, including those containing large fraction of noise or being partially overlapped.
There are several issues need further study.
First, the computation time of PWAN is still relatively high.
A possible approach to accelerate PWAN is to incorporate a forward generator network as in Sarode et al.
(2019) or to use the discriminative training as in Vongkulbhisal et al.
Second, it is interesting to explore PWAN in other PDM problems, such as domain adaption (Hu et al., 2020), pose estimation (Kuhnke & Ostermann, 2019) and multi-label learning (Yan & Guo, 2019).
A PPENDIX
In this appendix, we provide theoretical justiﬁcations of our algorithm and more experimental results.
The appendix is organized as follows.
We ﬁrst present a more general introduction of the optimal transport problem and ﬁx the notations in Sec.
A.
Then we introduce existing computational approach of optimal transport problem in Sec.
B. We derive our formulation in Sec. C. We further discuss the properties of our formulations in Sec.
D. Finally, we prove the theorems in the main text in Sec.
E and we present more details for the main text in Sec F.
A P RELIMINARIES
This section introduces the optimal transport (OT) problem and ﬁx the notations for later sections.
A. O PTIMAL TRANSPORT
OT is a classic problem dating back to Monge and Kantorovich.
It requires to solve the following transportation problem: let α be the distribution of warehouses storing some raw materials, and β be the distribution of factories requiring these materials.
Assuming the total mass of materials stored in the warehouse is m α , and the total mass of materials required by the factories is m β .
How to transport at least m ( m ≤ min( m α , m β ) ) mass materials from α to β so that the total cost is minimized?
Formally, let Ω be compact metric space and M + (Ω) be the set of non-negative measures deﬁned on Ω .
Deﬁne the source and target measures α, β ∈ M + (Ω) , and a continuous cost function c : Ω × Ω
→ R + .
OT seeks to solve the following optimization problem
[Equation]
(16) where Γ m ( α, β ) are the set of non-negative measures π deﬁned on Ω × Ω satisfying π ( A × Ω) ≤ α ( A ) , π (Ω × A ) ≤ β
( A ) and π (Ω × Ω) ≥ m for all measurable set A ⊆ Ω .
For ease of notations, we abbreviate m α = α (Ω) , m β = β (Ω) and m ( π ) =
π (Ω × Ω) .
In the complete OT problem, it is generally assumed that m α = m β
= m , i.e ., the source and the target distributions contain equal mass of materials, and all materials should be transported.
However, the more general partial optimal transport (POT) problem (Figalli, 2010; Caffarelli & McCann, 2010) only requires m ≤ min ( m α , m β ) , i.e ., the source and target distributions may contain different mass of materials, and only a partial of mass is required to transported.
A. P ARTIAL W ASSERSTEIN -1 DISCREPANCY
In this paper, we focus on a speciﬁc POT problem where the cost function is a distance in the metric space Ω , i.e ., c ( x, y ) =
d ( x, y ) , where d is the distance function deﬁned in Ω .
In this case, we called
L M,m the mass-type partial Wasserstein-1 (PW) discrepancy.
In the complete OT case, this type of OT problem deﬁnes the Wasserstein- 1 metric, which is also known as the earth move distance (EMD) between distributions.
According to the Kantorovich–Rubinstein duality (Kantorovich, 2006), the Wasserstein- 1 metric can be equivalently expressed as
[Equation] [Equation]
Note that this formulation offers a more efﬁcient implement of Wasserstein- 1 metric than the primal form 16, as it only requires to solve for a function with a local constraint in Ω instead of a transport plan with global constraints in Ω × Ω .
Several methods have been proposed to generalize (17) to the unbalanced OT (Chizat et al., 2018;
Schmitzer, 2019), i.e ., OT problems with extra regularizers.
See Schmitzer & Wirth (2019) for an uniﬁed framework of this type of generalizations.
Amongst these generalizations, KR metric (Lellmann et al., 2014) is closely related to the POT problem 16 considered in this paper, as it can be regarded as a Lagrangian of problem 16 where the mass constraint is soften.
Speciﬁcally, the primal form KR metric with parameter h > 0 is deﬁned as
[Equation]
Note the KR metric has several equivalent primal forms, such as the unbalanced optimal transport formulation with total-variation regularizer (Schmitzer & Wirth, 2019), or the ﬂow-based formulations in Lellmann et al.
It is important to notice that the KR metric has a natural explanation that it requires to ﬁnd a optimal plan whose transport distance does not exceed h .
This can be seen by re-writing problem 18 as KR h ( α, β ) =
inf (cid:82) ( d ( x, y ) − h )
dπ
( x, y )
+ const , and noticing that
if d ( x, y ) > h , the solution π ∗ to problem 18 should satisfy π ∗ ( x, y )
= 0 , because otherwise π ∗ can be further improved by setting π ∗ ( x, y ) to 0 .
Importantly, the KR metric is known to have an equivalent form (Lellmann et al., 2014; Schmitzer &
Wirth, 2019)
[Equation]
(19) which is very similar to (17), since they both involve the maximization of a Lipschitz function.
We called this type of formulations KR form s, and the solution to KR forms potential s. We refer readers to Villani (2009) for more historical contents. A. T HEORETICAL CONTRIBUTIONS
The contributions of this work are summarized as follows.
- We present the exact KR form of L M,m in Proposition. - We investigate some properties of the KR form of L M,m . In particular, we show that the potentials of L M,m exist in Sec.
D.1, which is the foundation of the gradient computation in KR form in Sec.
D.5.
Furthermore, some properties of the potentials are proved in Sec.
D.4, including the “ﬂatness” of the potentials, which justiﬁes the ability of our method to discard a fraction of distributions.
We also present the properties of L D,h as analogues of those of L M,m .
- We present the connections between various related formulations in Sec.
D.2 and D.3, including the primal, dual and KR forms of L M,m , L D,h and W 1 .
These results connect our method and WGAN, and they generalize the classic OT theories.
In particular, the c ( h ) -transform connecting the dual and the KR form is the partial version of the classic c -transform, and the resulting Propositionand 7 are analogues of the property of Kantorovich duality (Theorem 5.9 (c) in Villani (2009)).
A. N OTATIONS
- (Ω , d ) : a metric d associated to a compact metric space Ω .
For example, Ω can be a closed cubic in R 3 and d can be the Euclidean distance.
- C (Ω) : the set of continuous bounded function deﬁned on Ω equipped with the supreme norm.
- Lip (Ω) ⊆
C (Ω) : the set of 1-Lipschitz function deﬁned on Ω . - M ( X ) : the space of Radon measures on space X . - π 1# : The marginal of π on its ﬁrst variable.
Similarly, π 2# represent the marginal of π on its second variable. -
Given a function F : X → R ∪ + ∞ , the Fenchel conjugate of F is denoted as F ∗ and is given by:
[Equation]
where X ∗ is the dual space of X and < · > is the dual pairing.
B E XISTING COMPUTATIONAL APPROACHES
The computation of OT problem is an active ﬁeld in machine learning. In this section, we brieﬂy discuss three major classes of approaches and relate our method to the existing ones. One class of the most well-developed OT solver is based on the entropic regularizer.
This class of approaches relax the primal problem by adding an entropy regularizer to the transport plan, then solve the relaxed problem via the Sinkhorn algorithm (Cuturi, 2013).
However, the direct application of the algorithm in Cuturi (2013) has two drawbacks.
First, the entropic bias introduced by the regularizer always leads to undesired behaviors.
Second, the computational cost is high for large scale problems, since the Sinkhorn algorithm iteratively updates the whole transport plan.
Some works have been devoted to address these two issues.
To get rid of the bias, Genevay et al.
(2018; 2019) proposed the
Sinkhorn divergence as an unbiased version of the entropic OT.
To improve the efﬁciency, Schmitzer (2019) proposed some acceleration techniques such as muti-scale computing, and Fatras et al.
(2020) proposed to consider the mini-batch OT problem to avoid the computation of the complete transport plan.
The generalization of this type of approaches to unbalanced or partial OT problem was studied in Benamou et al.
; Chizat et al.
; Séjourné et al.
; Fatras et al.
The second class of approaches solve the sliced OT problem (Bonneel et al., 2015; Kolouri et al., 2016).
They avoid computing OT problems in high dimensional space by projecting the distributions onto random 1-dimensional lines, and solving a 1-dimensional OT problem each time.
Due to their simplicity and efﬁciency, this class of approaches have been applied to several ﬁelds in computer vision, such as generative modelling (Deshpande et al., 2018) and texture synthesis (Heitz et al., 2021).
Recently, Bonneel & Coeurjolly (2019) proposed an algorithm for a special case of partial sliced
OT problem, where a small distribution is completely matched to a fraction of a large distribution.
However, this method can not handle the general partial sliced OT problem where both distributions are not completely overlapped.
Our method belongs to the third class of approaches which focus on a speciﬁc type of OT problem: the
Wasserstein-1 type problem.
The foundation of this type of approach is the Kantorovich–Rubinstein duality (17), which claims that the Wasserstein-1 distance can be efﬁciently computed by estimating a Lipschitz potential function.
This property was directly exploited in the popular Wasserstein
GAN model (Arjovsky et al., 2017).
Some works (Lellmann et al., 2014; Schmitzer & Wirth, 2019) generalized this duality to the unbalanced Wasserstein-1 type problem and applied them to imaging problems, but the exact partial Wasserstein-1 problem ((16) with c = d ) has not been considered.
Our method completes these approaches in a sense that we solve the exact partial Wasserstein-1 problem.
Besides, unlike the existing works (Lellmann et al., 2014; Schmitzer & Wirth, 2019) which only handle the distributions in discrete image space, our method handles distributions in the continuous space, thus it is more suitable for applications in machine learning such as point sets registration.
In this section, we ﬁrst deﬁne the distance-type PW discrepancy by reformulating the KR metric.
Then we derive a KR form for the mass-type PW discrepancy 16 by utilizing the equivalence of the primal and the dual form of KR metric.
C. D ISTANCE - TYPE PW DISCREPANCY
First of all, we consider the Lagrangian POT problem by introducing a Lagrange multiplier h >conjugate to the minimal mass constraint in problem 16,
[Equation]
When c ( x, y )
=
d ( x, y ) , we immediately obtain the KR form of this problem by reformulating the
KR metric 18.
This is stated in the following proposition.
Proposition 1 (KR form of L D,h ) .
When c ( x, y )
=
d ( x, y ) , problem 21 can be reformulated as
[Equation]
Proof.
This is a direct result of the equivalence of (19) and (18).
As mentioned earlier, L D,h has a natural explanation that it requires to ﬁnd a optimal plan within the distance h , we thus called the formulation 22 distance-type PW discrepancy.
We note that the last term − hm β is a constant term, thus can be omitted if one only wishes to compute the potential f .
We deﬁne a functional associated to (22) as follows.
Deﬁnition 1 ( L D,h ) .
Deﬁne a functional L D,h associated to (22) as
[Equation]
With this deﬁnition, problem 22 becomes
[Equation]
Finally we remark that the maximizer of L α,βD,h exists.
Proposition 2 (Existence of optimizer Schmitzer & Wirth (2019)) .
For α, β ∈ M + (Ω) and h > 0 , there exists f ∈ C (Ω) such that L D,h ( α, β ) =
L α,βD,h ( f ) .
Proof.
This proposition can be seen as a special case of Proposition 2.11 in Schmitzer & Wirth (2019).
C. M ASS - TYPE PW DISCREPANCY
We now derive the KR form of L M,m ( α, β ) .
To this end, we ﬁrst derive the dual forms of L M,m ( α, β ) and L D,h ( α, β ) , which reveal a simple relation between these two problems.
By incorporating the
KR form 22 of L D,h ( α, β ) into this relation, we immediately obtain the KR form of L M,m ( α, β ) .
To begin with, we present the dual form of L M,m ( α, β ) and L D,h ( α, β ) in the following two propositions.
Proposition 3 (Dual form of L M,m ) .
Problem 16 can be equivalently expressed as
[Equation]
where the feasible set R is
[Equation]
In addition, the inﬁmum in Problem 16 is attained.
Proof.
We prove this proposition via Fenchel-Rockafellar duality.
We ﬁrst deﬁne space
E : C (Ω) ×
C (Ω) × R , space F : C
(Ω × Ω) , and a linear operator A : E → F as
[Equation]
Then we introduce a convex function H : F → R ∪ + ∞ as H ( [Equation]
[Equation]
∞ else
[Equation]
26) and L : E
→ R ∪ + ∞ as
[Equation]
We can check when f ≡ g ≡ 1 and h = − 1 , H is continuous at A ( f , g , h ) .
Thus by FenchelRockafellar duality, we have
[Equation]
We ﬁrst compute the Fenchel dual H ∗ ( − π ) and L ∗ ( A ∗ π ) in the right-hand side of (28).
For arbitrary π ∈ M (Ω × Ω) , we have
[Equation]
It is easy to see that if π is a non-negative measure, then this supremum is (cid:82) cdπ , otherwise it is + ∞ .
Thus
[Equation]
else (29)
Similarly, we have
[Equation]
If ( α − π 1# ) and ( β − π 2# ) are non-negative measures, and π (Ω × Ω) − m ≥ 0 , this supremum is 0 , otherwise it is + ∞ .
Thus
[Equation]
In addition, the left-hand side of (28) reads
[Equation]
[Equation]
Finally
[Equation]
by inserting these terms into (28) [Equation] we have
[Equation]
which proves (23).
In addition
[Equation]
we can also check right-hand side of (28) is ﬁnite, since we can always construct independent coupling (cid:101) [Equation] = [Equation]
β (Ω) β , such that (cid:101) π ∈ M + (Ω × Ω) , (cid:101) π 1# = mα (Ω) α ≤ α , (cid:101) π 2# = mβ (Ω) β ≤ β and (cid:101) π (Ω × Ω) = m .
Thus the Fenchel-Rockafellar duality suggests the inﬁmum is attained.
Proposition 4 (Dual form of L D,h ) .
Problem 21 can be equivalently expressed as
[Equation]
where the feasible set is
[Equation]
In addition, the inﬁmum in Problem 21 is attained.
Proof.
This proposition can be proved in similarly to Proposition 3, so we omit the proof here.
By comparing Proposition 4 with Proposition 3, we can see that L D,h and L M,m are related by
[Equation]
which suggests that L M,m can be regarded as L D,h whose parameter h is chosen adaptively according to m .
Thus we can derive the KR form for L M,m by inserting (22) into (33).
Formally, we have the following theorem.
Theorem 3 (KR form of L M,m ) .
When c ( x, y )
=
d ( x, y ) , problem 16 can be reformulated as
[Equation]
or equivalently as
[Equation]
Proof.
We obtain equation 34 by inserting (22) into (33) and merging two supremums.
As for equation 35, note that given a ﬁxed f ∈ C (Ω) in problem 34, the optimal h is simply − inf( f ) < + ∞ .
So we can replace h by − inf( f ) in problem 34 to obtain problem 35.
We deﬁne functionals associated to problem 34 and 35 as follows.
Deﬁnition 2 ( L M,m and L M,m ) .
Deﬁne functional L M,m associated to problem 34 as L α,βM,m [Equation] f ,
[Equation]
else, [Equation] 36) and functional L M,m associated to problem 35 as
[Equation]
With these notations, problem 34 and problem 35 are simpliﬁed as
[Equation]
and
[Equation]
respectively
[Equation] For clarity, we summarize the equivalent formulations of L M,m and L D,h discussed in this work in
Table.
We note that there are two equivalent KR forms of L M,m , and we use both of them in our theories.
However, the inﬁmum in L M,m is not easy to handle in practice, thus we only use L M,m in our algorithm.
[Table]
[Equation]
D P ROPERTIES
In the previous section, we have proposed the KR forms of L M,m and L D,h , which are the foundations of our algorithm.
To provide deeper understandings of these formulations, we investigate their properties in this section.
Speciﬁcally, we ﬁrst prove the solutions to the KR forms indeed exist.
Then we discuss the relations amongst some equivalent formulations.
We further discuss the connections amongst L M,m , L D,h and W 1 .
Finally, we present some more in-depth properties regarding the potentials of the KR forms, and the differentiability of the KR forms.
D. E XISTENCE OF OPTIMIZERS
As stated in Proposition 2, the maximizer of L α,βD,h exists.
In this section, we prove that the maximizer of L α,βM,m also exists.
Lemma 1 (Continuity) .
L α,βM,m is continuous on C (Ω) × R .
Proof.
Let ( f n , h n ) → ( f , h ) in C (Ω) × R .
Assume L α,βM,m ( f n , h n ) >
−∞ when n is sufﬁciently large.
We ﬁrst check L α,βM,m ( f , h ) >
−∞ as follows.
For arbitrary (cid:15) > 0 , there exists N > 0 such that for n > N , h n < h + (cid:15) , thus f n > − h n > − h − (cid:15) .
By taking n → ∞ , we see for arbitrary (cid:15) > 0 , f > − h − (cid:15) , which suggests f ≥ − h .
In addition, it is easy to see f ≤ 0 and h ≥ 0 .
It is also easy to see Lip ( f ) ≤ 1 due to the closeness of Lip (Ω) .
Thus according to the deﬁnition 36, we claim L α,βM,m ( f , h ) >
−∞ .
Furthermore, since − h − (cid:15) <
f n < 0 , and f n → f , by dominated convergence theorem, we have
[Equation]
Note we have h n ( m − m β ) → h ( m − m β ) .
We conclude the proof by combining these three terms and obtaining L α,βM,m ( f n , h n ) →
L α,βM,m ( f , h ) .
Proposition 5 (Existence of optimizers) .
For α, β ∈ M + (Ω) and m > 0 , there exists f ∈ C (Ω) and h ∈ R such that L M,m ( α, β ) =
L α,βM,m ( f , h ) .
Proof.
If we can ﬁnd a maximizing sequence ( f n , h n ) that converges to ( f , h ) ∈ C (Ω) × R , then Lemma 1 suggests that L M,m ( α, β ) =
sup f ,h L α,βM,m ( f , h ) =
lim n →∞ L α,βM,m ( f n , h n )
=
L α,βM,m ( f , h ) , which proves this proposition.
Therefore, we only need to show that it is always possible to construct such a maximizing sequence.
Let ( f n , h n ) be a maximizing sequence.
We abbreviate max( f n ) = max x ∈ Ω ( f n ( x )) and min( f n ) = min x ∈ Ω ( f n ( x )) .
We ﬁrst assume f n does not have any bounded subsequence, then there exists N > 0 , such that for all n > N , min( f n ) < − diam (Ω) (otherwise we can simply collect a subsequence of f n bounded by diam (Ω) ).
We can therefore construct (cid:102) f n = f n − (min( f n ) + diam (Ω)) and (cid:102) h n = h n + (min( f n ) + diam (Ω)) .
Note that max( (cid:102) f n ) ≤ min( (cid:102) f n ) + diam (Ω) =
− diam (Ω) + diam (Ω) = 0 , (cid:102) f n + (cid:102) h n = f n + h n ≥ 0 , and (cid:102) f n ∈ Lip (Ω) , so L α,βM,m ( (cid:102) f n , (cid:102) h n ) >
−∞ , and
[Equation]
[Equation]
which suggests that ( (cid:102) f n , (cid:102) h n ) is a better maximizing sequence than ( f n , h n ) .
Note (cid:102) f
n is uniformly bounded by diam (Ω) because 0 ≥ (cid:102) f n ≥ min( (cid:102) f n ) = − diam (Ω) .
As a result, we can always assume (cid:102) h n is also bounded by diam (Ω) .
Because otherwise we can construct h n = − min( (cid:102) f n ) ≤ diam (Ω) , and it is easy to show ( (cid:102) f n , h n ) is a better maximizing sequence than ( (cid:102) f n , (cid:102) h n ) .
In summary, we can always ﬁnd a maximizing sequence ( f n , h n ) , such that both f n and h n are bounded by diam (Ω) .
Finally, since f n is uniformly bounded and equicontinuous, f n converges uniformly (up to a subsequence) to a continuous function f .
In addition, h n has a convergent subsequence since it is bounded.
Therefore, we can always ﬁnd a maximizing sequence ( f n , h n ) that converges to some ( f , h ) ∈ C (Ω) × R , which ﬁnishes the proof.
Remark The proof of Proposition 5 is an analogue of Proposition 2.11 in Schmitzer & Wirth (2019).
The difference is that in our Proposition 5, besides f , we need to handle another variable h acting as the lower bound of f .
In contrast, Proposition 2.11 in Schmitzer & Wirth (2019) only handles the ﬁxed h .
D. C ONNECTIONS BETWEEN THE PRIMAL FORMS , THE KR FORMS AND THE DUAL FORMS
We begin with the discussion of the connection between the dual form and the KR form for L M,m .
Speciﬁcally, we have the following proposition.
Proposition 6 (KR-dual connection for L M,m ) .
Assume c
( x, y )
=
d ( x, y ) .
If ( f , h ) is a maximizer of L α,βM,m , then ( f , − f − h, h ) and ( − f − h, f , h ) are solutions to the corresponding dual form 23.
If ( f , g , h ) is a solution to the dual form 23, then ( g c ( h ) , h ) and ( − h − f c ( h ) , h ) are the maximizers of the corresponding L α,βM,m , where g c ( h ) is the c ( h ) -transformed g , and is deﬁned as
[Equation]
The proof of the second part of Proposition 6 requires a few lemmas regarding the c ( h ) -transform.
Lemma 2.
Given functions f and g deﬁned on Ω , we always have | inf f − inf g | ≤ sup | f − g | .
Lemma 3.
When c ( x, y )
=
d ( x, y ) , given a function f ≤ 0 and h ∈ R + , f c ( h ) has the following two properties, 1) − h ≤ f c ( h ) ≤ 0 . 2)
f c ( h ) ∈
Lip (Ω)
Proof.
On one hand, it is easy to see f c ( h ) ≤ 0 by deﬁnition.
On the other hand, by inserting d ( x, y ) ≥ 0 and f ≤ 0 into (40), we have
[Equation]
2) Considering the function f = inf x (cid:16) d ( x, y ) −
h − f ( x ) (cid:17) , we always have f ∈ Lip (Ω) , because for arbitrary y 1 , y 2 ∈ Ω ,
[Equation]
[Equation]
triangular inequality [Equation]
Furthermore, it is straightforward to check that
[Equation]
which suggests f c ( h ) ∈
Lip (Ω) .
Lemma 4.
Assume c ( x, y )
=
d ( x, y ) .
Given a function f satisfying the following two properties 1) − h ≤ f ≤ 0 , 2) f ∈ Lip (Ω) , the following two statements hold.
1) f c ( h ) can be explicitly written as f c ( h ) =
−
h − f . 2)
There exists a function g , such that f = g c ( h ) .
Proof.
Since f ∈ Lip (Ω) , for arbitrary x, y ∈ Ω , f ( y ) − d ( x, y ) ≤ f ( x ) ≤ f ( y )
+ d ( x, y ) .
Therefore,
[Equation]
Not the last equation holds because − f − h ≤ 0 .
Similarly, we have
[Equation]
d ( x, y ) −
h − f ( x ) (cid:17) , 0 (cid:111) ≤ min (cid:110) inf x (cid:16) d ( x, y ) −
h − f ( y )
+ d ( x, y ) (cid:17) , 0 (cid:111)
= min (cid:110) − f ( y ) − h, 0 (cid:111) =
− f ( y ) − h. Therefore, we have f c ( h ) = − h − f 2)
We construct g = − f − h .
It is easy to check − h ≤ g ≤ 0 and g ∈ Lip (Ω)
According to the ﬁrst part of this proposition, we claim g c ( h )
=
− h − g
= f , which concludes the proof.
Now we prove Proposition 6.
Proof.
(1) We only prove ( f , − f − h, h ) is a solution to the dual form 23, and the proof of ( − f − h, f , h ) is similar.
First, we check f ≤ 0 , − f − h ≤ 0 and h ≥ 0 .
In addition, we have
[Equation]
where the ﬁnal inequality holds due to f ∈ Lip (Ω) .
Thus ( f , − f − h, h ) ∈ R .
Finally, by re-arranging the terms, we have
[Equation]
By assumption, ( f , h ) is the maximizer of L α,βM,m , thus L α,βM,m ( f , h ) =
L M,m ( α, β ) , which completes our proof.
(2) We only prove ( − h − f c ( h ) , h ) is a maximizer of L α,βM,m , and the proof of ( g c ( h ) , h ) is similar.
For a solution ( f , g , h ) ∈ R to the dual form 23, it is easy to check ( f , f c ( h ) , h ) ∈ R .
Because we have f c ( h ) ∈
Lip (Ω) ⊆
C (Ω) , f c ( h ) ≤ 0 according to Lemma 3, and d ( x, y ) − h − f − f c ( h ) ≥by deﬁnition.
In addtion, for arbitrary x, y ∈ Ω , we have g ( y ) ≤
d ( x, y ) −
h − f ( x ) , thus g ( y ) ≤ inf x { d ( x, y )
−
h − f ( x ) } .
Combining this and the fact g ≤ 0 , we obtain g ( y ) ≤ f c ( h ) .
Since β is a non-negative measure, we have
[Equation]
which suggests ( f , f c ( h ) , h ) is also a solution to the dual form 23.
By repeating this argument, we claim (( f c ( h ) )
c ( h ) , f c ( h ) , h ) is also a solution.
According to Lemma 3, − h ≤ f c ( h ) ≤and f c ( h
) ∈ Lip (Ω) , therefore Lemma 4 suggests ( f c ( h ) )
c ( h ) =
− f c ( h ) − h , i.e ., ( − f c
( h ) − h, f c ( h ) , h ) =
(( f c ( h ) )
c ( h ) , f c ( h ) , h ) is a solution,
In other words,
[Equation]
Furthermore, since ( − f c ( h ) − h ) ∈ Lip (Ω) and − h ≤ ( − f c ( h ) − h ) ≤ 0 , the right-hand side of this equations equals L α,βM,m ( − f c ( h ) − h, h ) , which ﬁnishes our proof.
Remark It is worth noticing that Proposition 6 actually provide another way to construct the respective KR forms of L M,m and L D,h without the aid of KR metric.
Actually, the proof of
Proposition 6 is an analogue of the classic Kantorovich-Rubinstein duality, where the role played by c ( h ) -transform in our theory is similar to the role played by c -transform in classic OT literatures, e.g ., Villani (2009).
We note that from the proof of Proposition 6, we can see that the solution to the dual form can be always written as ( f , f c ( h ) , h ) , where ( f , h ) are the only two unknown variables.
We now present the relations between the dual form and the primal form.
Proposition 7 (Dual-primal connection for L M,m ) .
Let ( f , f c ( h ) , h ) be a solution to the dual form 31, and π be the solution to the corresponding primal form 16.
We have
[Equation]
Proof.
According to the equivalence of the dual and primal form, we have
[Equation]
Since f ≤ 0 and π 1# ≤ α , we have (cid:82) Ω f dα ≤ (cid:82) Ω f dπ 1# .
Similarly, we have (cid:82) Ω f c
( h ) dβ ≤ (cid:82) Ω f c ( h ) dπ 2# .
In addition, we have π (Ω × Ω) ≥ m and h ≥ 0 , so we have hm ≤ (cid:82) Ω × Ω hdπ .
By inserting these inequalities into the above equation, we have
[Equation]
By re-arranging this inequality, we have
[Equation]
In addition, since ( f , f c ( h ) , h ) ∈ R , we have c ( x, y ) − f ( x ) − f c ( h ) ( y ) − h ≥ 0 , thus
[Equation]
which suggests the equality is obtained, and ﬁnishes the proof.
As a direct consequence, we obtain the connection between the primal and the KR form for L M,m .
Corollary 1 (Primal-KR relation for L M,m ) .
Assume c
( x, y )
=
d ( x, y ) .
Let ( f , h ) be a maximizer of L M,m , and π be the solution to the corresponding primal form 16.
We have
[Equation] [Equation]
Proof.
According to Lemma 4, we have f c ( h ) =
− h − f .
Thus this corollary is a direct result of
Proposition 7.
We now present the results for L D,h .
We omit the proofs for these results since they can be proved in a similar way as their counterparts for L M,m .
Proposition 8 (KR-dual connection of L D,h ) .
Assume c
( x, y )
=
d ( x, y ) .
If f is a maximizer of L α,βD,h , then ( f , − f − h, h ) and ( − f − h, f , h ) are solutions to the corresponding dual form 31.
If ( f , g , h ) is a solution to the dual form 31, then g c ( h ) and − h − f c ( h ) are maximizers of the corresponding L α,βD,h , where ( · ) c ( h ) is the c ( h ) -transform deﬁned in (40) ,
Proposition 9 (Dual-primal connection for L D,h ) .
Let ( f , f c ( h ) ) be a solution to the dual form 31, and π be the solution to the corresponding primal form 16.
We have
[Equation]
Proposition 10 (Primal-KR relation for L D,h ) .
Assume c
( x, y )
=
d ( x, y ) .
Let f be a maximizer of
L D,h , and π be the solution to the corresponding primal form 16.
We have
[Equation]
[Equation]
D. C ONNECTIONS BETWEEN L M , L D AND W
We ﬁrst explain the relations between the KR forms of L M,m and L D,h , and then we discuss their relations to W 1 .
As can be seen in (33), the maximizers of L α,βD,h recovers to that of L α,βM,m when h is chosen properly.
We re-state this observation formally in the following proposition.
Proposition 11 (Relations between L M and L D ) .
Let f be a maximizer of L α,βM,m .
For the ﬁxed h ∗ = − inf( f ) , f is also a maximizer of L α,βD,h ∗ .
Proof.
First, it is easy to check that by deﬁnition, for arbitrary f ∈ C (Ω) , L α,βD, − inf( f ) ( f ) − inf( f ) m =
L α,βM,m ( f ) .
Then we proceed by contradiction.
If f is not a maximizer of L α,βD,h ∗ , then there exists f ∈ Lip (Ω) such that L α,βD, − inf( f ) ( f )
>
L α,βD, − inf( f ) ( f ) >
−∞ , thus inf( f ) ≥ inf( f ) , and we have
[Equation]
which contradicts the assumption that f is a maximizer of L α,βM,m .
Now we turn to the special cases of L M,m and L D,h .
To begin with, we deﬁne two KR forms as follows.
Deﬁnition 3 ( L α,βP ) .
Deﬁne the functional L α,βP as
[Equation]
and deﬁne L P ( α, β )
= sup f ∈ C (Ω) L α,βP ( f ) .
Deﬁnition 4 ( W α,β 1 ) .
Deﬁne the functional associated to Wasserstein-1 metric W 1 as
[Equation]
thus W 1 can be expressed as W 1 ( α, β ) =
sup f ∈ C (Ω) L α,βW ( f ) .
We note the maximizer of L α,βW is known to exist due to Kantorovich–Rubinstein duality.
The following proposition describes the relations between L M,m , L D,h , L P and W 1 .
Proposition 12 (Special cases of L M,m and L D,h ) .
(1) When m α ≥ m β
= m , L M,m can be equivalently expressed as L P .
(2) When m α ≥ m β and h ≥ diam (Ω) , L D,h can be equivalently expressed as L P − hm β .
(3) When m α = m β , L P becomes equivalent to W 1 deﬁned in (17)
Proof.
(1) It is trivial to check when m α ≥ m β
= m , L M,m becomes L P .
(2) Let f be a maximizer of L α,βD,h .
We only need to prove that f is also a maximizer of L α,βP , because this implies L D,h ( α, β )
=
L α,βD,h ( f )
=
L α,βP ( f ) − hm β =
L P ( α, β ) − hm β .
We proceed by contradiction.
Assume there exists f such that L α,βP ( f ) >
L α,βP ( f ) .
We construct (cid:101) f
= f − sup( f ) .
Since m α ≥ m β , we have L α,βP ( (cid:101) f ) ≥ L α,βP ( f ) >
L α,βP ( f ) .
In addition, since (cid:101) f ∈ Lip (Ω) , we have inf( (cid:101) f ) ≥ sup( (cid:101) f ) − diam (Ω) = − diam (Ω) ≥ − h , thus L α,βD,h
( (cid:101) f ) >
−∞ and L α,βD,h ( (cid:101) f )
=
L α,βP ( (cid:101) f ) − hm β >
L α,βP ( f ) − hm β
=
L α,βD,h ( f ) , which contradicts the assumption that f is a maximizer of L α,βD,h .
(3) Let f be a maximizer of L α,βW .
We construct (cid:101) f
= f − sup( f ) .
It is easy to see L α,βW ( f ) =
L α,βW ( (cid:101) f ) , thus (cid:101) f is also a maximizer of L α,βW .
We claim (cid:101) f is a maximizer of L α,βP , because otherwise there exists f such that L α,βP ( f ) >
L α,βP ( (cid:101) f ) , thus L α,βW ( f ) =
L α,βP ( f ) > L α,βP ( (cid:101) f )
=
L α,βW ( (cid:101) f ) , which contradicts the assumption that (cid:101) f is a maximizer of L α,βW .
As a result, we have W 1 ( α, β ) =
L α,βW ( (cid:101) f )
=
L α,βP ( (cid:101) f )
=
L P ( α, β ) .
Remark The signiﬁcance of Proposition 12 is in two folds.
On the one hand, it provides a new
KR form L P for the “semi-complete” Wasserstein problem, where all mass of β is transported to α .
This problem has many practical applications, such as the template matching problem, where all data points in an incomplete “data distribution” is matched to a complete “model distribution”.
Thus
L P may be interesting on its own in those applications.
We note Fig. 9(c) is an example of this case, because all mass in β is transferred.
On the other hand, Proposition 12 clearly suggests that both
L M,m and L D,h generalize W 1 .
We state it formally in the following corollary.
Corollary 2.
(1) When m α = m β
= m , L M,m is equivalent to W 1 .
(2) When m α = m β and h ≥ diam (Ω) , L D,h is equivalent to W 1 − hm β .
D. P ROPERTIES OF THE POTENTIALS
This subsection characterizes the maximizers of L M,m and L D,h in details.
We always assume c
= d in this subsection.
D.4. M ASS - TYPE PW DISCREPANCY
First of all, it is easy to see that the maximizers of L M,m generally lie on the boundary of the feasible set.
Formally, we have the following proposition.
Proposition 13.
Let f denote a maximizer of L M,m .
(1) If m α > m , sup( f ) = 0 .
If L M,m ( α, β ) > 0 , Lip ( f ) = 1 .
Proof.
The proof is straightforward.
(1) If sup( f ) < 0 , we construct ˜ f
= f − sup( f ) .
It is easy to check L M,m ( ˜ f ) >
−∞ and L M,m ( ˜ f ) − L M,m ( f ) = − sup( f )( m α − m ) > 0 , which contradicts the optimality of f .
(2) Similarly, if Lip ( f ) < 1 , we can construct ˜ f
= f /Lip ( f ) .
It is easy to check L M,m ( ˜ f ) >
−∞ and L M,m ( ˜ f )
=
L M,m ( f ) /Lip
( f ) > L M,m ( f ) , which contradicts the optimality of f .
We then discuss the symmetry of L M,m .
Since the functional L M,m is not symmetric w.r.t.
α and β , i.e ., L α,βM,m ( f ) (cid:54) =
L β,αM,m ( f ) , it is natural to ask what is the relation between the maximizer of L α,βM,m and that of L β,αM,m and whether we have L M,m ( α, β ) =
L M,m ( β, α ) .
These questions are answered in the following proposition.
Proposition 14 (Symmetry) .
Let f denote a maximizer of L α,β M,m .
˜
f
= − f + inf( f ) is a maximizer of L β,αM,m , In addition, we have L M,m ( α, β ) =
L M,m ( β, α ) .
Proof.
For the easy of notation, we abbreviate L α,β
= L α,βM,m .
Consider f (cid:48)
=
f − sup( f ) .
It is easy to check L α,β ( f (cid:48) )
≥ L α,β ( f ) , thus f (cid:48) is also a maximizer of L α,β .
Now we prove L β,α ( ˜ f ) =
L α,β ( f (cid:48) ) as follows.
First, we have L β,α ( ˜ f ) >
−∞ , because ˜ f ≤ − inf( f ) + inf( f ) = 0 , and
Lip ( ˜ f ) =
Lip ( f ) ≤ 1 .
Then, since inf( ˜ f ) = − sup ( f ) + inf( f ) = inf( f (cid:48) ) , we have
[Equation]
Finally, we complete the proof by contradiction.
If ˜ f is not a maximizer of L β,α , then there exists f ∗ such that L β,α ( f ∗ ) >
L β,α ( ˜ f ) .
By the above proof, we can construct f = − f ∗ + inf( f ∗ ) such that
L β,α ( f ∗ ) =
L α,β ( f ) , thus L α,β ( f ) =
L β,α ( f ∗ ) > L β,α ( ˜ f )
=
L α,β ( f [Equation] ) , which contradicts the optimality of f (cid:48) .
In addition, since we have proved ˜ f is a maximizer of L β,α , we have L M,m ( β, α ) =
L β,α ( ˜ f )
=
L α,β ( f (cid:48) )
=
L M,m ( α, β ) .
Finally, we characterize the “inactive” mass of L M,m .
One important property of L M,m is that it has some “inactive” mass that is not transported.
A direct observation is that if we throw away some inactive mass, the solution to the problem will not change.
This observation is formally stated as follows.
Proposition 15 (Inactive mass) .
Let π be the solution to the primal form of L M,m ( α, β ) .
Deﬁne α (cid:48) and β (cid:48) be measures satisfying
[Equation]
for an arbitrary measurable set A .
(1) π is also the solution to the primal form of L M,m ( α (cid:48) , β (cid:48) ) , thus L M,m ( α, β ) =
L M,m ( α (cid:48) , β (cid:48) ) .
(2) Let f be a maximizer of L α,βM,m .
Then f is also a maximizer of L α (cid:48) ,β (cid:48)
M,m .
Proof.
(1) We ﬁrst note π is feasible for L M,m ( α (cid:48) , β (cid:48) ) by construction, because π 1# ( A ) ≤ α (cid:48)
( A ) and π 2# ( A ) ≤ β (cid:48)
( A ) for an arbitrary measurable set A , and π (Ω × Ω) ≥ m .
Then we show π is the solution to L M,m ( α (cid:48) , β (cid:48) ) by contradiction.
Assume there exists a transport plan π (cid:48) that is better than π for L M,m ( α (cid:48) , β (cid:48) ) .
It is easy to see π (cid:48) is a feasible transport plan for L M,m ( α, β ) , because for an arbitrary measurable set A , π (cid:48) 1 # ( A ) ≤ α (cid:48)
( A ) ≤ α ( A ) , and similarly, π (cid:48) 2 # ( A ) ≤ β ( A ) .
Thus π (cid:48) is better than π for L M,m ( α, β ) , which contradicts the optimality of π .
Therefore, we conclude that π is the solution to L M,m ( α (cid:48) , β (cid:48) ) , and the equality of the transport costs follows.
(2) First, we show that f is a maximizer of L α (cid:48) ,βM,m .
We note f is in the feasible set of L α (cid:48) ,βM,m , thus we have
[Equation]
where the inequality holds because f ≤ 0 and α (cid:48) ≤ α .
In addition, according to the ﬁrst part of this proof, we have L M,m ( α (cid:48) , β ) =
L M,m ( α, β ) , which implies L α (cid:48) ,βM,m ( f ) ≥ L M,m ( α (cid:48) , β ) .
Thus we conclude that f is a maximizer of L α (cid:48) ,βM,m .
Then, according to the symmetry stated in Proposition 14, − f + inf( f ) is a maximizer of L β,α (cid:48)
M,m .
By repeating the argument above, we claim − f + inf( f ) is a also maximizer of L β (cid:48) ,α (cid:48)
M,m .
Finally, by using the symmetry again, we claim f is a maximizer of L α (cid:48) ,β (cid:48)
M,m .
Proposition 15 has several important consequences.
In particular, it implies the KR potentials are “ﬂat” on the inactive mass.
Formally, we have the following proposition.
Proposition 16 (Flatness of potentials) .
Let f be a maximizer of L α,βM,m , and π be the solution to the primal form of L α,βM,m .
Let α S and β S be measures satisfying
[Equation]
Then f =
α S -almost surly, and f = − inf( f ) β S -almost surly.
Proof.
According to Proposition 15, we claim that L α − α S ,β M,m =
L α,βM,m , and f is a maximizer of
L α − α S ,β M,m .
In other words, we have
[Equation]
By cleaning this equation, we obtain (cid:82) Ω f dα S = 0 .
Since f ≤ 0 and α S is a non-negative measure, we conclude that f =
α S -almost surely.
The statement for β S can be proved in a similar way.
We can further characterize the inactive region where the mass is inactive.
To begin with, we formally deﬁne the inactive region as follows
Deﬁnition 5 (Inactive region) .
A set A is inactive if it satisﬁes
[Equation]
where π is a solution to problem 16.
According to Corollary 1, as long as the optimal π transports mass from x to y , we have f ( x ) =
f ( y )
+ d ( x, y ) almost surely.
Thus, when a point x in α is not related to any point in β in this way, it must be inactive.
Formally, we have the following proposition.
Proposition 17 (Inactive region) .
Let f denote a maximizer of L α,βM,m .
Denote set
I [Equation] [Equation]
and
[Equation] [Equation]
Both I α
and I β are inactive.
Proof.
We prove I α is inactivate by contradiction.
If π ( I α × Ω) > 0 , then π ( I α × supp ( β )) =
π ( I α × Ω) > 0 , because π ( I α × (Ω \ supp ( β )))
≤ π (Ω ×
Ω \ supp ( β )))
≤ β (Ω \ supp ( β ))
= 0 .
As a result, for arbitrarily small (cid:15) > 0 , there exists a compact set K ⊆
I α × supp ( β ) such that π ( K ) >
π ( I α × supp ( β )) − (cid:15) > 0 .
Since d ( x, y ) − f ( x ) + f ( y ) is continuous and K is compact, there exists ( (cid:101) x, (cid:101) y
) ∈ K ⊆
I α × supp ( β ) such that inf
[Equation]
x
[Equation]
y [Equation] y ) > Note the last inequality holds by the deﬁnition of I α .
In addition, we have d ( x, y ) − f ( x )+ f ( y )
≥for all ( x, y ) ∈
Ω × Ω because f ∈ Lip (Ω) .
By combining these inequalities, we have
[Equation]
However, this contradicts Corollary 1, which claims that
[Equation]
We thus conclude π ( I α × Ω) = 0 .
In addition, it is easy to check I α ⊆ Ω \ supp ( β ) , since for all x ∈ supp ( β ) , there exists y = x
such that f ( x ) =
f ( y )
+ d ( x, y ) , which suggests x / ∈ I α .
Thus π (Ω × I α ) ≤ β
( I α ) ≤ β (Ω \ supp ( β ))
= 0 .
Finally, we can ﬁnally conclude that I α is inactive.
We omit the proof of I β since it can be proved in a similar way.
Based on Proposition 17, we can construct an inactive region as follows.
Corollary 3 (Boundary of active region) .
Let f denote a maximizer of L α,βM,m .
Denote
[Equation]
and
[Equation]
Both F α and F β are inactive.
Proof.
For arbitrary x ∈ F α and arbitrary y ∈ supp ( β ) , we have f ( y )
+
d ( x, y ) > f ( y ) − inf( f ) ≥ 0 ≥ f ( x ) , thus x ∈ I α .
Therefore, F α ⊆
I α .
According to Proposition 17, I α is inactive, thus so is
F α .
We omit the proof of F β since it can be proved in a similar way.
Intuitively, f is ﬂat in the inactive area described in Corollary 3.
This is formally stated as follows.
Corollary 4 (Flat region of potentials) .
Let f be a maximizer of L α,βM,m .
Deﬁne F α and F β as in
Corollary 3, and the corresponding measures α F and β F as
[Equation]
for an arbitrary measurable set A .
Then f
=
α F -almost surly, and f = − inf( f ) β F -almost surly.
Proof.
We only prove the statement for α F here and the statement for β F can be proved similarly.
According to Proposition 16, we only need to prove π 1# ≤ α − α F ≤ α .
Since the second inequality holds as α F ≥ 0 , we only need to prove the ﬁrst inequality.
In other words, we need to prove
[Equation]
for an arbitrary measurable set A .
For the ﬁrst term in the left hand side, we have π 1# ( A ) =
π 1# ( A ∩ supp ( π 1# )) ≤
α ( A ∩ supp ( π 1# )) .
In addition, since π 1# ( F α ) =
according to Corollary 3, and F α is open, we have F α ∩ supp ( π 1 # ) = ∅ .
Therefore, ( A ∩ supp ( π 1 # )) ∩ ( A ∩ F α ) = ∅ , thus α ( A ∩ supp ( π 1# ))
+ α ( A ∩ F α ) =
α (( A ∩ supp ( π 1# )) ∪ ( A ∩ F α ) ≤ α ( A ) , which ﬁnishes the proof.
Another consequence of Proposition 15 is that it implies L M,m actually solves a W 1 problem on the active measure.
This is formally stated in the following proposition.
Proposition 18 (Reduce to W 1 ) .
Let f be a maximizer of L M,m , and π be the solution to the corresponding primal form of L M,m 16.
(1) π is the solution to the primal form of W 1 ( π 1# , π 2# ) .
(2) f is a maximizer of W 1 ( π 1# , π 2# ) .
Proof.
First, we note that π only transport the necessary amount of mass, i.e ., m = π 1# (Ω) =
π 2# (Ω) =
π (Ω × Ω) , because if m < π (Ω × Ω) , we obtain a better solution π (cid:48)
= m
π (Ω × Ω) π , which contradicts the optimality of π .
Then let α (cid:48)
=
π 1# and β (cid:48)
=
π 2# , Proposition 15 suggests that π is respectively the solution for the primal forms L M,m ( π 1# , π 2# ) =
W 1 ( π 1# , π 2# ) .
In addition, since m = m β (cid:48) , Proposition 15 suggests f is a maximizer of W α (cid:48) ,β (cid:48)
=
L α (cid:48) ,β (cid:48) M,m , which ﬁnishes the proof.
In summary, Proposition 16 and 18 provide an intuitive characterization for L M,m .
Speciﬁcally, solving L M,m can be regarded as solving two sub-problems.
The ﬁrst sub-problem is to identify the active measures and the inactive measures.
The second sub-problem is to perform the complete optimal transport between the active measures, and to discard the inactive measures.
In this sense,
Corollary 2 becomes straightforward, because when both α and β are fully active, L M,m ( α, β ) simply becomes the second sub-problem, i.e ., W
( α, β ) .
D.4. D ISTANCE - TYPE PW DISCREPANCY
Now we discuss the properties of L D,h as analogues of those of L M,m .
We only present the results and omit the proof, since they can be proved in a very similar manner to their counterparts in the previous subsection.
Proposition 19.
Let f denote a maximizer to L α,βD,h .
If m α > m β , then sup( f ) = 0 ; If m β > m α , then inf( f )
= − h .
Proposition 20 (Symmetry) .
Let f denote a maximizer to L α,βD,h .
˜
f
= − f − h is a maximizer of
L β,αD,h .
In addition, we have L D,h ( α, β ) =
L D,h ( β, α ) .
Proposition 21 (Inactive mass) .
Let π be the solution to the primal form of L D,h ( α, β ) .
Deﬁne α (cid:48) and β (cid:48) be measures satisfying
[Equation]
for an arbitrary measurable set A .
(1) π is also the solution to the primal form of L D,h
( α (cid:48) , β (cid:48) ) , thus L D,h ( α, β ) =
L D,h ( α (cid:48) , β (cid:48) ) .
(2) Let f be a maximizer of L α,β D,h .
Then f is also a maximizer of L α (cid:48) ,β (cid:48)
D,h .
Proposition 22 (Flatness of potentials) .
Let f be a maximizer of L α,βD,h , and π be the solution to the primal form of L α,βD,h .
Let α S and β S be measures satisfying
[Equation]
Then f =
α S -almost surly, and f = − h β S -almost surly.
Proposition 23 (Inactive region) .
Let f denote a maximizer to L α,βD,h .
Denote set
[Equation]
[Equation]
and
I
[Equation] [Equation]
Both I α
and I β are inactive.
Corollary 5 (Boundary of active region) .
For L D,h , denote
[Equation]
and
[Equation]
[Equation]
Both F α and F β are inactive.
Corollary 6 (Flat region of potentials) .
Let f be a maximizer of L α,βD,h .
Deﬁne F α and F β as in
Corollary 5, and the corresponding measures α F and β F as
[Equation]
for an arbitrary measurable set A .
Then f
=
α F -almost surly, and f = − h β F -almost surly.
Proposition 24 (Reduce to W 1 ) .
Let f be a maximizer of L D,h , and π be the solution to the corresponding primal form of L D,h 21.
(1) π is the solution to the primal form of W 1 ( π 1# , π 2# ) .
(2) f is a maximizer of W 1 ( π 1# , π 2# ) .
D. D IFFERENTIABILITY
Now we consider the differentiability of the PW discrepancy.
We show that, when expressed in the
KR forms, if a mild assumption is satisﬁed, then L M,m and L D,h are differentiable and the gradients can be explicitly computed, which makes it feasible to optimize them via gradient based methods.
We consider a parametrized transformation T θ : R d × Ω → Ω , and denote β θ the corresponding push-forward measure of β .
We show in Proposition 25 and 26 that, if T θ is Lipschitz w.r.t.
θ , then the objective functions L D,h ( α, β θ ) and L M,m ( α, β θ ) is differentiable w.r.t.
θ , and the gradient can be computed explicitly.
Proposition 25 (Differentiability of L D,h ) .
If T θ : Ω → Ω is Lipschitz w.r.t.
to θ , then L D,h ( α, β θ ) is continuous w.r.t.
θ , and is differentiable almost everywhere.
Furthermore, we have
[Equation]
where f is a maximizer of L D,h .
Proof.
We ﬁrst prove L D,h ( α, β θ ) is continuous w.r.t.
θ .
By deﬁnition, we have L D,h ( α, β ) =
KR h ( α, β ) − h 2 ( m α + m β ) .
Thus by triangle inequality of the KR metric, for arbitrary θ, θ (cid:48) ∈
R
d , we have
[Equation]
Note that the second equality holds because T maintains the total mass, i.e ., m β θ (cid:48)
= m β θ = m β .
In addition, we have
[Equation]
By assumption, T θ is Lipschitz w.r.t.
θ , thus there exists a constant ∆ > 0 , such that d ( T θ ( y ) , T θ (cid:48) ( y ))
≤ ∆ || θ − θ (cid:48) || .
Thus KR h ( β θ (cid:48) , β θ ) ≤ m β ∆ ||
θ − θ (cid:48) || .
Finally, we have
[Equation]
which proves L D,h ( α, β θ ) is continuous w.r.t.
θ , and Radamacher’s theorem states that it is differentiable almost everywhere.
The rest of the proof follows that in Arjovsky et al.
The sketch of the proof is as follows.
We ﬁrst derive ∇ θ L D,h ( α, β θ ) as −∇ θ (cid:82) Ω f
( T θ ( x )) dβ using the envelope theorem (Milgrom & Segal, 2002).
Then we prove
[Equation]
when the right hand side of the equation is well deﬁned, which completes our proof.
Proposition 26 (Differentiability of L M,m ) .
If T θ : Ω → Ω is Lipschitz w.r.t.
to θ , then L M,m ( α, β θ ) is continuous w.r.t.
θ , and is differentiable almost everywhere.
Furthermore, we have
[Equation]
(58) where ( f , h ) is a maximizer of L M,m .
Proof.
To begin with, for arbitrary θ, θ (cid:48) , we consider the following two-step procedure that transports m unit mass from α to β θ (cid:48) .
First, we transport m mass from α to β θ according to π , which is a solution to L M,m ( α, β θ ) .
Second, we transport m mass received in β θ to β θ (cid:48)
according to a plan π (cid:48) deﬁned as
[Equation]
else, where (cid:101) π ( T θ ( x ) , T θ (cid:48) ( x )) =
β ( x ) .
Thus we have
[Equation]
Since T θ is Lipschitz w.r.t.
θ , there exists a constant ∆ > 0 , such that d ( T θ ( x ) , T θ (cid:48)
( x )) ≤ ∆ || θ − θ (cid:48) || .
Therefore, the cost of the second step can be bounded as
[Equation]
Denote cost 1 the overall cost of this two-step procedure.
We have
[Equation]
By applying the gluing lemma (Villani (2009) Sec.1) to π and π (cid:48) , we can construct a transport plan (cid:101) π that transports m unit mass from π 1# to π (cid:48) 2# .
Denote cost 2 = (cid:82) Ω × Ω d ( x, y ) d (cid:101) π ( x, y ) the cost of (cid:101) π .
One one hand, we have
L M,m ( α, β θ (cid:48) )
≤ costaccording to the deﬁnition of L M,m .
On the other hand, we have cost 2 ≤ cost 1 .
Because for arbitrary x, z ∈ Ω , the two-step procedure and (cid:101) π transport the same amount of mass between them.
However, the cost of transporting a unit mass from x to z is d ( x, y )
+
d ( y, z ) for a y ∈ Ω for the two-step procedure, but is d ( x, z ) for (cid:101) π , which is cheaper according to triangle inequality.
By combining these inequalities together, we have
[Equation]
By switching θ and θ (cid:48) and repeating the argument, we have
[Equation]
thus we have
[Equation]
which suggests L M,m ( α, β θ ) is continuous w.r.t.
θ , and Radamacher’s theorem states that it is differentiable almost everywhere.
The rest of the proof is similar to that of Proposition 25.
Remark The main idea used in the proofs in this section is based on that in Arjovsky et al.
However, a key difference is that unlike W 1 , neither L M,m nor L D,h is a metric, i.e ., they do not necessarily satisfy triangle inequality, thus the differences cannot be bounded directly.
This gap is bridged by using the KR metric in Proposition 25, and by constructing a transportation plan in
Proposition 26.
E A PPLICATION TO POINT REGISTRATION
In this section, we focus on the discrete distributions and prove the main results in the main text.
We ﬁrst present the proof of the main Theorems in the main text.
Proof of Theorem 1.
The KR formulation of L D,h is given in Proposition 1, and the existence of the optimizer is proved in Proposition 2.
The gradient of L D,h ( α, β θ ) is derived in Proposition 25.
Theorem 1 is proved by applying these results to discrete distributions.
Proof of Theorem 2.
The KR formulation of L M,m is given in Theorem 3, and the existence of the optimizer is proved in Proposition 5.
The gradient of L M,m ( α, β θ ) is derived in Proposition 26.
Theorem 2 is proved by applying these results to discrete distributions.
We ﬁnally verify that the parametrized transformation T θ used in our paper satisﬁes the regularity assumption, i.e ., it is Lipschitz w.r.t.
the parameter θ .
Lemma 5.
If T : R d × Ω → Ω is Lipschitz w.r.t.
to its ﬁrst variable, then for arbitrary y ∈ Y , and θ , θ (cid:48) ∈ R d , there exists a ∆ > 0 , such that
[Equation]
Proof.
The proof is straightforward.
By assumption, for arbitrary y i ∈ Y , we have ||T θ ( y i ) −
T θ (cid:48)
( y i ) ||
≤ ∆ i || θ − θ (cid:48)
|| .
Since Y is ﬁnite, we can simply let ∆ =
max i ( { ∆ i } y i ∈ Y ) .
Proposition 27.
Given a point set Y =
{ y i } ri =1 ⊆ Ω .
Let θ =
( A, t, V ) ∈
R 12+3 r , where A ∈ R 3 ×is a afﬁnity matrix, t ∈ R 1 × 3 is a translation vector, V ∈ R r × 3 is the offset vectors for all points in
Y , and V i represent the i -th row of V .
For each point y i ∈ Y , deﬁne T θ ( y i )
=
y i
A + t
+ V i , where
V i represents the i -th row of V .
T : R d × Ω → Ω is Lipschitz w.r.t.
to its ﬁrst variable.
Proof.
Note for arbitrary A , there exists δ > 0 , such that ||
A || 2 ≤ δ ||
A || F , where || · || F is the
Frobenius norm.
For arbitrary θ =
( A, t, V ) , (cid:101) θ =
( (cid:101) A, (cid:101) t, (cid:101) V ) and
y i ∈ Y , we have
[Equation]
|| 2 , which proves that T is Lipschitz w.r.t.
to its ﬁrst variable.
F M ORE DETAILS OF THE MAIN TEXT F. M ORE DETAILS OF S EC .
In contrast to the probabilistic methods, the correspondence-based methods explicitly estimate the correspondence between point sets, and then compute the transformation accordingly.
The iterative closest point (ICP) algorithm (Besl & McKay, 1992; Zhang, 1994) is one of the well-known methods in this class.
In each iteration, ICP estimates the correspondence based on the nearest distance criterion, and updates the transformation accordingly.
Despite its simplicity, ICP easily fails when the initial point sets are not roughly aligned.
In order to improve the robustness of ICP, several methods (Fitzgibbon, 2003; Granger & Pennec, 2002; Chui & Rangarajan, 2000; Stewart et al., 2003) have been proposed.
In particular, Chui & Rangarajan (2000) proposed to estimate the correspondence between all pairs of points, which improves the robustness at the expense of high computational cost.
Our method is also related to Wasserstein generative network (WGAN)
(Arjovsky et al., 2017), which is a popular method for large scale DM problems.
A recent survey of the applications of
WGAN can be found in Gui et al.
WGAN efﬁciently optimizes the Wasserstein-1 metric by approximating Wasserstein-1 potential using a neural network.
This technique is also used in our method to efﬁciently optimize the PW discrepancy.
In this sense, our method directly generalizes
WGAN to PDM problems.
F. M ORE DETAILS OF S EC . 4.
We present a simple example comparing the primal and the KR forms in Fig. We show the correspondence P obtained by the primal form in the 1-st row, where the black lines link the corresponding points. To estimate the KR forms, we parametrize the potential function f w,h by a neural network shown in Fig.
11, and learn f w,h by maximizing (9) or (10). The 2-nd row visualizes the learned potential function f w,h . The corresponding gradient norms |∇ f w,h | are shown in the 3-rd row. |∇ f w,h | are further visualized in the 4-th row, where the size of each point is approximately proportional to the gradient norm.
[Figure] We further evaluate the precision of the estimated KR forms. For each setting, we compute L M,m or L D,h in the KR forms by inserting the learned potential f w,h back into (9) or (10) (without the gradient penalty term). To obtain the true values of L M,m and L D,h , we compute their primal forms (1) and (2) using the POT tool box (Flamary et al., 2021).
The results are summarized in Tab. 2. As can be seen, our estimated PW discrepancies are close to the true values with the averaged relative error less than 0 . 2% , which we think is sufﬁciently precise for our application. As for the efﬁciency, we note that the KR formulation is not advantageous in this data scale, i.e ., tens of points, where the computation of the exact primal form only takes less than 1 sec on a CPU, while the computation of the KR form takes a few minutes on a mid-end GPU (due to the requirement of learning the potential network).
However, for large scale dataset, e.g ., ∼ 10 6 , which the primal form can not handle because the transport map (of size 10 6 × 10 6 ) does not even ﬁt into the memory, the
KR form can still apply (as validate in our experiments) using a 12G GPU.
Finally, we stress that since the implement of KR form depends on the underlying neural network, both the efﬁciency and precision of the KR forms naturally vary according to the structure of the neural network. This is fundamentally different from the solvers of the primal form, such as the Sinkhorn algorithm or the linear program, which are ﬁxed pipelines. Therefore, it is generally not possible to compare the efﬁciency and precision between the KR solvers and the primal solvers in a more rigorous way.
[Table]
F. M ORE DETAILS OF S EC . 4.
To estimate the gradient of the coherence energy fast, we ﬁrst decompose G as G ≈ Q Λ Q T via the
Nyström method (Williams & Seeger, 2000), where k (cid:28) r , Q ∈ R r × k , and Λ ∈ R k × k is a diagonal matrix.
Then we apply the Woodbury identity to ( σ I + Q Λ Q T ) − 1 and obtain
[Equation]
As a result, the gradient of the coherence energy can be approximated as
[Equation]
F. C ONNECTIONS BETWEEN PWAN AND WGAN
In theory, PWAN includes WGAN as a special case.
Because in the DM setting where distributions are required to be completely matched, i.e ., when m α = m β = m for L M,m , or m α = m β and h > diam (Ω) for L D,h , both L M,m and L D,h become W 1 (Corollary 2), thus the bound constraint of the potential is not needed, i.e ., PWAN becomes WGAN.
In practice, these two methods are only different in the structures of their potential networks. This is summarized in Tab. 3. [Table]
[Equation]
[Table] To provide a more in-depth characterization of the relations between PWAN and WGAN, we consider the toy 1-dimensional example in Fig.
4 again. We compare the potentials of WGAN and PWAN in
Fig.
As can be seen, both methods learn Lipschitz potentials. As discussed in the main text, the PWAN potential permits the existence of some ﬂat regions where gradient norm is 0 .
In contrast, the WGAN potential has gradient norm 1 almost surely (Corollary 1 in Gulrajani et al.
(2017)), i.e ., no ﬂat region exists. This difference explains the different behaviors of these two methods.
Speciﬁcally,
PWAN is a PDM method, because it will discard a fraction of points in β θ within in ﬂat region, while only moving the points outside the ﬂat region toward α (as discussed in the main text). However, WGAN is a DM method, because it will move ALL points in β θ toward α .
We note that PWAN has the following adversarial explanation.
Let us call the points in α real points, and those in β θ fake points.
In the registration process, f w,h is trying to discriminate real and fake points by assigning each of them a score in range [ − h, 0] , where real points have higher scores than fake points.
f w,h is so certain of a fraction of points, that it assigns the highest or lowest possible score ( 0 or − h ) to them and does not change these scores easily.
Meanwhile, T θ is trying to cheat f w,h by moving the fraction of fake points of which f w,h is not certain to obtain higher scores.
In addition, T θ is also trying to move all fake points to keep the coherence energy low.
The process ends when T θ cannot make further improvement on the fake points.
F. E XPERIMENTAL DETAILS
Data synthesis The data synthesis procedure mostly follows Hirose (2021b).
Speciﬁcally, given the original point set X ∈ R r × 3 the deformed point set Y ∈ R r × 3 is generated by:
[Equation]
(59) where (cid:15) is the Gaussian noise following N (0 , 0 . 02) , and V ∈ R r × 3 is the random offset vectors.
To ensure V varies smoothly in the space, we sample column vector V j from a distribution p :
[Equation]
(60) where G ∈ R r × r is a Gaussian kernel deﬁned as G ρ ( i, j ) =
e −|| y i
[Equation]
y j | [Equation]
2 /ρ .
To sample V from ditribution p , we compute V =
G 12 U , where U ∈ R r × 3 is a random matrix, of which each element follows N (0 , 1) , and G 12 ≈ Q Λ 12 is computed via the Nyström method.
We use λ = 10 or 50 and ρ = 2 in our experiments.
Network
structure
The network used in our experiment is a 5-layer point-wise multi-layer perceptron with a skip connection.
The detailed structure is shown in Fig.
Parameters We train the network f w,h using the Adam optimizer (Kingma & Ba, 2014), and we train the transformation T θ using the RMSprop optimizer (Tieleman & Hinton, 2012).
The learning rates of both optimizers are set to 10 − 4 .
For experiments in Sec.
5.3, the parameters as set as follows: For PWAN, we set ( ρ, λ, σ, T ) =
For TPS-RPM, we set T _ finalfac = 500 , frac = 1 , and T _ init = 1 . 5 .
For GMM-REG, we set sigma = 0 .
02 , Lambda = .
01 , max _ function _ evals = 50 , 50 , 100 and level = 3 .
For BCPD and CPD, we set ( β, λ, w ) =
For experiments in Sec.
5.4, the parameters as set as follows: For PWAN, we set ( ρ, λ, σ, T ) =
(1 . 0 , 10 − 3 , 1 . 0 , 3000) .
For BCPD and CPD, we set ( β, λ, w ) =
We also test w = 0 .for
these methods, but the difference is not obvious.
For experiments in Sec. F.9, the parameters as set as follows: For PWAN, we set ( ρ, λ, σ, T ) =
(0 . 5 , 5 × 10 − 4 , 1 . 0 , 3000) .
For BCPD and CPD, we set ( β, λ, w ) =
For experiments in Sec.
F.10, the parameters as set as follows: We use m-PWAN.
we set ( ρ, λ, σ, T ) =
(1 . 0 , 1 × 10 − 4 , 1 . 0 , 3000) .
For BCPD and CPD, we set ( β, λ, w ) = We also tried w = 0 . but the results are similar (not shown in our experiments). We note the parameters for CPD and BCPD are suggested in Hirose (2021a).
Parameter selection
The parameters m in m-PWAN and h in d-PWAN control the degree of alignment.
Speciﬁcally, increasing m or h will lead to the registration of larger fraction of distributions.
When m = 0 or h = 0 , PWAN does not align any point (the gradients are 0 for all points); When m = min { m α , m β } or h = ∞ , PWAN seeks to align the largest fraction of points, i.e ., it becomes a
DM method. Therefore, choosing overly large m or h may lead to the biased registration (like a DM method), while choosing overly small m or h may lead to the insufﬁcient registration where too few points are aligned. For uniform point sets, the parameters m and h can both be determined easily. For d-PWAN, if the averaged nearest distance between points is s , then h can be set near s .
Because for the aligned point sets, the points that are farther from the overlapped region than distance s are likely to be outliers.
For m-PWAN, one needs to estimate the overlap ratio ρ of one point set, e.g ., α , then m can be set as ρm α .
This is how we actually determined m and h in our experiments.
Reﬁnement In our experiments, we ﬁnd that a nearest-point-based reﬁnement sometimes slightly improves the performance.
Speciﬁcally, when the algorithm 1 is near convergence, we can optionally run a few steps of the nearest-point-based reﬁnement to improve the performance slightly.
Speciﬁcally, since algorithm 1 is near convergence, we can safely assume that the point sets are sufﬁciently aligned.
Therefore, each point is highly likely to correspond to its nearest neighborhood in the other set within the mass or distance threshold.
In other words, the correspondence matrix P ∈ R q × r can be written as
[Equation]
(61) for L D,h , and
P i,j = (cid:26) 1 if y j = NN ( x i ) and x i ∈
Nearest ( m ) 0 else (62) for L M,m , where NN () is the nearest neighborhood function, and Nearest ( m ) represent the nearest m points in X to Y .
Thus we can slightly reﬁne the result by switch the divergence term L to
[Equation]
(63) and update a few steps of the transformation T θ via gradient descent.
F. M ORE DETAILS OF S EC . 5.
The KL divergence and the L 2 distance between point set X and Y are formally deﬁned as
[Equation]
where φ ( ·| u, σ ) is the Gaussian distribution with mean u and variance σ .
For simplicity, we set σ =and ω = 0 .
2 for KL and L 2 .
F. M ORE DETAILS OF S EC . 5.
We report the standard deviation (STD) of the MSE in experiments Sec.
5.3 in Tab. 4 and 5.
It can be seen that our results generally have lower STDs than the baseline methods when the outlier ratio is high or when the overlap ratio is low (except for the bunny dataset in the ﬁrst experiment where
GMM-REG has the lowest STD).
This implies that our method is more stable than the baseline methods under these situations.
[Table] We present some results of these two experiments with N = 2000 in Fig.
12 and Fig. 13, and the corresponding quantitative results are shown in Tab. 6 and Tab. 7.
We do not show the results of
TPS-RPM on the second experiment, as it generally fails to converge.
As can be seen, PWAN successfully registers the point sets in all cases, while all baseline methods bias toward to the noise points or to the non-overlapped region when outlier ratio is high, except for TPS-RPM which shows strong robustness against noise points comparable with PWAN in the ﬁrst example.
To obtain a sense of the learned potential network f w,h , we visualize the potential at the end of the registration process.
The results are shown in Fig.
14 and Fig.
We represent the value of potential and the gradient norm at each point by colors, where brighter color indicates higher value.
As can be seen in Fig.
15(c) and 14(c), the network assigns higher values to the points in α while assigning [Table] lower values to the points in β θ .
In addition, as shown in Fig. 15(d) and 14(d), most of outliers, including noise points and the points in non-overlapping region, have low gradients,
i [Table]
e ., they are successfully discarded by the network.
[Table] [Table]
M ORE DETAILS OF S EC . 5.
We provide more details regarding the computation time of our method.
We ﬁrst sample q = r points from the bunny shape, where q = 10 5 or 7 × 10 5 .
We then run both the 1 GPU version and the 2 GPU version of PWAN 100 steps
[Table]
and report their computation time in Fig.
[Table]
where q - M GPU represents registering q points using M GPU.
As can be seen, the majority of time is spent on updating the network [Table] and the parallel training can effectively reduce the computation time for updating the network.
In addition, the gain of speed increase as the number of points, i.e ., we get 1 .
3 × speedup when q = 10 5 , while the speedup is 1 .
9 × when q = 7 × 10 5 , which is close to the theoretical speedup value 2 . [Figure] [Figure]
Finally [Figure] we evaluate PWAN on large scale point sets.
We generate noisy and partially overlapped armadillo datasets as stated in Sec.
We compare PWAN with BCPD and CPD, because they are the only baseline methods that are scalable in this experiment.
We present some registration results in
[Figure]
[Figure]
More training details of PWAN is shown in Fig.
18, Fig.
and Fig.
As can be seen in
Fig. 18(a), 19(a) and 20(a), the source sets are matched to the reference sets smoothly.
Fig.
18(b), 19(b) and 20(b) suggest that at the end of the registration process, most of outliers are correctly discarded. Initial sets [Equation]
[Figure] [Figure]
Initial sets
[Equation] [Figure] [Figure] [Figure]
[Figure]
[Figure]
[Equation]
[Table]
Besides, Fig.
18(c), 19(c) and 20(c) implies that the norm of gradient of the network is indeed controlled below 1 , i.e ., it is indeed Lipschitz.
In addition, both the training loss and the MSE decrease smoothly in the training process.
F. E XPERIMENT ON THE HUMAN FACE DATASET
We evaluate our method in the space-time faces dataset (Zhang et al., 2008), which consists of a time series of point sets sampled from a real human face.
Each face consists of 23 , 728 points and the true correspondence between faces are known.
We use the faces at time i and i + 20 as the source and the reference set, where i = 1 , ..., 20 .
All point sets in this dataset are the same size and are completely overlapped.
The registration results are shown in Tab. 8, where we can see PWAN outperforms both CPD and
BCPD.
We present the examples of the registration results in Fig.
As can be seen, PWAN successfully aligns the faces in different time points. [Figure] [Figure]
[Figure]
E XPERIMENT ON THE 3D HUMAN DATASET
We evaluate our method on a challenging human shape dataset (DataSet), which is taken from a
SHREC’19 track called “matching humans with different connectivity”.
This dataset consists ofshapes and we manually select 3 pairs of shapes for our experiments.
To generate a point set of each shape, we ﬁrst sample 50000 random points from the surface of the 3D mesh and then apply voxel grid ﬁltering to down-sample the point set to less than 10000 points.
The description for the selected point sets is presented in Tab.
We conduct the following two experiments.
In the ﬁrst experiment, we evaluate our method on registering the complete point sets.
In the second experiment, we evaluate our method on the more
[Figure]
[Figure]
[Table]
challenging partial matching problem, where we generate incomplete point sets by manually cropping a fraction of the no.30 and no.31 point sets.
We consider 3 types of partial matching problem, i.e ., match incomplete set to complete set, match complete set to incomplete set and match incomplete set to incomplete set.
For both of these two experiments, we compare our method with CPD and BCPD, and we only present qualitative registration results, because we do not know the true correspondence between point sets.
The results of the ﬁrst experiment is shown in Fig.
As can be seen, PWAN can handle both the local deformations (1-st row) and the articulated deformations (2-nd and 3-rd rows) well, and it produces good full-body registration results.
In contrast, although CPD and BCPD can handle local deformations relatively well (1-st row), they have difﬁculties aligning point sets with large articulated deformations, as signiﬁcant registration errors are observed near the limbs (2-nd and 3-rd rows).
The results of the second experiment is shown in Fig.
As can be seen, both CPD and BCPD fail in this experiment, as the non-overlapping points are seriously biased.
For example, in the 3 -rd row, they both wrongly match the left arm to the body, which causes highly unnatural artifacts.
In contrast, the proposed PWAN can handle the partial matching problem well, since it successfully maintains the shape of non-overlapping regions, which contributions to the natural registration results.
Finally, we note that although the proposed PWAN generally produces reasonable full-body registration results, it has some difﬁculties handling the local details.
For example, the hands in Fig.
22 and
[Figure]
are generally not well aligned.
This drawback might be alleviated by considering local constraints such as Ge et al.
(2014) in the future.
In addition, it is worth noticing that the aligned point sets in the second experiments are natural and do not exist in the original dataset.
These results suggests the potential of our method in other practical tasks such as point set completion and point sets merging.
[Figure]
[Figure]