Exact Shape Correspondence via 2D Graph Convolution
Abstract
For exact 3D shape correspondence (matching or alignment), i.e., the task ofmatching each point on a shape to its exact corresponding point on the other shape(or to be more specific, matching at geodesic error 0), most existing methods donot perform well due to two main problems.
First, on nearly-isometric shapes (i.e.,low noise levels), most existing methods use the eigen-vectors (eigen-functions)of the Laplace Beltrami Operator (LBO) or other shape descriptors to updatean initialized correspondence which is not exact, leading to an accumulation ofupdate errors.
Thus, though the final correspondence may generally be smooth,it is generally inexact.
Second, on non-isometric shapes (noisy shapes), existingmethods are generally not robust to noise as they usually assume near-isometry.
Inaddition, existing methods that attempt to address the non-isometric shape problem(e.g., GRAMPA) are generally computationally expensive and do not generalise tonearly-isometric shapes.
To address these two problems, we propose a 2D graphconvolution-based framework called 2D-GEM.
2D-GEM is robust to noise onnon-isometric shapes and with a few additional constraints, it also addresses theerrors in the update on nearly-isometric shapes.
We demonstrate the effectivenessof 2D-GEM by achieving a high accuracy of 90.5 % at geodesic error 0 on thenon-isometric benchmark SHREC16, i.e., TOPKIDS (while being much faster than
GRAMPA), and on nearly-isometric benchmarks by achieving a high accuracy of92.5 % on TOSCA and 84.9 % on SCAPE at geodesic error 0.
Introduction
3D shapes can undergo different types of deformations.
Deformations can be rigid or non-rigid.
Rigid deformations include rotations, translations, reflections, or any composition of these resultingin pose changes.
They preserve geometric properties of the shape such as segment lengths, angles,volumes, sizes, geodesic distances and others.
Non-rigid deformations do not preserve geometricproperties of the shape, they include dilations, or a composition of dilations and rigid deformations(e.g., stretching, scaling).
In this work we focus on non-rigid shapes that are either nearly-isometric(i.e.
, most geometric properties are preserved), or non-isometric (i.e., geometric properties are notpreserved, and contain strong topological noise).
Non-rigid shape matching is a very important task in machine learning, shape analysis and computervision.
It has important applications such as shape registration, comparison, recognition, and retrieval.3D shapes are usually represented as 2D manifolds embedded in R 3 .
Given two three dimensionalshapes M 1 and M 2 with n vertices each, the goal of non-rigid shape matching is to find a meaningfulcorrespondence T : M 1 → M 2 , where T is (a) bijective, (b) continuous in both directions in thesense that nearby points on M 1 should be matched to nearby points on M 2 (vice versa), and (c)similar points should be matched to each other [39].
In the discrete setting the correspondence (mapor match) T can be represented as a permutation matrix P ∈ { 0 , 1 } n × n [51, 4].
In more details, 3D shapes are usually represented using triangulations (e.g., the Delauney triangulation [18]) and xyz-coordinates representations in R 3 [77, 39].
It is hard to utilize these xyz-coordinatesand triangulations to directly perform 3D shape matching because of: (a) embedding ambiguities,i.e. shapes sharing the same metric (isometry) or very similar metrics (nearly-isometric) can havedrastically different xyz-coordinates representations in R 3 due to pose changes [77], and (b) nonisometric shapes with strong topological noise will have even more different xyz-coordinates andtriangulations.
As such, these 3D-coordinates together with the triangulations are usually used to extract point-wisedescriptors (which encode the similarity between points), and pair-wise descriptors (which encodeglobal and/or local relations between pairs of points).
Once these descriptors are built, the shapematching problem is reduced to an alignment problem over (a) the point-wise descriptors , or (b) thepair-wise descriptors , or (c) both (joint alignment) .
The joint alignment problem can be written as:
[Equation]
(1) where Π is the set of permutation matrices, E is the generalised energy, F is the energy over pointwise descriptors , H is the energy over pair-wise descriptors , and α is a trade-off parameter [39].
Several algorithms have been proposed to address point-wise alignment, [4, 67, 11, 52, 61], pairwise alignment [18, 16], as well as joint alignment [76, 39, 51, 55].
Due to space constraints, see
Appendix A for more details on the different alignments as well as related works.
To measure the performance of an algorithm on the 3D shape matching task, the geodesic erroris usually used as the error metric [18, 77].
Given that a point x on shape M 1 is mapped by analgorithm to y as its correspondence on shape M 2 , the geodesic error measures how far the matchy is from the true matching position for x on M 2 i.e., y ∗ (given by the true map).
When the mapreturned by algorithm is exact, it significantly improves performance and efficiency of other taskssuch as registration, comparison, motion transfer, style transfer.
However, most methods proposed ineach of the aforementioned categories for 3D shape matching do not perform well at geodesic error 0(i.e.
, the task of exact matching where y is the same point y ∗ ).
This is mainly due to two factors.
[Figure]
First, on non-isometric shapes, given that most existing methods assume near-isometry, they cannothandle noisy shapes (e.g., shapes with strong topological noise such as mesh “gluing” in areas ofcontact as in the TOPKIDS dataset [39]), since in this case geometric properties such as geodesicdistances are not preserved and hence the shapes are not nearly isometric.
An illustration can be seenin Figures 1(a) and 1(b), where the existing methods that assume near-isometry such as ZoomOut[51] and DIR [77] do not perform well at geodesic error
on the sample shape from TOPKIDS.
Theyeven produce a non-smooth map in the case of DIR.
Moreover, those methods that try to addressthis noise are computationally very expensive and do not generalize well to nearly-isometric shapes,because for nearly-isometric shapes stronger constraints (such as preserving geodesic distances)are needed for the match.
To illustrate this point, the spectral method GRAMPA is very expensiveas it needs to compute all the eigen-values and eigen-vectors of the adjacency matrices obtainedfrom the triangulations of each shape (see more details in Appendix B and time comparison with2D-GEM in Figures 2(a) and 2(b)).
Moreover, GRAMPA cannot generalize to cases of near-isometryas illustrated in Figure 3(a), which shows the matching of GRAMPA in an example from SCAPE, anearly-isometric shape dataset).
[Figure] [Figure]
Second, for nearly-isometric shapes, most existing methods attempt to first find an initial correspondence and then iteratively improve this initial correspondence using all points and the eigen-vectorscorresponding to lower frequencies (eigen-values) of the LBO [4, 55, 51] or using other shapedescriptors [61].
However, such an approach leads to an accumulation of errors in the correspondence,since the initial correspondence is mostly not accurate.
This problem has been discussed in detailsby Xiang et al.
[77], which attempted to address this via the Dual Iterative Refinement (DIR).
DIRfinds well matched points using the notion of Local Mapping Distortion (LMD) to identify points forwhich the geodesic distances are well preserved by the correspondence, and then use these points toupdate the correspondences of the other points via the notion of Functional maps [53].
Thus, DIRreduces the error accumulation.
However, the DIR approach only works for nearly-isometric shapesand not for non-isometric shapes since it is based on the assumption of preserving geodesic distanceswhich are not preserved in the case of non-isometries (see Figure 1(a), which shows that DIR does notperform well at geodesic 0 for non-isometric shapes).
Finally, even on nearly-isometric shapes, DIR,though effective, requires a lot of eigen-vectors of the LBO in order to perform well at geodesic error0.
This can be seen in Figure 3(b), where using up to 500 eigen-vectors gives only 25% accuracy atgeodesic error 0 on the sample shape from SCAPE, and even with up to 1000 eigen-vectors it onlyachieves 69 .
5% correspondence accuracy on SCAPE and a 59 .
8% on TOSCA at geodesic error
(asreported in the experiments in Section 5).
Given these problems, we propose to utilize 2D graph convolution for (a) non-isometric shapematching (since it is a de-noising operation), and (b) for isometric shape matching by furtherconstraining our framework to preserve geodesic distances.
To this end:
• we show how graph matching can be done via the 2D graph convolution, and re-define noisein the context of graph matching.
We further show empirically that this approach can matchthe recent spectral method GRAMPA on Erdos Regny random graph matching, as well as
outperform it in the presence of informative node features (see Appendix D),
• we propose a new 2D graph convolution-based approach to shape matching, called 2D-GEM,which is (a) robust to noise
and achieves high accuracy on exact matching of non-isometric
shapes while being orders of magnitude faster than GRAMPA, and (b) with some additionalconstraints also achieves high accuracy on exact matching of nearly-isometric shapes
2D graph Convolution for Joint Alignment
Here, we revisit graph convolution, show how the joint alignment problem can be done via 2D graphconvolution, and finally we re-define noise in the context of graph matching.
2.1 Graph Convolution
From the Graph Signal Processing [64] perspective, graph convolution is regarded as a frequencyfiltering operator over node attributes.
Given a graph, with node attribute matrix X ∈ R n × d , itsnormalized symmetric Laplacian (referred to as Laplacian in the rest of the paper) can be eigendecomposed as L = U Λ U T , where Λ = diag ( λ 1 , · · · , λ n ) ∈ R n × n are the eigen-values inincreasing order, and U =
[ u 1 , · · · , u n ] ∈ R n × n are the associated orthogonal eigen-vectors.
The graph convolution operation applied to the graph signal x (a column of X ) consists of threesteps [42, 22, 27]: (1) transform node attributes into the spectral domain (
i.e., z = U T x ); (2) removeattribute noise by scaling the spectrum with a function defined on eigen-values
(i.e., ¯ z
i
=
g
( λ i ) z i ); (3)perform inverse transformation on the scaled spectrum (i.e., ¯ x = U ¯ z ), where g ( λ ) is a scalar-valuedfunction on the eigen-values λ .
In short, graph convolution is able to remove the noisy signals byassigning smaller weight on higher eigen-value spectrum with a well defined function g ( λ ) .
Forconvenience, we denote g ( L ) =
U g ( Λ ) U T , so that we have the convolution as g ∗ X
= g ( L ) X .
Werefer to g ( L ) as the convolution function and g ( Λ ) as the filter function or simply filter.
2.2 Joint Alignment via 2D graph Convolution
Following previous works [20, 68] and Appendix C, the graph convolution is invariant to permutation.
As such, given two graphs A and B with graph convolution functions M A = g ( L A ) , M B = h ( L B ) ∈ 126 R n × n , features X A , X B ∈ R n × d , and having the functions g
and h to be the same function, the jointalignment problem in Equation (1) can instead be performed as:
[Equation]
see Appendix C for more details.
We observe that (2) reduces to the Linear Assignment Problem(LAP):
[Equation]
where Tr ( · ) is the matrix trace operator, the convolution functions capture the pair-wise descriptors,and the features capture the point-wise descriptors.
The first benefit of the convolution is to help do thejoint alignment as a Linear Assignment Problem (LAP), which is much cheaper than the traditional
Quadratic Assignment Problem (QAP) [21] used for graph matching.
Note that, M A
= U g (Λ) U T =
(cid:80) i g ( λ i ) u
i u Ti and M B = V h ( M ) V T =
(cid:80) j h ( µ j ) v j v Tj , where U , V are the orthogonal eigenvectors of M A and M B , and Λ , M are the diagonal matrices of their respective eigen-values.
Giventhese, one can rewrite our cost matrix C =
M A X
A X TB M B in (3) as:
[Equation]
When node features are not available, one can initialize all nodes with the same features in (3), i.e.,
X A = J A ∈ R n 1 × d , and X B = J B ∈ R n 2 × d , where n 1 may or may not be equal to n 2 for differentsize graphs, and J A , J B are all one rectangular matrices.
This is equivalent to assuming that nodeshave the same probability of being matched to each other.
Moreover, to save cost in the case whenthere is no feature, one could simply initialize the similarity matrix X A X TB =
J A,B ∈ R n 1 × n 2 (giventhat J
A J TB =
J A,B ∈ R n 1 × n 2 , where J A,B is a scalar matrix with constant [ J A,B ] ij = (cid:80) di
=1 1 = d )for all entries.
In the case where there is no feature, (4) becomes:
[Equation]
Our cost matrices in equations (4) and (5) can be seen as 2D graph convolution on the 2D graph signals
X A X TB (or J A,B in the case of no features) [43], where we: (1) transform the 2D graph signals intothe spectral domain to obtain the 2D graph spectrum (i.e., Z = U T X
A X T B V or Z
= U T J T A,B V )[43]; (2) remove noise (de-noise) from the 2D graph signals by scaling the 2D spectrum with afunction defined on each pair of eigen-values (i.e., ¯ z ij = w ( λ i , µ j ) z ij ); and (3) perform inversetransformation on the scaled 2D spectrum ( C =
U ¯ ZV T ).
Thus, our cost matrices are 2D filteringoperation on the 2D graph signals, where w ( λ i , µ j ) =
g ( λ i ) h ( µ j ) is the 2D filter (see Figures 4(a),4(b) and 4(c) for different 2D filters).
[Figure]
Redefining Noise in the Context of Graph Matching
In this section, we redefine noise in the context of graph matching and propose a suitable filter forgraph convolution with regards to the graph matching task.
It can be shown that graph convolution isa solution to the graph signal de-noising problem [22].
Formally, assuming that (1) the ground-truthnode features ˆ X are smooth w.r.t.
a graph G with adjacency A and Laplacian L , and that (2) the nodefeature noise has small magnitude, then the total variation of ˆ X should be small by assumption (1),and ||
ˆ X − X || can be upper-bounded by assumption (2).
Thus, we can model the problem ofrecovering the ground-truth signal as:
[Equation]
It can clearly be observed that ˆ X ∗ =
( I + α L ) − 1 X , where α is a hyperparameter.
This is a graphconvolution with graph convolution function given by ( I + α L ) − 1 .
This function can be decomposedas U ( I + α Λ) − 1 U T .
As well studied in [42, 75, 33], one can see that the filter ( I + α Λ) − 1 isdecaying, i.e., g ( λ i ) is lower for higher
λ i , which will mostly reduce the amplitude
z i of graphsignals u i belonging to larger eigen-values λ i .
Thus, one can conclude that noise corresponds tohigher order frequencies (eigen-values).
This kind of filter is commonly known as a low-pass filter[42].
It can be shown that for the 2D graph convolution, filters of the form w ( λ i , µ j ) =
g ( λ i ) h ( µ i ) ,where g ( λ i ) = (1 − αλ i ) k and h ( µ j )
= (1 − αµ j ) k will result in a 2D low-pass filter as well (see
Figures 4(a) and 4(b)).
These filters are referred to as k -hop low-pass filters (aggregating informationfrom the k -hop neighborhood of a node).
However, in the context of graph matching, graphs are isomorphic when they have the same eigenvalue, and in this case we want to align the eigen-vectors since two graphs A and B are isomorphiciff their eigen-vectors are permutations of each other up to a sign [69, 34].
This leads us to redefine noise in the graph matching context.
Given two graphs A and B , withconvolution functions M A , M B whose eigen-values are λ
i and µ j respectively, noise in graphmatching corresponds to 2D graph signals z i,j = u
Ti X A X TB v j or z i,
j = u
Ti J A,B v j such that 176 λ i ̸
= µ j , i.e., eigen-values that differ.
Thus, we should use a filter that kills off these signalscorresponding to eigen-values that do not match.
We choose the filter w ( λ i , µ j , e ) = 1 ( λ ei − µ ej ) 2 +1instead of g ( λ i ) h ( µ j ) .
This filter exactly kills off these signals corresponding to eigen-values thatdiffer (see Figure 4(c) to see an illustration of its action on signals).
A note here is that the amplitudeand the width of the filter response in Figure 4(c) depends on the spectra of the graphs to be matched.
For simplicity, we denote this filter as w ( λ i , µ j ) in the rest of the paper.
Thus, a noise robust 2Dgraph convolution based cost matrix for graph matching is given as:
[Equation]
Moreover, to reduce the time complexity, one may use the first k eigen-values Λ k = diag ( λ i , · · · , λ k ) ,
M k = diag ( µ i , · · · , µ k ) and eigen-vectors
U k
=
[ u i , · · · , u k ] , V k
= [ v i , · · · , v k ] , instead of thefull spectrum.
Doing so, the noise robust 2D graph convolution based cost matrix for graph matchingbecomes:
[Equation]
Due to space constraints, we refer the reader to Appendix D for the experimental validation of the 2Dgraph convolution approach for graph matching.
Our Algorithm
Having formulated the 2D graph convolution-based approach for graph matching as well as redefinednoise in the context of graph matching, we now formulate 2D-GEM (2D Graph convolution-basedframework for attempting Exact shape Matching) for non-isometric and nearly-isometric 3D shapematching.
4.1 2D-GEM
For our framework, given the xyz coordinates of two shapes M 1 and M 2 , we build suitable point-wisedescriptors X A and X B , as well as graph laplacians L A and L B using their triangulations.
We theninitialize the correspondence using the point-wise descriptors.
Then at each iteration, we first follow
GRAMPA and update the correspondence using the 2-hop neighborhood graph matching algorithm[81], which makes use of the Greedy Maximum Weight Matching (GMWM) with computationalcomplexity O ( n 2 log n ) (see Algorithm 2 and Appendix B).
The 2-hop adjacency matrices A
and
B 2 of the shapes M 1 and M 2 , respectively, are such that [ A 2 ] i,j = 1 and [ B 2 ] i,j =
if i andj are 2-hop neighbors, and 0 otherwise.
Second, we use the notion of conservation of geodesicdistances [61] to further constrain 2D-GEM, we capture this via LMD[77].
Finally, we adopt the 2Dgraph convolution in Equation (8) to filter the correspondence P rather than the feature similaritymatrix
X
A X TB in Equation (8), i.e., we use P rather than X A X TB as our 2D graph signals.
This can beseen as de-noising the correspondence at each iteration following the state-of-the-art Kernel-Matchingmethod [39].
In details (see Algorithm 1), we initialize the correspondence via SHOT[67], and then at each iteration( iter ):
• we update the correspondence using the 2-Hop Algorithm (2) with t = 1 ,
• we find l anchor pairs and p non-anchor pairs via LMD (see Section 4.2) and use them to update the 2D graph spectrum as
[Equation]
where U k [ l ] , V k
[ l ] ∈ R l × k and P l ∈ R l × l are the eigen-vectors and correspondence of thel -anchor pairs (landmarks found, and Z l is the updated k × k 2D
graph spectrum
• we use the updated spectrum for updating the correspondence of poorly matched points asmax P p
Tr ( P p C p ) , where
[Equation]
and u i [ p ] , v j [ p ] ∈ R p are the eigen-vectors of the non-landmark points p .
Computation complexity.
The complexity for computing k leading eigen-vectors of a n × nsparse matrix is O ( kn log n ) .
The complexity for checking LMD is O ( n ) following DIR [77].
The GMWM has a computational complexity of O ( n 2 log n ) .
Thus, our method is of complexity
O (( iter ) n 2 log n ) =
O ( n 2 log n ) , since iter is usually small, e.g., 10 or 60 in our experiments.
4.2 Local Mapping Distortion (LMD)
Here, we describe the LMD employed by 2D-GEM to get landmarks.
The LMD was introducedby [77] and [76] for nearly-isometric shapes.
Let T : M 1 → M 2 be a map function between twoshapes.
The LMD of the map T at the point x
i is given as follows:
[Equation] [Equation]
where B γ ( x ) = { y ∈ M 1 |
d M 1 ( x, y ) ≤ γ } is the γ -geodesic ball of x , A 1 is the area element ofthe mesh of shape M 1 , and DE T ( x, y ) =
|
d M 1 ( x, y ) − d M 2 ( T ( x ) , T ( y ))
| /γ represents a pair-wisedistance distortion of mapping nearby points x i and x j to T ( x i ) and T ( x j ) .
A smaller value of
D γ ( T )( x i ) means a better map continuity of T at the point x
i , in other words, the local distance atthe point x
i is well preserved.
Based on the above definition of LMD, one can check that if T is anisometric map, then D γ ( T )( x ) = 0 , ∀ x ∈ M 1 , γ > 0 .
Conversely, if D γ ( T )( x ) = 0 , ∀ x ∈ M 1 forsome γ > 0 , then T is isometric.
We use the LMD to find well matched pairs, i.e., pairs for which{ ( x i , T ( x i ))
|D γ ( T )( x i ) ≤ ϵ i } , where ϵ i is a threshold.
We call them anchor points (landmarks l ),and others non-anchor points (poor-points p ).
We use the landmarks to update the 2D graph spectrum
S kl , which we use to update the correspondence of non-landmark points (see Section 4.1).
It isimportant to note that at every iteration we gradually decrease the threshold following Xiang et al.[77], and hence not only is the correspondence T updated, but also the landmarks l .
For non-isometricshapes we relax this constraint by setting the thresholds
ϵ i arbitrarily large, such as 100 for 2D-GEMin the experiments (see Section 5.5).
Experiments
We report experimental results that validate the effectiveness and efficiency of 2D-GEM on exactmatching of non-isometric and nearly-isometric 3D shapes.
5.1 Experimental Set-up
Time comparison between 2D-GEM and GRAMPA is conducted in Matlab on a Windows 10 systemwith 16GB RAM and Intel(R) i5 11400 CPU @ 2.60-4.4GHz.
5.2 Datasets
We conduct experiments to evaluate the performance of our method on two nearly isometric benchmark datasets TOSCA [10], and SCAPE [5], as well as a benchmark non-isometric dataset SHREC’16(TOPKIDS)
TOSCA consists of 80 shapes in 8 different categories (human and animal shapes)with vertex numbers ranging from 4k to 50k.
SCAPE has 71 shapes (12,500 vertices for each) of thesame person with different poses.
TOPKIDS contains 25 shapes of the same class with up to 12Kvertices, undergoing near-isometric deformations in addition to large topological shortcuts (noise).
5.3 Evaluation Metrics
We use the geodesic error as our error metric [18].
Given that the correspondence of an algorithmmaps x ∈ M 1 to y ∈ M 2 , and the true correspondence maps x to y ∗ , the geodesic error is defined as
[Figure]
e ( x ) =
d M 2 ( y,y ∗ ) diam ( M 2 ) , where d M 2 denotes the geodesic distance on M 2 , and diam ( M 2 ) is the geodesicdiameter of M 2 .
5.4 Baselines
For non-isometric shape matching, i.e., for matching on the TOPKIDS dataset, we compare 2D-GEMwith the following methods: EM [59], GE [38], RF [58], PFM [57], FSPM [46], Kernel-Matching[39], and GRAMPA [18].
For near-isometric shape matching, i.e., for matching on the SCAPE and TOSCA datasets, wecompare 2D-GEM to the following methods: SGMDS[2], FM[53], BIM[31], Mobius[45], BestConformal [31], Kernel-Matching [39], DIR-500 [77] which uses 500 eigen-vectors, DIR-1000 [77]which uses 1000 eigen-vectors, and GRAMPA (on SCAPE, not on TOSCA due to its time complexityand the fact that 2D-GEM significantly outperforms it as shown on TOPKIDS and SCAPE).
5.5 Parameter Settings
For 2D-GEM , we initialize the correspondence by the GMWM on SHOT [67], and use e = 10 (see
Appendix D for more analysis on e ).
For nearly-isometric shapes (TOSCA and SCAPE), we use500 eigen-vectors of the LBO.
Following Xiang et al.
[77], we use the second ring neighborhoodsize for LMD criterion and set the maximum iteration number to iter = 10 .
We also set the LMDthresholds, we use [0 . 5280 , 0 . 4560 , 0 . 3840 , 0 .
For non-isometric shapes (TOPKIDS), we set the thresholds to 100 for all iterations, use iter = 60 ,and 10 LBO eigen-vectors.
For other baselines , we report their results as reported in their papers i.e., we use their curves.
Exceptfor GRAMPA where we re-run using the 2-hop improvement algorithm instead which performs muchbetter than the 1-hop, i.e., the adjacency.
For fair comparison with 2D-GEM , we set t = 60 for thisimprovement on GRAMPA.
5.6 Performance Analysis
Comparison on non-isometric shapes.
1 GRAMPA is the state-of-the-art non-isometric shapematching method that achieves an excellent performance on the exact matching task on the noisydataset TOPKIDS (Figure 5(a)).
It achieves 84 .
6% on this dataset at geodesic error 0.
This showsthat GRAMPA is very robust to noise as expected (note that this performance is much higher thanthe 60% reported in their original paper because we used the 2-hop adjacency instead of the 1-hopfor refinement as explained in Appendix B.2).
However, our method 2D-GEM further outperforms
GRAMPA and all other methods, including GRAMPA at geodesic error 0.
As reported in Figure 5(a),where
2D-GEM achieves a 90 . 5% accuracy.
Moreover, Figures 2(a) and 2(b) show that 2D-GEM issignificantly faster than GRAMPA since 2D-GEM only needs to compute a subset of the eigen-vectors,unlike GRAMPA which needs the whole spectrum and thus has a complexity of O ( n 3 ) .
Comparison on nearly-isometric shapes .
DIR is the state-of-the-art model for nearly-isometricshape matching.
DIR outperforms all the other baselines except 2D-GEM.
The better DIR-1000 using1000 eigen-vectors achieves 69 .
5% accuracy at geodesic error 0 on SCAPE, and 59 .
8% accuracyat geodesic error 0 on TOSCA.
Using the LMD proposed for the DIR model by Xiang et al.
[77]for extra constraints on 2D-GEM, 2D-GEM not only generalizes to nearly-isometric shapes, but itachieves the state-of-the-art results using only 500 eigen-vectors, outperforming even DIR-1000.2D-GEM achieves 92 .
5% accuracy (outperforming DIR-1000 by more than 40% ) at geodesic error 0on
the TOSCA dataset, as well as 84 .
9% accuracy on the SCAPE dataset (outperforming DIR-1000 byalmost 20% ) .
Note that these 2D-GEM’s accuracy improves using more eigen-vectors or increasingiter , but this also increases the running time.
Moreover, 2D-GEM’s gain over GRAMPA (the state-ofthe-art non-isometric shape matching method) can also be seen to extend to nearly-isometric shapes.
As noted earlier in Section 1, GRAMPA does not generalize to nearly-isometric shapes since theassumptions for nearly-isometric shape matching are different from those of non-sometric shapematching given that isometries preserve geometric properties which are lost by non-isometries.
Thisis seen by its poor performance on the SCAPE dataset (Figure 5(b), where even for exact matching,i.e., at geodesic error 0, it only achieves around 10% ).
In contrast, 2D-GEM (using the LMD for extraconstraints) generalizes to nearly-isometric shapes and achieves the state-of-the-art performance.
Wedo not attempt to use the LMD on GRAMPA since (1) it is not clear how to use GRAMPA in aniterative way as it is usually only used as an initialization for other algorithms (see Appendix B),(2) 2D-GEM outperforms GRAMPA even without the LMD as shown on the noisy TOPKIDS dataset(Figure 5(a)), and (3) 2D-GEM with or without the LMD is still faster than GRAMPA without the
LMD (see Figures 2(a) and 2(b)).
5.7 Limitations of 2D-GEM
There are three main limitations to this work.
First, for nearly-isometric shapes (shown in Figure 3(a)),2D-GEM sometimes produces maps that are not continuous (smooth), though much more accuratethan the other baselines.
Second, although the time complexity O ( n 2 log n ) of 2D-GEM is muchlower than the O ( n 3 ) of GRAMPA, it is much worse than the O ( n log n ) of other methods such as
ZoomOut and DIR.
Lastly, unlike GRAMPA, we do not provide rigorous theoretical exact recoveryguarantees for 2D-GEM on random graph matching.
Conclusions
We presented how the 2D graph convolution could be used for graph matching.
Using this 2Dgraph convolution, we re-defined noise in the context of graph matching and proposed a model(2D-GEM) that addresses the task of exact 3D shape matching on non-isometric (noisy) datasetsas well as on nearly-isometric datasets.
We then showed that 2D-GEM significantly outperformsexisting methods on both non-isometric shapes and nearly-isometric shapes.
For future work, wepropose to address two key limitations of 2D-GEM, i.e., further reducing the time complexity of
O ( n 2 log n ) and providing theoretical guarantees on its exact recovery performance.
Moreover, wealso propose to explore the generalization of 2D-GEM for correspondence tasks that allow usinggraph representation such as partial shape matching, image matching, and network de-anonymization.
Broader Impact
Given that our method needs no training, is unsupervised and based on intrinsic shape properties, itcan positively benefit the industry and society as a whole in that; (a) it may be easily generalized,and most importantly (b) it needs no human labels nor training, which can help reduce the cost andtime taken for the shape correspondence problems in different crucial fields such as medicine, andcomputer graphics.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes]
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] see Sections 1, 3 4, 5 and Appendix D
(b) Did you describe the limitations of your work?
[Yes] see Section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] see Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] see Sections 3,2.2 and Appendix C
(b) Did you include complete proofs of all theoretical results?
[Yes] see Sections listed inQuestion 2 (a) above.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] see code insupplementary material(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] see Section 5.5 amd Appendix D
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A](d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] see Section 5.14.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] see Section 5.5 andAppendix D.4(b)
Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]see code in supplementary material(d) Did you discuss whether
and how consent was obtained from people whose data you’reusing/curating?
[Yes] see Section 5.5 and Appendix {+D.
+}
(e) Did you discuss whether the data you are using/curating contains personally identifiable [Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?
[N/A]
[Table] (b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]
2D Graph
Appendices:A Related workA.1 point-wise Alignment
Given a set of point-wise descriptors X A ∈ R n 1 × d and X B ∈ R n 2 × d for two 3D shapes M 1and M 2 , such as the ones based on (1) the Laplacian Beltrami operator (LBO) e.g., Heat Kernel
Signatures (HKS) [11], Wave Kernel Signatures (WKS) [6], Global point Signatures (GPS)
[52],which all use a truncated basis of the eigen-functions (eigen-vectors) of the LBO rather than usingthe full basis, or given point-wise descriptors based on (2) the Geodesic distances e.g., Geodesicdistance descriptor (GDD) [61], or (3) others based on signatures of histograms such as SHOT [67],the point-wise alignment problem can be formulated as: [Equation]
[Equation]
As observed by Lähner et al.
[39], there are two main problems with this approach.
Which are:(a) despite the variety of point-wise shape descriptors proposed, most assume near-isometry (i.e.,assume no noise), and (b) shapes usually have intrinsic symmetries (e.g., humans left side and rightside are almost perfect copies of each other) which is not easily captured by point-wise descriptors.
The later problem will result in serious discontinuities in the map as some points will be mapped totheir desired location, but others to their symmetric counterparts e.g., the left eye and right eye of ahuman face both being mapped to the right eye of the other face.
As such For 2D-GEM we not onlyuse poit-wise descriptors, but combine them with pair-wise descriptors via the 2D-graph convolution.
A.2 pair
-wise Alignment.
Given a set of pair-wise descriptors A ∈ R n 1 × n 1 and B ∈ R n 2 × n 2 for two 3D shapes M 1 and M 2respectively, such as their; Laplacians, adjacency matrices, geodesic distance matrices, mass matrices,or stiffness matrices [76, 39], the alignment problem is given as the quadratic assignment problem(QAP)
[Equation]
The QAP problem is computationally intractable.
Thus, for 2D-GEM we employ the 2D-graphconvolution which helps us reduce the alignment problem to a Linear Assignment Problem (LAP).
It is notable to mention however that several relaxations have been proposed for the QAP problem(some of which are grouped as spectral methods and Quadratic Programming methods below).
Spectral Methods .
These methods replace the requirement for permutation matrices with a singleconstraint on the Frobenious norm of the matrices to obtain a maximal eigenvalue problem [62, 40, 47].
They are used both for testing graph isomorphism [34, 7] and for graph matching.
They usually reduce(13) into a Linear Assignment Problem (LAP) of the form max P Tr ( PC ) , where we try to find the
P permutation matrix that achieves the best alignment of the eigenvectors, where C is the similarity(cost) matrix of the eigen-vectors.
Given two graphs A and B with adjacencies A = (cid:80) i λ i u
i u Tiand B
=
(cid:80) j µ j v j v Tj , where λ i , u i are the eigenvalues and eigenvectors of A and µ j , v j those of
B , Umeyama [69] defined his cost function as C = (cid:80) i,j s i
u
i v Ti , for some appropriately chosensigns s i ∈ {± 1 } (similar to Xu and King [79]).
The motivation being that, this is the solution tothe orthogonal relaxation of (13), where the feasible set is relaxed to the set the orthogonal matrices[21]).
Furthermore Umeyama [69] suggested the construction C =
(cid:80) i,j | u i
|| v i | T , where | u i | is theelement wise absolute value of u i .
Kazemi et al.
[29] and Feizi et al.
[19] proposed other low rankcost functions, the simplest of which is C
= u 1 v T 1 .
Fan et al.
[18] proposed GRAMPA, suggestingthat using a combination of all eigenvectors rather than only those where i = j was significantly more resilient to noise and proposed C =
(cid:80) i,j w ( λ i , µ j ) u i
u Ti Jv j v Tj , where w ( λ, u ) is a monotonicallydecreasing function such as the Cauchy spectral weight w ( λ, µ )
= 1 ( λ − µ ) 2 + γ 2 , γ being a bandwidthparameter. Two other notable Spectral methods related to 2D-GEM are Fast Sinkhorn Filters [54] and
Structured Regularization of Functional Map Computations [56].
[54] provides a unifying frameworkfor spectral methods. Following the Meta Algorithm from [54], 2D-GEM can equally be seen asa spectral alignment regularised algorithm. [54] proposed the Sinkhorn filter instead of the KNNand Auction (and GMWM) matching.
It is worth considering to incorporate the sinkhorn filter into2D-GEM. Concerning the differences, besides the difference in the motivation for Fast Sinhorn Filters(Regularized optimal transport) and 2D-GEM (spectral graph theory), the main differences betweenthem are first the fact that the cost matrix proposed in Fast Sinkhorn Filters is a distance matrix(euclidean) between the eigen-vectors, while that in 2D-GEM is based on the spectrum of the graphsof the corresponding shapes. Second, rather than using single eigen-vectors on each shape, 2D-GEMuses pairs of eigen-vectors for its cost matrix. Finally, the regularizers used for 2D-GEM enforceboth neighbourhoods preservation (via the 2-Hop algorithm) and geodesic distortion preservation(via the LMD), while the regularizer in fast Sinkhorn Filters is mainly to make the problem strictlyconvex which gives the interpretation of a regularized transport plan.
[56] pointed out the fact thatonly penalizing the failure of the unknown map to commute with the Laplace-Beltrami operatoroperators on the source and target shapes may fail in some cases.
To address this problem, [56] furtherproposed additional regularizers to update the functional map (namely, a descriptor-commutativeregularizing term, an orientation-preserving term, and a resolvent term).
2D-GEM may be used toattempt to bridge the gap between the popular Functional Map and the 2D-Graph convolution.
In thisregard, 2D-GEM can also be interpreted as a regularised spectral alignment method to update themap.
Thus, it is related to [56] with the difference that the regularizers in our case are neighborhoodpreservation (enforced via the 2-Hop) and distortion preservation (enforced via the LMD) in additionto the fact that the map in our case is computed in the spectrum of the graphs associated with theshape’s triangulation rather than in the traditional shape LBO’s spectrum.
Quadratic Programming (QP), Doubly Stochastic (DS) and Semi Definite Programing (SDP)
Methods.
Here we have other relaxations of (13) e.g., denoting the set of permutation matrices in 703 R n × n by P , notice that (13) can be reformulated as:
[Equation]
Relaxing the set of permutation matrices Π to its convex hull (the Birkoff polytope of doublystochastic matrices P B ), leads to the doubly stochastic relaxation [66, 1].
Other methods here [16]add different regularization terms to (14) or use different relaxations like the Linear Programming(LP) [3] or the Semi Definite Programming (SDP) [30].
GRAMPA can also be placed in this categoryas its cost matrix is a solution to the regularized QP.
Neural Network Methods.
Various Graph neural networks (GNNs) have been proposed for variousgraph related tasks [32, 33, 24, 70, 68, 42, 72, 84, 75, 78, 72, 22].
It was recently shown that these
GNNs can work as the Weisfeiler-Lehman [78, 15].
It was shown that for the graph isomorphism taskamongst these GNNs, those that update a node’s representation as a sum of itself and its neighboursfeatures work better than those which use other aggregation techniques.
As such, many of these
GNNs have been used for the graph matching and graph similarity tasks [82, 25, 85, 73, 74, 8, 9, 44].
Many of these methods leverage the graph convolution for the shape matching task as well.
However,they perform the convolution on the features of each individual 3D shape as a way to combine thepair-wise and point-wise descriptors of each of the shapes and then feed these combined featuresinto a learning algorithm for supervised or unsupervised alignment using a suitable loss function[20].
Our method on the other hand is unsupervised and does not use neural nets, and our proposedfilter is derived from our new definition of noise in the context graph matching and can be regardedas a 2D graph convolution over the point-wise descriptor similarity space or over the initializedcorrespondence.
Other Methods.
Other related methods for the graph alignment task include; Symmetric-NonNegative-Matrix-Factorization methods for graph and bipartite graph matching [13, 37].
We equallyhave the popular Graduated Assignment [23] in this category, as well as the ISORANK algorithm[65] which is based on the adjacency matrix of product graphs.
Kollias et al.
[36] proposed Network
Similarity Decomposition (NSD) to approximate ISORANK by using decomposition techniques,thus avoiding the quadratic size of the product graph.
In “noisy” settings where the graphs are notexactly isomorphic, there have been many proposed algorithms [80, 48, 49, 14] besides GRAMPA.
A.3 Joint Alignment (
Both)
Using some of the aforementioned pair-wise descriptors A ∈ R n 1 × n 1 and B ∈ R n 2 × n 2 , and pointwise descriptors X A ∈ R n 1 × d and X B ∈ R n 2 × d for two 3D shapes M 1 and M 2 respectively , thesemethods are either of the form: [Equation]
[Equation]
such as SEQA, where the parameter α acts as a trade of parameter between the pair-wise alignmentand the point-wise alignment .
Or of the form:
[Equation]
[Equation]
such as Kernel-Match.
The first term in both (15) and (16) will normally lead to the QAP which iscomputationally intractable, but to avoid this, these two methods relax P to be in the set of doublystochastic matrices P B rather the set of permutation matrices Π .
One can observe that methods inthis category try to find a trade-off between the contribution of the point-wise descriptors and thoseof the pair-wise descriptors to the assignment problem.
Moreover these methods have traditionallybeen shown [39, 76] to outperform pair-wise alignment and point-wise alignment methods.
Hence,inspired by the performance of these methods, we proposed to leverage the 2D graph convolutioninstead for better performance at a relatively good time complexity as it equally leads to an LAPrather than the QAP.
Other methods in this category usually initialize the solution P from point-wise alignment by solving(12), and then iteratively update the solution using a relaxation of pair-wise alignment e.g., usinga truncated basis space via the notion of Functional Maps [53].
These methods include; ICP [4],
ZoomOut [51], and others [55, 61].
For nearly-isometric shapes, the approach used by most of these methods (i.e., first finding aninitial correspondence, and then iteratively improving this initial correspondence using all pointsand eigen-vectors corresponding to lower frequencies (eigen-values)), leads to an erroneous finalcorrespondence due to accumulation of errors in the updates given that the initial correspondence isusually not exact [77].
Moreover, these methods are usually not very noise robust as they usuallyassume near-isometry between the shapes, thus, they equally do not perform well on the task of exactmatching on non-isometric shapes.
Hence, for 2D-GEM we propose a 2D Graph convolution-basedframework that is robust to noise on non-isometric shapes, and can incorporate more tight constraints(such as preservation of geodesic distances) to reduce the error accumulation in the correspondenceupdate on near-isometric shapes.
B GRAMPA for Pair-wise Alignment
In this section we review GRAMPA [18] for 3D shape matching via pair-wise alignment .
This algorithm consists of two parts; (1) Formulation/Initialization of the permutation matrix, and (2) iterativeimprovement of the initialized correspondence.
B.1 Formulation
In graph theory, an isomorphism of graphs A and B is a bijection between the vertex sets of A and Bi.e.
, f : V ( A ) → V ( B ) ,
such that any two vertices u and v of A are adjacent in A if and only if f ( u )and f ( v ) are adjacent in B .
If an isomorphism exists between two graphs, then the graphs are calledisomorphic and denoted as A ≃ B .
From linear algebra, two graphs A and B are isomorphic iff theadjacency matrix of one can be expressed as the permutation of the rows and columns of the otheri.e., A = [Equation] BP
[Equation]
(17) where , and A , B ∈ R n × n are the adjacency matrices of A and B respectively.
As such, the graphmatching problem [41, 20, 73, 18, 12, 17] is the attempt at finding the permutation matrix P suchthat (17) holds.
This problem has several applications which include but are not limited to action identification incomputer vision [71], protein network matching in bioinformatics[63, 65], comparing molecules in cheminformatics [35], and linking user accounts (de-anonymization) in social network analysis [83].
GRAMPA was proposed to address this problem in a noisy setting, where both graphs may not beexactly isomorphic.
For 3D shape matching, graph matching can be seen as a pair-wise alignment problem in which theset of pair-wise descriptors are the adjacency matrices.
Using the adjacency matrices A ∈ R n 1 × n 1and B ∈ R n 2 × n 2 for two 3D shapes M 1 and M 2 respectively, the alignment problem is given as the
Quadratic Assignment Problem (QAP) [21] as in equation (13).
Several relaxations of this problem have been proposed [62, 40, 47, 69, 29, 19, 66, 1, 16, 3, 30].
Amongst the Spectral Relaxations, GRAMPA has been proven to achieve the SOTA results, beingtobust to high noise levels 2 .
GRAMPA’s relaxed alignment is max P
Tr ( P T C ) , where C is their costmatrix given as:
[Equation]
Tr ( · ) is the matrix Trace operator, w ( λ, µ ) =
1 ( λ − µ ) 2 + γ 2 , and γ the noise control parameter.
Thiscost matrix C in (18) was shown by Fan et al.
[18] to be the solution X to the regularized relaxed
QAP:
[Equation]
(19) first relaxing the set of permutations Π to its convex hull (the Birkhoff polytope of doubly stochasticmatrices), and further relaxing to only be required to satisfy the constraint 1X1 = n .
The proofbeing tedious, we refer the reader to their paper [18].
In the case where n 1 ̸ = n 2 , the permutation matrix P ∈ R n 2 × n 1 is adjusted to be a rectangularassignment matrix.
This problem has been widely addressed not only on 3D shapes, but also generallyon graphs.
B.2 2-Hop Iterative improvement
Once the permutation matrix P is obtained, GRAMPA proposed to use the iterative cleanup procedure:
P t +1 =
arg max
P ∈ Π
Tr ( P T AP t B ) .
More recently it has been suggested to use j-hop neighbourhood adjacency matrices A j and B j for the iterative improvement instead [81].
Specifically, YUet al.
[81] suggested to use the 2-hop neighborhood and gave performance guarantees.
The 2-hopadjacency matrices A 2 and B 2 (using the Delauney triangulations), are such that [ A 2 ] i,j = 1 and[ B 2 ] i,j = 1 if i and j are 2-hop neighbors, and
0 otherwise.
Using these, the iterative improvementbecomes P t +1 =
arg max
P ∈ Π
Tr ( P T A 2 P t B 2 ) .
B.3 Relationship Between 2D-GEM and GRAMPA
Looking at GRAMPA’s cost function in equation (18), one can see that using the adjacency instead ofthe Laplacian, as well as using no features, 2D-GEM as in equation (7) becomes exactly GRAMPAwith a similar but different filter w ( λ i , µ j ) .
As such, GRAMPA can also be seen as filtering [60] theinitialized indistinguishable 2D signals of graphs using the adjacency basis and a graph-matchingsuitable-filter (i.e., w ( λ, µ ) ), and then iteratively updating the correspondence i.e., GRAMPA isalso a filter (de-noise), then initialize, then update algorithm.
However, 2D-GEM as a 2D graphconvolution allows us to utilise features (such as the point-wise descriptors in the context of shapematching), which causes 2D-GEM to achieve better performance than GRAMPA when these featuresare informative.
Besides comparison on the 3D shape matching task, we equally do more empiricalcomparisons between 2D-GEM and GRAMPA on random graphs following the Erdos Regny model(see Appendix D).
These experiments show that indeed when we have features that are informative2D-GEM may not only outperform GRAMPA, but also the QP (i.e., the full Quadratic Assignment
Problem with full doubly stochastic constraints [18]) depending on how informative the features are.
C On the In-variance of the Graph Convolution to Permutation
In this section we want to review graph isomorphism from linear algebra and revisit the in-varianceof graph convolution to permutation.
C.1 Review of Isomorphism from Linear algebra
From linear algebra, to graphs A and B are isomorphic iff the adjacency matrix of one can beexpressed as the permutation of the rows and columns of the other i.e., A
[Equation]
(20) where P ∈ R n × n is a permutation matrix, and A , B ∈ R n × n are the adjacency matrices of A and B .
This implies that the diagonal degree matrix D A of A is also a permutation of the diagonal degreematrix D B of matrix B :
[Equation]
21) where d A , d B ∈ R n are the vectors of degree, 1 ∈ R n is the all ones vector, P1 = 1 .
This can bewritten in matrix form as:
[Equation]
where the extra P on the left of D B ensures that the permutation keeps the node degree with thecorresponding node index of the rows and columns of D B .
One can observe that a linear combinationof (20) and (22) will imply that isomorphism also implies that the Laplacian matrix of A is also apermutation of that of B , i.e., taking (22) - (20), we have:
[Equation]
(23) because Λ = M for isomorphic graphs, we also have g ( Λ ) =
h ( M ) where L A ∈ R n × n and
L B ∈ R n × n are the Laplacians of A and B respectively, Λ , M ∈ R n × n the eigen-value diagonalmatrices, g ( Λ ) , h ( M ) functions of the eigen-values (where g and h are the same function e.g.,g ( λ ) = 1 − αλ , and h ( µ ) =
1 − αµ ), and U , V and the eigen-vectors of the Laplacians L A and L Brespectively.
Moreover, if A and B are associated with node attributes X A ∈ R n × d and X B ∈ R n × d , one can alsoobserve that:
[Equation]
[Equation]
i.e., there exists a permutation of the node attribute matrix of B that reorders it into the node attributematrix of A .
However, one should note that here isomorphism implies equation (24) holds, butequation (24) holding does not necessarily imply isomorphism.
So, here we can be seen as usingsome form of relaxation.
Notice also that this relaxation is reasonable (and made by other baselines[39]) as in real life meaningful features often lead to some level of information on the permutationmatrix.
Putting (23) and (24) together, we have the in-variance of the graph convolution to permutation [20],i.e.
, for any convolution function h that allows the diagonalization h ( L B ) =
V h
( M ) V T we
havethat:
[Equation]
(25) where PP T =
P T P
=
I for permutation matrices P .
As such we have that for isomorphism,
[Equation]
(26) where M A =
U g ( Λ ) U T =
g ( L A ) is the convolution function for graph A , and likewise M B =
V h ( M ) V T = h ( L B ) is the convolution function for graph B .
As such the graph matching problemcan be reformulated using the convolution as:
[Equation]
[Figure] D Validation of 2D Graph Convolution for Graph Matching on Erdos RegnyRandom Graphs
In this section we conduct experiments to validate the effectiveness of utilizing features availablevia 2D graph convolution for graph matching.
We compare the 2D graph convolution (referred to as2D-GEM also in this Section, where we do not use any LMD nor 2-hop refinement, but just the costin equation (7) or equation (8) and then the normal rounding procedure "matchpairs" provided by
Matlab) with GRAMPA and few other methods (which all cannot utilize features).
We make use ofthe G ( n, q ) Erdos Regny random graphs model, where n determines the number of nodes and q thenumber of edges in the graph.
D.1 Datasets
Given a parent Erdos Regny random graph G ( n, q ) with adjacency G , we first sample a graph fromit which has the same number of nodes n but deleting edges (each independently) from it withprobability 1 − s , where s ∈ [0 , 1] to obtain a source graph with adjacency A ∼ G ( n, p ) , whereq ≜ ps .
We then define another graph (target graph) with adjacency B ∼ G ( n, p ) as:
[Equation]
where Bern ( s ) denotes a Bernoulli distribution with mean s .
Note that σ 2 = 1 − s can be seen as thesquare of the noise level determining the number of edges that differ between A and B .
Suppose weobserve a pair of graphs with adjacency matrices A and B ′ =
P T BP , where P is the permutationmatrix.
We then wish to recover P .
D.2 Evaluation Metrics
For these experiments on random Erdos Regny graphs, we visualise the % of correct correspondencesagainst the noise level σ .
D.3 Baselines
We compare 2D-GEM to GRAMPA [18], and others such as: Umeyama [69], Degree Profile (DP)[14], ISORANK [28], QP (where we follow [18] and use the relaxation of the permutation matrix inthe Quadratic Assignment Problem to a doubly stochastic matrix), EigenAlign [19], LowrankAlign[19] and TopEigenVec [18].
D.4 Parameter Settings
We use graphs with n = 100 and set p = 0 .
5 and σ starting from 0, and using a step size of 0.05 upto 0.8.
We repeat the experiments 10 times each and each time we generate features for the graphswe generate features X A for the graph with adjacency A and use X B = P T X
A
as features for B .
We conduct four different experiments and run each 10 times and plot the averages.
The details foreach setting for 2D-GEM are as follows (the other baselines are set following Fan et al.
[18] whokindly provided their source code):
• case (1): for figure (6(a))
we use indistinguishable features J A,B , and use all the eigen-valuesfor 2D-GEM as in equation (5) with e = 1 ,• case (2): for figure (6(b))
we use indistinguishable features J A,B , all eigen-values and usee = 10 ,• case (3): for figure figure (6(c))
we use features such that 10 % of randomly selected nodes
are distinguishable (i.e., 10 % of nodes in both graphs can be matched using these features)and use all the eigen-values for 2D-GEM as in equation (7), and e = • case (4): for figure figure (6(d))
we use features such that 10 % of randomly selected nodes
are distinguishable and use 20 eigen-values (instead of n = 100 eigen-values ) for 2D-GEM
as in equation (8), and e =
• case (5): for figure figure (6(e))
we use features such that all nodes are distinguishable and use e = 10 and all eigen-values for 2D-GEM as in equation (7), • case (6): for figure figure (6(f))
we use features such that all nodes are distinguishable and use e = 10 and 20 eigen-values (instead of n = 100 eigen-values ) for 2D-GEM as in equation (8),
D.5 Performance Analysis
[Table] Cases (1) and (2) show that without features, 2D-GEM with e = 1 performs worse than GRAMPAand even other methods such as DP and Umeyama, while with e = 10 , 2D-GEM performs on parwith GRAMPA.
This trend is in fact general and also occurs in other cases i.e., the performance of2D-GEM increases as e increases.
Cases (3)-(6) illustrate the fact that if we have features which area little informative even as little as 10 % (case (3)-(4)), 2D-GEM can outperform even GRAMPA,and if the features are fully informative (case (5)-(6)), 2D-GEM using all eigen-values (case (5)) canrecover the exact permutation matrix no mater the noise level in the adjacency, while 2D-GEM usingas little as 20 eigen-values also performs significantly well (case (6)).
These experiments demonstrate that using features in addition to the adjacency/Laplacian (i.e., pointwise descriptors together with pair-wise descriptors) will be more beneficial than using only either.
Moreover, these experiments also show that it is non-detrimental to assume that the permutation ofthe adjacency and those of the features correspond, as even when the features a non-informativesuch as in cases (1)-(2) where we used the all ones, the performance of 2D-GEM was on par with
GRAMPA, and the performance of 2D-GEM only got better as the features became more informative(cases (3)-(6)).
Moreover, in real life, most graphs are associated with features and for 3D shapes,one could extract point-wise descriptors as well as pair-wise descriptors.
A caution here is the casewhere the information contained in both do not agree with each other, though we did not test thiscase, it is obvious that in such a case the permutation matrix will be different.
Algorithm while 0 < i ≤ iter do and where C = (cid:80) ki,j w ( λ i , µ j ) u
i u Ti P i v j v j , end while return P t .
E Ablation Studies on 2D-GEM
Here we conduct ablation studies on 2D-GEM to show the effectiveness of the overall frameworkover its individual parts.
E.1 Set-up
We compare 5 different cases (scenarios).
For all cases, we use the same parameters as in Section 5.5,except that for all datasets we use 500 LBO eigen-vectors.
• case (1): we use 2D-GEM as it is,
• case (2): 2D-GEM without the 2-Hop improvement i
e., without step (2) in algorithm 1,• case (3): 2D-GEM without the LMD.
As in algorithm 3, with iter = 60 ,• case (4): 2D-GEM with only the 2-Hop algorithm i.e., without the LMD nor the 2D graphconvolution.
As in algorithm 3 without step (3),• case (5): 2D-GEM with only the 2D graph convolution i.e., without the LMD nor the 2-Hopalgorithm.
As in algorithm 3 without step (2). E.2 Comparison
[Figure]
Ablation on non-isometric shapes.
One can observe from Figure 7(b) that cases (1) and (4) haveexactly the same performance, since the plot of case (1) is shadowed by that of case (4).
Thissuggests that for non-isometric shape matching, given a good initialization, the 2-Hop algorithmis as strong as 2D-GEM’ framework (However, this is not the case on nearly-isometric shapes asseen in Figure 8(b) where it performs much worse than 2D-GEM).
This equally shows that the 2Dgraph convolution provides such a good initialization.
Moreover, it can be seen on this plot that the2D graph convolution by itself (i.e., case (5)) performs moderately at geodesic 0 though it producesnon-smooth maps.
On the other hand, it can be observed that 2D-GEM with only the 2D convolutionand the 2-Hop algorithm (i.e., case (3) ) performs a little worse than 2D-GEM as a whole and thanthe 2-Hop algorithm by itself.
It can also be seen on this plot that the LMD by itself (case (2) is reallynot suited for non-isometric shapes as for these shapes geodesic distances are not preserved.
Ablation on isometric shapes.
Figure 8(b) shows that 2D-GEM as a whole (i.e., case (1)) performsbest once again, and this time much better than the 2-Hop algorithm by itself (i.e., case (4)) since it [Figure] assumes conservation of geodesic distances (a constraint not accounted for by the 2-Hop algorithm).
Moreover, the 2D graph convolution by itself (i.e., case (5)) is also seen to be on par and evenoutmatching the 2-Hop algorithm, while being outmatched by the combination of both (i.e., case (3)).
Finally, it can equally be seen that the LMD (i.e., case (2)) is quite good for isometric shapes as ittakes into account the preservation of geometric properties such as geodesic distances.
However, the
LMD (i.e., case (2)) is still outmatched in the exact matching task by 2D-GEM (i.e., case (1)) as canbe seen on this figure at geodesic distance 0.
E.3 Ablation Conclusion
[Figure] Overall from the ablation ablation studies, one can see that 2D-GEM helps one with the flexibilityof incorporating stronger constraints such as preservation of geodesic distances or relaxing them,depending on the properties of the shapes at hand.
Moreover, one can also see that as on randomgraphs (see Section D), the 2D graph convolution can indeed serve as a good initialization. [Figure]
F More Comments and Initialization Studies
In this section we add some initialization studies for 2D-GEM 1.
For these studies we used SCAPE,
TOPKIDS and TOSCA-re-meshed [55], where TOSCA-re-meshed is the TOSCA dataset with eachshape re-meshed to 5,000 vertices.
[Figure]
As shown in Appendix D, 2D-GEM is robust to initializations on graphs and on shapes.
Whenno features or initialization are present, 2D-GEM is shown to be comparable with the spectralbaseline GRAMPA.
However, when an informative initialization or features is present, 2D-GEMperforms significantly better.
On 3D-shapes, 2D-GEm tends to maintain a good performance despitethe initializations as show in Figures 9(a), 9(b), and 10(a).
This means that 2D-GEM is robustto initializations on 3D-shapes.
Moreover, 2D-GEM itself following equation ?? .
An interesting
[Figure]
topic for future work is an in-depth investigation of the re-meshed shapes as well as shapes withvery different triangulation such as in SHRECK19 [50] (since a key component of 2D-GEM is theadjacency associated with the triangulation of the shapes).
For the plots in the figures, we used 100 eigen-vectors for the Heat Kernel Signature (HKS) and
Wave Kernel Signatures (WKS).
For GEM we used 500 eigen-vectors, maxiter 60, LMD 100 forall iterations on non-isometric sample from TOPKIDS, and LMD thresholds [0.5, 0.45, 0.4, 0.35,0.3] for non-isometric shapes where we repeat 0.3 for the rest of the iterations beyond 5.
DSHoTis directly using the shot feature (i.e., computing X
A X TB ) and using 2D-GEM as in equation 4 toinitialize the map and use this initialized map in the algorithm in Section 4.
See Figures 10(a), 9(a)and 9(b).
G On Non-Smooth Final Maps Returned By 2D-GEM
Concerning the non-smooth final maps produced by the algorithm, we will indeed try to dive deeperinto that in future extensions.
However, for now, we note that it may arise due to the initial mapbeing very poor and non smooth as pointed out by Consistent ZoomOut [26].
In most cases, this maybe reduced to an extent by (a) either using 2D-GEM in a ZoomOut-like manner, i.e., starting withlower eigen-values and iteratively increasing them (though this strategy works in some cases, it maynot always work as pointed out in [26]), or by (b) using more rounds of our pipeline, i.e., the LMD(since the LMD is an effective measure of the continuity and distortion of the map), the map updatevia convolution (for denoising the map), and the 2-Hop algorithm (since the 2-Hop algorithm alsoenforces neighbourhood preservation).
See Figures 11 and 12 where the initial map given by shotwas non-smooth.
H 2D-GEM’s Sensitivity to LMD Thresholds
As noted in Section 5, the LMD thresholds differently for 2D-GEM on isometric shapes and on
Isometric shape.
Our choices of parameters were motivated by the properties of the shape.
Namely,on non-isometric shapes where geometric properties are not preserved, we deactivate the constraint ofenforcing preservation of geometric distortions by setting higher values for the LMD (i.e., updatingthe 2D spectrum with more anchor points)
we used 100 in Section 5.
On isometric shapes we setlower values, thereby enforcing this constraint.
In this section, we conduct sensitivity studies on the
LMD using an isometric pair of shapes, i.e., two random meshes from the SCAPE dataset as well asa non-isometric pair of shapes, i.e., two shapes from the TOPKIDS dataset.
For all figures in this section we used 60 iterations and set 5 different ranges of values for the LMDthresholds as follows:
• +LMD1 corresponds to using thresholds as ϵ i = 100 for all iterations.• +LMD2 corresponds to using thresholds as ϵ i =
[100, 89, 78, 67, 56, 45, 34, 23, 12, 1] forthe first 10 iterations and 1 for the rest of the iterations.•
+LMD3 corresponds to using thresholds as ϵ i =
[1, 1.0, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2,0.1] for the first 10 iterations and
1 for the rest of the iterations.• +LMD4 corresponds to using thresholds as ϵ i =
[5.00, 4.6667, 4.3333, 4.0000, 3.6667,3.3333, 3.00, 2.6667, 2.3333, 2.00] for the first 10 iterations and 2.00 for the rest of theiterations.•
+LMD5 corresponds to using thresholds as ϵ i =
[3.00, 2.7333, 2.4667, 2.20, 1.9333, 1.6667,1.40, 1.1333, 0.8667, 0.60] for the first 10 iterations and 0.60 for the rest of the iterations.
Figures 13(a) and 13(b) show that +LMD3 achieves 100% accuracy on isometric shapes.
This showsthat values from 1 and decreasing at a good rate after every iterations are good, which validates the
[Figure]
use of the constraint of preserving geometric properties for isometric shape matching (as shown in
DIR[77]).
On non isometric shapes, +LMD1 achieves the best accuracy, showing that geometricdistortions are not preserved by non-isometrics, and so the LMD thresholds should be set high forthese shapes.