A Simple Way to Learn Metrics Between Attributed Graphs
Abstract
The choice of good distances and similarity measures between objects is important for many machine learning methods.
Therefore, many metric learning algorithms have been developed in recent years, mainly for Euclidean data, in order to improve performance of classiﬁcation or clustering methods.
However, due to difﬁculties in establishing computable, efﬁcient and differentiable distances between attributed graphs, few metric learning algorithms adapted to graphs have been developed despite the strong interest of the community.
In this paper, we address this issue by proposing a new Simple Graph Metric Learning - SGML - model with few trainable parameters based on Simple Graph Convolutional Neural NetworksSGCN - and elements of Optimal Transport theory.
This model allows us to build an appropriate distance from a database of labeled (attributed) graphs to improve the performance of simple classiﬁcation algorithms such as k -NN.
This distance can be quickly trained while maintaining good performance as illustrated by the experimental studies presented in this paper.
Introduction
Classiﬁcation of attributed graphs has received much attention in recent years because graphs are well suited to represent a broad class of data in ﬁelds such as chemistry, biology, computer science, etc [1, 2].
Advances were obtained, particularly thanks to the development of graph convolutional neural networks (GCN)
[3–6] of which many actually graph learning model can rely on [7, 8].
GCN has attracted interest in the recent years, due to their low computational cost, their ability to extract task-speciﬁc information, and their ease of training and integration into various models.
Some tackle classiﬁcation problems for attributed graphs by leveraging GCN: they characterize and build Euclidean representations for attributed graphs both in a supervised (e.g. [5, 9]) or unsupervised (e.g. [10, 11]) way.
Despite these achievements, classiﬁcation methods based on direct evaluation of similarity measures between graphs remain relevant since they can obtain similar, and in some cases even better, performance [12].
Currently, most of these methods work in a task-agnostic way.
However, given the diversity of graph datasets, we can not expect one similarity measure to be well suited for all of them, on all learning tasks.
Adapting similarity measures to speciﬁc datasets and related tasks help to improve their generality and their performance.
One of such approach is known as Metric Learning (hereafter ML), and has already been successful for Euclidean data.
Xing et al.
[13] were the ﬁrst to propose a Metric Learning method to improve a speciﬁc method and task ( k -means for clustering of Euclidean data).
This ﬁrst work sparked a strong interest in ML which led to the development of many methods [14–17] for
Euclidean data.
In contrast, few of these methods exist for attributed graphs.
Existing methods (e.g., [18]) rely on iterative procedures which are hardly differentiable, and this makes also scalability an issue.
In the state-of-the-art of classiﬁcation, neural networks tend to currently dominate in the literature, yet building simple and learned (hence adapted to data and task) similarity measures between attributed graphs remain a relevant issue for at least two reasons: it allows to step up simpler graph classiﬁcation algorithms, and also it allows to rely on graph kernels [1, 19] which are, as of today, as efﬁcient on numerous tasks as models relying on graph neural networks.
Our contribution.
To address the issue of scalability in Metric Learning for graphs, we propose here a novel graph ML method, called Simple Graph Metric Learning (SGML).
In the ﬁrst step, attributed graphs are coded as distributions by combining the attributes and the topology thanks to GCN.
Then, relying on Optimal Transport, we deﬁne a novel metric between these distributions, that we call Restricted Projected Wasserstein, RPW 2 for short.
RPW 2 is differentiable and has a quasi-linear complexity on the distribution size (in number of bins; this is also the number of nodes); it removes certain limitations of the well known Sliced Wasserstein (noted SW 2 )
RPW 2 is then used to build a parametric pseudo-metric between attributed graphs which then has also a quasi-linear complexity on the graph size (in the number of nodes).
The similarity measure proposed in SGML has a limited number of parameters, and it helps the model to scale efﬁciently.
Next, we focus on the the k-nearest neighbors (k-NN) method for classiﬁcation.
An advantage of using k-NN is that, if the learning set grows, one can exploit it at near zero additional cost (since it only requires to store these new data) on the contrary of SVMs that would require to retrain the whole data (a task quadratic in size).
Since many real datasets (e.g., graphs from social networks, or to detect anomalies on computer networks) are expected to have a growing size, this property is important for continual learning, and from an energetic and environmental stance to avoid costly retraining.
In order to use k-NN and train the distance, we propose a novel softmax-based loss function over class point clouds.
It appears to be novel in the context of graph ML and it leads to better results in the explored setting than the usual ML losses (i.e., those speciﬁcally built to improve k-NN for Euclidean data).
Our experiments show that SGML learns a metric increasing signiﬁcantly the k-NN performance, compared to state-of-art algorithms for graph similarity measures.
The article is organized as follows.
In Section 2, we discuss related works on graph metric learning and on optimal transport theory applied to the construction of attributed graphs similarity measures.
Section 3 provides useful notations and deﬁnitions needed for the present work.
The SGML model is deﬁned in Section 4.
Finally, in Section 5, we present various numerical experiments assessing the efﬁciency of our model.
These experiments show that in various conditions, SGML has great ability to build accurate distance with competitive performance with the state-of-the-art in classiﬁcation of graphs, both in context of k-NN and kernel-based methods, and that despite its limited number of parameters.
A main advantage of the proposed SGML method is also its simplicity, hence leading to a scalable and efﬁcient method for graph Metric Learning.
We conclude in Section 6.
Societal Impact
The contribution is essentially fundamental, and we do not see any direct and immediate potential negative societal impact.
Conversely, the scalability of the method will help to alleviate the energy consumption of ML on graphs.
Related Works 2.1 Graph Metric Learning
About ML for graphs, we can notably mention a series of works [21–23] that consist in learning a metric through Graph Edit Distance (GED).
The major disadvantage of these methods is the complexity of the computation of the GED which can be only done for very small graphs.
Following the introduction of GCN, an approach based on Siamese neural networks has been proposed in [24] for the study of brain connectivity signals, represented as graphs signals.
In this speciﬁc case, all graphs are the same and they differ only by the signal they carry.
This makes this method not applicable to most of datasets.
More recently models without neural networks have been proposed: [18] present Interpretable Graph Metric Learning which builds a similarity measure by counting the most relevant subgraphs to perform a classiﬁcation task.
However, their method cannot handle large graphs.
[25] proposes to learn a kernel based on graph persistent homology.
The resulting model is also efﬁcient, but it has the disadvantage of not being able to deal with discrete features in graphs.
As seen, existing work on graph ML are either limited by the assumptions made to build their model, or too costly, or not suitable to actually leverage simple (classiﬁcation) algorithms and increase their performance.
To obtain a simple graph ML procedure that is not itself too costly, we need to have a similarity measure between graphs that can be computed quickly.
To construct such a distance, recent works suggest that Optimal Transport is an appropriate tool.
2.2 Optimal Transport for Graphs
Optimal Transport (OT) has been put forward as a good approach to quickly compute similarity measures between graphs, relying on the the fact that it provides tools for computing metric between distributions [26].
Recent studies have shown that efﬁcient distances and kernels for graphs can be built from this theory.
Fused-Gromov-Wasserstein [27] is such a metric (distance in a mathematical sense) using OT to compare graphs through both their structures and attributes.
Notably it allows one to compute barycenter of a set of graphs, and interpolation between graphs.
Experimentally, it leads to good results in classiﬁcation.
Its bi-quadratic complexity in the size of graphs is its main drawback, even if it can be reduced to cubic cost with entropic regularization.
In [28] an OT based approach to compare graphs is developed.
It uses OT between speciﬁc signals on the graphs.
Thanks to a Gaussian distribution hypothesis, the analytical expression of the OT between these signals is derived.
While the model provides good results, it is initially limited to graphs having the same size, and a task of node alignment (which has a cubic complexity) must be performed.
[29] relaxes the condition on size, yet the focus remains on graph alignment of non attributed graphs.
[12] has proposed the Wasserstein Weisfeiler-Lehman (WWL) method which can be seen as an evolution of the previous work [28] without the two hypotheses, neither on the size of the graphs nor on the speciﬁcity of the graph signals.
In addition, a non trainable GCN is used to build task-agnostic characteristics which are then compared through OT.
This pseudo-metric is then used to build an efﬁcient kernel for graph classiﬁcation.
Unfortunately this model requires the computation of the optimal transport map which has a cubic cost (or quadratic with entropy regularization).
While these models are efﬁcient on classiﬁcation tasks, their complexity remains high, and they are not fast enough (being quadratic or more) to be incorporated in a framework of Metric Learning.
A part of our contribution is to provide such an optimal transport-based fast similarity measure for attributed graphs, with no restriction on the nature of the graphs (and their attributes) to be compared.
Background on Metric Learning and Optimal Transport
Notations.
Let us consider a ﬁnite dataset X
= { x i } | X | i =1 whose elements are in R q .
The dataset comes with a set of labels E
= { e i } | E | i
=1 and a labeling function E : X !
E .
We note P ( X ) ⇢ P ( R q ) the set of discrete probability over
X ⇢ R q .
x is the Dirac distribution centered in x .
We note d a metric on X .
It veriﬁes the following properties: Symmetry - 8 ( x , y )
2 X 2 , d ( x , y ) =
d ( y , x ) ; Identity of indiscernibles - 8 ( x , y ) 2 X 2 , d ( x , y ) = 0 , x = y ;
Triangle inequality - 8 ( x , y , z )
X 3 , d ( x , z )  d ( x , y )
+
d ( y , z ) .
d is referred to as a pseudo-metric when it follows these properties except the identity of indiscernibles.
In this article, the term "distance" will be used sometimes in an informal way as a synonym of discrepancy or measures of similarity.
Additionally the term "distribution" will always refer to discrete distribution.
3.1 Learning a metric
For ML, we suppose that a dataset X is given with the knowledge of two sets: S (similar) and D (dissimilar), containing pairs of some elements of X .
The goal is to build a parametric distance d ✓ in such a way that the pairs of elements in S should be close while the pairs in D should be far away 1 .
These sets are often built from the labeling function of X such that { x i , x j } 2 S if E ( x i ) =
E ( x j ) otherwise { x i , x j } 2 D .
An optimization problem depending on d ✓ , S and D is then deﬁned with a loss function
F suitable for the purpose:
[Equation]
We denote ✓ ⇤ the optimal parameters.
The interest for building such a distance d ✓ ⇤ with respect to information in D and S lies in the fact that X is often included in a larger set, containing elements which are not labeled.
The goal is that the obtained distance d ✓ ⇤ will ease learning algorithm to ﬁnd these missing labels.
A part of our work will be to introduce a new and suitable loss function F in metric learning literature for the problem of metric learning for graphs.
3.2 Optimal transport
Let us consider two ﬁnite datasets X , X 0 , and two distributions µ 2 P ( X ) et ⌫ 2 P
( X 0 ) on these sets:
[Equation]
with a i  0 , b i  0 , n = | X | , n 0 = | X 0 | , and
[Equation]
[Equation]
b i = 1 .
Given a continuous cost function c : R q ⇥ R q !
R + , one can build from optimal transport a metric between distributions with support in R q , the so-called 2-Wasserstein distance W 2 :
[Equation]
⇧ a,b is the set of joint distributions on X ⇥ X 0 , ⇡ =
P n,n 0 i,j =1 ⇡ i,j  ( x i , x [Equation] j ) whose marginals are the distributions µ =
P x 0 i 2 X 0 ⇡ ( · , x 0 i ) and ⌫ =
P x i 2 X ⇡ ( x i , · ) .
We note ⇡ ⇤ 2 ⇧ a,b the optimal distribution (or coupling, or map) giving the solution of this problem.
The cost function c is taken as 2-norm: c ( x i , x 0 j ) =
||
x i  x
j || 2 , leading hence to the 2-Wasserstein distance.
This deﬁnes an efﬁcient way to compare distributions.
One could use differentiable versions (w.r.t the parameters of a distribution) by considering the 1-Wasserstein [30] or the entropic regularization of W 2 [26, 31].
Still, they are not suitable for metric learning because of the (initial) complexity (when n = n 0 ) in
O ( n 3 log n ) , or O ( n 2 log( n )) with entropic regularization thanks to the Sinkhorn algorithm [26].
Sliced Wasserstein distance ( SW 2 ).
In order to drastically reduce the cost for computing the OT, [20] has proposed a modiﬁed metric SW 2 which consists to compare the measures µ and ⌫ via their one dimensional projections.
Let ✓ 2 S q  1 be a vector of the unit sphere of R q .
Distributions µ and ⌫ projected along ✓ are denoted µ ✓ =
[Equation]
i
[Equation]
i [Equation] ✓ and ⌫ ✓ = P x
[Equation]
2 X 0 b
i
x i · ✓ .
SW 2 is deﬁned as follows:
[Equation]
The advantage of this formulation stems from the quasi-linearity in n (or n 0 ) of the computation cost of W 2 distance between one dimensional distributions.
The integral can be estimated via a
Monte-Carlo sampling.
The complexity is then (when n 0  n ) at most O ( M ( n log n )) with M the number of samples (uniformly) drawn from S q  1 .
However, [32] shows that SW 2 is a biased downwards compared to W 2 , since the vector ✓ for projection determines at the same time the OT plans and also the cost of transport; this leads to a less effective distance.
Projected Wasserstein distance ( PW 2 ).
When n = n 0 , PW 2 is introduced by [32] in answer to previous limitations.
PW 2 is computed similarly as SW 2 , but for each projection ✓ , the one dimensional optimal transport plan ⇡ ✓ , ⇤ between µ ✓ and ⌫ ✓ is used with the original distributions µ and ⌫ so as to compute the transport cost:
[Equation]
They show that this formulation deﬁnes a metric, has good properties and is more suitable for several learning tasks, e.g. generative tasks or reinforcement learning.
Unfortunately their result holds only for uniform distributions of the same size.
Our method rely on an extended version of this deﬁnition, involving distributions of different sizes and not necessarily uniform.
A Simple and Scalable Graph Metric Learning
Let us consider a dataset G of attributed graphs with labeling set E and labeling function E .
For a given graph G 2 G having A as adjacency matrix, we call n the number of node of the graph.
Each node i of G carry features
X ( i, :) 2 R q ; thus X 2 R n ⇥ q is the matrix of attributes of the graph.
4.1 From graph to distribution
Previous works using OT (pseudo-)metric have shown that comparing graphs through the signal they carry is a good way to compare them; we follow this path.
The ﬁrst step of our learning method consists in the generation of features jointly representative of the structure of each graph G and the attributes of their nodes X .
We use for this purpose Simple GCN [6], a streamlined version of GCN in which all the intermediate non-linearities have been removed.
This choice is dictated by the need to strongly reduce the number of trainable parameters, and it accelerates the training without degrading its performance compared to other GCN.
This Simple GCN creates features as:
[Equation]
where X 2 R n ⇥ q are the initial attributes of the nodes, e
A =
A + I n (where I n is the identity matrix of R n ) and Y 2 R n ⇥ p are the features computed by SGCN.
The neighborhood exploration depth r of this GCN is one of the hyperparameters of the method, along with the dimension p of the extracted features Y .
The coefﬁcients of the matrix ⇥ 2 R q ⇥ p of this GCN are the (only) trainable weights of the method.
We will always choose p  q , so the method has at most q 2 trainable parameters.
From the extracted features Y , we deﬁne a uniform distribution whose suport is the nodes’ characteristics:
[Equation]
This ﬁrst step is similar to WWL [12], except that we consider a trainable GCN, ⇥ being the trainable parameters.
In eq.
(7), both the structure G and the attributes X are accounted for.
Next, we propose a novel way to evaluate the similarity between attributed graphs using these distributions.
4.2 From distributions to distance
The distances between graphs are computed as a distance between their representative distributions (Eq.
(7)) with OT; speciﬁcally, we propose a novel one, called Restricted Projected Wasserstein (and noted RPW 2 ) extending PW 2 previously introduced in [32].
Restricted Projected Sliced-Wasserstein.
In [32], PW 2 is only deﬁned for uniform distributions when n = n 0 .
We extend this deﬁnition to cases n 6 = n 0 , and we remove the constraint of uniformity.
We show in Appendix A.1 that it still deﬁnes a metric (on discrete distribution space).
In order to compute this quantity, we could rely on Monte-Carlo sampling, and the complexity would be O
( Mpn log( n )) .
This can be prohibitive due to the term pM .
In order to have an even more scalable model, we restrict the projections to be alongside the basis vectors { u k } p k =1 of R p only.
This choice stems from a spanning constraint that allows us to deﬁne a quantity (named Restricted PW 2 , or RPW 2 for short) verifying the identity of indiscernibles without increasing signiﬁcantly the computing time (see Appendix A.1).
This guarantees that
RPW 2 is also a metric of discrete distribution space as PW 2 (see also Appendix A.1). Therefore it can always distinguish distributions that are different but also (associated with the continuity) that, when two distributions are getting closer, then RPW 2 tends towards 0; this is important in ML context.
RPW 2 is expressed as:
[Equation]
Note that RPW 2 deﬁnes a metric therefore it should considered as such and not as an attempt to approximate PW 2 .
A major advantage of RPW 2 is that it is deﬁned by a deterministic formula; this avoids the variability introduced by a Monte-Carlo sampling (when one would need to evaluate PWor SW 2 ).
Anyway, we can notice that for a given u k , many ⇡ u k i,j may be optimal for the projected distribution on u k , while they may lead to different values when computing Eq.
In order to have an unambiguous and deterministic deﬁnition, in such cases we can choose among admissible optimal transport maps the one which minimizes Eq.
In fact it is even necessary for it to be a metric.
However since this case is quite rare, in our implementation we simply took the ﬁrst one returned by our sorting algorithm.
The complexity of RPW 2 is given by O ( p 2 n log( n )) which saves a factor Mp as compared to PW 2 and this term is often greater than 10.
From RPW 2 , we deﬁne a parametric distance d RPW 2 ⇥ between two attributed graphs ( G , X ) and ( G 0 , X 0 ) :
[Equation]
All the metric learning experiments will be conducted using this distance, excepted in an ablative study where we report the use of SW 2 and PW 2 .
Note that d RPW 2 ⇥ is a pseudo-metric, since there are different graphs that can have the same features outputted by the GCN, hence leading to similar representative distribution, therefore the identity of indiscernibles is not veriﬁed.
4.3 Loss for training distance: the Nearest Class Cloud Metric Learning
The last element to complete our model is to deﬁne the loss function F for Eq.
We propose here a new loss function for the purpose of improving the k -nearest neighbors method.
Actually there are classical losses already efﬁcient for this purpose: one can notably mention Large Margin
Nearest Neighbor (LMNN) [15] and Neighbourhood Component Analysis (NCA)[14].
However, the optimization is done using a gradient descent algorithm.
Since computing all pairwise distances between graphs at each step of gradient descent would be intractable for large datasets, we have to train our loss in a batch way.
In this context, LMNN may be not relevant since this method works locally and a batch is often not representative of the true neighborhood of an element of the dataset.
On the contrary NCA loss can be trained in a batch way, as it relies on a probability model which tends to attract elements with the same label with each other, wherever they are.
However, preliminary experiments showed only a slight improvement of the k-NN with NCA.
Therefore we have constructed a new loss which proposes a different way to ensure the same condition (see
Appendix A.2) and which experimentally works better in our setting (see Ablative study, Sec. 5.4).
The model is called Nearest Cloud Class Metric Learning (NCCML); the probability of being labeled by e 2 E for a graph G depends on the distance to the point clouds of a class (hence the name of the method):
[Equation]
Given this probability, we want to construct the distance d RPW 2 ⇥ maximizing the probability that the labeled graphs in the dataset have the correct labels, which leads to solve the following problem:
[Equation]
By maximizing this loss, we construct a distance which, for each element, favors its relative distance to elements of the same labels compared to those of different labels.
This should favor k-NN, especially when k > 1 .
We will show in the experiments that, in this speciﬁc context, NCCML exhibits better performance than NCA.
More details on NCCML can be found in Appendix A.2.
4. Computational aspects
We test, in the next Section, the proposed metric learning method with RPW 2 (and SW 2 or PW 2 in ablative studies).
Optimization.
In terms of optimization, we can differentiate directly with respect to one dimensional distribution parameters of Wasserstein distance, thus we can also differentiate through RPW(Eq.
(8)) (and also approximation of SW 2 (Eq.
(4)) or PW 2 ).
Self-differentiation techniques can be used on these expressions (see [26]).
We implemented our algorithm in tensorflow 2 . The Algorithm
⇤ . Require: A dataset of attributed graphs G and their labeling function E . for each epoch
e 2 { 1 , . . .
, E } do Build a partition: [ k B k
=
G such that B k \ B k 0 = ; .
for each batch B k do for each graph pair ( G , G 0 )
2 B k ⇥ B k do
Compute distance d RPW 2 ⇥ ( G , G 0 ) (Eq. Compute  F B k ⇥ (Eq. and apply an iteration of Adam descent algorithm.
return all pairwise distance d RPW 2 ⇥ ⇤ in G .
minimization of the loss is performed by batch and stochastic gradient descent (in particular with the optimizer Adam [33]).
Parameters.
The following default parameters are used (unless otherwise indicated in the text): learning rate l r = 0 .
⇤ 10  2 , number of epochs E = 10 , batch size B = 8 , and the GCN output features size p = min(5 , q ) .
For experiments involving SW 2 and PW 2 , the sampling number is set to M = 50 which is a common value used in the literature.
Time complexity.
Theoretically, the training time is negligible compared to the computation of all pairwise distances; therefore we focus on this last step for the time complexity analysis (see
Appendix A.5 for runtimes per dataset).
If we denote ˜ n the number of average nodes of a graph, the total complexity of this computation with RPW 2 (resp.
SW 2 ) is given by O ( | G | ˜ n ( p 2 + ˜ nrp )
+ | G | 2 p 2 ˜ n log ˜ n ) (resp.
O ( | G | ˜ n ( p 2 + ˜ nrp )
+ | G | 2 pM ˜ n log ˜ n ) ).
The ﬁrst terms occur for application of GCN and the latest for computing distances.
In practice, for not too large ˜ n values, a quadratic implementation exploiting vectorization can be faster (see section 5.2).
Furthermore, one can see that the GCN becomes the limiting element for scaling (on graph sizes); in practice, the sparsity of the adjacency matrix and the optimizations on GPUs limit this problem.
However, it is still an active research topic to determine the less expensive ways to characterize the nodes [34, 35].
Spatial Complexity.
Our quadratic implementation mentioned above requires to store in memory a tensor of size O (˜ n 2 p ) for RPW 2 and O (˜ n 2 M ) for SW 2 or PW 2 .
The sequential implementation has a O (˜ n ) spatial complexity (more details on these implementations are in Appendix A.3).
Anyway for both implementations, for the dataset of graphs considered, SGML is very cheap in terms of memory consumption in regards of actual GPU capability.
Experiments 5.1 Datasets
For the experiments, we use a large panel of data sets from the literature [2] 3 : ENZYMES, PROTEINS,
IMDB-B, IMDB-M, MUTAG, BZER, COX2 and NCI1.
More information on these datasets can be found in Appendix A.4.
Additional details about the following experiments can be found in
Appendix A.7 for reproducibility.
When a dataset has discrete features, they are one-hot encoded.
5.2 RPW 2 Running times
We have generated uniform random (normal) distributions with support in R 5 of size ranging from 10 1 to 10 6 .
These sizes of the distributions correspond to graph sizes n (number of nodes).
The choice of R 5 is motivated by the usual good performance of ML when performed in small dimension.
We compare the running time to compute the distance between these distributions with W 2 , W e 2 , ( W 2 with entropic regularization parameter  = 100 ), SW 2 using POT [36] library, PW 2 and
RPW 2 .
For RPW 2 we compare both the quadratic and the sequential ( numpy ) implementations we developed.
The results can be found on Figure 1.
Additional details and results are given in Appendix A.6.
As expected SW 2 , PW 2 and RPW 2 are the methods scaling the best: we obtain the expected (quasi) linear slope for all three methods O ( n log n ) .
As soon as n > 10 4 , These three methods allows to compute distances between distributions of several orders of magnitude larger for the same time as W 2 and W e 2 .
Although SW 2 , PW 2 and RPW 2 scale mostly the same, SW 2 seems a bit faster than PW 2 and RPW 2 . However, the slope of RPW 2 is a little bit smaller than SW 2 one. (See Appendix A.6.)
Anyway, we will show in the next experiment (Sec. 5.4) that
RPW 2 builds better metrics than SW 2 .
Finally, note that the quadratic implementation of RPW 2 is the fastest for samples with less than 200 instances, which is the case for the datasets considered in the following experiments.
[Table]
We evaluate the method in two ways: by using k-NN directly on the computed distances, and by using a SVM with a custom kernel built from the model proposed.
We eventually compare the method to several (pseudo-) metric and distances from literature such as NetLSD [37], WWL [12], FGW [27].
k-Nearest Neighbors.
Datasets are split in a training (90%) and test set (10%).
For each of them we train RPW 2 following Algorithm 1 (in the appendix) on the training set with only one graphs datasets. The best accuracy are in bold green. Note that for PROTEINS, ENZYMES and
CUNEIFORM we concatenate continuous attributes with discrete attributes to build an extended continuous attributes (see Appendix A.7 for more details)
[Table]
hyperparameter to adjust: the depth of SGCN taken as r = { 1 , 2 , 3 , 4 } for all datasets, except for
MUTAG for which we go up to 7 .
The training is done for each parameter r during 10 epochs.
A 5-fold cross validation of the number of neighbors k
= { 1 , 2 , 3 , 5 , 7 } to be considered is performed on the training set using the considered distance.
Then for the best k ⇤ , we keep the associated validation accuracy, and we ﬁnally train a k-NN on the whole training set and evaluate its accuracy on test set.
This experiment is averaged on 10 runs.
The ﬁnal test accuracy retained is the one associated to the largest validation accuracy.
In this procedure, test set labels were never seen during neither training nor validation.
Results are given in the ﬁrst lines of Table 1 for graphs having labeled nodes and of Table 2 for graphs with continuous attributes.
The learning metric framework combined with k-NN allows us to obtain good performance in classiﬁcation tasks, in particular for datasets of graphs with continuous attributes.
The exception is ENZYMES where we can see a lower net performance.
For discrete attributes, SGML performs slightly below the state-of-the-art, yet it outperforms the existing distances classically combined with k-NN.
Experiments show that our graph ML distance framework is efﬁcient.
Note: This procedure is very similar to the one used by WWL, except that the parameter k is replaced by the corresponding parameters of their kernel (see next section).
SVM.
To compare to graph kernel methods, the experiment described in the previous section is reproduced using a SVM for classiﬁcation.
The kernel K RPW 2 = exp(   d RPW 2 ⇥ ⇤ ) is built from the constructed distance.
In this experiment, kernel hyperparameter  and SVM hyperparameter C are tuned similarly as the parameter k above.
The set of possible  (resp.
C ) values are 6 (resp.
12) regularly spaced values between 10  4 and 10 1 (resp.
10  4 and 10 5 including 1).
The results are provided in Table 1 (bottom part).
In this part of the table, one can see that the distance learned with our model performs as well as other
OT distances when used as a kernel, on the majority of the datasets.
We reach or are slightly above state of the art results on 5 datasets over 6 but are still below on NCI1.
We recall that our method is speciﬁcally designed for the k-nearest neighbors method and that its computational complexity is much lower than many of the best methods on these datasets (notably WWL and FGW ).
5.4 Ablative study
We perform experiments to justify the design choice of our model.
Speciﬁcally we show that these choices effectively help to improve k-NN performance by reproducing the experiments above (with k-NN) on different versions of the method without some (or all) of our propositions.
Raw model.
Without any of our novel propositions, the method would be equivalent to WWL, which corresponds to use the Wasserstein distance between distributions of Eq.
(7), where Y is generated with GIN [5], a non trainable GCN.
This speciﬁc case corresponds to the ﬁrst column denoted WWL of Table 3.
We see that even if there are datasets where there is a loss of performance, others beneﬁt from the learned metrics.
Moreover we remind that our distance is much less expensive to use than
W 2 on which WWL is based.
[Table]
SGML with SW 2 .
This second ablative study is in the second column, denoted SGML - SW 2 , of
Table 3, and is related to replacing RPW 2 by SW 2 .
The result clearly validates our choice to use
RPW 2 instead of SW 2 .
Our model is the best one except on one dataset.
SGML with NCA .
For this experiment we replaced the loss NCCML by the NCA loss.
The result is in the third column, SGML - NCA of Table 3.
It appears that NCCML is often more appropriate than NCA in our speciﬁc ML framework.
SGML with PW 2 .
For this ﬁnal experiment we used PW 2 instead of RPW 2 .
This experiments show that
PW 2 and RPW 2 have equivalent results.
This suggests that projecting only on the canonical basis is sufﬁciently informative while still being less costly.
Globally, the ablative study is in favor of the choices proposed for SGML.
The driving idea in the present article of choosing simple and scalable methods over more complex ones, leads to competitive performance while allowing full scalability.
Conclusion
In this article, we proposed a metric learning method for attributed graphs, speciﬁcally to increase the performance of k-NN.
We have shown experimentally that it can indeed achieve performance similar or even superior to the state of the art.
However, a theoretical work on the properties of RPW 2 will be useful to allow us to better understand when it does not perform well.
Appendix A.9 presents some additional elements on the limits of the work.
In addition, further work may easily adapt SGML to perform other tasks like graph clustering or regression, with an appropriate (and probably different)
ML loss.
Acknowledgements
The work has been was supported by the ACADEMICS Grant given by the IDEXLYON project of the Université de Lyon, as part of the "Programme Investissements d’Avenir" ANR-16-IDEX-0005, the ANR-19-CE48-0002 DARLING grant, and the GraphNEx CHIST-ERA ANR-21-CHR4grant.